Divergence detection method for branch roads, model training method, and device

Abstract

A divergence detection method for branch roads, a model training method, and a device, which can apply artificial intelligence technology to the field of autonomous driving. In the method, image or point cloud data of a surrounding environment of a vehicle can be acquired; and when the vehicle enters a preset range of a fork in a road, whether the vehicle diverges when driving through the fork in the road is determined by means of a machine learning model and on the basis of the image or point cloud data of the surrounding environment of the vehicle. Even in traffic scenarios such as tunnels, urban canyons, elevated roads and small-angle parallel branch roads, a vehicle can still acquire high-quality image or point cloud data of a surrounding environment, and can then determine, by means of a machine learning model, whether the vehicle has a divergence behavior when passing through a fork in the road, thereby helping to effectively detect whether the vehicle has a divergence behavior in more traffic scenarios.

Description

A method for detecting yaw of a bifurcated road, a model training method, and a device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on February 29, 2024, with application number 202410234583.8 and invention name “A yaw detection method for a forked road, a model training method and an apparatus”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of intelligent driving, and in particular to a method for detecting a yaw error on a forked road, a model training method, and a device. Background Art

[0003] With the rapid development of navigation technology, navigation systems have become an indispensable tool for vehicle driving. However, for various reasons, vehicles may still deviate from the original navigation route during driving, especially when passing through forks in the road. Therefore, it is necessary to detect vehicle deviation during the navigation process. For example, the global navigation satellite system (GNSS) can be used to obtain vehicle position information in real time. By comparing the vehicle position information with the navigation route, the vehicle can be determined whether the vehicle has deviated.

[0004] However, due to the generally weak GNSS signals in traffic scenarios such as tunnels, urban canyons, and under overpasses, vehicles may not be able to obtain accurate position information, making it difficult to effectively determine whether the vehicle has veered off course at a fork in the road. Furthermore, in scenarios with narrow parallel forks, the forks are close together, and the limited GNSS signal accuracy makes it impossible to distinguish which lane the vehicle is in. Therefore, a solution to accurately determine whether a vehicle has veered off course at a fork in the road is urgently needed. Summary of the Invention

[0005] The present application provides a method for detecting a yaw error at a fork in the road, a model training method, and a device, which are used to provide a solution for determining whether a vehicle has yawed when passing a fork in the road, which is conducive to effectively detecting whether a vehicle has yawed in more traffic scenarios.

[0006] This application provides the following technical solutions:

[0007] In a first aspect, the present application provides a method for detecting yaw at a fork in the road, which can apply artificial intelligence technology to the field of autonomous driving. In this method, an execution device can obtain environmental information around the vehicle, which can include images or point cloud data of the vehicle's surrounding environment; when the vehicle enters a preset range of a fork in the road, the execution device can determine whether the vehicle yaws when passing through the fork in the road based on the environmental information around the vehicle and the vehicle's navigation information through a machine learning model (hereinafter referred to as the "first machine learning model" for ease of description), wherein the vehicle's navigation information can determine the correct direction of the fork when the vehicle enters the fork in the road. Exemplarily, the execution device can be specifically manifested as a vehicle or a processor in the vehicle. A navigation map can be deployed in the execution device, and then it can be determined whether the vehicle has entered the preset range of the fork in the road based on the navigation map; for example, when the distance between the vehicle and the fork in the road is less than or equal to the preset distance, the vehicle is deemed to have entered the preset range of the fork in the road.

[0008] In this implementation, when the vehicle enters a preset range of a fork in the road, it can obtain images or point cloud data of the vehicle's surroundings, and then based on the images or point cloud data of the vehicle's surroundings, it can determine through a first machine learning model whether the vehicle has yawed when passing through the fork in the road according to the images or point cloud data of the vehicle's surroundings and the vehicle's navigation information. That is, a solution is provided for determining whether the vehicle has yawed when passing through a fork in the road. In addition, even in traffic scenarios such as tunnels, urban canyons, elevated roads, and small-angle parallel forks in the road, the vehicle can obtain high-quality images of the surrounding environment and high-quality point cloud data of the surrounding environment. Therefore, it can determine through the first machine learning model whether the vehicle has yawed when passing through the fork in the road, which is conducive to effectively detecting whether the vehicle has yawed in more traffic scenarios and providing users with a better navigation experience.

[0009] In one possible implementation, a forked road may include at least two forking directions, and an execution device determines whether the vehicle has veered off course at the fork in the road using a first machine learning model based on environmental information surrounding the vehicle and the vehicle's navigation information. Specifically, the execution device may generate prediction information using a first machine learning model based on the environmental information surrounding the vehicle, the prediction information being used to determine whether the vehicle has entered a first forking direction of the at least two forking directions; and then, based on the first forking direction and the vehicle's navigation information, determine whether the vehicle has veered off course at the fork in the road. For example, the execution device may determine a second forking direction that the vehicle needs to enter based on the navigation information, where the at least two forking directions include the second forking direction, i.e., the second forking direction represents the forking direction that the vehicle needs to enter based on the navigation information. If the first forking direction and the second forking direction are inconsistent, it can be determined that the vehicle has veered off course at the fork in the road. If the first forking direction and the second forking direction are consistent, it can be determined that the vehicle has not veered off course at the fork in the road.

[0010] In this implementation, if the prediction information generated by the first machine learning model is used to indicate which branch direction of the fork in the road the vehicle has entered, that is, the first machine learning model is used to select a branch direction from at least two directions included in the fork in the road, the implementation idea of ​​generating the aforementioned prediction information can be completed with the help of a classifier, that is, the first machine learning model can include a classifier, and the classifier outputs the prediction information. Then, when the prediction information indicates which branch direction of the fork in the road the vehicle has entered, it is an easier to implement solution.

[0011] In one possible implementation, the execution device acquires environmental information surrounding the vehicle, which may include: after determining that the vehicle has entered a preset range of a fork in the road, while the vehicle continues to travel, the execution device may acquire a forward-view image of the vehicle at a preset frequency (hereinafter referred to as the "first preset frequency" for ease of distinction), and the image of the vehicle's surrounding environment is the vehicle's forward-view image; since the execution device continuously acquires the vehicle's forward-view image at the first preset frequency, all acquired forward-view images may include a current forward-view image and historical forward-view images, where the current forward-view image refers to the forward-view image acquired at the current moment, and the historical forward-view image refers to the forward-view image acquired before the current moment. The execution device generates prediction information based on the environmental information surrounding the vehicle using a machine learning model, which may include: the vehicle inputting feature information of N historical forward-view images and the current forward-view image into a first machine learning model, and generating prediction information using the first machine learning model. Wherein, N is an integer greater than or equal to 1, for example, N can be 5, 6, 7, 8, or other values, and the feature information of each historical forward-view image is obtained by extracting features from the historical forward-view image. Optionally, the N historical forward-viewing images may be the N historical forward-viewing images acquired most recently in time with the current forward-viewing image.

[0012] In this implementation, the feature information of the historical forward-view image and the current forward-view image are input into the first machine learning model. The first machine learning model can then comprehensively utilize the information of multiple forward-view images to determine which fork direction the vehicle has entered. Since multiple historical forward-view images and the current forward-view image can better reflect the vehicle's driving process, that is, they can better reflect the process of the vehicle passing through the fork intersection, it is beneficial to improve the reliability of the first fork direction finally determined.

[0013] In one possible implementation, the execution device generates prediction information through a first machine learning model based on environmental information around the vehicle, which may include: in the process of continuously collecting forward-looking images at a first preset frequency, the execution device may repeatedly execute the input of feature information of each of the N historical forward-looking images and the current forward-looking image into the first machine learning model at a second preset frequency until a first preset condition is met; that is, the vehicle will repeatedly execute the input of feature information of each of the N historical forward-looking images and the current forward-looking image into the first machine learning model at least twice, and when the first preset condition is met, the vehicle can obtain at least two first prediction information through the first machine learning model. After obtaining at least two first prediction information, since each of the at least two first prediction information indicates that the vehicle has entered one of the at least two diverging directions included in the fork road, the vehicle can determine the first diverging direction by cumulative voting based on the at least two first prediction information.

[0014] In this implementation, since the vehicle obtains the vehicle's forward-view image at a preset frequency, the vehicle can continuously obtain new forward-view images. The vehicle repeatedly inputs the feature information of the historical forward-view image and the current forward-view image into the first machine learning model at least twice to obtain at least two first prediction information through the first machine learning model, and then based on the at least two first prediction information, the first bifurcation direction is finally determined by cumulative voting, that is, the first bifurcation direction is finally determined by integrating multiple first prediction information generated by the first machine learning model multiple times, which is beneficial to improving the reliability of the first bifurcation direction finally obtained.

[0015] Furthermore, when the vehicle repeatedly inputs the feature information of the historical forward view image and the current forward view image into the first machine learning model at least twice, the feature extraction of the historical forward view image has been performed in the historical rounds. Then, the feature information of the historical forward view image can be directly obtained from the historical rounds, and then the feature information of the historical forward view image and the current forward view image can be input into the first machine learning model, which saves the step of extracting the features of the historical forward view image through the first machine learning model, is beneficial to saving computer resources, and is also beneficial to improving the efficiency of the process of obtaining the first bifurcation direction.

[0016] In one possible implementation, the execution device obtains environmental information around the vehicle, including: during the driving process of the vehicle, obtaining a forward-view image of the vehicle at a first preset frequency, the image of the vehicle's surrounding environment is the forward-view image of the vehicle, and the forward-view image of the vehicle includes a current forward-view image and a historical forward-view image. The execution device generates prediction information through a first machine learning model based on the environmental information around the vehicle, including: the execution device inputs N historical forward-view images and the current forward-view image into the first machine learning model, and generates prediction information through the first machine learning model. In this implementation, the feature information of the historical forward-view image and the current forward-view image are input into the first machine learning model, and the first machine learning model can comprehensively utilize the information of multiple forward-view images to determine which branch direction of the forked road the vehicle has entered. Since multiple historical forward-view images and the current forward-view image can better reflect the driving process of the vehicle, that is, can better reflect the process of the vehicle passing through the fork in the road, it is beneficial to improve the reliability of the first branch direction finally determined.

[0017] In one possible implementation, the execution device generates prediction information using a first machine learning model based on environmental information surrounding the vehicle. This includes: the execution device generates prediction information using the first machine learning model based on the environmental information surrounding the vehicle and the vehicle's driving trajectory. In this implementation, not only the environmental information surrounding the vehicle but also the vehicle's driving trajectory are used to comprehensively determine which direction of the fork in the intersection the vehicle has entered, which helps improve the reliability of the ultimately determined first fork direction.

[0018] In one possible implementation, the environmental information surrounding the vehicle includes an image of the vehicle's surroundings, which can specifically be a forward-view image of the vehicle. While acquiring the forward-view image, the vehicle can also trigger the start of acquisition of the vehicle's driving trajectory for a period of time until a third preset condition is satisfied. The third preset condition can be that the vehicle's forward distance reaches a first distance after acquisition of the vehicle's driving trajectory begins, or that the vehicle's forward duration reaches a first duration after acquisition of the vehicle's driving trajectory begins, or that the vehicle reaches a third preset distance after passing a fork in the road.

[0019] The execution device generates prediction information through a first machine learning model based on environmental information around the vehicle and the vehicle's driving trajectory, including: the execution device projects the vehicle's driving trajectory into the vehicle's front view image to obtain a front view image carrying trajectory information; and then inputs the front view image carrying the trajectory information into the first machine learning model to generate prediction information through the first machine learning model.

[0020] In this implementation, since the vehicle's driving trajectory can well reflect the vehicle's position changes, after mapping the vehicle's driving trajectory to the vehicle's front view image, it can better reflect how the vehicle moves when passing through the fork in the road, thereby helping to improve the reliability of the first branching direction finally determined; in addition, another implementation scheme for generating the first branching direction is provided, which improves the implementation flexibility of this scheme.

[0021] In one possible implementation, the environmental information surrounding the vehicle includes images or point cloud data of the roadside surrounding the vehicle; optionally, the environmental information surrounding the vehicle also includes images or point cloud data of lane markings surrounding the vehicle. It should be noted that the images of the roadside surrounding the vehicle and the images of the lane markings surrounding the vehicle can be located in the same image or in different images; similarly, the point cloud data of the roadside surrounding the vehicle and the point cloud data of the lane markings surrounding the vehicle can be located in the same point cloud data or in different point cloud data. In this implementation, while acquiring the environmental information surrounding the vehicle, the vehicle can also trigger the start of acquisition of the vehicle's driving trajectory for a period of time in the future until a third preset condition is met.

[0022] The execution device generates prediction information through a first machine learning model based on environmental information around the vehicle and the vehicle's driving trajectory, including: the execution device obtains a first image based on the image or point cloud data of the curb around the vehicle and the vehicle's driving trajectory, and then inputs the first image into the first machine learning model to generate prediction information through the first machine learning model; wherein the first image includes a first line corresponding to the curb around the vehicle and a second line corresponding to the vehicle's driving trajectory.

[0023] Optionally, the execution device obtains a first image based on the image or point cloud data of the roadside around the vehicle, the vehicle's driving trajectory, and the image or point cloud data of the lane lines around the vehicle, and the first image also includes a third line corresponding to the lane lines around the vehicle.

[0024] In this implementation, since the vehicle's driving trajectory can well reflect the vehicle's position change, the vehicle's driving trajectory and the curbs around the vehicle are integrated into the same image, and the vehicle's driving trajectory and the curbs around the vehicle are displayed in the form of lines in the first image, which can intuitively reflect how the vehicle moves when passing through the fork in the road, which is conducive to improving the reliability of the first branching direction finally determined; and provides another implementation scheme for generating the first branching direction, which improves the implementation flexibility of this scheme.

[0025] In one possible implementation, the vehicle's trajectory includes the vehicle's displacement in a predetermined direction, where the predetermined direction includes at least one of the following: forward, left, right, or rearward. For example, the predetermined directions may include forward, left, right, and rearward. Optionally, the vehicle may also obtain velocity, acceleration, yaw angle, or other information over a period of time to assist in determining the vehicle's trajectory.

[0026] In this implementation, the information included in the vehicle's driving trajectory is clarified, which is conducive to improving the feasibility of this solution. In addition, the vehicle's displacement in various directions can be obtained without the help of a positioning system. The vehicle's driving trajectory is determined as the vehicle's displacement in a preset direction, which is conducive to obtaining an accurate driving trajectory in various traffic scenarios, and thus helps to improve the reliability of the final yaw detection result.

[0027] In one possible implementation, in this method, when determining whether the vehicle deviates when passing through a fork in the road, the execution device can also output prompt information through a display interface, and the prompt information includes a first fork direction and a second fork direction, and at least two fork directions include the second fork direction, and the second fork direction is the fork direction that the vehicle needs to enter based on the navigation information.

[0028] For example, the display interface may be a human-machine interface (HMI) display interface, a head-up display (HUD) or other types of display interfaces.

[0029] This prompt information can be used by the user to confirm whether the first bifurcation direction is correct, that is, to confirm whether the yaw detection result is correct. Optionally, the first bifurcation direction and the second bifurcation direction displayed in the display interface can be displayed in different colors, for example, the first bifurcation direction can be displayed in red and the second bifurcation direction can be displayed in green. Optionally, the first bifurcation direction can also be displayed in a flashing manner in the display interface.

[0030] In this implementation, when it is detected whether the vehicle has yawed when passing a fork in the road, the first fork direction confirmed by the vehicle and the second fork direction planned by the navigation can be promptly displayed to the user through the display interface, so that the user can confirm whether the yaw detection result obtained by the vehicle is accurate, and then determine whether to modify the navigation route. This is conducive to avoiding modification of the navigation route based on erroneous yaw detection results, and is also conducive to providing users with a better navigation experience.

[0031] In a second aspect, the present application provides a method for detecting yaw at a forked road, which can apply artificial intelligence technology to the field of autonomous driving. In this method, a training device obtains training data, the training data including environmental information surrounding a vehicle and a desired bifurcation direction corresponding to the environmental information surrounding the vehicle; based on the environmental information surrounding the vehicle, a machine learning model generates prediction information, the prediction information being used to determine whether the vehicle has entered a first bifurcation direction of at least two bifurcation directions included in a bifurcation intersection; and a loss function is used to train the machine learning model to obtain a trained machine learning model. The goal of the loss function training includes improving the similarity between the first bifurcation direction and the desired bifurcation direction, where the desired bifurcation direction represents the correct bifurcation direction of the at least two bifurcation directions entered by the vehicle.

[0032] In one possible implementation, the desired branching direction is determined based on the vehicle's trajectory and a high-precision map. Specifically, for a forked road area where a high-precision map is available, the vehicle's trajectory can be matched with the high-precision map to automatically determine which of the at least two branching directions included in the fork in the road the vehicle has entered (i.e., determine the desired branching direction).

[0033] In this implementation, the vehicle's driving trajectory and high-precision map can be combined to automatically perform true value labeling, that is, training data can be obtained without manual intervention, which is conducive to improving the convenience of the process of obtaining training data.

[0034] In the second aspect of the present application, the training device can also execute the steps performed by the device in the various possible implementation methods of the first aspect. For the meaning of the nouns in the second aspect of the present application and the various possible implementation methods of the second aspect, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the first aspect, and will not be repeated here one by one.

[0035] Thirdly, the present application provides a device for detecting a yaw error at a fork in the road, which can apply artificial intelligence technology to the field of autonomous driving. The device includes an acquisition module for acquiring environmental information surrounding a vehicle, including images or point cloud data of the vehicle's surroundings; and a determination module for determining, based on the environmental information and navigation information of the vehicle, whether the vehicle has yawed when passing through a fork in the road, using a machine learning model.

[0036] In one possible implementation, a forked road includes at least two forked directions, and a determination module is specifically used to generate prediction information through a machine learning model based on environmental information around the vehicle, and the prediction information is used to determine whether the vehicle has entered a first forked direction of at least two forked directions; based on the first forked direction and the navigation information of the vehicle, it is determined whether the vehicle has deviated at the fork in the road.

[0037] In one possible implementation, the acquisition module is specifically used to acquire the vehicle's forward-view image at a preset frequency, and the image of the vehicle's surrounding environment is the vehicle's forward-view image, and the vehicle's forward-view image includes a current forward-view image and a historical forward-view image; the determination module is specifically used to input feature information of the historical forward-view image and the current forward-view image into a machine learning model, and generate prediction information through the machine learning model, and the feature information of the historical forward-view image is obtained after feature extraction of the historical forward-view image.

[0038] In one possible implementation, the determination module is specifically used to repeatedly input the feature information of the historical forward-view image and the current forward-view image into the machine learning model at least twice to obtain at least two prediction information through the machine learning model, each prediction information indicating one of at least two bifurcation directions; based on the at least two prediction information, the first bifurcation direction is determined by cumulative voting.

[0039] In one possible implementation, the acquisition module is specifically used to acquire the vehicle's forward-view image at a preset frequency during vehicle driving, and the image of the vehicle's surrounding environment is the vehicle's forward-view image, and the vehicle's forward-view image includes a current forward-view image and a historical forward-view image; the determination module is specifically used to input the historical forward-view image and the current forward-view image into a machine learning model to generate prediction information through the machine learning model.

[0040] In one possible implementation, the determination module is specifically configured to generate prediction information through a machine learning model based on environmental information surrounding the vehicle and the vehicle's driving trajectory.

[0041] In one possible implementation, the image of the vehicle's surroundings is a front view image of the vehicle, and the determination module is specifically used to project the vehicle's driving trajectory into the vehicle's front view image to obtain a front view image carrying trajectory information; the front view image carrying the trajectory information is input into a machine learning model, and prediction information is generated through the machine learning model.

[0042] In one possible implementation, the environmental information around the vehicle includes images or point cloud data of the curb around the vehicle. The determination module is specifically used to obtain a first image based on the images or point cloud data of the curb around the vehicle and the vehicle's driving trajectory, where the first image includes a first line corresponding to the curb around the vehicle and a second line corresponding to the vehicle's driving trajectory; the first image is input into a machine learning model, and prediction information is generated through the machine learning model.

[0043] In one possible implementation, the driving trajectory of the vehicle includes the displacement of the vehicle in a preset direction, where the preset direction includes at least one of the following directions: front, left, or right.

[0044] In one possible implementation, the yaw detection device for a fork in the road also includes: an output module 1203, which is used to output prompt information through a display interface when determining whether a vehicle yaws when passing a fork in the road, the prompt information including a first fork direction and a second fork direction, at least two fork directions including the second fork direction, and the second fork direction being the fork direction that the vehicle needs to enter determined based on navigation information.

[0045] The specific implementation methods, meanings of terms and beneficial effects of the steps in the third aspect can all be referred to the first aspect and will not be repeated here.

[0046] In a fourth aspect, the present application provides a model training device that can apply artificial intelligence technology to the field of autonomous driving. The model training device includes: an acquisition module for acquiring training data, the training data including environmental information surrounding a vehicle and a desired bifurcation direction corresponding to the environmental information surrounding the vehicle; a generation module for generating prediction information using a machine learning model based on the environmental information surrounding the vehicle, the prediction information being used to determine whether the vehicle has entered a first bifurcation direction of at least two bifurcation directions included in a bifurcation intersection; and a training module for training the machine learning model using a loss function to obtain a trained machine learning model, wherein the goal of training using the loss function includes improving the similarity between the first bifurcation direction and the desired bifurcation direction, the desired bifurcation direction representing the correct bifurcation direction of the at least two bifurcation directions entered by the vehicle.

[0047] In one possible implementation, the desired bifurcation direction is determined based on the vehicle's driving trajectory and a high-precision map during driving.

[0048] In the fourth aspect of this application, the training device of the model is also used to execute the steps performed by the training device in the second aspect and various possible implementation methods of the second aspect. The specific implementation methods of the steps in the fourth aspect, the meanings of the terms and the beneficial effects brought about can all be referred to the second aspect and will not be repeated here.

[0049] In a fifth aspect, an embodiment of the present application provides a device comprising a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs, and the processor is used to execute the programs in the memory, so that the device executes the method described in the first or second aspect above.

[0050] In a sixth aspect, an embodiment of the present application provides a vehicle comprising a processor and a memory, wherein the processor is coupled to the memory, the memory being used to store programs; and the processor being used to execute programs in the memory, so that the vehicle executes the method described in the first aspect above.

[0051] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first or second aspect above.

[0052] In an eighth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program runs on a computer, it enables the computer to execute the method described in the first or second aspect above.

[0053] In a ninth aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the terminal device or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] FIG1 is a schematic diagram of the structure of an artificial intelligence main framework provided in an embodiment of the present application;

[0055] FIG2 is a schematic diagram of an architecture of a forked road yaw detection system provided in an embodiment of the present application;

[0056] FIG3 is a flow chart of a method for detecting a yaw error on a bifurcated road according to an embodiment of the present application;

[0057] FIG4 is a schematic diagram of a fork in the road provided in an embodiment of the present application;

[0058] FIG5 is a schematic diagram of a front view image acquired by a vehicle at a preset frequency according to an embodiment of the present application;

[0059] FIG6 is a schematic diagram of obtaining a plurality of first prediction information according to an embodiment of the present application;

[0060] FIG7 is a schematic diagram of a bifurcated road provided in an embodiment of the present application;

[0061] FIG8 is a schematic diagram of a front-view image carrying trajectory information provided by an embodiment of the present application;

[0062] FIG9 is a schematic diagram of a first image provided in an embodiment of the present application;

[0063] FIG10 is another schematic flow chart of a method for detecting a yaw error on a bifurcated road according to an embodiment of the present application;

[0064] FIG11 is a schematic diagram of prompt information provided in an embodiment of the present application;

[0065] FIG12 is a flow chart of a method for training a model according to an embodiment of the present application;

[0066] FIG13 is a schematic structural diagram of a device for detecting a yaw of a bifurcated road provided in an embodiment of the present application;

[0067] FIG14 is a schematic structural diagram of a model training device provided in an embodiment of the present application;

[0068] FIG15 is a schematic structural diagram of a device provided in an embodiment of the present application;

[0069] FIG16 is another schematic diagram of the structure of the device provided in an embodiment of the present application;

[0070] FIG17 is a schematic structural diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all of the embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0072] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances. This is merely a way of distinguishing when describing objects with the same properties in the embodiments of the present application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, so that a process, method, system, product or apparatus that includes a series of units is not necessarily limited to those units, but may include other units not expressly listed or inherent to these processes, methods, products or apparatuses.

[0073] In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information (such as the indication information described below) is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated can also be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated; it is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance, for example, the indication of specific information can be achieved with the help of the arrangement order of each information agreed in advance (such as predefined by the protocol), thereby reducing the indication overhead to a certain extent. The present application does not limit the specific method of indication. It is understandable that, for the sender of the indication information, the indication information can be used to indicate the information to be indicated, and for the receiver of the indication information, the indication information can be used to determine the information to be indicated.

[0074] First, let's describe the overall workflow of an AI system. See Figure 1, which shows a schematic diagram of the AI ​​framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0075] (1) Infrastructure

[0076] The infrastructure provides computing power for AI systems, enabling communication with the outside world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by smart chips, which can specifically adopt hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs); the basic platform includes related platform guarantees and support such as distributed computing frameworks and networks, and can include cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to obtain data, which is then provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0077] (2) Data

[0078] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0079] (3) Data processing

[0080] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0081] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0082] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0083] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0084] (4) General ability

[0085] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0086] (5) Smart products and industry applications

[0087] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart manufacturing, smart transportation, smart homes, smart medical care, smart security, autonomous driving, smart cities, etc.

[0088] The method provided in the present application can be applied to application scenarios in which whether a vehicle deviates from its course when passing a fork in the road is detected. For example, in the field of autonomous driving, the vehicle can provide a navigation function, and there may be a scenario in which whether a vehicle deviates from its course when passing a fork in the road is detected. The vehicle may be a car, truck, motorcycle, bus, boat, lawn mower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, airplane, helicopter, etc., and the embodiments of the present application do not impose any special limitations.

[0089] Related technologies often rely on GNSS to obtain real-time vehicle location information, which is then compared with the navigation route to determine whether the vehicle has deviated. However, in traffic scenarios such as tunnels, urban canyons, elevated roads, and narrow-angle parallel forks, the GANN signal received by the vehicle may be interfered with or completely blocked, resulting in the vehicle being unable to obtain accurate location information. Furthermore, in narrow-angle parallel forks, the forks are closely spaced, and the GNSS signal accuracy is limited, making it impossible to distinguish which lane the vehicle is in. Consequently, it is impossible to detect whether the vehicle has deviated when passing through a fork.

[0090] To address the above issues, the present application provides a method for detecting yaw at a forked road, which can accurately detect whether a vehicle has yawed when passing a forked road. The present application discloses: based on image or point cloud data of the vehicle's surrounding environment, a machine learning model (hereinafter referred to as the "first machine learning model" for ease of description) is used to detect whether a vehicle has yawed when passing a forked road.

[0091] Before providing a detailed introduction to the methods provided herein, the architecture of the forked road yaw detection system provided herein will be described. Please refer to Figure 2, which is a schematic diagram of the architecture of a forked road yaw detection system provided herein. As shown in Figure 2, forked road yaw detection system 200 includes a training device 210, a database 220, an execution device 230, and a data storage system 240. Execution device 230 includes a computing module 231.

[0092] The database 220 stores a training data set. During the training phase of the first machine learning model 201, the training device 210 generates the first machine learning model 201 and iteratively trains the first machine learning model 201 using the training data set, thereby obtaining a trained first machine learning model 201. The "trained first machine learning model 201" may also be referred to as the "trained first machine learning model 201." The first machine learning model 201 may be specifically represented by a neural network or a non-neural network model. In the embodiments of the present application, the first machine learning model 201 is described as a neural network.

[0093] The first machine learning model 201 obtained by the training device 210 after performing the training operation can be deployed to the computing module 231 of the execution device 230. The execution device 230 can call data, code, etc. in the data storage system 240, or store data, instructions, etc. in the data storage system 240. The data storage system 240 can be placed in the execution device 230, or the data storage system 240 can be an external memory relative to the execution device 230.

[0094] During the application stage of the first machine learning model 201, the execution device 230 can determine whether the vehicle deviates when passing through a fork in the road based on the image or point cloud data of the vehicle's surrounding environment through the trained first machine learning model 201.

[0095] In some embodiments of the present application, referring to FIG. 2 , the execution device 230 and the client device may be integrated into the same device, so that the user can directly interact with the execution device 230. For example, when the client device is a vehicle, the execution device 230 may be a module in the vehicle's host CPU that performs data processing using the first machine learning model. The execution device 230 may also be a graphics processing unit (GPU) or a neural network processor (NPU) in the vehicle. The GPU or NPU is mounted on the host processor as a coprocessor, and the host processor assigns tasks.

[0096] It is worth noting that FIG2 is only an architectural diagram of a yaw detection system for a fork in the road provided by an embodiment of the present invention, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in other embodiments of the present application, the execution device 230 and the client device can be independent devices, and the execution device 230 is configured with an input / output (I / O) interface to exchange data with the client device. The client device sends an image or point cloud data of the vehicle's surrounding environment to the execution device 230 through the I / O interface. After the execution device 230 determines whether the vehicle has yawed when passing through the fork in the road through the first machine learning model 201 in the computing module 231, it can return the result of the yaw detection to the client device through the I / O interface.

[0097] In combination with the above description, the following describes the specific implementation process of the training phase and the application phase in the method provided by this application.

[0098] 1. Application stage

[0099] In the embodiments of the present application, the application stage refers to the process in which the execution device uses the trained first machine learning model to determine whether the vehicle has yawed when passing through a fork in the road. The execution device can be specifically manifested as a vehicle or a processor in the vehicle. The subsequent embodiments are only described using the execution device manifested as a vehicle as an example. When the execution device manifests as other product forms, it can be understood by reference. The embodiments of the present application will not be described in detail. Specifically, please refer to Figure 3. Figure 3 is a flow chart of a fork in the road yaw detection method provided in the embodiments of the present application. The fork in the road yaw detection method provided in the embodiments of the present application may include:

[0100] 301. Acquire environmental information around the vehicle, where the environmental information around the vehicle includes images or point cloud data of the vehicle's surrounding environment.

[0101] 302. When the vehicle enters a preset range of a fork in the road, based on environmental information around the vehicle and navigation information of the vehicle, a first machine learning model is used to determine whether the vehicle deviates when passing through the fork in the road.

[0102] In an embodiment of the present application, after the vehicle obtains the environmental information around the vehicle, when the vehicle enters a preset range of a fork in the road, it can determine whether the vehicle deviates when passing through the fork in the road based on the environmental information around the vehicle and the navigation information of the vehicle through a trained first machine learning model.

[0103] Exemplarily, the first machine learning model can specifically adopt a convolutional neural network, a fully connected neural network, a residual neural network or other types of machine learning models, such as support vector machines (SVM), or a combination of the above various machine learning models, etc., which is not limited in the embodiments of the present application.

[0104] Among them, the fork road includes at least two fork directions. For example, the at least two fork directions may include a left fork direction and a right fork direction; for another example, the at least two fork directions may include a left fork direction and a straight fork direction; for another example, the at least two fork directions may include a straight fork direction and a right fork direction; for another example, the at least two fork directions may include a left fork direction, a straight fork direction and a right fork direction, etc. The specific fork directions can be determined in combination with the specific situation of the fork road and are not limited here.

[0105] For a more intuitive understanding of this solution, please refer to Figure 4, which is a schematic diagram of a fork in the road provided in an embodiment of the present application. The fork in the road shown in Figure 4 includes three forking directions: a left forking direction, a straight-ahead forking direction, and a right forking direction. It should be understood that the example in Figure 4 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0106] A specific implementation method for determining whether a vehicle has entered a preset range at a fork in the road. For example, in one implementation, a navigation map may be deployed in the vehicle, and the navigation map may be used to determine whether the vehicle has entered the preset range of the fork in the road. For example, when the distance between the vehicle and the fork in the road is less than or equal to the preset distance, the vehicle is considered to have entered the preset range of the fork in the road. For example, the preset distance may be 100 meters, 80 meters, 50 meters, or other values. The specific value of the preset distance may be determined in combination with actual application scenarios and is not limited in the embodiments of this application.

[0107] In another implementation, a trained second machine learning model may be deployed in the vehicle. The vehicle may capture forward-facing images and input the captured forward-facing images into the second machine learning model to obtain prediction information generated by the second machine learning model. The prediction information generated by the second machine learning model is used to indicate whether the vehicle has entered the preset range of the fork in the road. It should be noted that other methods may also be used to determine whether the vehicle has entered the preset range of the fork in the road. The examples provided here are merely to demonstrate the feasibility of this solution and are not intended to limit this solution.

[0108] Specifically, in one implementation, step 302 may include: the vehicle generates first prediction information through a first machine learning model based on environmental information around the vehicle, and the first prediction information is used to determine that the vehicle has entered a first diverging direction among at least two diverging directions; and then the vehicle can determine whether the vehicle has deviated at the fork in the road based on the first diverging direction and the vehicle's navigation information.

[0109] Exemplarily, the vehicle can determine the second divergence direction that the vehicle needs to enter based on the navigation information, and at least two divergence directions include the second divergence direction, that is, the second divergence direction represents the divergence direction that the vehicle needs to enter based on the navigation information; if the first divergence direction and the second divergence direction are inconsistent, it can be determined that the vehicle has deviated at the fork in the road; if the first divergence direction and the second divergence direction are consistent, it can be determined that the vehicle has not deviated at the fork in the road.

[0110] In an embodiment of the present application, if the prediction information generated by the first machine learning model is used to indicate which branch direction of the fork in the road the vehicle has entered, that is, the first machine learning model is used to select a branch direction from at least two directions included in the fork in the road, the implementation idea of ​​generating the aforementioned prediction information can be completed with the help of a classifier, that is, the first machine learning model can include a classifier, and the classifier outputs the prediction information. Then, when the prediction information indicates which branch direction of the fork in the road the vehicle has entered, it is an easier to implement solution.

[0111] Furthermore, in one case, the environmental information around the vehicle can be specifically expressed as a front-view image of the vehicle, that is, the environmental information around the vehicle includes an image of the environment around the vehicle, and the image of the environment around the vehicle is specifically expressed as a front-view image of the vehicle. Step 301 may include: after the vehicle is determined to have entered a preset range of a fork in the road, while the vehicle continues to travel, the front-view image of the vehicle can be acquired at a preset frequency (hereinafter referred to as the "first preset frequency" for the convenience of distinction). Since the vehicle continuously acquires the front-view image of the vehicle at the first preset frequency, all acquired front-view images may include a current front-view image and a historical front-view image, wherein the current front-view image refers to the front-view image acquired at the current moment, and the historical front-view image refers to the front-view image acquired before the current moment.

[0112] Among them, the first preset frequency can be that the vehicle performs a forward-view image collection operation once every first preset distance, for example, the first preset distance can be 5 meters, 8 meters, 10 meters or other values, etc.; or, the vehicle can also perform a forward-view image collection operation once every first time interval, for example, the first time interval can be 5 seconds, 10 seconds or other durations, etc., which are not limited in this application.

[0113] In order to more intuitively understand the concept of a vehicle acquiring a forward-looking image according to a first preset frequency, please refer to Figure 5. Figure 5 is a schematic diagram of a forward-looking image acquired by a vehicle according to a preset frequency according to an embodiment of the present application. Figure 5 takes three forward-looking images of the vehicle (i.e., image 1, image 2, and image 3 in Figure 5) as an example. The acquisition times of the three forward-looking images are t0, t1, and t2, respectively. As shown in Figure 5, the vehicle is getting closer and closer to the road sign on the roadside, where t2 represents the current moment, and t0 and t1 both represent the moment before the current moment. Image 3 is the current forward-looking image, and image 1 and image 2 are both historical forward-looking images. It should be understood that the example in Figure 5 is only for the convenience of understanding this solution and is not used to limit this solution.

[0114] In step 302, in one implementation, the vehicle generates first prediction information through a first machine learning model based on environmental information around the vehicle, which may include: the vehicle inputs feature information of each of N historical forward-view images and the current forward-view image into the first machine learning model to generate the first prediction information through the first machine learning model.

[0115] Wherein, N is an integer greater than or equal to 1. For example, the value of N can be 5, 6, 7, 8, or other values, and is not limited in the embodiments of the present application. The feature information of each historical foresight image is obtained by extracting features from the historical foresight image. Optionally, the N historical foresight images can be the N historical foresight images that were acquired most recently as the current foresight image.

[0116] Optionally, the first machine learning model may include a feature extraction network and a feature processing network. The feature information of each historical forward-view image may be obtained by extracting features from the historical forward-view image using the feature extraction network of the first machine learning model. Alternatively, an additional machine learning model may be deployed in the vehicle specifically for feature extraction of historical forward-view images.

[0117] Optionally, in the process of the vehicle continuously collecting forward-view images at a first preset frequency, the vehicle may repeatedly execute the input of the feature information of each of the N historical forward-view images and the current forward-view image into the first machine learning model at a second preset frequency until the first preset condition is met; that is, the vehicle will repeatedly execute the input of the feature information of each of the N historical forward-view images and the current forward-view image into the first machine learning model at least twice, and when the first preset condition is met, the vehicle can obtain at least two first prediction information through the first machine learning model. After obtaining at least two first prediction information, since each of the at least two first prediction information indicates that the vehicle has entered one of the at least two diverging directions included in the fork road, the vehicle can determine the first diverging direction by cumulative voting based on the at least two first prediction information.

[0118] For example, the second preset frequency can be consistent with the first preset frequency. That is, after each new forward-view image is captured, the vehicle once inputs the feature information of each of the N historical forward-view images and the current forward-view image into the first machine learning model, thereby obtaining first prediction information output by the first machine learning model. Alternatively, the second preset frequency can be inconsistent with the first preset frequency, which is not limited in the embodiments of the present application.

[0119] The first preset condition can be that the vehicle reaches a second preset distance after passing the fork in the road. For example, the second preset distance can be 50 meters, 100 meters, 150 meters or other values; or, the second preset condition can also be that the vehicle reaches a certain preset time after passing the fork in the road. The specific condition can be flexibly determined in combination with the actual application scenario and is not limited in the embodiments of the present application.

[0120] For a more intuitive understanding of this solution, please refer to FIG6 , which is a schematic diagram of obtaining multiple first prediction information provided by an embodiment of the present application. Because when the vehicle executes the input of the feature information of each historical forward-view image in the N historical forward-view images and the current forward-view image into the first machine learning model in the current round, the current forward-view image in the previous round becomes the historical forward-view image in the current round. In FIG6 , the vehicle will use the feature information of the current forward-view image obtained when the feature information of each historical forward-view image in the N historical forward-view images and the current forward-view image were input into the first machine learning model in the previous round as the feature information of the historical forward-view image in the current round. Therefore, before the vehicle inputs the feature information of each of the N historical forward-view images and the current forward-view image into the first machine learning model each time, it will obtain the feature information of the current forward-view image generated in the N rounds before the current round, and use the feature information of the current forward-view image generated in the N rounds before the current round as the feature information of each of the N historical forward-view images in the current round; then, the feature information of each of the N historical forward-view images and the current forward-view image are input into the first machine learning model to obtain the first prediction information generated in the current round; after the vehicle completes the operation of the current round, it will also update the feature information of the N historical forward-view images. It should be noted that since the vehicle cannot obtain the feature information of the N historical forward-view images when obtaining the first N forward-view images, the first N prediction information obtained by the vehicle when performing the first N operations can be discarded (that is, regarded as invalid information). It should be understood that the example in Figure 6 is only for the convenience of understanding this solution and is not used to limit this solution.

[0121] In step 302, in another implementation, the vehicle generates prediction information through a first machine learning model based on environmental information around the vehicle, which may include: the vehicle inputs N historical forward-view images and the current forward-view image into the first machine learning model, and generates first prediction information through the first machine learning model.

[0122] Optionally, in the process of the vehicle continuously collecting forward-view images at a first preset frequency, the vehicle may repeatedly input N historical forward-view images and the current forward-view image into the first machine learning model at a second preset frequency until the first preset condition is met; that is, the vehicle will repeatedly input N historical forward-view images and the current forward-view image into the first machine learning model at least twice, and when the first preset condition is met, the vehicle can obtain at least two first prediction information through the first machine learning model. After obtaining at least two first prediction information, since each of the at least two first prediction information indicates that the vehicle has entered one of the at least two diverging directions included in the fork road, the vehicle can determine the first diverging direction by cumulative voting based on the at least two first prediction information.

[0123] It should be noted that in the above two implementation methods, some prediction information generated by the first machine learning model may also indicate that it is impossible to distinguish which branching direction the vehicle is entering. For example, if the N historical forward-view images and the current forward-view image are all forward-view images before the vehicle enters the fork in the road, then the first machine learning model may not be able to distinguish which branching direction the vehicle is entering, that is, the first machine learning model generates invalid prediction information; for another example, if the N historical forward-view images and the current forward-view image are all forward-view images after the vehicle has passed the fork in the road, then the first machine learning model may also not be able to distinguish which branching direction the vehicle is entering; then the above-mentioned "at least two first prediction information" can represent at least two first prediction information obtained after screening all the prediction information generated by the first machine learning model, and each of the at least two first prediction information after screening can indicate that the vehicle has entered one of the at least two branching directions.

[0124] For a more intuitive understanding of this solution, please refer to FIG7 , which is a schematic diagram of a forked road provided in an embodiment of the present application. As shown in FIG7 , when the vehicle is far away from the forked road, or after the vehicle has passed a certain distance from the forked road, the prediction information generated by the first machine learning model based on the acquired N historical front-view images and the current front-view image may be invalid prediction information; when the vehicle is near the forked road, the prediction information generated by the first machine learning model based on the acquired N historical front-view images and the current front-view image may be valid prediction information. It should be understood that the example in FIG7 is only for the convenience of understanding this solution and is not used to limit this solution.

[0125] Regarding a specific implementation of "determining a first diverging direction based on the at least two first prediction information using a cumulative voting method" performed by a vehicle, for each of the at least two first prediction information, the vehicle may determine a score corresponding to each first prediction information; optionally, the vehicle may determine the score corresponding to each first prediction information based on the time at which each first prediction information was generated, wherein the closer the time at which each first prediction information was generated is to the current moment, the greater the score corresponding to the first prediction information. Since a forked road includes at least two diverging directions, and each of the at least two first prediction information indicates that the vehicle has entered one of the at least two diverging directions, the vehicle may divide the at least two first prediction information into at least one group based on the diverging directions, with first prediction information in the same group pointing to the same diverging direction, and first prediction information in different groups pointing to different diverging directions. The vehicle may sum the scores of all first prediction information in the same group to obtain a total score corresponding to the diverging direction; and the vehicle may determine the first diverging direction based on the total score corresponding to each of the at least two diverging directions.

[0126] Exemplarily, at least two diverging directions include a left diverging direction, a straight diverging direction, and a right diverging direction, and at least two first prediction information include first prediction information 1, first prediction information 2, first prediction information 3, first prediction information 4, first prediction information 5, and first prediction information 6, wherein the score of the first prediction information 1 is 1, and the first prediction information 1 indicates that the vehicle has entered the left diverging direction, the score of the first prediction information 2 is 1.2, and the first prediction information 2 indicates that the vehicle has entered the left diverging direction, the score of the first prediction information 3 is 1.6, and the first prediction information 3 indicates that the vehicle has entered the right diverging direction, and the score of the first prediction information 4 is 1. The score of the first prediction information 4 is 2, and the first prediction information 4 indicates that the vehicle has entered the right branching direction, the score of the first prediction information 5 is 3, and the first prediction information 5 indicates that the vehicle has entered the right branching direction, the score of the first prediction information 6 is 5, and the first prediction information 6 indicates that the vehicle has entered the right branching direction; the total score corresponding to the left branching direction is 2.2, the total score corresponding to the straight branching direction is 0, and the total score corresponding to the right branching direction is 11.6, then the vehicle can determine that the vehicle has entered the right branching direction when passing the fork intersection (that is, the first branching direction is the right branching direction). It should be understood that the examples here are only for the convenience of understanding this scheme and are not used to limit this scheme.

[0127] Further optionally, in order to improve the reliability of the first branching direction finally obtained, the vehicle can determine a score threshold. Only when the difference between the total score value corresponding to the first branching direction and the total score value corresponding to each branching direction other than the first branching direction in at least two branching directions is greater than or equal to the score threshold, the first branching direction is determined to be a valid detection result; if the difference between the total score value corresponding to the first branching direction and the total score value corresponding to any branching direction other than the first branching direction in at least two branching directions is less than the score threshold, it is determined that no valid detection result is obtained through the first machine learning model, that is, the first machine learning model cannot determine which branching direction the vehicle entered when passing the fork intersection.

[0128] For example, at least two bifurcation directions include a left bifurcation direction, a straight bifurcation direction, and a right bifurcation direction. The score threshold is 5, the total score corresponding to the left bifurcation direction is 2.2, the total score corresponding to the straight bifurcation direction is 0, and the total score corresponding to the right bifurcation direction is 11.6. Since the difference between the total score of the right bifurcation direction and the total score corresponding to the left bifurcation direction is greater than 5, and the difference between the total score of the right bifurcation direction and the total score corresponding to the straight bifurcation direction is greater than 5, the left bifurcation direction is determined to be a valid detection result.

[0129] For another example, at least two branching directions include a left branching direction, a straight branching direction, and a right branching direction. The score threshold is 5, the total score corresponding to the left branching direction is 0, the total score corresponding to the straight branching direction is 5, and the total score corresponding to the right branching direction is 8. Since the difference between the total score of the right branching direction and the total score corresponding to the straight branching direction is less than 5, it is determined that the first machine learning model cannot determine which branching direction the vehicle entered when passing the fork intersection. It should be understood that the example here is only for the convenience of understanding this solution and is not used to limit this solution.

[0130] Exemplarily, the at least two first prediction information can be specifically M first prediction information, where M is an integer greater than or equal to 2. In one case, the larger the value of M, the larger the value of the score threshold can be; in another case, the score threshold can be a fixed value that does not change with the change of M.

[0131] In an embodiment of the present application, the feature information of the historical forward-view image and the current forward-view image are input into the first machine learning model. The first machine learning model can then comprehensively utilize the information of multiple forward-view images to determine which fork direction the vehicle has entered. Since multiple historical forward-view images and the current forward-view image can better reflect the vehicle's driving process, that is, they can better reflect the process of the vehicle passing through the fork in the road, it is beneficial to improve the reliability of the first fork direction that is finally determined.

[0132] Since the vehicle obtains the vehicle's forward-view image at a preset frequency, the vehicle can continuously obtain new forward-view images. The vehicle repeatedly inputs the feature information of the historical forward-view image and the current forward-view image into the first machine learning model at least twice to obtain at least two first prediction information through the first machine learning model, and then based on the at least two first prediction information, the first bifurcation direction is finally determined by cumulative voting. That is, the first bifurcation direction is finally determined by integrating multiple first prediction information generated by the first machine learning model multiple times, which is beneficial to improving the reliability of the first bifurcation direction finally obtained.

[0133] Furthermore, when the vehicle repeatedly inputs the feature information of the historical forward view image and the current forward view image into the first machine learning model at least twice, the feature extraction of the historical forward view image has been performed in the historical rounds. Then, the feature information of the historical forward view image can be directly obtained from the historical rounds, and then the feature information of the historical forward view image and the current forward view image can be input into the first machine learning model, which saves the step of extracting the features of the historical forward view image through the first machine learning model, is beneficial to saving computer resources, and is also beneficial to improving the efficiency of the process of obtaining the first bifurcation direction.

[0134] In another case, the vehicle generates first prediction information through a first machine learning model based on environmental information around the vehicle, which may include: the vehicle generates first prediction information through a first machine learning model based on environmental information around the vehicle and the vehicle's driving trajectory.

[0135] For example, the vehicle's trajectory may include the vehicle's displacement in a preset direction, where the preset direction includes at least one of the following: front, left, right, or rear. For example, the preset directions may include front, left, right, and rear. Optionally, the vehicle may also obtain speed, acceleration, yaw angle, or other information over a period of time to assist in determining the vehicle's trajectory.

[0136] In the embodiment of the present application, it is clarified what information the vehicle's driving trajectory includes, which is conducive to improving the feasibility of the present solution; in addition, the vehicle's displacement in various directions can be obtained without the help of a positioning system, and the vehicle's driving trajectory is determined as the vehicle's displacement in a preset direction, which is conducive to obtaining an accurate driving trajectory in various traffic scenarios, and further helps to improve the reliability of the final yaw detection result.

[0137] Specifically, in one implementation, the environmental information surrounding the vehicle includes an image of the vehicle's surrounding environment, which can be specifically expressed as a front view image of the vehicle. While acquiring the front view image, the vehicle can also trigger the start of acquiring the vehicle's driving trajectory for a period of time in the future until a third preset condition is met; the third preset condition can be that the vehicle's forward distance reaches a first distance after the vehicle's driving trajectory is acquired; or, the third preset condition can be that the vehicle's forward duration reaches a first duration after the vehicle's driving trajectory is acquired; or, the third preset condition can be that the vehicle reaches a third preset distance after passing a fork in the road, etc., which can be determined in combination with actual application scenarios and are not limited in the embodiments of this application. In this implementation, the vehicle generates first prediction information through a first machine learning model based on the environmental information surrounding the vehicle and the vehicle's driving trajectory, which can include: the vehicle projects the vehicle's driving trajectory onto the vehicle's front view image to obtain a front view image carrying trajectory information; and then inputs the front view image carrying trajectory information into the first machine learning model to generate the first prediction information through the first machine learning model.

[0138] For a more intuitive understanding of this solution, please refer to Figure 8, which is a schematic diagram of a front-view image carrying trajectory information provided in an embodiment of the present application. Figure 8 includes two left and right sub-schematic diagrams. The left sub-schematic diagram and the right sub-schematic diagram in Figure 8 both represent front-view images carrying trajectory information. The difference between the left sub-schematic diagram and the right sub-schematic diagram in Figure 8 is that the distance between the vehicle and the fork in the road is farther in the left sub-schematic diagram, while the distance between the vehicle and the fork in the road is closer in the right sub-schematic diagram. It should be understood that the example in Figure 8 is only for the purpose of facilitating understanding of this solution and is not intended to limit this solution.

[0139] Optionally, the vehicle may acquire a forward-view image at a third preset frequency in step 301, and each time the forward-view image is acquired, trigger the start of acquiring the vehicle's driving trajectory for a period of time in the future until a third preset condition is met; wherein the meaning of the third preset frequency is similar to that of the first preset frequency, except that the third preset frequency is lower than the first preset frequency. For example, the third preset frequency may be executed once every 50 meters the vehicle travels. After each time the vehicle obtains a new forward-view image carrying trajectory information, the vehicle inputs the forward-view image carrying trajectory information into the first machine learning model, generates a first prediction information through the first machine learning model, and then determines the first bifurcation direction based on all the first prediction information generated by the first machine learning model.

[0140] It should be noted that since there may be discrepancies between the navigation map and the actual road conditions, some of the front-view images obtained by the vehicle are too far away from the fork in the road. After the front-view image carrying the trajectory information is input into the first machine learning model, the first machine learning model may not be able to determine which fork direction the vehicle has entered at the fork in the road. That is, some prediction information generated by the first machine learning model in this implementation may indicate that it is impossible to distinguish which fork direction the vehicle has entered. The operations of obtaining the front-view image, obtaining the driving trajectory, and inputting the front-view image carrying the trajectory information into the first machine learning model can be repeated multiple times according to the third preset frequency, so that multiple first prediction information generated by the first machine learning model can be obtained, and then the first fork direction can be determined based on all the first prediction information generated by the first machine learning model.

[0141] Exemplarily, since the third preset frequency is low, the vehicle determines the first divergence direction based on all the first prediction information generated by the first machine learning model, which may include: the vehicle filters out valid first prediction information from all the first prediction information generated by the first machine learning model, and the valid first prediction information represents a divergence direction that can clearly instruct the vehicle to drive into at least one of the two divergence directions; and determines the divergence direction pointed to by the valid first prediction information as the first divergence direction.

[0142] Optionally, if all the filtered valid first prediction information points to different bifurcation directions, the first bifurcation direction may be finally determined by cumulative voting based on all the valid first prediction information.

[0143] In an embodiment of the present application, since the vehicle's driving trajectory can well reflect the vehicle's position change, after mapping the vehicle's driving trajectory to the vehicle's front view image, it can better reflect how the vehicle moves when passing through the fork in the road, thereby helping to improve the reliability of the first branching direction finally determined; in addition, another implementation scheme for generating the first branching direction is provided, which improves the implementation flexibility of this scheme.

[0144] In another implementation, the environmental information surrounding the vehicle includes images or point cloud data of the roadside surrounding the vehicle; optionally, the environmental information surrounding the vehicle also includes images or point cloud data of lane lines surrounding the vehicle; it should be noted that the images of the roadside surrounding the vehicle and the images of the lane lines surrounding the vehicle can be located in the same image or in different images; similarly, the point cloud data of the roadside surrounding the vehicle and the point cloud data of the lane lines surrounding the vehicle can be located in the same point cloud data or in different point cloud data. In this implementation, while acquiring the surrounding environmental information, the vehicle can also trigger the start of acquiring the vehicle's driving trajectory for a period of time in the future until a third preset condition is met; the meaning of the "third preset condition" can be found in the above description and is not elaborated here.

[0145] In this implementation, the vehicle generates first prediction information through a first machine learning model based on environmental information around the vehicle and the vehicle's driving trajectory, which may include: the vehicle obtains a first image based on images or point cloud data of the roadside around the vehicle and the vehicle's driving trajectory (optionally, also including images or point cloud data of lane lines around the vehicle), the first image including first lines corresponding to the roadside around the vehicle and obtaining second lines corresponding to the vehicle's driving trajectory, optionally, the first image also including third lines corresponding to the lane lines around the vehicle; the vehicle inputs the first image into the first machine learning model, and generates first prediction information through the first machine learning model.

[0146] In an embodiment of the present application, since the vehicle's driving trajectory can well reflect the vehicle's position change, the vehicle's driving trajectory and the curbs around the vehicle are integrated into the same image, and the vehicle's driving trajectory and the curbs around the vehicle are displayed in the form of lines in the first image, which can intuitively reflect how the vehicle moves when passing through the fork in the road, which is conducive to improving the reliability of the first branching direction finally determined; and provides another implementation scheme for generating the first branching direction, which improves the implementation flexibility of this scheme.

[0147] The process of the vehicle acquiring the first image can be understood as: the vehicle synthesizing the roadside around the vehicle and the vehicle's driving trajectory (optionally, also including the lane lines around the vehicle) into the same image. Optionally, the first image can be a grayscale image, and the vehicle can determine the coordinate information of the roadside around the vehicle based on the image or point cloud data of the roadside around the vehicle; optionally, the vehicle can also determine the coordinate information of the lane lines around the vehicle based on the image or point cloud data of the lane lines around the vehicle; the vehicle can normalize the coordinate information of the vehicle's driving trajectory and the coordinate information of the lane lines around the vehicle (optionally, also including the coordinate information of the lane lines around the vehicle), that is, mapping the coordinate information of the vehicle's driving trajectory and the coordinate information of the lane lines around the vehicle (optionally, also including the coordinate information of the lane lines around the vehicle) to the same value range, and then generating a grayscale image (i.e., the first image), the grayscale image including a first line corresponding to the roadside around the vehicle and a second line corresponding to the vehicle's driving trajectory. Optionally, the grayscale image also includes a third line corresponding to the lane lines around the vehicle. The vehicle can determine different grayscale values ​​for different types of data. For example, the grayscale value of the first line is 0.3, the grayscale value of the second line is 1.0, and the grayscale value of the third line is 0.6. It should be understood that the examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0148] For a more intuitive understanding of this solution, please refer to Figure 9, which is a schematic diagram of the first image provided in an embodiment of the present application. In Figure 9, the gray lines of different shades represent the first line, the second line, and the third line, respectively. Among them, the darkest gray line in Figure 9 represents the first line corresponding to the curb around the vehicle, the lightest gray line represents the third line corresponding to the lane line around the vehicle, and the gray line in the middle represents the second line corresponding to the vehicle's driving trajectory. As shown in Figure 9, the vehicle should have entered the left fork direction of the fork in the intersection. It should be understood that the example in Figure 9 is only for the convenience of understanding this solution and is not used to limit this solution.

[0149] Optionally, in step 301, the vehicle can obtain surrounding environmental information at a third preset frequency, and each time the surrounding environmental information is obtained, it triggers the start of obtaining the vehicle's driving trajectory in the future period until the third preset condition is met; after each new first image is obtained, the vehicle inputs the first image into the first machine learning model, generates a first prediction information through the first machine learning model, and then determines the first bifurcation direction based on all the first prediction information generated by the first machine learning model.

[0150] Exemplarily, since the third preset frequency is low, the vehicle determines the first divergence direction based on all the first prediction information generated by the first machine learning model, which may include: the vehicle filters out valid first prediction information from all the first prediction information generated by the first machine learning model, and the valid first prediction information represents a divergence direction that can clearly instruct the vehicle to drive into at least one of the two divergence directions; and determines the divergence direction pointed to by the valid first prediction information as the first divergence direction.

[0151] Optionally, if all the filtered valid first prediction information points to different bifurcation directions, the first bifurcation direction may be finally determined by cumulative voting based on all the valid first prediction information.

[0152] In the embodiment of the present application, not only the environmental information around the vehicle but also the driving trajectory of the vehicle are used to comprehensively determine which branch direction of the fork intersection the vehicle has entered, which is beneficial to improving the reliability of the first branch direction finally obtained.

[0153] In another implementation, step 302 may include: the vehicle generates second prediction information through a first machine learning model based on surrounding environmental information and navigation information, and the second prediction information indicates whether the vehicle deviates when passing a fork in the road.

[0154] Specifically, in one case, the vehicle inputs feature information of N historical forward-view images, the current forward-view image, and navigation information into a first machine learning model, and generates second prediction information through the first machine learning model.

[0155] In another case, the vehicle inputs N historical forward-view images, current forward-view images, and navigation information into a first machine learning model, and generates second prediction information through the first machine learning model.

[0156] In another case, the vehicle inputs the forward-view image carrying trajectory information and navigation information into the first machine learning model, and generates second prediction information through the first machine learning model.

[0157] In another case, the vehicle inputs the first image and navigation information into a first machine learning model, and generates second prediction information through the first machine learning model.

[0158] It should be noted that the specific implementation method of the vehicle obtaining the "characteristic information of the historical forward-view image", "the current forward-view image", "the historical forward-view image", "the forward-view image carrying trajectory information" and "the first image" can be found in the above description and will not be repeated here.

[0159] In an embodiment of the present application, when a vehicle enters a preset range of a fork in the road, it can obtain images or point cloud data of the vehicle's surroundings, and then based on the images or point cloud data of the vehicle's surroundings, it can determine through a first machine learning model whether the vehicle has yawed when passing through the fork in the road according to the images or point cloud data of the vehicle's surroundings and the vehicle's navigation information. That is, a solution is provided for determining whether the vehicle has yawed when passing through a fork in the road. In addition, even in traffic scenarios such as tunnels, urban canyons, elevated roads, and small-angle parallel forks in the road, the vehicle can obtain high-quality images of the surrounding environment and high-quality point cloud data of the surrounding environment. Therefore, it can determine through the first machine learning model whether the vehicle has yawed when passing through the fork in the road, which is conducive to effectively detecting whether the vehicle has yawed in more scenarios and providing users with a better navigation experience.

[0160] Optionally, based on the embodiment corresponding to FIG. 3 , please refer to FIG. 10 , which is another flow chart of a method for detecting a yaw of a bifurcated road provided in an embodiment of the present application. The method for detecting a yaw of a bifurcated road provided in an embodiment of the present application may include:

[0161] 1001. Acquire environmental information surrounding a vehicle, where the environmental information surrounding the vehicle includes images or point cloud data of the vehicle's surrounding environment.

[0162] 1002. When the vehicle enters a preset range of a fork in the road, based on the environmental information around the vehicle and the navigation information of the vehicle, a first machine learning model is used to determine whether the vehicle deviates when passing through the fork in the road.

[0163] In the embodiment of the present application, the specific implementation of steps 1001 and 1002 can refer to the description in the embodiment corresponding to Figure 3 above, and will not be repeated here.

[0164] 1003. When it is determined that the vehicle deviates when passing through a fork in the road, a prompt message is output through the display interface. The prompt message includes a first fork direction and a second fork direction. The at least two fork directions include the second fork direction. The second fork direction is the fork direction that the vehicle needs to enter based on the navigation information.

[0165] In the embodiment of the present application, step 1003 is an optional step. For example, the display interface can be a display interface of a human-machine interface (HMI), a head-up display (HUD), or other types of display interfaces.

[0166] This prompt information can be used by the user to confirm whether the first bifurcation direction is correct, that is, to confirm whether the yaw detection result is correct. Optionally, the first bifurcation direction and the second bifurcation direction displayed in the display interface can be displayed in different colors, for example, the first bifurcation direction can be displayed in red and the second bifurcation direction can be displayed in green. Optionally, the first bifurcation direction can also be displayed in a flashing manner in the display interface.

[0167] For a more intuitive understanding of this solution, please refer to Figure 11, which is a schematic diagram of the prompt information provided in an embodiment of the present application. As shown in Figure 11, the first bifurcation direction and the second bifurcation direction are simultaneously shown through the display interface. It should be understood that the example in Figure 11 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0168] Optionally, the prompt information also includes prompting the user to confirm whether the navigation route needs to be modified through a display interface or voice; when the user does not confirm whether the navigation route needs to be modified within a preset time period, the navigation route can be modified by default, and then the new navigation route can be used for navigation.

[0169] Optionally, when the vehicle determines that the vehicle has deviated when passing a fork in the road, it can also output voice information to prompt the user that the vehicle has deviated when passing the fork in the road. For example, the voice message can be "The current GNSS signal is weak, and the vehicle has deviated using intelligent vision recognition."

[0170] In an embodiment of the present application, when it is detected whether the vehicle has yawed when passing through a fork in the road, the first fork direction confirmed by the vehicle and the second fork direction planned by the navigation can be promptly displayed to the user through a display interface, so that the user can confirm whether the yaw detection result obtained by the vehicle is accurate, and then determine whether to modify the navigation route. This is beneficial to avoid modifying the navigation route based on erroneous yaw detection results, and is also beneficial to providing users with a better navigation experience.

[0171] 2. Training Phase

[0172] Specifically, please refer to FIG12, which is a flow chart of a model training method provided in an embodiment of the present application. The model training method provided in an embodiment of the present application may include:

[0173] 1201. Acquire training data, where the training data includes environmental information surrounding a vehicle and an expected branching direction corresponding to the environmental information surrounding the vehicle. The expected branching direction represents a correct branching direction among at least two branching directions into which the vehicle may enter.

[0174] In an embodiment of the present application, a training data set may be deployed on a training device. The training data set includes multiple training data. Each training data set may include environmental information surrounding a vehicle and an expected bifurcation direction corresponding to the environmental information surrounding the vehicle. The "expected bifurcation direction corresponding to the environmental information surrounding the vehicle" may also be referred to as a true value corresponding to the environmental information surrounding the vehicle, a label corresponding to the environmental information surrounding the vehicle, or the like.

[0175] Among them, the environmental information around the vehicle included in the training data can be collected by the vehicle during driving; optionally, the expected branching direction corresponding to the environmental information around the vehicle can be determined based on the vehicle's driving trajectory and high-precision map during driving. Specifically, for a forked road area where a high-precision map exists, the vehicle's driving trajectory can be matched with the high-precision map to automatically determine which of the at least two branching directions included in the fork in the road the vehicle has entered (that is, the expected branching direction is determined). In the embodiment of the present application, the true value annotation can be automatically performed in combination with the vehicle's driving trajectory and the high-precision map, that is, the training data can be obtained without manual intervention, which is conducive to improving the convenience of the process of obtaining training data.

[0176] Further optionally, when the input of the first machine learning model is the above-mentioned first image, the first image can also be manually constructed. In this case, the manually constructed first image can be input into the first machine learning model in step 1202, thereby performing data enhancement on the training data set of the first machine learning model to expand the diversity of the training data.

[0177] 1202. Based on environmental information surrounding the vehicle, prediction information is generated through a first machine learning model, where the prediction information is used to determine whether the vehicle has entered a first diverging direction of at least two diverging directions included in a fork in the road.

[0178] In the embodiment of the present application, the specific implementation method of the training device executing step 1202 can refer to the description of the vehicle executing step 302 in the embodiment corresponding to Figure 3 above, and will not be repeated here.

[0179] 1203. Use a loss function to train the first machine learning model to obtain a trained first machine learning model. The goal of using the loss function for training includes improving the similarity between the first bifurcation direction and the expected bifurcation direction.

[0180] In an embodiment of the present application, after obtaining the first bifurcation direction, the vehicle may iteratively train the machine learning model using a loss function until a convergence condition is satisfied, thereby obtaining a trained first machine learning model. The loss function indicates the similarity between the first bifurcation direction and the desired bifurcation direction. The convergence condition may include satisfying a convergence condition of the loss function and / or the number of iterative training iterations reaching a predetermined number.

[0181] Exemplarily, during each training process of the first machine learning model, the training device may first generate the function value of the loss function, and use the backpropagation algorithm to update the weight parameters of the first machine learning model to achieve one training of the first machine learning model.

[0182] For example, in the process of training the first machine learning model, the predicted value generated by the first machine learning model can be compared with the desired expected value, and the weight vector of each layer of the neural network can be updated according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the first machine learning model). For example, if the predicted value of the network is high, the weight vector is adjusted to make it predict lower, and the adjustment is continued until the first machine learning model can generate the expected value that is really desired or a value very close to the expected value that is really desired. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the expected value", which is the loss function or objective function, which are important equations for measuring the difference between the predicted value and the expected value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and the training of the first machine learning model becomes a process of minimizing this loss as much as possible.

[0183] Machine learning models can use the backpropagation algorithm to correct the size of the initial machine learning model parameters during training, reducing the reconstruction error loss of the machine learning model. Specifically, forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the initial machine learning model parameters, thereby converging the error loss. The backpropagation algorithm is a backward propagation movement dominated by error loss, aiming to obtain the optimal parameters of the machine learning model.

[0184] On the basis of the embodiments corresponding to Figures 1 to 12, in order to better implement the above-mentioned scheme of the embodiment of the present application, the following also provides related equipment for implementing the above-mentioned scheme. Specifically referring to Figure 13, Figure 13 is a structural schematic diagram of a yaw detection device for a forked road provided in an embodiment of the present application, and the yaw detection device 1300 for a forked road includes: an acquisition module 1301 for acquiring environmental information around the vehicle, the environmental information around the vehicle including images or point cloud data of the vehicle's surrounding environment; a determination module 1302 for determining whether the vehicle yaws when passing through a forked road based on the environmental information around the vehicle and the vehicle's navigation information when the vehicle enters a preset range of the forked road.

[0185] Optionally, the fork in the road includes at least two fork directions, and the determination module 1302 is specifically used to generate prediction information through a machine learning model based on environmental information around the vehicle, and the prediction information is used to determine whether the vehicle has entered the first fork direction of at least two fork directions; based on the first fork direction and the navigation information of the vehicle, determine whether the vehicle has deviated at the fork in the road.

[0186] Optionally, the acquisition module 1301 is specifically used to acquire the forward-view image of the vehicle at a preset frequency, the image of the vehicle's surrounding environment is the forward-view image of the vehicle, and the forward-view image of the vehicle includes the current forward-view image and the historical forward-view image; the determination module 1302 is specifically used to input the feature information of the historical forward-view image and the current forward-view image into a machine learning model, and generate prediction information through the machine learning model, and the feature information of the historical forward-view image is obtained after feature extraction of the historical forward-view image.

[0187] Optionally, the determination module 1302 is specifically used to repeatedly input the feature information of the historical forward-view image and the current forward-view image into the machine learning model at least twice to obtain at least two prediction information through the machine learning model, each prediction information indicating one of at least two bifurcation directions; based on the at least two prediction information, the first bifurcation direction is determined by cumulative voting.

[0188] Optionally, the acquisition module 1301 is specifically used to acquire the vehicle's forward-view image at a preset frequency during the vehicle's driving process, and the image of the vehicle's surrounding environment is the vehicle's forward-view image, and the vehicle's forward-view image includes a current forward-view image and a historical forward-view image; the determination module 1302 is specifically used to input the historical forward-view image and the current forward-view image into a machine learning model, and generate prediction information through the machine learning model.

[0189] Optionally, the determination module 1302 is specifically used to generate prediction information through a machine learning model based on environmental information around the vehicle and the vehicle's driving trajectory.

[0190] Optionally, the image of the vehicle's surroundings is a front view image of the vehicle, and the determination module 1302 is specifically used to project the vehicle's driving trajectory into the vehicle's front view image to obtain a front view image carrying trajectory information; input the front view image carrying trajectory information into a machine learning model, and generate prediction information through the machine learning model.

[0191] Optionally, the environmental information around the vehicle includes images or point cloud data of the curb around the vehicle, and the determination module 1302 is specifically used to obtain a first image based on the images or point cloud data of the curb around the vehicle and the driving trajectory of the vehicle, the first image including a first line corresponding to the curb around the vehicle and a second line corresponding to the driving trajectory of the vehicle; the first image is input into the machine learning model, and prediction information is generated through the machine learning model.

[0192] Optionally, the driving trajectory of the vehicle includes the displacement of the vehicle in a preset direction, and the preset direction includes at least one of the following directions: front, left or right.

[0193] Optionally, the yaw detection device 1300 for a fork in the road also includes: an output module 1303, which is used to output prompt information through a display interface when determining whether a vehicle yaws when passing a fork in the road, the prompt information including a first fork direction and a second fork direction, at least two fork directions including the second fork direction, and the second fork direction being the fork direction that the vehicle needs to enter based on the navigation information.

[0194] It should be noted that the information interaction, execution process, etc. between the modules / units in the yaw detection device 1300 for the bifurcated road are based on the same concept as the various method embodiments corresponding to Figures 1 to 12 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0195] Refer to Figure 14, which is a structural diagram of a training device for a model provided in an embodiment of the present application. The model training device 1400 includes: an acquisition module 1401, used to acquire training data, the training data including environmental information around the vehicle and an expected bifurcation direction corresponding to the environmental information around the vehicle; a generation module 1402, used to generate prediction information through a machine learning model based on the environmental information around the vehicle, the prediction information being used to determine whether the vehicle has entered the first bifurcation direction of at least two bifurcation directions included in the bifurcation intersection; a training module 1403, used to train the machine learning model using a loss function to obtain a trained machine learning model, wherein the goal of training using a loss function includes improving the similarity between the first bifurcation direction and the expected bifurcation direction, and the expected bifurcation direction represents the correct bifurcation direction of at least two bifurcation directions that the vehicle has entered.

[0196] Optionally, the expected branching direction is determined based on the vehicle's driving trajectory and a high-precision map during driving.

[0197] It should be noted that the information interaction, execution process, etc. between the modules / units in the model training device 1400 are based on the same concept as the various method embodiments corresponding to Figures 1 to 12 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0198] Next, a device provided in an embodiment of the present application is introduced. When the device is specifically an execution device, please refer to Figure 15, which is a schematic structural diagram of a device provided in an embodiment of the present application. Specifically, the device 1500 includes: a receiver 1501, a transmitter 1502, a processor 1503, and a memory 1504 (wherein the number of processors 1503 in the device 1500 can be one or more, and Figure 15 uses one processor as an example). The processor 1503 may include an application processor 15031 and a communication processor 15032. In some embodiments of the present application, the receiver 1501, the transmitter 1502, the processor 1503, and the memory 1504 may be connected via a bus or other means.

[0199] Memory 1504 may include read-only memory and random access memory, and provides instructions and data to processor 1503. A portion of memory 1504 may also include non-volatile random access memory (NVRAM). Memory 1504 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0200] Processor 1503 controls the operation of the device. In specific applications, the various components of the device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0201] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1503. Processor 1503 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 1503. The above processor 1503 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1503 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1504, and processor 1503 reads the information in memory 1504 and, in conjunction with its hardware, completes the steps of the above method.

[0202] Receiver 1501 can be used to receive input digital or character information and generate signal input related to device settings and function control. Transmitter 1502 can be used to output digital or character information through the first interface. Transmitter 1502 can also be used to send instructions to the disk pack through the first interface to modify data on the disk pack. Transmitter 1502 can also include a display device such as a display screen.

[0203] In the embodiment of the present application, processor 1503 is used to execute the method executed by the vehicle in the embodiments corresponding to Figures 1 to 12. It should be noted that the specific manner in which application processor 15031 in processor 1503 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to Figures 1 to 12 of the present application, and the technical effects achieved are the same as the various method embodiments corresponding to Figures 1 to 12 of the present application. For details, please refer to the description of the method embodiments shown above in the present application, and will not be repeated here.

[0204] In the case where the device is specifically manifested as a second device, please refer to Figure 16, which is another structural diagram of the device provided in an embodiment of the present application. Specifically, the device 1600 is implemented by one or more servers. The device 1600 may have relatively large differences due to different configurations or performances. It may include one or more central processing units (CPUs) 1622 (for example, one or more processors) and a memory 1632, and one or more storage media 1630 (for example, one or more mass storage devices) storing application programs 1642 or data 1644. Among them, the memory 1632 and the storage medium 1630 can be temporary storage or permanent storage. The program stored in the storage medium 1630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the device. Furthermore, the central processing unit 1622 can be configured to communicate with the storage medium 1630 to execute a series of instruction operations in the storage medium 1630 on the device 1600.

[0205] The device 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 1658, and / or one or more operating systems 1641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0206] In the embodiments of the present application, central processing unit 1622 is configured to execute the map construction method performed by the training device in the embodiments corresponding to Figures 1 to 12. It should be noted that the specific manner in which central processing unit 1622 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to Figures 1 to 12 of the present application, and the technical effects achieved are the same as those of the various method embodiments corresponding to Figures 1 to 12 of the present application. For details, please refer to the description of the method embodiments shown above in this application, and will not be repeated here.

[0207] The present application also provides a vehicle. Please refer to FIG17 , which is a schematic structural diagram of a vehicle provided by the present application. Vehicle 100 is configured for a fully or partially autonomous driving mode. For example, vehicle 100 can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the possibility of the other vehicle performing the possible behavior, and control vehicle 100 based on the determined information. When vehicle 100 is in autonomous driving mode, vehicle 100 can also be set to operate without human interaction.

[0208] The vehicle 100 may include various subsystems, such as a travel system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power source 110, a computer system 112, and a user interface 116. Alternatively, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the vehicle 100 may be interconnected via wired or wireless connections.

[0209] Travel system 102 may include components that provide powered movement for vehicle 100. In one embodiment, travel system 102 may include engine 118, power source 119, transmission 120, and wheels / tires 121.

[0210] The engine 118 may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy. Examples of the energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source 119 may also provide energy for other systems of the vehicle 100. The transmission 120 may transmit the mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other devices, such as a clutch. The drive shaft may include one or more shafts that can be coupled to one or more wheels 121.

[0211] Sensor system 104 may include several sensors that sense information about the environment surrounding vehicle 100. For example, sensor system 104 may include a positioning system 122 (the positioning system may be a global positioning system (GPS), a BeiDou system, or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. Sensor system 104 may also include sensors for internal systems of monitored vehicle 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors may be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). This detection and recognition is a key function for the safe operation of autonomous vehicle 100.

[0212] Among them, the positioning system 122 can be used to estimate the geographic location of the vehicle 100. The IMU 124 is used to sense the position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope. The radar 126 can use radio signals to sense objects in the surrounding environment of the vehicle 100, and can specifically be a millimeter wave radar or a laser radar. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or direction of travel of objects. The laser rangefinder 128 can use lasers to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. The camera 130 can be used to capture multiple images of the surrounding environment of the vehicle 100. The camera 130 can be a still camera or a video camera.

[0213] Control system 106 controls the operation of vehicle 100 and its components. Control system 106 may include various components, including a steering system 132 , a throttle 134 , a brake unit 136 , a computer vision system 140 , a lane control system 142 , and an obstacle avoidance system 144 .

[0214] The steering system 132 is operable to adjust the direction of travel of the vehicle 100. For example, in one embodiment, it may be a steering wheel system. The throttle 134 is used to control the operating speed of the engine 118 and, in turn, the speed of the vehicle 100. The brake unit 136 is used to control the deceleration of the vehicle 100. The brake unit 136 may use friction to slow the wheels 121. In other embodiments, the brake unit 136 may convert the kinetic energy of the wheels 121 into electrical current. The brake unit 136 may also take other forms to slow the rotation speed of the wheels 121 to control the speed of the vehicle 100. The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the vehicle 100. These objects and / or features may include traffic signs, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 can be used to map the environment, track objects, estimate their speed, and so on. The route control system 142 is used to determine the route and speed of the vehicle 100. In some embodiments, the route control system 142 may include a lateral planning module 1421 and a longitudinal planning module 1422, which are respectively used to determine the route and speed for the vehicle 100 by combining data from the obstacle avoidance system 144, GPS 122, and one or more predetermined maps. The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise navigate obstacles in the environment of the vehicle 100. The aforementioned obstacles can specifically be represented by actual obstacles and virtual moving objects that may collide with the vehicle 100. In one embodiment, the control system 106 may include additional or alternative components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0215] Vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users via peripheral devices 108. Peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152. In some embodiments, peripheral devices 108 provide a means for the user of vehicle 100 to interact with user interface 116. For example, onboard computer 148 may provide information to the user of vehicle 100. User interface 116 may also operate onboard computer 148 to receive user input. Onboard computer 148 may be operated via a touchscreen. In other cases, peripheral devices 108 may provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (e.g., voice commands or other audio input) from the user of vehicle 100. Similarly, speaker 152 may output audio to the user of vehicle 100. Wireless communication system 146 may wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system 146 may utilize 3G cellular communications, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communications, such as LTE. Or 5G cellular communications. The wireless communication system 146 may utilize wireless local area network (WLAN) communications. In some embodiments, the wireless communication system 146 may utilize infrared links, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols, such as various vehicle communication systems, may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0216] Power source 110 can provide power to various components of vehicle 100. In one embodiment, power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such batteries can be configured as a power source to provide power to various components of vehicle 100. In some embodiments, power source 110 and energy source 119 can be implemented together, such as in some all-electric vehicles.

[0217] Some or all functions of vehicle 100 are controlled by computer system 112. Computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium, such as memory 114. Computer system 112 may also be a plurality of computing devices that control individual components or subsystems of vehicle 100 in a distributed manner. Processor 113 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, processor 113 may be a specialized device, such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although FIG. 17 functionally illustrates the processor, memory, and other components of computer system 112 in the same block, those skilled in the art will appreciate that the processor or memory may actually include multiple processors or memories that are not stored in the same physical housing. For example, memory 114 may be a hard drive or other storage medium located in a different housing than computer system 112. Therefore, references to processor 113 or memory 114 should be understood to include references to a collection of processors or memories that may or may not operate in parallel. Rather than using a single processor to perform the steps described herein, some components, such as the steering assembly and the retarding assembly, may each have its own processor that performs only calculations related to the functionality of the component specific component.

[0218] In various aspects described herein, the processor 113 may be located remotely from the vehicle 100 and in wireless communication with the vehicle 100. In other aspects, some of the processes described herein are performed on the processor 113 disposed within the vehicle 100 while others are performed by the remote processor 113, including taking the necessary steps to perform a single maneuver.

[0219] In some embodiments, memory 114 may contain instructions 115 (e.g., program logic) that are executable by processor 113 to perform various functions of vehicle 100, including those described above. Memory 114 may also contain additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of travel system 102, sensor system 104, control system 106, and peripherals 108. In addition to instructions 115, memory 114 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other such vehicle data, as well as other information. This information may be used by vehicle 100 and computer system 112 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes. A user interface 116 is provided for providing information to or receiving information from a user of vehicle 100. Optionally, user interface 116 may include one or more input / output devices within the set of peripherals 108, such as wireless communication system 146, onboard computer 148, microphone 150, and speaker 152.

[0220] Computer system 112 may control functions of vehicle 100 based on input received from various subsystems (e.g., travel system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 may utilize input from control system 106 to control steering system 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 may be operable to provide control over many aspects of vehicle 100 and its subsystems.

[0221] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 100. For example, the memory 114 may be partially or completely separate from the vehicle 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0222] Optionally, the above components are just an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 17 should not be understood as a limitation on the embodiments of the present application. A vehicle traveling on a road, such as vehicle 100 above, can identify objects in its surrounding environment to determine the adjustment of the current speed. The object can be another vehicle, a traffic control device, or other types of objects. In some examples, each identified object can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, distance from the vehicle, etc., it can be used to determine the speed to be adjusted for the vehicle.

[0223] Optionally, the vehicle 100 or a computing device associated with the vehicle 100, such as the computer system 112, computer vision system 140, and memory 114 of Figure 17, can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can be considered together to predict the behavior of a single identified object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the vehicle 100 can determine what stable state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 on the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc. In addition to providing instructions to adjust the speed of the vehicle, the computing device may also provide instructions to modify the steering angle of the vehicle 100 so that the vehicle 100 follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the vehicle 100 (e.g., cars in adjacent lanes on the road).

[0224] In the embodiment of the present application, the processor 113 in the vehicle 100 is used to execute the method executed by the vehicle in the embodiments corresponding to Figures 1 to 12. It should be noted that the specific manner in which the processor 113 executes the aforementioned steps is based on the same concept as the various method embodiments corresponding to Figures 1 to 12 of the present application, and the technical effects it brings are the same as the various method embodiments corresponding to Figures 1 to 12 of the present application. For specific details, please refer to the description of the method embodiments shown above in this application, and will not be repeated here.

[0225] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program, which, when executed on a computer, enables the computer to execute the steps executed by the vehicle in the method described in the embodiments shown in Figures 1 to 12 above, or enables the computer to execute the steps executed by the training device in the method described in the embodiments shown in Figures 1 to 12 above.

[0226] Also provided in an embodiment of the present application is a computer program product, which includes a program. When the computer program product is run on a computer, it enables the computer to execute the steps performed by the vehicle in the method described in the embodiments shown in Figures 1 to 12 above, or enables the computer to execute the steps performed by the training device in the method described in the embodiments shown in Figures 1 to 12 above.

[0227] An embodiment of the present application further provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the method described in the embodiments shown in Figures 1 to 12 above.

[0228] The execution device, training device or yaw detection device for a bifurcated road provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer-executable instructions stored in the storage unit so that the chip executes the method described in the embodiments shown in Figures 1 to 12 above. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0229] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0230] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0231] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CLUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, disk or optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0232] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0233] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive (SSD)).

Claims

1. A method for detecting a yaw of a bifurcated road, characterized in that: The method comprises: Acquiring environmental information around the vehicle, wherein the environmental information around the vehicle includes an image or point cloud data of the vehicle's surrounding environment; When a vehicle enters a preset range of a fork in the road, a machine learning model is used to determine whether the vehicle deviates when passing the fork in the road based on environmental information around the vehicle and navigation information of the vehicle.

2. The method according to claim 1, characterized in that The forked road includes at least two forking directions, and determining whether the forked road has yawed by using a machine learning model based on environmental information around the vehicle and navigation information of the vehicle includes: Based on environmental information surrounding the vehicle, generating prediction information through a machine learning model, wherein the prediction information is used to determine that the vehicle has entered a first branching direction of the at least two branching directions; It is determined whether the vehicle deviates from the fork in the road according to the first branch direction and navigation information of the vehicle.

3. The method according to claim 2, characterized in that The obtaining of environmental information around the vehicle includes: Acquire a front view image of the vehicle at a preset frequency, wherein the image of the vehicle's surrounding environment is the front view image of the vehicle, and the front view image of the vehicle includes a current front view image and a historical front view image; The generating the prediction information by a machine learning model based on the environmental information around the vehicle includes: The feature information of the historical foresight image and the current foresight image are input into the machine learning model, and the prediction information is generated by the machine learning model, wherein the feature information of the historical foresight image is obtained by extracting features from the historical foresight image.

4. The method according to claim 3, characterized in that The generating the prediction information by a machine learning model based on the environmental information around the vehicle includes: repeatedly inputting the feature information of the historical forward-view image and the current forward-view image into the machine learning model at least twice, so as to obtain at least two pieces of prediction information through the machine learning model, each piece of prediction information indicating one of the at least two bifurcation directions; Based on the at least two pieces of prediction information, the first bifurcation direction is determined by cumulative voting.

5. The method according to claim 2, characterized in that The obtaining of environmental information around the vehicle includes: Acquiring a front view image of the vehicle at a preset frequency during vehicle travel, wherein the image of the vehicle's surrounding environment is the front view image of the vehicle, and the front view image of the vehicle includes a current front view image and a historical front view image; The generating the prediction information by a machine learning model based on the environmental information around the vehicle includes: The historical forward-view image and the current forward-view image are input into the machine learning model, and the prediction information is generated by the machine learning model.

6. The method according to claim 2, characterized in that The generating the prediction information by a machine learning model based on the environmental information around the vehicle includes: The prediction information is generated by a machine learning model based on the environmental information around the vehicle and the vehicle's driving trajectory.

7. The method according to claim 6, characterized in that The image of the vehicle's surroundings is a front view image of the vehicle, and the prediction information is generated by a machine learning model based on the environmental information of the vehicle's surroundings and the vehicle's driving trajectory, including: Projecting the driving trajectory of the vehicle onto the front view image of the vehicle to obtain a front view image carrying the trajectory information; The forward-looking image carrying the trajectory information is input into the machine learning model, and the prediction information is generated by the machine learning model.

8. The method according to claim 6, characterized in that The environmental information surrounding the vehicle includes images or point cloud data of a roadside around the vehicle. The generating of the prediction information by a machine learning model based on the environmental information surrounding the vehicle and the driving trajectory of the vehicle includes: Acquire a first image based on the image or point cloud data of the roadside around the vehicle and the driving trajectory of the vehicle, wherein the first image includes a first line corresponding to the roadside around the vehicle and a second line corresponding to the driving trajectory of the vehicle; The first image is input into the machine learning model, and the prediction information is generated by the machine learning model.

9. The method according to any one of claims 6 to 8, characterized in that The driving trajectory of the vehicle includes the displacement of the vehicle in a preset direction, and the preset direction includes at least one of the following directions: front, left or right.

10. The method according to any one of claims 2 to 8, characterized in that The method further comprises: When determining whether the vehicle deviates when passing the fork in the road, prompt information is output through the display interface, and the prompt information includes the first fork direction and the second fork direction. The at least two fork directions include the second fork direction, and the second fork direction is the fork direction that the vehicle needs to enter based on the navigation information.

11. A model training method, characterized in that: The method comprises: Acquiring training data, the training data including environmental information surrounding a vehicle and an expected bifurcation direction corresponding to the environmental information surrounding the vehicle; Based on environmental information surrounding the vehicle, generating prediction information through a machine learning model, the prediction information being used to determine that the vehicle has entered a first branching direction of at least two branching directions included in the fork intersection; The machine learning model is trained using a loss function to obtain a trained machine learning model. The goals of training using the loss function include improving the similarity between the first bifurcation direction and the expected bifurcation direction, where the expected bifurcation direction represents the correct bifurcation direction among the at least two bifurcation directions into which the vehicle enters.

12. The method according to claim 11, characterized in that The desired branching direction is determined based on the vehicle's driving trajectory and high-precision map during driving.

13. A device for detecting a yaw of a bifurcated road, characterized in that: The device comprises: An acquisition module is used to acquire environmental information around the vehicle, wherein the environmental information around the vehicle includes an image or point cloud data of the vehicle's surrounding environment; The determination module is used to determine whether the vehicle deviates when passing through the fork in the road based on the environmental information around the vehicle and the navigation information of the vehicle through a machine learning model when the vehicle enters a preset range of the fork in the road.

14. A model training device, characterized in that: The device comprises: an acquisition module, configured to acquire training data, the training data including environmental information surrounding the vehicle and an expected bifurcation direction corresponding to the environmental information surrounding the vehicle; a generating module, configured to generate prediction information using a machine learning model based on environmental information surrounding the vehicle, wherein the prediction information is used to determine that the vehicle has entered a first branching direction of at least two branching directions included in the fork intersection; A training module is used to train the machine learning model using a loss function to obtain a trained machine learning model, wherein the goals of training using the loss function include improving the similarity between the first bifurcation direction and the expected bifurcation direction, wherein the expected bifurcation direction represents the correct bifurcation direction among the at least two bifurcation directions into which the vehicle enters.

15. A device, characterized in that The method comprises a processor coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 12 is implemented.

16. A vehicle, characterized in that: The method comprises a processor coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 10 is implemented.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 12.

18. A computer program product, characterized in that The computer program product comprises a program, which, when run on a computer, causes the computer to perform the method according to any one of claims 1 to 12 .

Images