Decision-making method and device for realizing automatic driving, automobile, equipment and storage medium

Abstract

The invention provides a decision-making method and device for realizing automatic driving, an automobile, equipment and a storage medium, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring an outside image of an automobile collected at a current decision moment, and determining image features of the outside image; if it is determined that the image outside the automobile is an image where driving decision making cannot be conducted based on the image features, the state feature of the automobile at the previous decision making moment is determined as the current state feature of the automobile at the current decision making moment; if it is determined that the vehicle exterior image is an image capable of performing driving decision based on the image features, determining the current state features of the vehicle at the current decision moment based on the image features; inputting the current state characteristics into a strategy prediction model to obtain a current driving strategy, output by the strategy prediction model, of the automobile at the current decision moment; the strategy prediction model is obtained based on sample vehicle exterior image training. According to the invention, the decision accuracy of automatic driving can be improved, so that the automatic driving safety of the automobile is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a decision-making method, device, vehicle, equipment and storage medium for realizing autonomous driving. Background Art

[0002] With the rapid development of science and technology, people have higher and higher requirements for cars. At present, the autonomous driving function is an important demand of users, so the realization of safe autonomous driving is a technical demand that needs to be solved urgently.

[0003] Currently, autonomous driving is achieved by collecting clear images of the road condition in front of the car and inputting them into a computer vision model to obtain the final autonomous driving command. However, real road conditions are not always ideal. For example, sudden objects may block the line of sight, and scenes with extremely low visibility may cause inaccurate autonomous driving commands to be obtained based on the extremely blurred road condition images, which may lead to driving safety risks. Summary of the invention

[0004] The present invention provides a decision-making method, device, automobile, equipment and storage medium for realizing autonomous driving, so as to solve the defect of low decision-making accuracy of autonomous driving in scenes with extremely blurred vision in the prior art and realize highly safe autonomous driving.

[0005] The present invention provides a decision-making method for realizing automatic driving, comprising: Acquire an image outside the vehicle collected at the current decision-making moment, and determine image features of the image outside the vehicle; If it is determined based on the image features that the image outside the vehicle is an image in which a driving decision cannot be made, the state features of the vehicle at the last decision moment are determined as the current state features of the vehicle at the current decision moment; If it is determined based on the image features that the image outside the vehicle is an image for making a driving decision, determining the current state features of the vehicle at the current decision moment based on the image features; The current state feature is input into a strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model; the strategy prediction model is trained based on sample external vehicle images.

[0006] According to a decision-making method for realizing automatic driving provided by the present invention, the current state feature is input into a strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model, including: In the case where the image outside the vehicle is an image in which a driving decision cannot be made, determining the accumulated reward at the previous decision moment as the accumulated reward at the current decision moment; When the image outside the vehicle is an image that can be used to make a driving decision, determining the cumulative reward at the current decision moment; Determine an input sequence based on the historical state characteristics of the car at historical decision moments, the historical driving strategies of the car at historical decision moments, the accumulated rewards at historical decision moments, the current state characteristics and the accumulated rewards at the current decision moment; the historical decision moment is the decision moment before the current decision moment; Inputting the input sequence into a strategy prediction model to obtain the current driving strategy output by the strategy prediction model; Among them, the cumulative reward at any decision moment is the cumulative value of the instant rewards at each decision moment from the decision moment to the future termination decision moment. The instant reward at any decision moment is used to characterize the value of the driving strategy at the decision moment.

[0007] According to a decision-making method for realizing automatic driving provided by the present invention, the input sequence is determined based on the historical state characteristics of the car at the historical decision-making moments, the historical driving strategies of the car at the historical decision-making moments, the accumulated rewards at the historical decision-making moments, the current state characteristics and the accumulated rewards at the current decision-making moment, including: In a case where the vehicle exterior image is an image for which a driving decision cannot be made, determining the discard span embedding vector at the current decision moment based on the discard span embedding vector at the previous decision moment; In the case where the vehicle exterior image is an image for making a driving decision, determining a preset embedding vector as a discard span embedding vector at the current decision moment; Determine a current state encoding feature based on the current state feature, the current decision moment, and the discard span embedding vector of the current decision moment, determine a cumulative reward encoding vector of the current decision moment based on the cumulative reward of the current decision moment, the current decision moment, and the discard span embedding vector of the current decision moment, and determine a historical driving strategy encoding vector of the previous decision moment based on the historical driving strategy of the previous decision moment and the current decision moment; Determine an input sequence based on the historical state coding features of the vehicle at the historical decision moments, the historical driving strategy coding vectors of the vehicle at the historical decision moments, the cumulative reward coding vectors at the historical decision moments, the current state coding features and the cumulative reward coding vector at the current decision moment; Among them, the discarded span embedding vector at any decision moment is used to represent the number of images of the vehicle exterior images for which driving decisions cannot be made continuously; the preset embedding vector is used to represent that the number of images of the vehicle exterior images for which driving decisions cannot be made continuously is 0.

[0008] According to a decision-making method for realizing automatic driving provided by the present invention, whether the image outside the vehicle is an image for making a driving decision is determined based on the following method: Inputting the vehicle exterior image into a feature extraction model to obtain image features output by the feature extraction model; The image features are input into a binary classification model to obtain a classification result output by the binary classification model; the classification result includes a first classification result or a second classification result, the first classification result is used to indicate that the image outside the vehicle is an image for which driving decisions cannot be made, and the second classification result is used to indicate that the image outside the vehicle is an image for which driving decisions can be made.

[0009] According to a decision method for realizing automatic driving provided by the present invention, the step of inputting the external image of the vehicle into a feature extraction model to obtain image features output by the feature extraction model includes: Dividing the vehicle exterior image into a plurality of image blocks, and determining position encoding vectors of the plurality of image blocks; For any of the image blocks, embedding the position coding vector of the image block into the feature vector of the image block to obtain the feature coding vector of the image block; Determining a feature sequence based on the feature coding vectors of the plurality of image blocks; The feature sequence is input into a feature extraction model to obtain image features output by the feature extraction model.

[0010] According to a decision-making method for realizing automatic driving provided by the present invention, before inputting the current state feature into a strategy prediction model to obtain the current driving strategy of the vehicle at the current decision moment output by the strategy prediction model, the method further includes: If the exterior image of the vehicle collected at the current decision-making moment is not obtained, the state feature of the vehicle at the previous decision-making moment is determined as the current state feature of the vehicle at the current decision-making moment.

[0011] The present invention also provides a decision-making device for realizing automatic driving, comprising: An image acquisition module is used to acquire an image outside the vehicle collected at the current decision moment and determine image features of the image outside the vehicle; a feature determination module, configured to determine, if it is determined based on the image feature that the image outside the vehicle is an image for which a driving decision cannot be made, a state feature of the vehicle at a previous decision moment as a current state feature of the vehicle at the current decision moment; The feature determination module is further configured to determine the current state feature of the vehicle at the current decision moment based on the image feature if the image outside the vehicle is determined to be an image for making a driving decision based on the image feature; A strategy prediction module is used to input the current state features into a strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model; the strategy prediction model is trained based on sample external vehicle images.

[0012] The present invention also provides a car, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a decision-making method for realizing automatic driving as described in any one of the above.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a decision-making method for realizing autonomous driving as described in any one of the above-mentioned methods.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a decision-making method for realizing autonomous driving as described in any one of the above.

[0015] The decision-making method, device, automobile, equipment and storage medium for realizing automatic driving provided by the present invention obtain an image outside the vehicle collected by the automobile at the current decision moment, determine the image features of the image outside the vehicle, if the image outside the vehicle is determined to be an image for making a driving decision based on the image features, determine the current state features of the automobile at the current decision moment based on the image features, if the image outside the vehicle is determined to be an image for making a driving decision based on the image features, determine the state features of the automobile at the previous decision moment as the current state features of the automobile at the current decision moment, thereby ensuring that the current state features are determined based on the image features of the image outside the vehicle for making a driving decision, and further ensuring that the current state features for making a driving decision are input into a strategy prediction model, and obtaining the current driving strategy of the automobile at the current decision moment output by the strategy prediction model, that is, ensuring that the current driving strategy is determined based on the image features of the image outside the vehicle for making a driving decision, that is, ensuring that an accurate driving strategy is obtained, thereby improving the decision accuracy of automatic driving, and ultimately improving the safety of automatic driving of the automobile. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the decision-making method for realizing automatic driving provided by the present invention.

[0018] Figure 2 This is the second flow chart of the decision-making method for realizing automatic driving provided by the present invention.

[0019] Figure 3 This is the third flow chart of the decision-making method for realizing automatic driving provided by the present invention.

[0020] Figure 4 It is a schematic diagram of the structure of a decision-making device for realizing automatic driving provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] At present, autonomous driving is achieved by collecting clear road images in front of the car and inputting them into a computer vision model to obtain the final autonomous driving command. However, real road conditions are not always ideal, such as sudden objects blocking the line of sight and scenes with extremely low visibility. The existing technology is mainly aimed at rain, snow and foggy days with high visibility, but real road conditions are often strange and it is far from enough to only apply to some of the above scenes. Therefore, based on the road image with extremely blurred vision, it may not be possible to obtain accurate autonomous driving commands, which in turn leads to driving safety risks.

[0024] In view of the above problems, the present invention proposes the following embodiments. Figure 1-Figure 3 The decision-making method for realizing autonomous driving of the present invention is described.

[0025] Figure 1 is one of the flowcharts of the decision-making method for realizing automatic driving provided by the present invention, such as Figure 1 As shown, the decision-making method for realizing autonomous driving includes the following steps 110, 120, 130 and 140.

[0026] Step 110, obtaining an image outside the vehicle collected at the current decision moment, and determining image features of the image outside the vehicle.

[0027] Here, the car is an autonomous driving car. The current decision moment is the moment when the autonomous driving decision is currently made. It should be understood that the current decision moment is the current moment, so as to achieve real-time autonomous driving decision and ensure the safety of autonomous driving.

[0028] Here, the vehicle exterior image is an image of the vehicle exterior environment. The vehicle exterior image captured at the current decision moment may be one or more images. If there are multiple images, they may be vehicle exterior images from different perspectives, such as images of the road conditions ahead of the vehicle in the forward direction, images of the side of the vehicle, images of the rear of the vehicle, and the like. In one embodiment, the vehicle exterior image is an image of the road conditions ahead of the vehicle in the forward direction. In one embodiment, the vehicle exterior image is captured by an image capture device (such as a camera) of the vehicle.

[0029] In one embodiment, when the car is in an automatic driving state, an image outside the car collected by the car at the current decision-making moment is obtained.

[0030] In some embodiments, the image outside the vehicle is input into a feature extraction model to obtain image features output by the feature extraction model.

[0031] In one embodiment, the image outside the vehicle is divided into multiple image blocks, a feature sequence is determined based on the feature vectors of the multiple image blocks, and the feature sequence is input into a feature extraction model to obtain image features output by the feature extraction model. The feature extraction model can be a transformer model. Based on this, by dividing the image outside the vehicle into multiple image blocks and converting each image block into a vector representation as an input sequence, the contextual dependencies of different positions in the image outside the vehicle can be captured, that is, the feature extraction capability is improved, and then the representation capability of the image features is improved, and finally the decision accuracy of the autonomous driving is improved.

[0032] Furthermore, the image outside the vehicle is divided into multiple image blocks, and the position coding vectors of the multiple image blocks are determined; for any image block, the position coding vector of the image block is embedded in the feature vector of the image block to obtain the feature coding vector of the image block; based on the feature coding vectors of the multiple image blocks, the feature sequence is determined. Based on this, by dividing the image outside the vehicle into multiple image blocks, and converting each image block into a vector representation as an input sequence, and also embedding the position coding vector of the image block in the feature vector of the image block, the context dependency of different positions in the image outside the vehicle can be more accurately captured, the feature extraction capability can be further improved, and then the representation capability of image features can be further improved, and finally the decision accuracy of autonomous driving can be improved.

[0033] Step 120: If it is determined based on the image features that the image outside the vehicle is an image in which a driving decision cannot be made, the state features of the vehicle at the last decision moment are determined as the current state features of the vehicle at the current decision moment.

[0034] Here, images that make it impossible to make driving decisions may include, but are not limited to, at least one of the following: images with blocked vision, images with extremely low visibility (such as sandstorm scenes), and images with extremely blurred vision, etc.

[0035] In one embodiment, the image features are input into a binary classification model to obtain a classification result output by the binary classification model; the classification result includes a first classification result or a second classification result, the first classification result is used to indicate that the image outside the vehicle is an image in which a driving decision cannot be made, and the second classification result is used to indicate that the image outside the vehicle is an image in which a driving decision can be made. The binary classification model is trained based on sample images outside the vehicle and their corresponding classification result labels.

[0036] Here, the last decision moment is the decision moment before the current decision moment, which is the moment when the driving strategy was last determined. It should be understood that the state characteristics of the last decision moment are determined in the same way as the current state characteristics of the current decision moment.

[0037] Taking into account that the image outside the vehicle collected at the current decision-making moment is an image that cannot make a driving decision, the image features of the image outside the vehicle are no longer suitable for determining the current state features at the current decision-making moment. Therefore, the state features at the previous decision moment are determined as the current state features of the vehicle at the current decision moment, thereby ensuring that the current state features are determined based on the image features of the image outside the vehicle on which a driving decision can be made, and then ensuring that the current driving strategy is determined based on the image features of the image outside the vehicle on which a driving decision can be made, that is, ensuring that an accurate driving strategy is obtained, thereby improving the safety of the vehicle's automatic driving.

[0038] Here, the state feature at any decision moment is determined based on the image feature of the vehicle exterior image collected at the decision moment. The state feature can be used to characterize the state information of the vehicle exterior environment. In one embodiment, the image feature is directly determined as the state feature. In another embodiment, the image feature is further processed to obtain the state feature.

[0039] It should be noted that if the image outside the vehicle collected at the last decision moment is also an image that cannot make a driving decision, the state feature at the last decision moment is the state feature of the last two decision moments.

[0040] Step 130 , if it is determined based on the image features that the image outside the vehicle is an image for making a driving decision, the current state features of the vehicle at the current decision moment are determined based on the image features.

[0041] Considering that the outside image of the vehicle collected at the current decision-making moment is an image that can be used for driving decision-making, that is, the image features of the outside image of the vehicle are suitable for determining the current state features at the current decision-making moment, therefore, the current state features at the current decision-making moment can be determined based on the image features.

[0042] In one embodiment, the image features are directly determined as the current state features. In another embodiment, the image features are further processed to obtain the current state features.

[0043] For example, when the image outside the vehicle is an image that can be used for driving decision making 0, when the image outside the vehicle is such that driving decisions cannot be made is 1, and the current state characteristics at the current decision moment are determined based on the following formula: ; In the formula, represents the current state characteristics at the current decision moment, represents the state characteristics of the previous decision moment, represents the real state feature determined based on the image features of the vehicle exterior image at the current decision moment. are state characteristics that need to be input into the model.

[0044] Step 140, inputting the current state feature into a strategy prediction model, and obtaining the current driving strategy of the automobile at the current decision moment output by the strategy prediction model.

[0045] The strategy prediction model is obtained by training based on sample external vehicle images.

[0046] Here, the current driving strategy is used to instruct the car to perform automatic driving, that is, to control the car based on the current driving strategy. For example, the current driving strategy includes a car control command, and the car control command may include a steering control command, a braking command, a refueling command, etc., which are not specifically limited here.

[0047] In one embodiment, an input sequence is determined based on the historical state features and current state features of the car at historical decision moments. The historical decision moment is the decision moment before the current decision moment, and the historical decision moment includes at least the previous decision moment. The number of the historical decision moments can be set according to actual needs. For example, the strategy prediction model can be built based on the transformer model. Based on this, the state features of each decision moment are used as the input sequence to capture the contextual dependency of the state features at different decision moments, that is, to improve the feature extraction capability, thereby improving the accuracy of subsequent strategy predictions, and ultimately improving the decision accuracy of autonomous driving.

[0048] In one embodiment, the current state characteristics and the instant reward at the current decision moment are input into the strategy prediction model to obtain the current driving strategy output by the strategy prediction model. Considering that the reinforcement learning model needs to interact with the environment, it is not suitable in the real-time scenario of autonomous driving. Based on this, the strategy prediction model can be an offline reinforcement learning model, thereby improving the safety of autonomous driving.

[0049] The decision-making method for realizing automatic driving provided by the embodiment of the present invention obtains the outside image of the car collected at the current decision moment, determines the image features of the outside image, and if it is determined based on the image features that the outside image is an image for making a driving decision, the current state features of the car at the current decision moment are determined based on the image features; if it is determined based on the image features that the outside image is an image for which a driving decision cannot be made, the state features of the car at the previous decision moment are determined as the current state features of the car at the current decision moment, thereby ensuring that the current state features are determined based on the image features of the outside image for which a driving decision can be made, and further ensuring that the current state features for which a driving decision can be made are input into the strategy prediction model, and the current driving strategy of the car at the current decision moment output by the strategy prediction model is obtained, that is, ensuring that the current driving strategy is determined based on the image features of the outside image for which a driving decision can be made, that is, ensuring that an accurate driving strategy is obtained, thereby improving the decision accuracy of automatic driving, and ultimately improving the safety of automatic driving of the car.

[0050] Based on any of the above embodiments, Figure 2 FIG. 2 is a flow chart of the decision-making method for realizing automatic driving provided by the present invention. Figure 2 As shown, the above step 140 includes step 141, step 142, step 143 and step 144.

[0051] Step 141, when the image outside the vehicle is an image in which a driving decision cannot be made, the accumulated reward at the previous decision moment is determined as the accumulated reward at the current decision moment.

[0052] Among them, the cumulative reward at any decision moment is the cumulative value of the instant rewards at each decision moment from the decision moment to the future termination decision moment. The instant reward at any decision moment is used to characterize the value of the driving strategy at the decision moment.

[0053] Here, the cumulative reward (return to go) reflects the sum of all immediate rewards from the current time step (decision moment) to the time step at the end of the sequence (future termination decision moment). Furthermore, the immediate rewards of future time steps will be discounted, thereby reducing the impact of future immediate rewards, thereby balancing the rewards of the current time step with the rewards of future time steps, thereby guiding the policy prediction model to make more reasonable decisions in pursuit of long-term goals. Based on this, we not only care about the immediate rewards of the current time step, but also the rewards of future time steps, that is, the long-term cumulative rewards, so that we can understand the impact of the actions taken at the current time step on the future, so that in the field of autonomous driving, we can better meet its safety requirements. Furthermore, during the verification phase of the policy prediction model's training process, its cumulative rewards can be determined manually, thereby guiding its trajectory so that the model's effect is as close to the cumulative rewards as possible.

[0054] Taking into account that the image outside the vehicle collected at the current decision-making moment is an image that cannot make a driving decision, the cumulative reward at the previous decision-making moment is determined as the cumulative reward at the current decision-making moment, thereby ensuring an accurate driving strategy and improving the safety of the car's autonomous driving.

[0055] It should be noted that if the image outside the vehicle collected at the last decision moment is also an image that cannot make a driving decision, the cumulative reward at the last decision moment is the cumulative reward of the last two decision moments.

[0056] Step 142, when the image outside the vehicle is an image that can be used for driving decision making, determine the cumulative reward at the current decision moment.

[0057] Considering that the image outside the vehicle collected at the current decision-making moment is an image that can be used for driving decision-making, that is, the image features of the image outside the vehicle are suitable for determining the current state features at the current decision-making moment, therefore, the cumulative reward (real cumulative reward) at the current decision-making moment can be directly determined.

[0058] For example, when the image outside the vehicle is an image that can be used for driving decision making 0, when the image outside the vehicle is such that driving decisions cannot be made is 1, and the cumulative reward at the current decision moment is determined based on the following formula: ; In the formula, represents the cumulative reward at the current decision moment, represents the cumulative reward at the last decision moment, Represents the actual cumulative reward at the current decision moment. Among them, the cumulative reward at the current decision moment is the reward that needs to be input into the model.

[0059] The actual cumulative reward at the current decision moment can be determined based on a pre-trained reward function, or it can be defaulted to a preset value. Of course, other methods can also be used to determine the cumulative reward at the current decision moment, which is not specifically limited here.

[0060] Step 143, determining an input sequence based on the historical state characteristics of the car at historical decision-making moments, the historical driving strategies of the car at historical decision-making moments, the accumulated rewards at historical decision-making moments, the current state characteristics and the accumulated rewards at the current decision-making moment.

[0061] The historical decision moment is the decision moment before the current decision moment. The historical decision moment includes at least the previous decision moment, and the number of the historical decision moments can be set according to actual needs; for example, based on the historical state characteristics of the car at multiple historical decision moments, the historical driving strategy of the car at multiple historical decision moments, the cumulative rewards of multiple historical decision moments, the current state characteristics and the cumulative rewards of the current decision moment, the input sequence is determined, and it should be understood that multiple historical decision moments are continuous decision moments.

[0062] Here, the method for determining the historical state characteristics at the historical decision moments is the same as the method for determining the current state characteristics at the current decision moment; the method for determining the historical driving strategies at the historical decision moments is the same as the method for determining the current driving strategies at the current decision moment.

[0063] Specifically, the historical state features of the car at the historical decision-making moments, the historical driving strategies of the car at the historical decision-making moments, the accumulated rewards at the historical decision-making moments, the current state features, and the accumulated rewards at the current decision-making moment are combined and encoded into an input sequence. Based on this, the state features, accumulated rewards, and driving strategies (not available at the current decision-making moment) at each decision-making moment are used as input sequences by using sequence modeling, thereby capturing the contextual dependencies at different decision-making moments, that is, improving the feature extraction capability, thereby improving the accuracy of subsequent strategy predictions, and ultimately improving the decision-making accuracy of autonomous driving.

[0064] Step 144, inputting the input sequence into a strategy prediction model to obtain the current driving strategy output by the strategy prediction model.

[0065] For example, the policy prediction model can be built based on the transformer model.

[0066] In one embodiment, the policy prediction model is an offline reinforcement learning model, so that there is no need to interact with the environment, thereby improving the safety of autonomous driving. For example, the offline reinforcement learning model can be built based on the GPT model.

[0067] It should be noted that when training the strategy prediction model, some time steps of the sample data need to be masked according to a certain percentage; specifically, some time steps are identified as being unable to make driving decisions, and then the corresponding state features and cumulative rewards are set to the state features and cumulative rewards corresponding to the previous time step, thereby ensuring that in the application stage, even if the image outside the vehicle is an image that cannot make a driving decision, a reliable current driving strategy can be output.

[0068] The decision-making method for realizing automatic driving provided by the embodiment of the present invention determines the cumulative reward of the previous decision moment as the cumulative reward of the current decision moment when the image outside the vehicle is an image in which a driving decision cannot be made, and determines the cumulative reward of the current decision moment when the image outside the vehicle is an image in which a driving decision can be made, thereby ensuring that an accurate driving strategy is obtained and improving the safety of automatic driving of the vehicle; based on the historical state characteristics of the vehicle at historical decision moments, the historical driving strategies of the vehicle at historical decision moments, the cumulative rewards at historical decision moments, the current state characteristics and the cumulative rewards at the current decision moment, an input sequence is determined, and the input sequence is input into a strategy prediction model to obtain the current driving strategy output by the strategy prediction model, thereby taking the state characteristics, cumulative rewards and driving strategies at each decision moment as the input sequence, thereby capturing the contextual dependencies at different decision moments, that is, improving the feature extraction capability, thereby improving the accuracy of subsequent strategy predictions, and ultimately improving the decision accuracy of automatic driving.

[0069] Based on any of the above embodiments, in the method, Figure 3 FIG. 3 is a flow chart of a decision-making method for realizing automatic driving provided by the present invention. Figure 3 As shown, the above step 143 includes: step 1431, step 1432, step 1433 and step 1434.

[0070] Step 1431, when the external image of the vehicle is an image for which a driving decision cannot be made, the discard span embedding vector of the current decision moment is determined based on the discard span embedding vector of the previous decision moment.

[0071] Among them, the discarded span embedding vector at any decision moment is used to represent the number of images of the vehicle exterior for which driving decisions cannot be made continuously.

[0072] When the image outside the vehicle is an image in which a driving decision cannot be made, the difference between the number of images represented by the discarded span embedding vector at the current decision moment and the number of images represented by the discarded span embedding vector at the previous decision moment is 1.

[0073] It should be understood that the discarded span embedding vector at the current decision moment is used to characterize the number of images of the vehicle exterior for which driving decisions cannot be made consecutively at the current decision moment. Combined with this embedding vector, the current driving strategy at the current decision moment can be better determined, ultimately improving the decision accuracy of autonomous driving.

[0074] Step 1432, when the external image of the vehicle is an image for making a driving decision, a preset embedding vector is determined as the discard span embedding vector at the current decision moment.

[0075] The number of images of the preset embedding vector used to represent the continuous vehicle exterior images in which driving decisions cannot be made is 0.

[0076] When the image outside the vehicle is an image that can be used for driving decision-making, the number of images represented by the discarded span embedding vector at the current decision moment is 0. Therefore, combining this embedding vector can better determine the current driving strategy at the current decision moment, ultimately improving the decision accuracy of autonomous driving.

[0077] Step 1433, based on the current state feature, the current decision moment and the discarded span embedding vector of the current decision moment, determine the current state encoding feature, and based on the cumulative reward at the current decision moment, the current decision moment and the discarded span embedding vector of the current decision moment, determine the cumulative reward encoding vector of the current decision moment, and based on the historical driving strategy at the previous decision moment and the current decision moment, determine the historical driving strategy encoding vector of the previous decision moment.

[0078] Considering that the current state feature may be the state feature of the previous decision moment, based on this, the current state encoding feature is determined based on the current state feature, the current decision moment and the discarded span embedding vector of the current decision moment, so as to fully consider the current decision moment and the discarded span embedding vector of the current decision moment, and then the current driving strategy at the current decision moment can be better determined, and finally the decision accuracy of autonomous driving can be improved.

[0079] In one embodiment, feature aggregation processing is performed on the current state feature, the time step embedding vector at the current decision moment, and the discarded span embedding vector at the current decision moment to obtain the current state encoding feature.

[0080] Considering that the cumulative reward at the current decision moment may be the cumulative reward at the previous decision moment, based on this, the cumulative reward encoding vector at the current decision moment is determined based on the cumulative reward at the current decision moment, the current decision moment and the discarded span embedding vector at the current decision moment, so as to fully consider the current decision moment and the discarded span embedding vector at the current decision moment, and then better determine the current driving strategy at the current decision moment, and ultimately improve the decision accuracy of autonomous driving.

[0081] In one embodiment, the cumulative reward at the current decision moment, the time step embedding vector at the current decision moment, and the discarded span embedding vector at the current decision moment are feature aggregated to obtain the cumulative reward encoding vector at the current decision moment.

[0082] Considering that the historical driving strategies at any decision-making moment are real driving strategies, based on this, we only need to determine the historical driving strategy encoding vector of the previous decision moment based on the historical driving strategy of the previous decision moment and the current decision moment. Therefore, considering the current decision moment, we can better determine the current driving strategy at the current decision moment, and ultimately improve the decision accuracy of autonomous driving.

[0083] In one embodiment, feature aggregation processing is performed on the historical driving strategy at the last decision moment and the time step embedding vector at the current decision moment to obtain the historical driving strategy encoding vector at the last decision moment.

[0084] Step 1434, determine the input sequence based on the historical state coding features of the car at the historical decision-making moments, the historical driving strategy coding vector of the car at the historical decision-making moments, the cumulative reward coding vector at the historical decision-making moments, the current state coding features and the cumulative reward coding vector at the current decision-making moment.

[0085] Here, the method for determining the historical state coding features at the historical decision moment is the same as the method for determining the current state coding features, the method for determining the historical driving strategy coding vector at the historical decision moment is the same as the method for determining the historical driving strategy coding vector at the previous decision moment, and the method for determining the cumulative reward coding vector at the historical decision moment is the same as the method for determining the cumulative reward coding vector at the current decision moment.

[0086] Specifically, the historical state encoding features of the car at the historical decision-making moment, the historical driving strategy encoding vector of the car at the historical decision-making moment, the cumulative reward encoding vector at the historical decision-making moment, the current state encoding features and the cumulative reward encoding vector at the current decision-making moment are combined and encoded into an input sequence. Based on this, the state encoding features, cumulative reward encoding vectors and driving strategy encoding vectors (not available at the current decision moment) at each decision moment are used as input sequences by using sequence modeling, thereby capturing the contextual dependencies at different decision moments, that is, improving the feature extraction capability, thereby improving the accuracy of subsequent strategy predictions, and ultimately improving the decision accuracy of autonomous driving.

[0087] The decision-making method for realizing automatic driving provided by an embodiment of the present invention, when the image outside the vehicle is an image for which a driving decision cannot be made, the discard span embedding vector of the current decision moment is determined based on the discard span embedding vector of the previous decision moment, and when the image outside the vehicle is an image for which a driving decision can be made, a preset embedding vector is determined as the discard span embedding vector of the current decision moment, and the discard span embedding vector of any decision moment is used to represent the number of images outside the vehicle for which a driving decision cannot be made continuously, and the preset embedding vector is used to represent the number of images outside the vehicle for which a driving decision cannot be made continuously is 0, so that in combination with the embedding vector, the current driving strategy at the current decision moment can be better determined, and ultimately the decision accuracy of automatic driving can be improved; based on the current state features, the current decision moment and the discard span at the current decision moment The current state encoding feature is determined based on the cumulative reward at the current decision moment, the current decision moment and the discard span embedding vector at the current decision moment, and the historical driving strategy encoding vector at the previous decision moment is determined based on the historical driving strategy at the previous decision moment and the current decision moment. The input sequence is determined based on the historical state encoding features of the car at the historical decision moments, the historical driving strategy encoding vector of the car at the historical decision moments, the cumulative reward encoding vector at the historical decision moments, the current state encoding features and the cumulative reward encoding vector at the current decision moment, so as to fully consider the current decision moment and the discard span embedding vector at the current decision moment, thereby better determining the input sequence at the current decision moment, and ultimately improving the decision accuracy of autonomous driving.

[0088] Based on any of the above embodiments, in the method, whether the image outside the vehicle is an image for making a driving decision is determined based on the following method: Inputting the vehicle exterior image into a feature extraction model to obtain image features output by the feature extraction model; The image features are input into a binary classification model to obtain a classification result output by the binary classification model; the classification result includes a first classification result or a second classification result, the first classification result is used to indicate that the image outside the vehicle is an image for which driving decisions cannot be made, and the second classification result is used to indicate that the image outside the vehicle is an image for which driving decisions can be made.

[0089] The binary classification model is trained based on sample vehicle exterior images and their corresponding classification result labels. For example, the binary classification model is built based on MLP (Multilayer Perceptron).

[0090] In one embodiment, the image outside the vehicle is divided into multiple image blocks, a feature sequence is determined based on the feature vectors of the multiple image blocks, and the feature sequence is input into a feature extraction model to obtain image features output by the feature extraction model. The feature extraction model can be a transformer model. Based on this, by dividing the image outside the vehicle into multiple image blocks and converting each image block into a vector representation as an input sequence, the contextual dependencies of different positions in the image outside the vehicle can be captured, that is, the feature extraction capability is improved, and then the representation capability of the image features is improved, and finally the decision accuracy of the autonomous driving is improved.

[0091] In the decision-making method for realizing autonomous driving provided by an embodiment of the present invention, the image features output by the feature extraction model are not only used to determine the current state features of the car at the current decision-making moment, but also used to determine whether the image outside the car is an image for making a driving decision. Thus, the extraction of image features once can be used for two purposes, simplifying the model structure and improving the decision-making efficiency of autonomous driving.

[0092] Based on any of the above embodiments, in the method, inputting the vehicle exterior image into a feature extraction model to obtain image features output by the feature extraction model includes: Dividing the vehicle exterior image into a plurality of image blocks, and determining position encoding vectors of the plurality of image blocks; For any of the image blocks, embedding the position coding vector of the image block into the feature vector of the image block to obtain the feature coding vector of the image block; Determining a feature sequence based on the feature coding vectors of the plurality of image blocks; The feature sequence is input into a feature extraction model to obtain image features output by the feature extraction model.

[0093] Specifically, the vehicle exterior image is divided into a plurality of image patches. Further, the vehicle exterior image is divided into a plurality of image patches of the same size.

[0094] The position coding vector is used to represent the position information of each image block in the complete vehicle exterior image. In one embodiment, the position coding vectors of the plurality of image blocks are determined according to the division method of the vehicle exterior image.

[0095] Here, the feature vector of the image block is a vector for extracting features from the image block. In one embodiment, the image block is compressed into a feature vector of a certain dimension through a fully connected network.

[0096] The position encoding vector of the image block is embedded in the feature vector of the image block, so that a position information is appended to the feature vector of the image block, so as to better capture the contextual dependencies of different positions in the image outside the vehicle, that is, improve the feature extraction capability, and then improve the representation capability of image features, and finally improve the decision accuracy of autonomous driving.

[0097] Specifically, the feature coding vectors of the plurality of image blocks are combined and encoded to obtain a feature sequence. Further, based on the feature coding vectors of the plurality of image blocks and a preset classification vector (Class Token), a feature sequence is determined.

[0098] For example, the feature sequence is input into a plurality of encoding layers connected sequentially in the feature extraction model, and the last encoding layer outputs the image features.

[0099] In one embodiment, a VIT (Vision Transformer) can be used to construct an architecture for extracting image features, thereby fully considering the advantage of a large amount of data in autonomous driving scenarios and ultimately improving the accuracy of autonomous driving decisions.

[0100] The decision-making method for realizing autonomous driving provided by an embodiment of the present invention divides the external image of the vehicle into multiple image blocks, converts each image block into a vector representation as an input sequence, and embeds the position encoding vector of the image block on the feature vector of the image block, so as to more accurately capture the contextual dependency relationship of different positions in the external image of the vehicle, further improve the feature extraction capability, and then further improve the representation capability of image features, and ultimately improve the decision accuracy of autonomous driving.

[0101] Based on any of the above embodiments, considering that the image outside the vehicle may not be collected at the current decision moment, or the image outside the vehicle at the current decision moment may not be obtained due to communication reasons, resulting in the inability to make a driving decision; based on this, before the above step 140, the method further includes: If the exterior image of the vehicle collected at the current decision-making moment is not obtained, the state feature of the vehicle at the previous decision-making moment is determined as the current state feature of the vehicle at the current decision-making moment.

[0102] Considering that if the outside image collected by the car at the current decision-making moment is not obtained, the state characteristics at the previous decision-making moment are determined as the current state characteristics of the car at the current decision-making moment, thereby ensuring that the current state characteristics are determined based on the image characteristics of the outside image of the car for driving decisions, and then ensuring that the current driving strategy is determined based on the image characteristics of the outside image of the car for driving decisions, that is, ensuring that an accurate driving strategy is obtained, thereby improving the safety of the car's autonomous driving.

[0103] It should be noted that if the external image of the car collected at the last decision moment is not obtained, the state characteristics at the last decision moment are the state characteristics of the last two decision moments.

[0104] Accordingly, the above-mentioned step 140 includes: in the case where the outside image of the car collected at the current decision moment is not obtained, determining the cumulative reward of the previous decision moment as the cumulative reward of the current decision moment; in the case where the outside image of the car is an image in which a driving decision cannot be made, determining the cumulative reward of the previous decision moment as the cumulative reward of the current decision moment; in the case where the outside image of the car is an image in which a driving decision can be made, determining the cumulative reward of the current decision moment; determining the input sequence based on the historical state characteristics of the car at historical decision moments, the historical driving strategies of the car at historical decision moments, the cumulative rewards at historical decision moments, the current state characteristics and the cumulative rewards at the current decision moment; the historical decision moments are decision moments before the current decision moment; inputting the input sequence into the strategy prediction model to obtain the current driving strategy output by the strategy prediction model.

[0105] Correspondingly, the input sequence is determined based on the historical state characteristics of the car at the historical decision moments, the historical driving strategies of the car at the historical decision moments, the accumulated rewards at the historical decision moments, the current state characteristics and the accumulated rewards at the current decision moment, including: when the external image of the car collected at the current decision moment is not obtained, the discard span embedding vector of the current decision moment is determined based on the discard span embedding vector of the previous decision moment; when the external image is an image that cannot make a driving decision, the discard span embedding vector of the current decision moment is determined based on the discard span embedding vector of the previous decision moment; when the external image is an image that can make a driving decision, a preset embedding vector is determined as the discard span of the current decision moment. embedding vector; determine the current state encoding feature based on the current state feature, the current decision moment and the discarded span embedding vector of the current decision moment, and determine the cumulative reward encoding vector of the current decision moment based on the cumulative reward of the current decision moment, the current decision moment and the discarded span embedding vector of the current decision moment, and determine the historical driving strategy encoding vector of the previous decision moment based on the historical driving strategy of the previous decision moment and the current decision moment; determine the input sequence based on the historical state encoding feature of the car at the historical decision moment, the historical driving strategy encoding vector of the car at the historical decision moment, the cumulative reward encoding vector of the historical decision moment, the current state encoding feature and the cumulative reward encoding vector of the current decision moment.

[0106] The decision-making method for realizing automatic driving provided by an embodiment of the present invention determines the state characteristics of the car at the previous decision moment as the current state characteristics of the car at the current decision moment if the outside image of the car collected at the current decision moment is not obtained, thereby ensuring that the current state characteristics are determined based on the image characteristics of the outside image of the car for driving decision making, and further ensuring that the current state characteristics for driving decision making are input into the strategy prediction model, and obtaining the current driving strategy of the car at the current decision moment output by the strategy prediction model, that is, ensuring that the current driving strategy is determined based on the image characteristics of the outside image of the car for driving decision making, that is, ensuring that an accurate driving strategy is obtained, thereby improving the decision accuracy of automatic driving, and ultimately improving the safety of automatic driving of the car.

[0107] Based on the above embodiments, automatic driving can be achieved in scenes with extremely blurred or even invisible vision, and automatic driving can be achieved in scenes with temporary failure of the car camera, that is, the vehicle can continue to operate when the vision disappears for a short time.

[0108] The decision-making device for implementing autonomous driving provided by the present invention is described below. The decision-making device for implementing autonomous driving described below and the decision-making method for implementing autonomous driving described above can be referenced to each other.

[0109] Figure 4 is a schematic diagram of the structure of the decision-making device for realizing automatic driving provided by the present invention, such as Figure 4 As shown, the decision-making device for realizing autonomous driving includes an image acquisition module 410 , a feature determination module 420 and a strategy prediction module 430 .

[0110] The image acquisition module 410 is used to acquire the image outside the vehicle collected at the current decision moment and determine the image features of the image outside the vehicle.

[0111] The feature determination module 420 is used to determine the state feature of the car at the last decision moment as the current state feature of the car at the current decision moment if it is determined based on the image feature that the image outside the car is an image that cannot make a driving decision.

[0112] The feature determination module 420 is further used to determine the current state feature of the vehicle at the current decision moment based on the image feature if the image outside the vehicle is determined to be an image for making a driving decision based on the image feature.

[0113] The strategy prediction module 430 is used to input the current state features into the strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model; the strategy prediction model is obtained based on sample external vehicle image training.

[0114] The decision-making device for realizing automatic driving provided by the embodiment of the present invention obtains an image outside the vehicle collected by the vehicle at the current decision moment, determines the image features of the image outside the vehicle, and if it is determined based on the image features that the image outside the vehicle is an image for making a driving decision, the current state features of the vehicle at the current decision moment are determined based on the image features; if it is determined based on the image features that the image outside the vehicle is an image for which a driving decision cannot be made, the state features of the vehicle at the previous decision moment are determined as the current state features of the vehicle at the current decision moment, thereby ensuring that the current state features are determined based on the image features of the image outside the vehicle for which a driving decision can be made, and further ensuring that the current state features for which a driving decision can be made are input into a strategy prediction model, and the current driving strategy of the vehicle at the current decision moment output by the strategy prediction model is obtained, that is, ensuring that the current driving strategy is determined based on the image features of the image outside the vehicle for which a driving decision can be made, that is, ensuring that an accurate driving strategy is obtained, thereby improving the decision accuracy of automatic driving, and ultimately improving the safety of automatic driving of the vehicle.

[0115] Based on any of the above embodiments, the strategy prediction module 430 is further used for: In the case where the image outside the vehicle is an image in which a driving decision cannot be made, determining the accumulated reward at the previous decision moment as the accumulated reward at the current decision moment; When the image outside the vehicle is an image that can be used to make a driving decision, determining the cumulative reward at the current decision moment; Determine an input sequence based on the historical state characteristics of the car at historical decision moments, the historical driving strategies of the car at historical decision moments, the accumulated rewards at historical decision moments, the current state characteristics and the accumulated rewards at the current decision moment; the historical decision moment is the decision moment before the current decision moment; Inputting the input sequence into a strategy prediction model to obtain the current driving strategy output by the strategy prediction model; Among them, the cumulative reward at any decision moment is the cumulative value of the instant rewards at each decision moment from the decision moment to the future termination decision moment. The instant reward at any decision moment is used to characterize the value of the driving strategy at the decision moment.

[0116] Based on any of the above embodiments, the strategy prediction module 430 is further used for: In a case where the vehicle exterior image is an image for which a driving decision cannot be made, determining the discard span embedding vector at the current decision moment based on the discard span embedding vector at the previous decision moment; In the case where the vehicle exterior image is an image for making a driving decision, determining a preset embedding vector as a discard span embedding vector at the current decision moment; Determine a current state encoding feature based on the current state feature, the current decision moment, and the discard span embedding vector of the current decision moment, determine a cumulative reward encoding vector of the current decision moment based on the cumulative reward of the current decision moment, the current decision moment, and the discard span embedding vector of the current decision moment, and determine a historical driving strategy encoding vector of the previous decision moment based on the historical driving strategy of the previous decision moment and the current decision moment; Determine an input sequence based on the historical state coding features of the vehicle at the historical decision moments, the historical driving strategy coding vectors of the vehicle at the historical decision moments, the cumulative reward coding vectors at the historical decision moments, the current state coding features and the cumulative reward coding vector at the current decision moment; Among them, the discarded span embedding vector at any decision moment is used to represent the number of images of the vehicle exterior images for which driving decisions cannot be made continuously; the preset embedding vector is used to represent that the number of images of the vehicle exterior images for which driving decisions cannot be made continuously is 0.

[0117] Based on any of the above embodiments, the image acquisition module 410 is further used for: Inputting the vehicle exterior image into a feature extraction model to obtain image features output by the feature extraction model; The image features are input into a binary classification model to obtain a classification result output by the binary classification model; the classification result includes a first classification result or a second classification result, the first classification result is used to indicate that the image outside the vehicle is an image for which driving decisions cannot be made, and the second classification result is used to indicate that the image outside the vehicle is an image for which driving decisions can be made.

[0118] Based on any of the above embodiments, the image acquisition module 410 is further used for: Dividing the vehicle exterior image into a plurality of image blocks, and determining position encoding vectors of the plurality of image blocks; For any of the image blocks, embedding the position coding vector of the image block into the feature vector of the image block to obtain the feature coding vector of the image block; Determining a feature sequence based on the feature coding vectors of the plurality of image blocks; The feature sequence is input into a feature extraction model to obtain image features output by the feature extraction model.

[0119] Based on any of the above embodiments, the feature determination module 420 is further used to: If the exterior image of the vehicle collected at the current decision-making moment is not obtained, the state feature of the vehicle at the previous decision-making moment is determined as the current state feature of the vehicle at the current decision-making moment.

[0120] The present invention also provides a car, which may include: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein a decision method for automatic driving is implemented when the processor executes the computer program, the method comprising: acquiring an image outside the car collected by the car at a current decision moment, and determining image features of the image outside the car; if it is determined based on the image features that the image outside the car is an image for which a driving decision cannot be made, determining the state features of the car at the previous decision moment as the current state features of the car at the current decision moment; if it is determined based on the image features that the image outside the car is an image for which a driving decision can be made, determining the current state features of the car at the current decision moment based on the image features; inputting the current state features into a strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model; the strategy prediction model is obtained through training based on sample images outside the car.

[0121] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a decision-making method for realizing automatic driving, the method comprising: acquiring an image outside the vehicle collected by the vehicle at the current decision moment, and determining the image features of the image outside the vehicle; if the image outside the vehicle is determined to be an image that cannot make a driving decision based on the image features, determining the state features of the vehicle at the previous decision moment as the current state features of the vehicle at the current decision moment; if the image outside the vehicle is determined to be an image that can make a driving decision based on the image features, determining the current state features of the vehicle at the current decision moment based on the image features; inputting the current state features into a strategy prediction model to obtain the current driving strategy of the vehicle at the current decision moment output by the strategy prediction model; the strategy prediction model is obtained based on training of sample vehicle images outside the vehicle.

[0122] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0123] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the decision-making method for realizing automatic driving provided by the above-mentioned methods, which includes: obtaining an image outside the vehicle collected by the vehicle at the current decision-making moment, and determining the image features of the image outside the vehicle; if it is determined based on the image features that the image outside the vehicle is an image in which a driving decision cannot be made, the state features of the vehicle at the previous decision moment are determined as the current state features of the vehicle at the current decision moment; if it is determined based on the image features that the image outside the vehicle is an image in which a driving decision can be made, the current state features of the vehicle at the current decision moment are determined based on the image features; the current state features are input into a strategy prediction model to obtain the current driving strategy of the vehicle at the current decision moment output by the strategy prediction model; the strategy prediction model is trained based on sample images outside the vehicle.

[0124] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the decision-making method for realizing automatic driving provided by the above-mentioned methods, the method comprising: obtaining an image outside the vehicle collected by the vehicle at the current decision-making moment, and determining the image features of the image outside the vehicle; if it is determined based on the image features that the image outside the vehicle is an image in which a driving decision cannot be made, determining the state features of the vehicle at the previous decision moment as the current state features of the vehicle at the current decision moment; if it is determined based on the image features that the image outside the vehicle is an image in which a driving decision can be made, determining the current state features of the vehicle at the current decision moment based on the image features; inputting the current state features into a strategy prediction model to obtain the current driving strategy of the vehicle at the current decision moment output by the strategy prediction model; the strategy prediction model is trained based on sample images outside the vehicle.

[0125] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0126] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A decision-making method for realizing automatic driving, characterized in that: include: Acquire an image outside the vehicle collected at the current decision-making moment, and determine image features of the image outside the vehicle; If it is determined based on the image features that the image outside the vehicle is an image in which a driving decision cannot be made, the state features of the vehicle at the last decision moment are determined as the current state features of the vehicle at the current decision moment; If it is determined based on the image features that the image outside the vehicle is an image for making a driving decision, determining the current state features of the vehicle at the current decision moment based on the image features; The current state feature is input into a strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model; the strategy prediction model is trained based on sample external vehicle images.

2. The decision-making method for realizing automatic driving according to claim 1, characterized in that: The step of inputting the current state feature into a strategy prediction model to obtain a current driving strategy of the vehicle at the current decision moment output by the strategy prediction model includes: In the case where the image outside the vehicle is an image in which a driving decision cannot be made, determining the accumulated reward at the previous decision moment as the accumulated reward at the current decision moment; When the image outside the vehicle is an image that can be used to make a driving decision, determining the cumulative reward at the current decision moment; Determine an input sequence based on the historical state characteristics of the car at historical decision moments, the historical driving strategies of the car at historical decision moments, the accumulated rewards at historical decision moments, the current state characteristics and the accumulated rewards at the current decision moment; the historical decision moment is the decision moment before the current decision moment; Inputting the input sequence into a strategy prediction model to obtain the current driving strategy output by the strategy prediction model; Among them, the cumulative reward at any decision moment is the cumulative value of the instant rewards at each decision moment from the decision moment to the future termination decision moment. The instant reward at any decision moment is used to characterize the value of the driving strategy at the decision moment.

3. The decision-making method for realizing automatic driving according to claim 2, characterized in that: The step of determining an input sequence based on the historical state characteristics of the vehicle at historical decision-making moments, the historical driving strategies of the vehicle at historical decision-making moments, the accumulated rewards at historical decision-making moments, the current state characteristics, and the accumulated rewards at the current decision-making moment includes: In a case where the vehicle exterior image is an image for which a driving decision cannot be made, determining a discard span embedding vector at the current decision moment based on a discard span embedding vector at a previous decision moment; In the case where the vehicle exterior image is an image for making a driving decision, determining a preset embedding vector as a discard span embedding vector at the current decision moment; Determine a current state encoding feature based on the current state feature, the current decision moment, and the discard span embedding vector of the current decision moment, determine a cumulative reward encoding vector of the current decision moment based on the cumulative reward of the current decision moment, the current decision moment, and the discard span embedding vector of the current decision moment, and determine a historical driving strategy encoding vector of the previous decision moment based on the historical driving strategy of the previous decision moment and the current decision moment; Determine an input sequence based on the historical state coding features of the vehicle at the historical decision moments, the historical driving strategy coding vectors of the vehicle at the historical decision moments, the cumulative reward coding vectors at the historical decision moments, the current state coding features and the cumulative reward coding vector at the current decision moment; Among them, the discarded span embedding vector at any decision moment is used to represent the number of images of the vehicle exterior images for which driving decisions cannot be made continuously; the preset embedding vector is used to represent that the number of images of the vehicle exterior images for which driving decisions cannot be made continuously is 0.

4. The decision-making method for realizing automatic driving according to claim 1, characterized in that: The vehicle exterior image is determined to be an image for making driving decisions based on the following method: Inputting the vehicle exterior image into a feature extraction model to obtain image features output by the feature extraction model; The image features are input into a binary classification model to obtain a classification result output by the binary classification model; the classification result includes a first classification result or a second classification result, the first classification result is used to indicate that the image outside the vehicle is an image for which driving decisions cannot be made, and the second classification result is used to indicate that the image outside the vehicle is an image for which driving decisions can be made.

5. The decision-making method for realizing automatic driving according to claim 4, characterized in that: The step of inputting the vehicle exterior image into a feature extraction model to obtain image features output by the feature extraction model includes: Dividing the vehicle exterior image into a plurality of image blocks, and determining position encoding vectors of the plurality of image blocks; For any of the image blocks, embedding the position coding vector of the image block into the feature vector of the image block to obtain the feature coding vector of the image block; Determining a feature sequence based on the feature coding vectors of the plurality of image blocks; The feature sequence is input into a feature extraction model to obtain image features output by the feature extraction model.

6. The decision-making method for realizing automatic driving according to claim 1, characterized in that: Before inputting the current state feature into the strategy prediction model to obtain the current driving strategy of the automobile at the current decision moment output by the strategy prediction model, the method further includes: If the exterior image of the vehicle collected at the current decision-making moment is not obtained, the state feature of the vehicle at the previous decision-making moment is determined as the current state feature of the vehicle at the current decision-making moment.

7. A decision-making device for realizing automatic driving, characterized in that: include: An image acquisition module is used to acquire an image outside the vehicle collected at the current decision moment and determine image features of the image outside the vehicle; a feature determination module, configured to determine, if it is determined based on the image feature that the image outside the vehicle is an image for which a driving decision cannot be made, a state feature of the vehicle at a previous decision moment as a current state feature of the vehicle at the current decision moment; The feature determination module is further configured to determine the current state feature of the vehicle at the current decision moment based on the image feature if the image outside the vehicle is determined to be an image for making a driving decision based on the image feature; A strategy prediction module is used to input the current state features into a strategy prediction model to obtain the current driving strategy of the car at the current decision moment output by the strategy prediction model; the strategy prediction model is trained based on sample external vehicle images.

8. An automobile, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the decision-making method for realizing autonomous driving as described in any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the decision-making method for realizing autonomous driving as described in any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the decision-making method for realizing autonomous driving as described in any one of claims 1 to 6 is implemented.

Images