Vehicle control method and device, electronic equipment, computer readable storage medium and vehicle

Abstract

The invention relates to a vehicle control method and device, electronic equipment, a computer readable storage medium and a vehicle, and the vehicle control method comprises the steps: obtaining the driving data of the vehicle and the environment data of the driving environment; performing braking scene recognition based on the driving data and the environment data by using a scene recognition network obtained by deep learning, and determining at least one braking scene identifier; the at least one braking scene identifier is used for representing a target object state related to vehicle braking in a driving environment and / or a vehicle driving state; a braking command is determined based on the at least one braking scenario identification. According to the method, the classification precision of the emergency braking scene and the system response speed are improved, so that the braking effect and the driving safety are improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle control method, device, electronic equipment, computer-readable storage medium and vehicle. Background Art

[0002] The research and development of autonomous driving technology has made significant progress in the field of vehicle systems. However, for autonomous driving systems, how to accurately and efficiently identify danger and emergency braking scenarios remains a major challenge. Especially when dealing with emergency braking scenarios, the system must be able to quickly and accurately identify emergency events and respond appropriately to ensure the safety of the vehicle and passengers.

[0003] Most of the existing patents and technologies use image recognition and machine learning methods to identify targets and analyze dangerous situations. However, the intelligent models obtained by current machine learning methods may not be able to handle complex natural environments and emergencies, and there is also much room for improvement in recognition accuracy and response speed. Summary of the invention

[0004] One of the objects of the present invention is to provide a vehicle control method to improve the classification accuracy and system response speed of emergency braking scenarios, thereby improving the braking effect; the second object is to provide a vehicle control device; the third object is to provide an electronic device; the fourth object is to provide a computer-readable storage medium; and the fifth object is to provide a vehicle.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A vehicle control method, comprising:

[0007] Acquire the driving data of the vehicle and the environmental data of the driving environment;

[0008] Using a scene recognition network obtained by deep learning, performing braking scene recognition based on the driving data and the environmental data, and determining at least one braking scene identifier; the at least one braking scene identifier is used to characterize a target object state related to vehicle braking and / or a vehicle driving state in the driving environment;

[0009] Based on the at least one braking scenario identification, a braking command is determined.

[0010] According to the above technical means, the scene recognition network obtained by deep learning is used to identify the target object state related to vehicle braking and / or the vehicle driving state in the current driving environment of the vehicle based on the driving data during the vehicle driving process and the environmental data in the driving environment, and output it in the form of at least one scene identification. Since deep learning can more accurately identify and track targets, the recognition accuracy and system response speed of emergency braking scenes can be improved. In addition, based on at least one braking scene identification, a braking instruction that matches the current braking scene can be adaptively generated, so that the system can adapt to accurate braking instructions in a timely manner when facing scenes of different complexities, improve the accuracy of vehicle control, and thus improve the braking effect and driving safety.

[0011] Furthermore, before the scene recognition network obtained by deep learning performs braking scene recognition based on the driving data and the environmental data and determines at least one braking scene identifier, the method further includes:

[0012] Standardization processing is performed on the original sample sets collected under various driving environments to determine the training sample set; the original samples in the original sample set include sample driving data, sample environmental data and braking scene identification true value; the sample driving data includes at least one of vehicle position, vehicle motion state, vehicle control data, braking distance and collision situation; the sample environmental data includes at least one of target object attribute data, target object position, target object distance, target object motion state and target object occlusion situation; the braking scene identification true value is used to characterize the state of at least one target object related to vehicle braking and at least one vehicle driving state in the original sample;

[0013] The training sample set is used to perform deep learning training on the initial scene recognition network to obtain the scene recognition network.

[0014] According to the above technical means, a training sample set is obtained through standardized processing, and the training sample set is used to perform deep learning training on the initial scene recognition network. This allows the initial scene recognition network to self-learn and extract features from a large amount of data to handle tasks in high-dimensional and complex environments in actual driving environments, thereby more accurately identifying and tracking targets, greatly improving the classification accuracy and system response speed of emergency braking scenarios.

[0015] Furthermore, after determining the training sample set, the method further includes:

[0016] The training data set is data enhanced to perform deep learning training on the initial scene recognition network using the data enhanced training sample set to obtain the scene recognition network.

[0017] According to the above-mentioned technical means, through data enhancement, the network can fully explore the characteristics of various aspects of the training samples and improve generalization, thereby improving the recognition accuracy of the scene recognition network, and then improving the accuracy of at least one braking scene identification and the braking effect of the braking instruction generated based on at least one braking scene identification.

[0018] Further, the using the training sample set to perform deep learning training on the initial scene recognition network to obtain the scene recognition network includes:

[0019] Performing network reasoning based on training samples in the training sample set through the initial scene recognition network to obtain at least one braking scene identifier corresponding to the training sample;

[0020] Determine the learning rate corresponding to each round of training; the learning rate decreases round by round;

[0021] Based on at least one braking scene identifier corresponding to the training sample and the learning rate corresponding to each round of training, the network parameters of the initial scene recognition network are adjusted, and the next round of training is continued until the preset training conditions are met to obtain the scene recognition network.

[0022] According to the above technical means, reducing the learning rate round by round during the network training process can reduce oscillation and training error, balance the training convergence speed and training error, and improve the training effect.

[0023] Further, performing network inference based on the training samples in the training sample set to obtain at least one braking scene identifier corresponding to the training samples includes:

[0024] Batch normalization processing is performed on the input features of each network layer in the initial scene recognition network, and network inference is performed on each network layer based on the input features after the batch normalization processing to obtain at least one braking scene identification corresponding to the training sample.

[0025] According to the above technical means, batch regularization processing can effectively alleviate the phenomenon of gradient explosion and vanishing in deep neural networks, accelerate the speed of training convergence, and make network training more stable.

[0026] Furthermore, the scene recognition network includes: at least one of a residual network, a gated recurrent unit network, and a network model combining a residual network and a gated recurrent unit network.

[0027] According to the above technical means, by utilizing the structure of the residual network and / or the gated recurrent unit network, the gradient vanishing phenomenon in network training can be reduced, and the recognition effect of the trained scene recognition network can be improved.

[0028] Further, the determining the braking instruction based on the at least one braking scenario identifier includes:

[0029] A braking instruction is determined based on at least one braking scenario identifier and at least one preset braking strategy; the at least one braking strategy includes a braking strategy for one braking scenario identifier and / or a braking strategy for a combination of multiple braking scenario identifiers.

[0030] According to the above technical means, braking instructions are generated through targeted braking strategies for different emergency braking scenarios, which further improves the braking effect and driving safety.

[0031] A vehicle control device, comprising:

[0032] An acquisition module, used to acquire the driving data of the vehicle and the environmental data of the driving environment;

[0033] an identification module, configured to use a scene recognition network obtained through deep learning to perform braking scene recognition based on the driving data and the environmental data, and determine at least one braking scene identifier; the at least one braking scene identifier is used to characterize a target object state related to vehicle braking and / or a vehicle driving state in the driving environment;

[0034] A determination module is used to determine a braking instruction based on the at least one braking scenario identifier.

[0035] According to the above technical means, the scene recognition network obtained by deep learning is used to identify the target object state related to vehicle braking and / or the vehicle driving state in the current driving environment of the vehicle based on the driving data during the vehicle driving process and the environmental data in the driving environment, and output it in the form of at least one scene identification. Since deep learning can more accurately identify and track targets, the recognition accuracy and system response speed of emergency braking scenes can be improved. In addition, based on at least one braking scene identification, a braking instruction that matches the current braking scene can be adaptively generated, so that the system can adapt to accurate braking instructions in a timely manner when facing scenes of different complexities, improve the accuracy of vehicle control, and thus improve the braking effect and driving safety.

[0036] Furthermore, the vehicle control device also includes a training module, which is used to use the scene recognition network obtained by deep learning to perform braking scene recognition based on the driving data and the environmental data. Before determining at least one braking scene identification, the original sample set collected under various driving environments is standardized to determine the training sample set; the original samples in the original sample set include sample driving data, sample environmental data and braking scene identification true value; the sample driving data includes at least one of vehicle position, vehicle motion state, vehicle control data, braking distance, and collision situation; the sample environmental data includes at least one of target object attribute data, target object position, target object distance, target object motion state, and target object occlusion situation; the braking scene identification true value is used to characterize the state of at least one target object related to vehicle braking and at least one vehicle driving state in the original sample; using the training sample set, the initial scene recognition network is deep learned and trained to obtain the scene recognition network.

[0037] According to the above technical means, a training sample set is obtained through standardized processing, and the training sample set is used to perform deep learning training on the initial scene recognition network. This allows the initial scene recognition network to self-learn and extract features from a large amount of data to handle tasks in high-dimensional and complex environments in actual driving environments, thereby more accurately identifying and tracking targets, greatly improving the classification accuracy and system response speed of emergency braking scenarios.

[0038] Furthermore, after determining the training sample set, the training module is also used to perform data enhancement on the training data set, so as to perform deep learning training on the initial scene recognition network using the data enhanced training sample set to obtain the scene recognition network.

[0039] According to the above-mentioned technical means, through data enhancement, the network can fully explore the characteristics of various aspects of the training samples and improve generalization, thereby improving the recognition accuracy of the scene recognition network, and then improving the accuracy of at least one braking scene identification and the braking effect of the braking instruction generated based on at least one braking scene identification.

[0040] Furthermore, the training module is also used to perform network inference based on the training samples in the training sample set through the initial scene recognition network to obtain at least one braking scene identifier corresponding to the training samples; determine the learning rate corresponding to each round of training; reduce the learning rate from round to round; adjust the network parameters of the initial scene recognition network based on the at least one braking scene identifier corresponding to the training samples and the learning rate corresponding to each round of training, and continue the next round of training until the preset training conditions are met to obtain the scene recognition network.

[0041] According to the above technical means, reducing the learning rate round by round during the network training process can reduce oscillation and training error, balance the training convergence speed and training error, and improve the training effect.

[0042] Furthermore, the training module is also used to perform batch normalization processing on the input features of each network layer in the initial scene recognition network, and perform network inference based on the input features after batch normalization processing through each network layer to obtain at least one braking scene identification corresponding to the training sample.

[0043] According to the above technical means, batch regularization processing can effectively alleviate the phenomenon of gradient explosion and vanishing in deep neural networks, accelerate the speed of training convergence, and make network training more stable.

[0044] Furthermore, the scene recognition network includes: at least one of a residual network, a gated recurrent unit network, and a network model combining a residual network and a gated recurrent unit network.

[0045] According to the above technical means, by utilizing the structure of the residual network and / or the gated recurrent unit network, the gradient vanishing phenomenon in network training can be reduced, and the recognition effect of the trained scene recognition network can be improved.

[0046] Furthermore, the determination module is used to determine a braking instruction based on at least one braking scenario identifier and at least one preset braking strategy; the at least one braking strategy includes a braking strategy for one braking scenario identifier and / or a braking strategy for a combination of multiple braking scenario identifiers.

[0047] According to the above technical means, braking instructions are generated through targeted braking strategies for different emergency braking scenarios, which further improves the braking effect and driving safety.

[0048] Beneficial effects of the present invention:

[0049] The scene recognition network obtained by deep learning is used to identify the target object state related to vehicle braking and / or the vehicle driving state in the current driving environment of the vehicle based on the driving data during the vehicle driving process and the environmental data in the driving environment, and output it in the form of at least one scene identification. Since deep learning can more accurately identify and track targets, the recognition accuracy and system response speed of emergency braking scenes can be improved. In addition, based on at least one braking scene identification, a braking instruction that matches the current braking scene can be adaptively generated, so that the system can adapt to accurate braking instructions in a timely manner when facing scenes of different complexities, improve the accuracy of vehicle control, and thus improve the braking effect and driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of an optional flow chart of the vehicle control method provided by the present invention;

[0051] Figure 2 A schematic diagram of an optional flow chart of the vehicle control method provided by the present invention;

[0052] Figure 3 A schematic diagram of the change of training error during the network training process provided by the present invention;

[0053] Figure 4 A schematic diagram of the change of learning rate during the network training process provided by the present invention;

[0054] Figure 5 A schematic diagram of the change in accuracy during the network training process provided by the present invention;

[0055] Figure 6 A schematic diagram of an optional flow chart of applying the vehicle control method provided by the present invention to an actual scenario;

[0056] Figure 7 A schematic diagram of the structure of a vehicle control device provided by the present invention;

[0057] Figure 8 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0058] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0059] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0060] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0061] 1. Vulnerable Road Users (VRU): VRU in the automotive field refers to "vulnerable road users". These users include pedestrians, cyclists and motorcyclists, who are relatively vulnerable and easily injured in road traffic.

[0062] 2. Autonomous Emergency Braking (AEB) is an active safety technology for automobiles, mainly composed of three modules, including control module, distance measurement module, and braking module. The core of the distance measurement module includes microwave radar, face recognition technology and video system, which can provide safe, accurate and real-time images and road condition information of the road ahead.

[0063] 3. F1 Score is an indicator used in statistics to measure the accuracy of a binary classification model. It takes into account both the precision and recall of the classification model.

[0064] 4. Top-1 accuracy: The Top-1 accuracy of a neural network refers to the probability that the model predicts the correct category to be at the top of the output list in a test image. Specifically, if the model ranks the correct category first when predicting a certain image, the prediction is considered correct, otherwise it is wrong.

[0065] An embodiment of the present invention provides a vehicle control method, such as Figure 1 As shown, including S101-S103, as follows:

[0066] S101. Acquire driving data of a vehicle and environmental data of a driving environment.

[0067] In an embodiment of the present invention, during the driving process of the vehicle, the current form data of the vehicle and the environmental data of the current driving environment are obtained in real time. In some embodiments, the environmental data includes image data obtained by an image sensor capturing images of the driving environment and / or point cloud data obtained by a laser radar scanning the driving environment; the driving data includes the driving status of the vehicle collected by the vehicle's sensors and the control signal data of the vehicle. Exemplarily, the driving data may include the vehicle's speed, current location, the vehicle's steering system, the signal flag of the braking system, etc., and the specific selection is made according to the actual situation, and the embodiment of the present invention is not limited.

[0068] S102. Using a scene recognition network obtained through deep learning, perform braking scene recognition based on driving data and environmental data to determine at least one braking scene identifier.

[0069] In the embodiment of the present invention, the scene recognition network is obtained by pre-training the initial scene recognition network in a deep learning manner. Compared with traditional machine learning, deep learning can self-learn and extract features from a large amount of data, and is suitable for processing tasks in high-dimensional and complex environments. Therefore, the scene recognition network obtained by deep learning can more accurately identify and track targets, greatly improving the classification accuracy and system response speed of emergency braking scenes.

[0070] In an embodiment of the present invention, at least one braking scene identifier is used to characterize the state of a target object related to vehicle braking in a driving environment and / or the driving state of the vehicle. In some embodiments, the state of the target object includes: the motion state (such as motion, stillness, motion speed, etc.) of at least one target object in the driving environment, attribute data (such as the type, size, depth information, quantity distribution, etc. of the target object), the position of the target object, the distance between the target object and the vehicle, and the occlusion of the target object (such as occlusion, no occlusion, degree of occlusion), etc. In some embodiments, the driving state of the vehicle includes: the vehicle position (the current positioning of the vehicle), the vehicle motion state (motion direction, such as straight, reverse, turn, vehicle speed, vehicle driving trajectory, etc.), and vehicle control data (brake system control data, such as AEB flag, steering system control data, such as steering wheel angle range, whether the steering wheel is in the center, etc.). Other or more target object states or vehicle driving state data can also be defined according to actual conditions, which is not limited in the embodiment of the present invention. It can be understood that it is necessary to fully consider the state of the target object in the current driving environment and make braking decisions for automatic driving in combination with the driving state of the vehicle.

[0071] Exemplarily, the scene recognition network performs scene recognition based on the environmental data and the driving data, and at least one braking scene identification output in the form of a label may be as follows:

[0072] The labels representing the state of the target object include: bit0: unknown motion state, bit1: motion, bit2: stationary, bit3: motion to stationary, bit4: slow movement, bit5: ghost head, bit6: crossing to straight, bit7: straight to crossing, bit8: dense target, bit9: high-speed occlusion crossing, bit10: VRU is blocked by other obstacles, bit11: pedestrian flag on tricycle, etc. The labels representing the driving state of the vehicle include: bit12: AEB output flag, bit13: self-vehicle reversing, bit14: self-vehicle high-speed forward, etc. It can be seen that through the above labels, the motion state (speed, direction), number (dense or not), type (pedestrian or non-motor vehicle), occlusion, etc. of the target objects related to vehicle braking in the driving environment can be represented, as well as the driving state of the self-vehicle related to vehicle braking (direction, speed, control signal flag), so as to provide accurate environment-related parameters for the selection of braking strategy.

[0073] It should be noted that the scene recognition network outputs at least one braking scene identifier, that is, one or more braking scene identifiers can be output. When determining the braking strategy, it is necessary to comprehensively consider the adaptive braking strategy based on one or more braking scene identifiers.

[0074] S103: Determine a braking instruction based on at least one braking scenario identifier.

[0075] In an embodiment of the present invention, a braking strategy is adapted and decided based on the at least one braking scenario identifier, a braking instruction is generated according to the determined braking strategy, and the braking instruction is used to control a vehicle module that performs a braking action to brake the vehicle.

[0076] In some embodiments, a braking strategy for a braking scenario represented by one braking scenario identifier and a braking strategy for a braking scenario represented by a combination of multiple braking scenario identifiers can be formulated in advance based on the types of braking scenario identifiers that can be output by the scene recognition network. In this way, during the driving process of the vehicle, at least one braking scenario identifier output in real time by the scene recognition network can be matched in at least one preset braking strategy, and a corresponding braking instruction will be generated according to the braking strategy with the highest degree of adaptability.

[0077] The braking strategy may include: the judgment conditions for triggering braking, the degree of braking (such as deceleration acceleration, etc.). In some embodiments, the corresponding braking strategy can be fully matched according to at least one braking scene identifier. When the braking strategy cannot be fully matched, the braking strategy can also be matched according to the braking scene identifier with a high weight in at least one braking scene identifier. The weight of at least one braking scene identifier can be pre-set according to actual needs, and the embodiment of the present invention does not limit it.

[0078] In some embodiments, based on at least one braking scene identifier obtained in real time, the braking strategy can be adjusted in real time according to different scenarios. For example, on an open road, the system can identify and predict the movement trajectory of pedestrians crossing in the distance in advance, and give braking scene identifiers such as "open scene", "pedestrian crossing", and "occluded". The braking system will adjust the braking strategy according to the specific scene, reduce the judgment conditions for triggering braking, and brake in time to avoid collision. For example, in view of the small length and width of the pedestrian target detection frame, the target is regarded as a point when calculating the coverage; for the occluded crossing scene, the number of multi-frame confirmation frames of the trigger threshold is reduced; because the target appears very suddenly due to occlusion, the duration threshold of the pedestrian target detection frame in the target selection is reduced at this time to ensure that the pedestrian can be selected in time, avoiding the accident caused by not braking in time when confirming whether the pedestrian target detection frame reaches the duration threshold. In this way, the system can improve the braking effect of the scene without affecting the false triggering of other scenes.

[0079] It is understandable that the scene recognition network obtained by deep learning can identify the target object state related to vehicle braking and / or the vehicle driving state in the current driving environment of the vehicle based on the driving data during the vehicle driving process and the environmental data in the driving environment, and output it in the form of at least one scene identification. Since deep learning can more accurately identify and track targets, it can improve the recognition accuracy and system response speed of emergency braking scenes. In addition, based on at least one braking scene identification, a braking instruction that matches the current braking scene can be adaptively generated, so that the system can adapt to accurate braking instructions in a timely manner when facing scenes of different complexities, improve the accuracy of vehicle control, and thus improve the braking effect and driving safety.

[0080] In some embodiments, the Figure 2 In the method shown, before S102, a scene recognition network is trained as follows:

[0081] S001. Standardize the original sample sets collected under various driving environments to determine the training sample sets.

[0082] In S001, sample collection can be performed in a variety of driving environments in a simulation environment or a real environment, and the collected sample data can be screened to extract environmental data and vehicle driving data with low correlation with the braking scene to form an original sample set. The original samples in the original sample set include sample driving data, sample environmental data and braking scene identification true value; the sample driving data includes at least one of vehicle position, vehicle motion state, vehicle control data, braking distance, and collision situation; the sample environmental data includes at least one of target object attribute data, target object position, target object distance, target object motion state, and target object occlusion situation; the braking scene identification true value is used to characterize the state of at least one target object related to vehicle braking and at least one vehicle driving state in the original sample. Exemplarily, the vehicle motion state may include vehicle speed, driving direction, steering data, driving trajectory data (such as straight road, curve, curve radius), etc., the vehicle control data may include whether the steering wheel is returning to the center, braking force, AEB status flag, etc., and the collision situation may include how long it will collide without braking, etc. The target object attribute data may include the type of target object (such as motor vehicle, non-motor vehicle, pedestrian, etc.), the size of the target object (such as the size of the detection box, etc.), the depth data of the target object, the quantity distribution of the target object, etc.; the specific selection shall be made according to the actual situation, and the embodiment of the present invention is not limited.

[0083] By sorting out the environmental data and driving data of each training sample in the original sample set, the characteristics and ranges of the environmental data and driving data under different braking scenarios can be sorted out. According to the characteristics and ranges of the environmental data and driving data under different braking scenarios, the training samples are classified and labeled with different scene names to obtain the scene identification truth value corresponding to each training sample.

[0084] In some embodiments, the original sample set is standardized into the data input format required by the initial scene recognition network, and sliding filtering and denoising are performed on it to obtain a training sample set, thereby completing sample preparation for network training.

[0085] S002. Use the training sample set to perform deep learning training on the initial scene recognition network to obtain a scene recognition network.

[0086] In some embodiments, the initial scene recognition network includes: at least one of a residual network, a gated recurrent unit network, and a network model combining a residual network and a gated recurrent unit network. That is, the scene recognition network obtained by training the initial scene recognition network includes: at least one of a residual network, a gated recurrent unit network, and a network model combining a residual network and a gated recurrent unit network.

[0087] The training process of the neural network includes the process of feature extraction and information collection, and it goes through several convolutional layers and pooling layers for feature extraction, and finally is summarized and integrated in the fully connected layer to output the result of each network training, that is, at least one braking scene identification corresponding to each training sample. For traditional network structures, as the number of neural network layers continues to increase, the objective function used for parameter adjustment will gradually deviate from the global optimal solution. Moreover, as the number of network layers increases, the gradient vanishing phenomenon will become more and more serious. The embodiment of the present invention utilizes the network structure of the residual network and the gated recurrent unit network to realize the initial scene recognition network, which can avoid the problem of gradient vanishing. The complex mapping that the original neural network needs to learn can be expressed as H(x,w h ), where x represents the data input into the neural network, and w h Represents network parameters, such as weights. Neural networks need to learn to map from x to H(x,w) through training. h ) capability. The residual network can directly pass the input x backward through a shortcut, so that the entire mapping becomes: y = H(x,w h )+x. In this way, the original expected output is the complex mapping H(x,w h ), and the residual network changes the learning goal and directly passes the input x backward through a shortcut, so that the optimal solution becomes the difference between H(x,w) and the input x, that is, the residual y = H(x,w h )+x. This structure solves the problem of information loss during transmission and ensures the completeness of information. In addition, the shortcut does not set weights, so that the network can remain stable and easy to learn as the number of layers increases, thereby improving the network training effect and the scene recognition effect of the trained scene recognition network.

[0088] In some embodiments, data enhancement may be performed on the training data set to perform deep learning training on the initial scene recognition network using the data enhanced training sample set to obtain a scene recognition network.

[0089] In an embodiment of the present invention, in order to improve the model training effect, the training data set can be enhanced: by processing limited samples through various means, various features in the samples are mined, and the generalization of the samples is improved. These methods include: random cropping, adding noise, zooming in and out, spatial transformation, rotation, flipping, etc., striving to obtain as much useful information as possible from limited samples. During the training process, the best recognition effect is obtained by continuously optimizing the model parameters to avoid the situation where the number of parameters exceeds the input samples during the training process. In this way, the generalization ability of the model can be greatly improved, that is, the model's ability to explain the output of unknown samples can be improved, thereby reducing the time consumption of network training and improving the network training effect.

[0090] In some embodiments, through an initial scene recognition network, network inference is performed based on training samples in a training sample set to obtain at least one braking scene identifier corresponding to the training sample; a learning rate corresponding to each round of training is determined; the learning rate is reduced round by round; based on at least one braking scene identifier corresponding to the training sample and the learning rate corresponding to each round of training, network parameters of the initial scene recognition network are adjusted, and the next round of training is continued until a preset training condition is reached to obtain a scene recognition network.

[0091] For the set network structure, the difficulty of deep learning lies mainly in parameter selection and adjustment. Since it is difficult to determine the corresponding parameters at the beginning, in order to achieve the best training effect, it is necessary to continuously adjust the parameter values ​​according to the feedback of the results through repeated network training. The neural network can be expressed by a simple linear function, and the expression can be: f(x) = w*x+b. Wherein f(x) is used to represent the neural network, x represents the data input to the neural network, w represents the weight, b represents the bias value, and the w and b parameters need to be continuously adjusted during network training. The adjustment of network parameters needs to be adjusted according to the learning rate, and the learning rate represents the amplitude of each adjustment of the parameters. Of course, the learning rate is not the larger the better. Although increasing the learning rate will increase the convergence speed and reduce the training time, when the learning rate reaches a certain level, it will produce oscillations in the training and increase the error. If the learning rate is set too small, it is not conducive to network learning and increases the convergence time. In an embodiment of the present invention, the best effect can be achieved by making the learning rate gradient decay or continuous decay during the network training process.

[0092] In some embodiments, batch normalization is performed on the input features of each network layer in the initial scene recognition network, and network inference is performed on each network layer based on the input features after batch normalization to obtain at least one braking scene identification corresponding to the training sample.

[0093] In a neural network, the connection between layers is very close, and any slight change in the previous layer will be continuously amplified as the number of network layers increases. At the same time, the continuous change of parameters will affect the input distribution of the current layer network, and the network of the previous layer needs to continuously adjust the output to match the changes in these distributions. These reasons increase the difficulty of training deep neural networks. The embodiment of the present invention performs batch regularization on the input features of each network layer in the initial scene recognition network, so that the input batch feature matrix of each layer can be adjusted through normalization and linear transformation so that the mean of its distribution is 1 and the variance is 0. This makes the data standards between network layers consistent, and the networks of the front and back layers do not need to be constantly adjusted to adapt to each other's changes. This method can stabilize the mean and variance of the output, making the training of the neural network more stable, and can effectively alleviate the phenomenon of gradient explosion and disappearance in deep neural networks, and accelerate the convergence speed.

[0094] According to the network training method described above in the embodiment of the present invention, the training error of the initial scene recognition network during the training process can be shown in Figure 4. It can be seen that as the network training progresses, the training error steadily decreases. The training error of the initial scene recognition network during the training process can be shown in Figure 4. Figure 3 shown. Figure 3 The horizontal axis represents the training time, and the vertical axis represents the training error. It can be seen that as the network training progresses, the training error steadily decreases. The learning rate change of the initial scene recognition network during training can be shown as follows: Figure 4 shown. Figure 4 The horizontal axis represents the training time, and the vertical axis represents the learning rate. It can be seen that a larger learning rate in the early stage can reduce the training time and increase the model convergence speed; a smaller learning rate in the later stage can reduce oscillation and training errors. The accuracy of the output results of the initial scene recognition network during training can be as follows Figure 5 As shown, Figure 5 The horizontal axis represents the training time, and the vertical axis represents the TOP-1 accuracy. It can be seen that as the network training progresses, the accuracy of the network output results gradually improves and tends to be stable. For example, the effect of the scene recognition network (such as ResNet101) obtained after training is shown in Table 1.

[0095] Table 1

[0096] Accuracy Recall F1 score 0.9583 0.9500 0.9487

[0097] Next, combine Figure 6 , introduces a vehicle control method applied in a practical scenario provided by an embodiment of the present invention.

[0098] like Figure 6 As shown in FIG. 1 , a historical data set is first obtained and data preprocessed, such as standardization and sliding filter denoising, to obtain a training set (equivalent to a training sample set). The initial scene recognition network of the ResNet101 structure is trained based on the training set to learn various important features of the driving scene in a deep learning manner, and the scene label classification is performed on each training sample in the training set, that is, the braking scene recognition is performed to obtain at least one braking scene identifier corresponding to each training sample, such as Figure 6 The Flag label 1, Flag label 2, Flag label N, etc. in the image. Different braking scene identifiers can represent different emergency braking scenes. Furthermore, using the different braking scene identifiers output by the trained scene recognition network, targeted braking strategies can be adopted, such as braking strategy 1, braking strategy 2, braking strategy N, etc. By executing the braking instructions corresponding to the braking strategy, the system can dynamically adjust the recognition and classification strategies according to the current scene in the actual driving environment, improve the system's action accuracy and response speed, and ensure driving safety.

[0099] An embodiment of the present invention provides a vehicle control device 1, such as Figure 7 As shown, including:

[0100] The acquisition module 11 is used to acquire the driving data of the vehicle and the environmental data of the driving environment;

[0101] The recognition module 12 is used to use the scene recognition network obtained by deep learning to perform braking scene recognition based on the driving data and the environmental data, and determine at least one braking scene identifier; the at least one braking scene identifier is used to characterize the target object state related to vehicle braking and / or the vehicle driving state in the driving environment;

[0102] The determination module 13 is configured to determine a braking instruction based on the at least one braking scenario identifier.

[0103] In some embodiments, the vehicle control device also includes a training module, which is used to use a scene recognition network to perform braking scene recognition based on the driving data and the environmental data. Before determining at least one braking scene element, the original sample set collected under multiple driving environments is standardized to determine a training sample set; the original samples in the original sample set include sample driving data, sample environmental data and a braking scene identification true value; the sample driving data includes at least one of vehicle position, vehicle motion state, vehicle control data, braking distance, and collision condition; the sample environmental data includes at least one of target object attribute data, target object position, target object distance, target object motion state, and target object occlusion condition; the braking scene identification true value is used to characterize the state of at least one target object related to vehicle braking and at least one vehicle driving state in the original sample; using the training sample set, the initial scene recognition network is deep learned and trained to obtain the scene recognition network.

[0104] In some embodiments, after determining the training sample set, the training module is further used to perform data enhancement on the training data set, so as to perform deep learning training on the initial scene recognition network using the data enhanced training sample set to obtain the scene recognition network.

[0105] In some embodiments, the training module is also used to perform network inference based on the training samples in the training sample set through the initial scene recognition network to obtain at least one braking scene identifier corresponding to the training samples; determine the learning rate corresponding to each round of training; reduce the learning rate from round to round; adjust the network parameters of the initial scene recognition network based on the at least one braking scene identifier corresponding to the training samples and the learning rate corresponding to each round of training, and continue the next round of training until the preset training conditions are met to obtain the scene recognition network.

[0106] In some embodiments, the training module is also used to perform batch normalization processing on the input features of each network layer in the initial scene recognition network, and perform network inference based on the input features after batch normalization processing through each network layer to obtain at least one braking scene identification corresponding to the training sample.

[0107] In some embodiments, the scene recognition network includes at least one of a residual network, a gated recurrent unit network, and a network model combining a residual network and a gated recurrent unit network.

[0108] In some embodiments, the determination module 13 is used to determine a braking instruction based on at least one braking scenario identifier and at least one preset braking strategy; the at least one braking strategy includes a braking strategy for one braking scenario identifier and / or a braking strategy for a combination of multiple braking scenario identifiers.

[0109] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0110] An embodiment of the present invention further provides an electronic device, Figure 8 This is an optional structural diagram of an electronic device 3 provided in an embodiment of the present invention. Figure 8 As shown, the electronic device 3 includes: a memory 32 and a processor 33. The memory 32 and the processor 33 are connected via a communication bus 34; the memory 32 is used to store executable instructions; the processor 33 is used to implement the vehicle control method provided by the embodiment of the present invention when executing the executable instructions stored in the memory 32.

[0111] An embodiment of the present invention provides a vehicle, the vehicle comprising: Figure 7 The vehicle control device shown, or, including Figure 8 Electronic equipment shown.

[0112] An embodiment of the present invention provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by the above-mentioned processor, the above-mentioned processor will be caused to execute the vehicle control method provided by the embodiment of the present invention.

[0113] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0114] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0115] As examples, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As examples, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0116] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0117] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A vehicle control method, characterized in that: include: Acquire the driving data of the vehicle and the environmental data of the driving environment; Using a scene recognition network obtained through deep learning, performing braking scene recognition based on the driving data and the environmental data, and determining at least one braking scene identifier; The at least one braking scenario identifier is used to characterize a target object state related to vehicle braking and / or a vehicle driving state in the driving environment; Based on the at least one braking scenario identification, a braking command is determined.

2. The method according to claim 1, characterized in that Before the scene recognition network obtained by deep learning performs braking scene recognition based on the driving data and the environmental data and determines at least one braking scene identifier, the method further includes: Standardization processing is performed on the original sample sets collected under various driving environments to determine the training sample set; the original samples in the original sample set include sample driving data, sample environmental data and braking scene identification true value; the sample driving data includes at least one of vehicle position, vehicle motion state, vehicle control data, braking distance and collision situation; the sample environmental data includes at least one of target object attribute data, target object position, target object distance, target object motion state and target object occlusion situation; the braking scene identification true value is used to characterize the state of at least one target object related to vehicle braking and at least one vehicle driving state in the original sample; The training sample set is used to perform deep learning training on the initial scene recognition network to obtain the scene recognition network.

3. The method according to claim 2, characterized in that After determining the training sample set, the method further includes: The training data set is data enhanced to perform deep learning training on the initial scene recognition network using the data enhanced training sample set to obtain the scene recognition network.

4. The method according to claim 2 or 3, characterized in that: The method of using the training sample set to perform deep learning training on the initial scene recognition network to obtain the scene recognition network includes: Performing network reasoning based on training samples in the training sample set through the initial scene recognition network to obtain at least one braking scene identifier corresponding to the training sample; Determine the learning rate corresponding to each round of training; the learning rate decreases round by round; Based on at least one braking scene identifier corresponding to the training sample and the learning rate corresponding to each round of training, the network parameters of the initial scene recognition network are adjusted, and the next round of training is continued until the preset training conditions are met to obtain the scene recognition network.

5. The method according to claim 4, characterized in that The performing network reasoning based on the training samples in the training sample set to obtain at least one braking scene identifier corresponding to the training samples includes: Batch normalization processing is performed on the input features of each network layer in the initial scene recognition network, and network inference is performed on each network layer based on the input features after the batch normalization processing to obtain at least one braking scene identification corresponding to the training sample.

6. The method according to any one of claims 1 to 3 or claim 5, characterized in that: The scene recognition network includes: at least one of a residual network, a gated recurrent unit network, and a network model combining a residual network and a gated recurrent unit network.

7. The method according to any one of claims 1 to 3 or claim 5, characterized in that: The determining of the braking instruction based on the at least one braking scenario identifier includes: A braking instruction is determined based on at least one braking scenario identifier and at least one preset braking strategy; the at least one braking strategy includes a braking strategy for one braking scenario identifier and / or a braking strategy for a combination of multiple braking scenario identifiers.

8. A vehicle control device, characterized in that: The device comprises: An acquisition module, used to acquire the driving data of the vehicle and the environmental data of the driving environment; an identification module, configured to use a scene recognition network obtained through deep learning to perform braking scene recognition based on the driving data and the environmental data, and determine at least one braking scene identifier; the at least one braking scene identifier is used to characterize a target object state related to vehicle braking and / or a vehicle driving state in the driving environment; A determination module is used to determine a braking instruction based on the at least one braking scenario identifier.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the computer executable instructions stored in the memory.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising computer executable instructions, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A vehicle, characterized in that: The vehicle comprises: the vehicle control device as claimed in claim 8, or the electronic device as claimed in claim 9.

Images