A hierarchical feedback self-learning method for vehicle artificial intelligence

Abstract

The application belongs to the technical field of artificial intelligence, and discloses a layered feedback self-learning method for vehicle artificial intelligence, which comprises the following steps: acquiring a video stream collected by a camera of a vehicle during driving of the vehicle; performing the following steps by using a vehicle artificial intelligence perception network model: extracting feature information of multiple dimensions of each video frame in the video stream; calculating first difference information between first feature information and second feature information; generating motion space information based on the first difference information and motion information of the vehicle, and predicting feature information of a next video frame based on the motion space information; determining a layered feedback online autonomous learning strategy based on second difference information between the predicted feature information and the extracted feature information; and performing layered feedback self-learning on the vehicle artificial intelligence perception network model based on the layered feedback online autonomous learning strategy. The application realizes layered feedback self-learning of the artificial intelligence perception network model, and improves the perception accuracy of the model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and particularly relates to a layered feedback self-learning method for vehicle artificial intelligence. BACKGROUND

[0002] In recent years, with the rapid development of artificial intelligence, artificial intelligence is gradually applied to vehicles. Specifically, vehicle driving data can be collected offline, and the collected driving data is used as training data for model training to obtain a pre-trained model. During vehicle driving, a prediction result is obtained through the pre-trained model, and the vehicle is controlled based on the prediction result.

[0003] As can be seen from the above description, in the related art, a large amount of vehicle driving data is required to train the pre-trained model offline, resulting in a high training cost of the pre-trained model. Moreover, during vehicle driving, if a driving scenario not involved in the pre-trained model is encountered, the pre-trained model cannot make accurate prediction, resulting in a low prediction accuracy of the pre-trained model, thereby causing a safety hazard to the vehicle. SUMMARY

[0004] The purpose of the present application is to realize layered feedback self-learning of vehicle artificial intelligence perception network model, so that the perception accuracy of the vehicle artificial intelligence perception network model after online training is higher, the safety of the vehicle is improved, and the training cost of the vehicle artificial intelligence perception network model is reduced without the need for special collection of training data. Online layered and collaborative learning realizes the self-adaptation and generalization ability of vehicle intelligent perception and intelligent decision-making. The layered feedback self-learning vehicle artificial intelligence technology is suitable for safe vehicle intelligent driving.

[0005] In a first aspect, an embodiment of the present application provides a layered feedback self-learning method for vehicle artificial intelligence, comprising:

[0006] During vehicle driving, a video stream collected by a camera of the vehicle is obtained;

[0007] The video stream is input into a vehicle artificial intelligence perception network model to be trained online, and the following steps are performed by the vehicle artificial intelligence perception network model:

[0008] Feature information of multiple dimensions of each video frame in the video stream is extracted;

[0009] For each dimension of feature information of a current video frame, first difference information between first feature information and second feature information is calculated; the first feature information is the dimension of feature information of the current video frame, and the second feature information is the dimension of feature information of a next video frame of the current video frame; the current video frame is any video frame of the video stream;

[0010] generate motion space information of the vehicle based on the first difference information and motion information of the vehicle, and predict feature information of the dimension of the next video frame based on the motion space information;

[0011] determine a hierarchical feedback online autonomous learning strategy based on second difference information between the predicted feature information of the dimension of the next video frame and the extracted feature information of the dimension of the next video frame;

[0012] based on the hierarchical feedback online autonomous learning strategy, perform hierarchical feedback self-learning on the vehicle artificial intelligence perception network model to obtain an online hierarchical feedback self-learned vehicle artificial intelligence perception network model.

[0013] Optionally, the vehicle artificial intelligence perception network model comprises a two-dimensional feature information extraction sub-model, a three-dimensional feature information extraction sub-model, and a four-dimensional feature information extraction sub-model.

[0014] The extracting the feature information of multiple dimensions of each video frame in the video stream comprises:

[0015] extracting two-dimensional feature information of each video frame in the video stream by a two-dimensional feature information extraction sub-model, wherein the two-dimensional feature information is used to represent plane feature information of the video frame.

[0016] extracting three-dimensional feature information of each video frame in the video stream by a three-dimensional feature information extraction sub-model, wherein the three-dimensional feature information is used to represent spatial feature information of the video frame.

[0017] extracting four-dimensional feature information of each video frame in the video stream by a four-dimensional feature information extraction sub-model, wherein the four-dimensional feature information is used to represent spatial feature information and time feature information of the video frame.

[0018] Optionally, the determining the hierarchical feedback online autonomous learning strategy based on the second difference information between the predicted feature information of the dimension of the next video frame and the extracted feature information of the dimension of the next video frame comprises:

[0019] determining first parameter adjustment gradient information of the two-dimensional feature information extraction sub-model based on second difference information between predicted two-dimensional feature information of the next video frame and extracted two-dimensional feature information of the next video frame, and constructing a two-dimensional feature feedback network based on the first parameter adjustment gradient information.

[0020] determine second parameter adjustment gradient information corresponding to the three-dimensional feature information extraction sub-model based on second difference information between the predicted three-dimensional feature information of the next video frame and the extracted three-dimensional feature information of the next video frame, and construct a three-dimensional feature feedback network based on the second parameter adjustment gradient information;

[0021] determine third parameter adjustment gradient information corresponding to the four-dimensional feature information extraction sub-model based on second difference information between the predicted four-dimensional feature information of the next video frame and the extracted four-dimensional feature information of the next video frame, and construct a four-dimensional feature feedback network based on the third parameter adjustment gradient information;

[0022] construct an inter-layer feedback network based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network;

[0023] determine a hierarchical feedback online autonomous learning strategy based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network.

[0024] Optionally, the determining of the hierarchical feedback online autonomous learning strategy based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network comprises:

[0025] determine first parameter adjustment information of the two-dimensional feature information extraction sub-model based on the two-dimensional feature feedback network and the inter-layer feedback network;

[0026] determine second parameter adjustment information of the three-dimensional feature information extraction sub-model based on the three-dimensional feature feedback network and the inter-layer feedback network;

[0027] determine third parameter adjustment information of the four-dimensional feature information extraction sub-model based on the four-dimensional feature feedback network.

[0028] Optionally, the hierarchical feedback online autonomous learning strategy is used to perform hierarchical feedback self-learning on the vehicle artificial intelligence perception network model to obtain an online hierarchical feedback self-learned vehicle artificial intelligence perception network model, which comprises:

[0029] perform hierarchical feedback self-learning on the two-dimensional feature information extraction sub-model through the first parameter adjustment information;

[0030] perform hierarchical feedback self-learning on the three-dimensional feature information extraction sub-model through the second parameter adjustment information;

[0031] perform hierarchical feedback self-learning on the four-dimensional feature information extraction sub-model through the third parameter adjustment information.

[0032] In a second aspect, an embodiment of the present application provides a vehicle control method based on vehicle artificial intelligence, the method comprising:

[0033] acquiring a video stream collected by a camera of the vehicle during driving of the vehicle;

[0034] inputting the video stream into a vehicle artificial intelligence perception network model after online hierarchical feedback self-learning, to output a perception result through the vehicle artificial intelligence perception network model after online hierarchical feedback self-learning;

[0035] controlling the vehicle based on the perception result.

[0036] In a third aspect, an embodiment of the present application provides a hierarchical feedback self-learning device of vehicle artificial intelligence, the device comprising:

[0037] a video stream acquisition module, configured to acquire a video stream collected by a camera of the vehicle during driving of the vehicle;

[0038] a video stream input module, configured to input the video stream into a vehicle artificial intelligence perception network model to be trained online, and execute the following steps through the vehicle artificial intelligence perception network model:

[0039] extracting feature information of multiple dimensions of each video frame in the video stream;

[0040] for feature information of each dimension of a current video frame, calculating first difference information between first feature information and second feature information; the first feature information is feature information of the dimension of the video frame, the second feature information is feature information of the dimension of a next video frame of the video frame; the current video frame is any video frame of the video stream;

[0041] generating motion space information of the vehicle based on the first difference information and motion information of the vehicle, and predicting the feature information of the dimension of the next video frame based on the motion space information;

[0042] determining a hierarchical feedback online autonomous learning strategy based on second difference information between the predicted feature information of the dimension of the next video frame and the extracted feature information of the dimension of the next video frame;

[0043] performing hierarchical feedback self-learning on the vehicle artificial intelligence perception network model based on the hierarchical feedback online autonomous learning strategy, to obtain a vehicle artificial intelligence perception network model after online hierarchical feedback self-learning.

[0044] In a fourth aspect, an embodiment of the present application provides a vehicle control device based on vehicle artificial intelligence, the device comprising:

[0045] a video stream acquisition module, configured to acquire a video stream collected by a camera of the vehicle during driving of the vehicle;

[0046] a model online hierarchical feedback self-learning module, configured to input the video stream into a vehicle artificial intelligence perception network model after online hierarchical feedback self-learning, so as to output a perception result through the vehicle artificial intelligence perception network model after online hierarchical feedback self-learning;

[0047] a vehicle control module, configured to control the vehicle based on the perception result.

[0048] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising:

[0049] at least one processor;

[0050] a memory for storing instructions executable by the at least one processor;

[0051] The at least one processor is configured to execute the instructions to implement the method in the first aspect or the second aspect.

[0052] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the method in the first aspect or the second aspect.

[0053] In a seventh aspect, an embodiment of the present application provides a computer program product, comprising a computer program, when the computer program is executed by a processor, the method in any one of the first aspect or the second aspect is implemented.

[0054] The technical scheme provided by the embodiment of the present application acquires a video stream collected by a camera of a vehicle during driving of the vehicle; inputs the video stream into a vehicle artificial intelligence perception network model to be trained online, and executes the following steps through the vehicle artificial intelligence perception network model: extracts feature information of multiple dimensions of each video frame in the video stream; calculates first difference information between first feature information and second feature information for each dimension of feature information of a current video frame; generates motion space information of the vehicle based on the first difference information and motion information of the vehicle, and predicts dimension feature information of a next video frame based on the motion space information; determines a hierarchical feedback online self-learning strategy based on second difference information between the predicted dimension feature information of the next video frame and the extracted dimension feature information of the next video frame; and performs hierarchical feedback self-learning on the vehicle artificial intelligence perception network model based on the hierarchical feedback online self-learning strategy, to obtain a vehicle artificial intelligence perception network model after online hierarchical feedback self-learning.

[0055] It can be seen that in the embodiment of the present application, during the movement of the vehicle, the layered feedback online autonomous learning strategy can be determined in real time, the vehicle artificial intelligence perception network model is realized to perform layered feedback self-learning, the perception accuracy of the vehicle artificial intelligence perception network model after online training is higher, the safety of the vehicle is improved, and the online training does not need to specially collect training data, thereby reducing the training cost of the vehicle artificial intelligence perception network model. The online layered and collaborative learning realizes the self-adaptation and generalization ability of the vehicle intelligent perception and intelligent decision. The layered feedback self-learning vehicle artificial intelligence technology is suitable for safe vehicle intelligent driving. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flowchart of a layered feedback self-learning method of vehicle artificial intelligence provided by the embodiment of the present application is shown in FIG. 1.

[0057] Figure 2 A flowchart of a specific implementation of S160 in the embodiment of the present application is shown in FIG. 2. Figure 1

[0058] A process schematic diagram of layered feedback self-learning of vehicle artificial intelligence provided by the embodiment of the present application is shown in FIG. 3. Figure 3

[0059] Another process schematic diagram of layered feedback self-learning of vehicle artificial intelligence provided by the embodiment of the present application is shown in FIG. 4. Figure 4

[0060] A flowchart of a vehicle control method based on vehicle artificial intelligence provided by the embodiment of the present application is shown in FIG. 5. Figure 5

[0061] A structural schematic diagram of a layered feedback self-learning device of vehicle artificial intelligence provided by the embodiment of the present application is shown in FIG. 6. Figure 6

[0062] A vehicle control device based on vehicle artificial intelligence provided by the embodiment of the present application is shown in FIG. 7. Figure 7

[0063] A structural schematic diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 8. Figure 8 DETAILED DESCRIPTION

[0064] The present application will be described in detail in the following embodiments.

[0065] ​In recent years, with the rapid development of artificial intelligence, artificial intelligence is gradually applied to vehicles. However, the artificial intelligence in the prior art does not have the ability of online learning or real-time updating, and as a pre-trained model, it can only continuously iterate based on pre-training, which means that its knowledge and ability are trained based on data before a specific time point. And in the process of use, generative artificial intelligence cannot actively learn or remember new information, and cannot update its knowledge base in real time according to user interaction. This makes the safety of the vehicle face great challenges when encountering a scenario not involved in pre-training during driving.

[0066] As can be known from the above description, the training cost of the pre-trained model in the prior art is huge, a large amount of offline data needs to be prepared for pre-training, and the trained model is then put into application. Moreover, due to the extremely complex internal structure of the model, the training process cannot be analyzed, and the reliability of the result still needs to be considered. Moreover, if a scenario not involved in pre-training is encountered during driving, the prediction accuracy of the pre-trained model is low, which will cause incorrect control of the vehicle and will bring serious consequences. Therefore, the reliability and safety of the application of artificial intelligence to vehicles is a great challenge.

[0067] In order to solve the above technical problems, the embodiment of the present application provides a vehicle artificial intelligence capable of online learning. Compared with the existing artificial intelligence technology, the present application reduces the complexity of the model through online feedback learning technology, can analyze the internal model to a certain extent, and makes its safety and reliability meet the driving requirements. This not only promotes the development of artificial intelligence theory, but also promotes the industrial application of intelligent technology in the field of intelligent driving.

[0068] Specifically, the embodiment of the present application introduces a multi-level dynamic feedback optimization network to improve the adaptability and reliability of the vehicle artificial intelligence perception network model to unknown environments. The present application realizes real-time optimization through local-level dynamic feedback, and realizes global-level feedback through the synergy between local optimization and local functions. The real-time feedback learning mechanism is the core to ensure the stability and robustness of the model, and the dynamic feedback and adjustment mechanism is the most effective measure for the model to shield noise and adapt to the environment, and is applied to the verification of vehicle active safety control. Moreover, the online learning technology of the present application can significantly reduce the amount of data required for model training, thereby reducing the training cost of the model. The online hierarchical and collaborative learning realizes the self-adaptation and generalization ability of vehicle intelligent perception and intelligent decision-making. The hierarchical feedback self-learning vehicle artificial intelligence technology is suitable for safe vehicle intelligent driving.

[0069] The specific implementation mode of the hierarchical feedback self-learning method of the vehicle artificial intelligence provided by the embodiment of the present application will be described in detail below.

[0070] As Figure 1As shown, a flowchart of a layered feedback self-learning method of vehicle artificial intelligence provided by an embodiment of the present application, which can include the following steps:

[0071] S110, during vehicle driving, acquiring a video stream collected by a camera of the vehicle.

[0072] Specifically, the vehicle artificial intelligence perception network model is trained by the online learning technology, therefore, the continuous video stream is the basis for learning and perception of the vehicle artificial intelligence perception network model. During vehicle driving, the camera of the vehicle can collect a video stream in real time, therefore, the video stream collected by the camera of the vehicle can be acquired.

[0073] S120, inputting the video stream into the vehicle artificial intelligence perception network model to be trained online, and executing the following steps S130 to S170 by the vehicle artificial intelligence perception network model.

[0074] Before training the vehicle artificial intelligence perception network model online, the vehicle artificial intelligence perception network model can be pre-trained, and the pre-training process is an offline training process. In order to improve the perception accuracy of the vehicle artificial intelligence perception network model obtained by pre-training, in the embodiment of the present application, the video stream collected by the camera of the vehicle is input into the vehicle artificial intelligence perception network model obtained by pre-training during vehicle driving, so as to train the vehicle artificial intelligence perception network model obtained by pre-training online. Because various new scenes will be encountered during vehicle driving, by training the vehicle artificial intelligence perception network model online, the vehicle artificial intelligence perception network model can adapt to various new scenes, and thus the perception accuracy of the vehicle artificial intelligence perception network model can be improved. Moreover, online training does not need to collect a large amount of training data specially, and can reduce the training cost of the vehicle artificial intelligence perception network model.

[0075] After the vehicle artificial intelligence perception network model to be trained online receives the video stream, the layered feedback self-learning will be performed by steps S130 to S160.

[0076] S130, extracting feature information of multiple dimensions of each video frame in the video stream.

[0077] Specifically, after the vehicle artificial intelligence perception network model receives the video stream, the two-dimensional feature information, three-dimensional feature information and four-dimensional feature information of each video frame in the video stream can be extracted. The two-dimensional feature information can be the plane feature information of the video frame, for example, the feature information in the X-axis and Y-axis directions. The three-dimensional feature information can be the space feature information of the video frame, for example, the feature information in the X-axis, Y-axis and Z-axis directions. The four-dimensional feature information can be the space feature information and time feature information of the video frame.

[0078] In an embodiment, the vehicle artificial intelligence perception network model can include a two-dimensional feature information extraction sub-model, a three-dimensional feature information extraction sub-model, and a four-dimensional feature information extraction sub-model.

[0079] At this time, S130, the feature information of each video frame in the video stream is extracted, which can include the following three steps, steps 1 to 3:

[0080] Step 1, extracting two-dimensional feature information of each video frame in the video stream by a two-dimensional feature information extraction sub-model, wherein the two-dimensional feature information is used to represent the planar feature information of the video frame.

[0081] Step 2, extracting three-dimensional feature information of each video frame in the video stream by a three-dimensional feature information extraction sub-model, wherein the three-dimensional feature information is used to represent the spatial feature information of the video frame.

[0082] Step 3, extracting four-dimensional feature information of each video frame in the video stream by a four-dimensional feature information extraction sub-model, wherein the four-dimensional feature information is used to represent the spatial feature information and the time feature information of the video frame.

[0083] Wherein, the two-dimensional feature information extraction sub-model, the three-dimensional feature information extraction sub-model and the four-dimensional feature information extraction sub-model are respectively pre-trained by a traditional back propagation training method.

[0084] S140, for each dimension of the feature information of the current video frame, calculating the first difference information between the first feature information and the second feature information.

[0085] Wherein, the first feature information is the dimension of the feature information of the current video frame, and the second feature information is the dimension of the feature information of the next video frame of the current video frame; the current video frame is any video frame of the video stream.

[0086] Specifically, since the video stream is a continuous plurality of video frames, for any video frame in the plurality of video frames, the feature information of the plurality of dimensions of the video can be extracted by step S130, for example, the two-dimensional feature information, the three-dimensional feature information, and the four-dimensional feature information of the video frame can be extracted.

[0087] For any video frame in the video stream, the video frame can be taken as a current video frame, and difference information of two-dimensional feature information of the video frame and two-dimensional feature information of a next video frame of the video frame can be calculated, denoted as first difference information. Similarly, for three-dimensional feature information of any video frame in the video stream, difference information of three-dimensional feature information of the video frame and three-dimensional feature information of a next video frame of the video frame can be calculated, also denoted as first difference information. For four-dimensional feature information of any video frame in the video stream, difference information of four-dimensional feature information of the video frame and four-dimensional feature information of a next video frame of the video frame can be calculated, denoted as first difference information.

[0088] For example, assuming that the feature information is a position of a vehicle in a video frame, after obtaining the position of the vehicle in any video frame, difference information of the position of the vehicle in the video frame and the position of the vehicle in a next video frame can be calculated to obtain a position change of the vehicle in different video frames.

[0089] In S150, motion space information of the vehicle is generated based on the first difference information and motion information of the vehicle, and the feature information of the dimension of the next video frame is predicted based on the motion space information.

[0090] Since the first difference information is a scalar, in order to more accurately predict the feature information of the next video frame, motion space information of the vehicle can be generated based on the first difference information and motion information of the vehicle, the motion space information being a vector, and the feature information of the next video frame can be accurately predicted through the motion space information. For example, the first difference information is a difference value between the position of the vehicle in any video frame and the position of the vehicle in a next video frame, and the difference value is a scalar; the motion information of the vehicle can be a motion speed of the vehicle, and the motion space information of the vehicle generated based on the first difference information and the motion information of the vehicle is a vector. The feature information of the next video frame can be predicted through the motion space information.

[0091] Specifically, if the first difference information is difference information of two-dimensional feature information of a video frame and two-dimensional feature information of a next video frame of the video frame, then the two-dimensional feature information of the next video frame can be predicted based on the motion space information generated based on the first difference information and the motion information of the vehicle. If the first difference information is difference information of three-dimensional feature information of a video frame and three-dimensional feature information of a next video frame of the video frame, then the three-dimensional feature information of the next video frame can be predicted based on the motion space information generated based on the first difference information and the motion information of the vehicle. If the first difference information is difference information of four-dimensional feature information of a video frame and four-dimensional feature information of a next video frame of the video frame, then the four-dimensional feature information of the next video frame can be predicted based on the motion space information generated based on the first difference information and the motion information of the vehicle.

[0092] S160, determining the hierarchical feedback online self-learning strategy based on the second difference information between the feature information of the predicted dimension of the next video frame and the feature information of the dimension of the extracted next video frame.

[0093] For the sake of clear description of the scheme, the next video frame can be referred to as the video frame corresponding to time t, which can be any time in the continuous video stream. As can be known from the above description, the two-dimensional feature information, the three-dimensional feature information and the four-dimensional feature information of the video frame at time t can be extracted through step S130, and the two-dimensional feature information, the three-dimensional feature information and the four-dimensional feature information of the video frame at time t can be predicted through step S140.

[0094] For the video frame at time t, the difference information between the extracted two-dimensional feature information and the predicted two-dimensional feature information can be calculated to obtain the prediction error of the two-dimensional feature extraction sub-model, and the two-dimensional feature feedback network can be constructed based on the prediction error. The difference information between the extracted three-dimensional feature information and the predicted three-dimensional feature information can be calculated to obtain the prediction error of the three-dimensional feature extraction sub-model, and the three-dimensional feature feedback network can be constructed based on the prediction error. The difference information between the extracted four-dimensional feature information and the predicted four-dimensional feature information can be calculated to obtain the prediction error of the four-dimensional feature extraction sub-model, and the four-dimensional feedback network can be constructed based on the prediction error. The inter-layer feedback network can be constructed through the three-dimensional feedback network and the four-dimensional feedback network.

[0095] Then, the parameter adjustment information of the two-dimensional feature sub-model is determined according to the two-dimensional feature feedback network and the inter-layer feedback network; the parameter adjustment information of the three-dimensional feature sub-model is determined according to the three-dimensional feature feedback network and the inter-layer feedback network; and the parameter adjustment information of the four-dimensional feature sub-model can also be determined according to the four-dimensional feature feedback network.

[0096] For the sake of clear and complete description of the scheme, the specific embodiments of S160 will be described in detail in the following examples.

[0097] S170, performing hierarchical feedback self-learning on the vehicle artificial intelligence perception network model based on the hierarchical feedback online self-learning strategy to obtain the vehicle artificial intelligence perception network model after online hierarchical feedback self-learning.

[0098] Specifically, after obtaining the hierarchical feedback online autonomous learning strategy of the vehicle artificial intelligence perception network model, the hierarchical feedback learning can be performed on the vehicle artificial intelligence perception network model according to the hierarchical feedback online autonomous learning strategy. This process is a model parameter adjustment process of the vehicle artificial intelligence perception network model, and an online hierarchical feedback self-learning vehicle artificial intelligence perception network model is obtained. Assuming that the two-dimensional feature extraction sub-model in the vehicle artificial intelligence perception network model needs to adjust three parameters a, b and c, and the hierarchical feedback online autonomous learning strategy is that a is unchanged, b is increased, and c is decreased. Then, based on the hierarchical feedback online autonomous learning strategy, a is kept unchanged, b is increased, and c is decreased. For the three-dimensional feature extraction sub-model and the four-dimensional feature extraction sub-model in the vehicle artificial intelligence perception network model, the model parameters can be adjusted accordingly according to the corresponding hierarchical feedback online autonomous learning strategy. The perception accuracy of the online trained vehicle artificial intelligence perception network model is higher, and the online training does not need to collect training data specially, thereby reducing the training cost of the vehicle artificial intelligence perception network model.

[0099] The technical scheme provided by the embodiment of the application comprises the following steps: in the process of vehicle driving, video stream collected by a camera of the vehicle is obtained; the video stream is input into a vehicle artificial intelligence perception network model to be trained online, and the following steps are executed by the vehicle artificial intelligence perception network model: multi-dimensional feature information of each video frame in the video stream is extracted; first difference information between first feature information and second feature information of each dimension of the current video frame is calculated; motion space information of the vehicle is generated based on the first difference information and motion information of the vehicle, and the dimension of feature information of the next video frame is predicted based on the motion space information; hierarchical feedback online autonomous learning strategy is determined based on second difference information between the predicted dimension of feature information of the next video frame and the extracted dimension of feature information of the next video frame; and hierarchical feedback self-learning is performed on the vehicle artificial intelligence perception network model based on the hierarchical feedback online autonomous learning strategy, so as to obtain an online hierarchical feedback self-learning vehicle artificial intelligence perception network model.

[0100] It can be seen that in the embodiment of the application, the hierarchical feedback online autonomous learning strategy can be determined in real time in the process of vehicle motion, the hierarchical feedback self-learning of the vehicle artificial intelligence perception network model is realized, the perception accuracy of the online trained vehicle artificial intelligence perception network model is higher, the safety of the vehicle is improved, the online training does not need to collect training data specially, and the training cost of the vehicle artificial intelligence perception network model is reduced. The online hierarchical and collaborative learning realizes the self-adaptation and generalization ability of vehicle intelligent perception and intelligent decision. The hierarchical feedback self-learning vehicle artificial intelligence technology is suitable for safe vehicle intelligent driving.

[0101] For the sake of clear and complete description of the scheme, the specific implementation of S160 will be described in detail in the following embodiments.

[0102] S160, based on the second difference information between the predicted feature information of the dimension of the next video frame and the extracted feature information of the dimension of the next video frame, determines a hierarchical feedback online self-learning strategy, such as Figure 2 As shown, it can include the following steps:

[0103] S161, based on the second difference information between the predicted two-dimensional feature information of the next video frame and the extracted two-dimensional feature information of the next video frame, determines the first parameter adjustment gradient information corresponding to the two-dimensional feature information extraction sub-model, and constructs a two-dimensional feature feedback network based on the first parameter adjustment gradient information.

[0104] Specifically, after predicting the two-dimensional feature information of the next video frame, the predicted two-dimensional feature information of the next video frame can be compared with the real two-dimensional feature information of the extracted next video frame to obtain the second difference information of the two, and based on the second difference information, the parameter adjustment gradient information corresponding to the two-dimensional feature information extraction sub-model is determined. In order to clearly describe the scheme, the parameter adjustment gradient information can be called the first parameter adjustment gradient information, and a two-dimensional feature feedback network is constructed based on the first parameter adjustment gradient information. The two-dimensional feature feedback network can feed back the first parameter adjustment gradient information, i.e. the parameter adjustment information, to the two-dimensional feature information extraction sub-model, so that the two-dimensional feature information extraction sub-model can adjust the model parameters based on the parameter adjustment information.

[0105] S162, based on the second difference information between the predicted three-dimensional feature information of the next video frame and the extracted three-dimensional feature information of the next video frame, determines the second parameter adjustment gradient information corresponding to the three-dimensional feature information extraction sub-model, and constructs a three-dimensional feature feedback network based on the second parameter adjustment gradient information.

[0106] Specifically, after predicting the three-dimensional feature information of the next video frame, the predicted three-dimensional feature information of the next video frame can be compared with the real three-dimensional feature information of the extracted next video frame to obtain the second difference information of the two, and based on the second difference information, the parameter adjustment gradient information corresponding to the three-dimensional feature information extraction sub-model is determined. In order to clearly describe the scheme, the parameter adjustment gradient information can be called the second parameter adjustment gradient information, and a three-dimensional feature feedback network is constructed based on the second parameter adjustment gradient information. The three-dimensional feature feedback network can feed back the second parameter adjustment gradient information, i.e. the parameter adjustment information, to the three-dimensional feature information extraction sub-model, so that the three-dimensional feature information extraction sub-model can adjust the model parameters based on the parameter adjustment information.

[0107] S163, determining third parameter adjustment gradient information corresponding to the four-dimensional feature information extraction sub-model based on the second difference information between the predicted four-dimensional feature information of the next video frame and the extracted four-dimensional feature information of the next video frame, and constructing a four-dimensional feature feedback network based on the third parameter adjustment gradient information.

[0108] Specifically, after predicting the four-dimensional feature information of the next video frame, the predicted four-dimensional feature information of the next video frame can be compared with the real four-dimensional feature information of the extracted next video frame to obtain the second difference information of the two, and the parameter adjustment gradient information corresponding to the four-dimensional feature information extraction sub-model is determined based on the second difference information. In order to clearly describe the scheme, the parameter adjustment gradient information can be referred to as third parameter adjustment gradient information, and a four-dimensional feature feedback network is constructed based on the third parameter adjustment gradient information. The four-dimensional feature feedback network can feed back the third parameter adjustment gradient information, i.e. the parameter adjustment information, to the four-dimensional feature information extraction sub-model, so that the four-dimensional feature information extraction sub-model can adjust the model parameters based on the parameter adjustment information.

[0109] The embodiments of the present application will be described below with Figure 3 as an example. Figure 3 The image space in the image space is the image contained in each video frame in the video stream collected by the camera of the vehicle, and the video input is the input of the video stream into the vehicle artificial intelligence perception network model, Figure 3 The feature extraction in the feature extraction can be two-dimensional feature information extracted by a two-dimensional feature information extraction sub-model, three-dimensional feature information extracted by a three-dimensional feature information extraction sub-model, or four-dimensional feature information extracted by a four-dimensional feature information extraction sub-model. The feature space corresponding to time t-1 can be two-dimensional feature information, three-dimensional feature information or four-dimensional feature information, and the motion information is the vehicle motion information.

[0110] The difference space can be the first difference information described above, which can be the difference information between the two-dimensional feature information of the video frame at time t-1 and the two-dimensional feature information of the video frame at time t, or the difference information between the three-dimensional feature information of the video frame at time t-1 and the three-dimensional feature information of the video frame at time t, or the difference information between the four-dimensional feature information of the video frame at time t-1 and the four-dimensional feature information of the video frame at time t. The motion space can be the vehicle motion space information at time t-1 generated based on the first difference information and the vehicle motion information.

[0111] The feature space corresponding to the prediction time t can be two-dimensional feature information, three-dimensional feature information or four-dimensional feature information obtained by prediction. For example, if the first difference information is the difference information between the two-dimensional feature information of the video frame at time t-1 and the two-dimensional feature information of the video frame at time t, the feature space corresponding to the prediction time t can be the two-dimensional feature information obtained by prediction. The feature space corresponding to the time t can be the real two-dimensional feature information, real three-dimensional feature information or real four-dimensional feature information extracted.

[0112] By calculating the difference information (the second difference information) between the two-dimensional feature information obtained by prediction and the real two-dimensional feature information, the first parameter adjustment gradient information corresponding to the two-dimensional feature information extraction sub-model can be determined, that is, Figure 3 The gradient space is shown in the figure. Assuming that the model parameters to be adjusted are three, a, b and c, the gradient space can be a constant, b increasing and c decreasing. Of course, this is only an example. How to adjust needs to be determined according to the actual situation. After obtaining the first parameter adjustment gradient information, the two-dimensional feature feedback network can be constructed according to the first parameter adjustment gradient information. The two-dimensional feature feedback network can feed back the first parameter adjustment gradient information to the two-dimensional feature information extraction sub-model.

[0113] Similarly, by calculating the difference information (the second difference information) between the three-dimensional feature information obtained by prediction and the real three-dimensional feature information, the second parameter adjustment gradient information corresponding to the three-dimensional feature information extraction sub-model can be determined, that is, Figure 3 The gradient space is shown in the figure. After obtaining the second parameter adjustment gradient information, the three-dimensional feature feedback network can be constructed according to the second parameter adjustment gradient information. The three-dimensional feature feedback network can feed back the second parameter adjustment gradient information to the three-dimensional feature information extraction sub-model. By calculating the difference information (the second difference information) between the four-dimensional feature information obtained by prediction and the real four-dimensional feature information, the third parameter adjustment gradient information corresponding to the four-dimensional feature information extraction sub-model can be determined, that is, Figure 3 The gradient space is shown in the figure. After obtaining the third parameter adjustment gradient information, the four-dimensional feature feedback network can be constructed according to the third parameter adjustment gradient information. The four-dimensional feature feedback network can feed back the third parameter adjustment gradient information to the four-dimensional feature information extraction sub-model.

[0114] S164, constructing an inter-layer feedback network based on the three-dimensional feature feedback network and the four-dimensional feature feedback network.

[0115] Specifically, in the embodiment of the application, not only the two-dimensional feature feedback network, the three-dimensional feature feedback network and the four-dimensional feature feedback network can be constructed, but also the inter-layer feedback network can be constructed according to the three-dimensional feature feedback network and the four-dimensional feature feedback network.

[0116] S165, determine the hierarchical feedback online autonomous learning strategy based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network and the inter-layer feedback network.

[0117] As an implementation manner of the embodiment of the present application, S165, determining the hierarchical feedback online autonomous learning strategy based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network and the inter-layer feedback network, can include the following three steps, which are step 1 to step 3 respectively:

[0118] Step 1, determine the first parameter adjustment information of the two-dimensional feature information extraction sub-model based on the two-dimensional feature feedback network and the inter-layer feedback network.

[0119] Step 2, determine the second parameter adjustment information of the three-dimensional feature information extraction sub-model based on the three-dimensional feature feedback network and the inter-layer feedback network.

[0120] Step 3, determine the third parameter adjustment information of the four-dimensional feature information extraction sub-model based on the four-dimensional feature feedback network.

[0121] Specifically, the two-dimensional feature feedback network and the inter-layer feedback network can feedback parameter adjustment information to the two-dimensional feature information extraction sub-model cooperatively, and therefore, the first parameter adjustment information of the two-dimensional feature information extraction sub-model can be determined based on the two-dimensional feature feedback network and the inter-layer feedback network. That is, the two-dimensional feature feedback network and the inter-layer feedback network can determine the first parameter adjustment information of the two-dimensional feature information extraction sub-model through weighted summation. The weight of the two-dimensional feature feedback network and the inter-layer feedback network can be determined according to actual conditions.

[0122] Similarly, the three-dimensional feature feedback network and the inter-layer feedback network can feedback parameter adjustment information to the three-dimensional feature information extraction sub-model cooperatively, and therefore, the second parameter adjustment information of the three-dimensional feature information extraction sub-model can be determined based on the three-dimensional feature feedback network and the inter-layer feedback network. That is, the three-dimensional feature feedback network and the inter-layer feedback network can determine the second parameter adjustment information of the three-dimensional feature information extraction sub-model through weighted summation. The weight of the three-dimensional feature feedback network and the inter-layer feedback network can also be determined according to actual conditions.

[0123] The four-dimensional feature feedback network feedbacks parameter adjustment information to the four-dimensional feature information extraction sub-model, and therefore, the third parameter adjustment information of the four-dimensional feature information extraction sub-model can be determined based on the four-dimensional feature feedback network.

[0124] The following will be described taking Figure 4 as an example, Figure 4The image space in the image space is the image contained in each video frame in the video stream collected by the camera of the vehicle, and the video input is the input of the video stream into the vehicle artificial intelligence perception network model. The 2D network refers to a two-dimensional feature information extraction sub-model, the 2D feature space refers to two-dimensional feature information, the 2D-3D network refers to a three-dimensional feature information extraction sub-model, the 3D feature space refers to three-dimensional feature information, the 3D-4D network refers to a four-dimensional feature information extraction sub-model, and the 4D feature space refers to four-dimensional feature information. The 2D feedback network is a two-dimensional feature feedback network, the 3D feedback network is a three-dimensional feature feedback network, the 4D feedback network is a four-dimensional feature feedback network, and the inter-layer feedback network can be constructed based on the 3D feedback network and the 4D feedback network. From Figure 4 It can be seen that the 2D feedback network and the inter-layer feedback network cooperatively feed back the online autonomous learning strategy, i.e., the parameter adjustment information, to the 2D network. The 3D feedback network and the inter-layer feedback network cooperatively feed back the online autonomous learning strategy, i.e., the parameter adjustment information, to the 2D-3D network. The 4D feedback network feeds back the online autonomous learning strategy, i.e., the parameter adjustment information, to the 3D-4D network.

[0125] On the basis of the above-mentioned embodiments, in an implementation, S170, the hierarchical feedback online autonomous learning strategy is used to perform hierarchical feedback self-learning on the vehicle artificial intelligence perception network model, and an online hierarchical feedback self-learned vehicle artificial intelligence perception network model is obtained, which can include the following steps:

[0126] The first parameter adjustment information is used to perform hierarchical feedback self-learning on the model parameters of the two-dimensional feature information extraction sub-model.

[0127] The second parameter adjustment information is used to perform hierarchical feedback self-learning on the model parameters of the three-dimensional feature information extraction sub-model.

[0128] The third parameter adjustment information is used to perform hierarchical feedback self-learning on the model parameters of the four-dimensional feature information extraction sub-model.

[0129] As can be seen from the above description, the first parameter adjustment information of the two-dimensional feature information extraction sub-model can be obtained, and therefore, the first parameter adjustment information can be used to perform hierarchical feedback self-learning on the two-dimensional feature information extraction sub-model, i.e., to adjust the model parameters of the two-dimensional feature information extraction sub-model. Similarly, the second parameter adjustment information can be used to perform hierarchical feedback self-learning on the three-dimensional feature information extraction sub-model, i.e., to adjust the model parameters of the three-dimensional feature information extraction sub-model, and the third parameter adjustment information can be used to perform hierarchical feedback self-learning on the four-dimensional feature information extraction sub-model, i.e., to adjust the model parameters of the four-dimensional feature information extraction sub-model.

[0130] It can be seen that in the embodiment of the application, during the movement of the vehicle, the layered feedback online autonomous learning strategy can be determined in real time, the vehicle artificial intelligence perception network model is realized to perform layered feedback self-learning, the perception accuracy of the vehicle artificial intelligence perception network model after online training is higher, the safety of the vehicle is improved, and the online training does not need to specially collect training data, thereby reducing the training cost of the vehicle artificial intelligence perception network model. The online layered and collaborative learning realizes the self-adaptation and generalization ability of the intelligent perception and intelligent decision of the vehicle. The layered feedback self-learning vehicle artificial intelligence technology is suitable for safe vehicle intelligent driving.

[0131] The embodiment of the application also provides a vehicle control method based on vehicle artificial intelligence, as shown in the method comprises the following steps: Figure 5

[0132] S510, acquiring a video stream collected by a camera of the vehicle during the driving of the vehicle.

[0133] S520, inputting the video stream into the vehicle artificial intelligence perception network model after online layered feedback self-learning, so as to output a perception result through the vehicle artificial intelligence perception network model after online layered feedback self-learning.

[0134] S530, performing corresponding control on the vehicle based on the perception result.

[0135] Specifically, in actual application, during the driving of the vehicle, the video stream collected by the camera of the vehicle is input into the vehicle artificial intelligence perception network model after online training obtained through the above embodiment. After the vehicle artificial intelligence perception network model after online training receives the video stream, the two-dimensional feature information, the three-dimensional feature information and the four-dimensional feature information of each video frame in the video stream are extracted, the prediction structure is output based on the four-dimensional feature information, and then the vehicle is controlled according to the prediction result. For example, if the prediction result is that there is an obstacle in front of the vehicle, the vehicle can be controlled to avoid the obstacle. It can be understood that there are many prediction results, and there are many control modes for the vehicle, which are not listed here.

[0136] It can be seen that in the embodiment of the application, during the movement of the vehicle, the vehicle artificial intelligence perception network model can be trained online in real time, and the model parameters of the vehicle artificial intelligence perception network model are adjusted, thereby realizing online feedback and autonomous learning of the vehicle artificial intelligence perception network model. The prediction accuracy of the vehicle artificial intelligence perception network model after online training is higher, the safety of the vehicle is improved, and the online training does not need to specially collect training data, thereby reducing the training cost of the vehicle artificial intelligence perception network model. The online layered and collaborative learning realizes the self-adaptation and generalization ability of the intelligent perception and intelligent decision of the vehicle. The layered feedback self-learning vehicle artificial intelligence technology is suitable for safe vehicle intelligent driving.​

[0137] The embodiment of the present application also provides a layered feedback self-learning device 60 of vehicle artificial intelligence, as shown in the figure, the device comprises: Figure 6

[0138] A video stream acquisition module 610 is configured to acquire a video stream collected by a camera of a vehicle during driving of the vehicle;

[0139] A video stream input module 620 is configured to input the video stream into a vehicle artificial intelligence perception network model to be trained online, and perform the following steps by using the vehicle artificial intelligence perception network model:

[0140] Extracting feature information of multiple dimensions of each video frame in the video stream;

[0141] For the feature information of each dimension of a current video frame, calculating first difference information between first feature information and second feature information; the first feature information is the feature information of the dimension of the video frame, the second feature information is the feature information of the dimension of a next video frame of the video frame; the current video frame is any video frame in the video stream;

[0142] Generating motion space information of the vehicle based on the first difference information and motion information of the vehicle, and predicting the feature information of the dimension of the next video frame based on the motion space information;

[0143] Determining a layered feedback online autonomous learning strategy based on second difference information between the predicted feature information of the dimension of the next video frame and the extracted feature information of the dimension of the next video frame;

[0144] Performing layered feedback self-learning on the vehicle artificial intelligence perception network model based on the layered feedback online autonomous learning strategy, to obtain an online layered feedback self-learning vehicle artificial intelligence perception network model.

[0145] The embodiment of the present application also provides a vehicle control device 70 based on vehicle artificial intelligence, as shown in the figure, the device comprises: Figure 7

[0146] A video stream acquisition module 710 is configured to acquire a video stream collected by a camera of a vehicle during driving of the vehicle;

[0147] An online layered feedback self-learning model module 720 is configured to input the video stream into an online layered feedback self-learning vehicle artificial intelligence perception network model, to output a perception result by using the online layered feedback self-learning vehicle artificial intelligence perception network model;

[0148] ​​The vehicle control module 730 is configured to control the vehicle according to the perception result.

[0149] In a fifth aspect, an electronic device 800 is provided, comprising: Figure 8

[0150] at least one processor 801;

[0151] a memory 802 configured to store instructions executable by the at least one processor;

[0152] The at least one processor is configured to execute the instructions to implement the method described above.

[0153] In a sixth aspect, a computer readable storage medium is provided, when instructions in the computer readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method described above.

[0154] In a seventh aspect, a computer program product is provided, comprising a computer program, when the computer program is executed by a processor, the method described above is implemented.

[0155] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments without departing from the principles and spirit of the present application within the scope of the present application.​

Claims

1. A hierarchical feedback self-learning method of vehicle artificial intelligence, characterized by, The method comprises: During driving of the vehicle, a video stream collected by a camera of the vehicle is acquired; The video stream is input into a vehicle artificial intelligence perception network model to be trained online, and the following steps are performed by the vehicle artificial intelligence perception network model: Feature information of multiple dimensions of each video frame in the video stream is extracted, the vehicle artificial intelligence perception network model comprises a two-dimensional feature information extraction sub-model, a three-dimensional feature information extraction sub-model and a four-dimensional feature information extraction sub-model, and the feature information of the multiple dimensions comprises two-dimensional feature information, three-dimensional feature information and four-dimensional feature information; For feature information of each dimension of a current video frame, first difference information between first feature information and second feature information is calculated, the first feature information is the feature information of the dimension of the current video frame, the second feature information is feature information of the dimension of a next video frame of the current video frame, and the current video frame is any video frame of the video stream; Motion space information of the vehicle is generated based on the first difference information and motion information of the vehicle, and the feature information of the dimension of the next video frame is predicted based on the motion space information; Based on second difference information between the predicted two-dimensional feature information of the next video frame and the extracted two-dimensional feature information of the next video frame, first parameter adjustment gradient information corresponding to the two-dimensional feature information extraction sub-model is determined, and a two-dimensional feature feedback network is constructed based on the first parameter adjustment gradient information; Based on second difference information between the predicted three-dimensional feature information of the next video frame and the extracted three-dimensional feature information of the next video frame, second parameter adjustment gradient information corresponding to the three-dimensional feature information extraction sub-model is determined, and a three-dimensional feature feedback network is constructed based on the second parameter adjustment gradient information; Based on second difference information between the predicted four-dimensional feature information of the next video frame and the extracted four-dimensional feature information of the next video frame, third parameter adjustment gradient information corresponding to the four-dimensional feature information extraction sub-model is determined, and a four-dimensional feature feedback network is constructed based on the third parameter adjustment gradient information; Based on the three-dimensional feature feedback network and the four-dimensional feature feedback network, an inter-layer feedback network is constructed; Based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network and the inter-layer feedback network, a hierarchical feedback online self-learning strategy is determined; Based on the hierarchical feedback online self-learning strategy, hierarchical feedback self-learning is performed on the vehicle artificial intelligence perception network model, and an online hierarchical feedback self-learned vehicle artificial intelligence perception network model is obtained.

2. The method of claim 1, wherein, The extraction of the feature information of the multiple dimensions of each video frame in the video stream comprises: Two-dimensional feature information of each video frame in the video stream is extracted by a two-dimensional feature information extraction sub-model, wherein the two-dimensional feature information is used to represent plane feature information of a video frame; Three-dimensional feature information of each video frame in the video stream is extracted by a three-dimensional feature information extraction sub-model, wherein the three-dimensional feature information is used to represent space feature information of a video frame; The four-dimensional feature information extraction sub-model extracts four-dimensional feature information of each video frame in the video stream, wherein the four-dimensional feature information is used to represent spatial feature information and time feature information of the video frame.

3. The method of claim 1, wherein, The layered feedback online autonomous learning strategy is determined based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network, and includes: The first parameter adjustment information of the two-dimensional feature information extraction sub-model is determined based on the two-dimensional feature feedback network and the inter-layer feedback network; The second parameter adjustment information of the three-dimensional feature information extraction sub-model is determined based on the three-dimensional feature feedback network and the inter-layer feedback network; The third parameter adjustment information of the four-dimensional feature information extraction sub-model is determined based on the four-dimensional feature feedback network.

4. The method of claim 3, wherein, The layered feedback online autonomous learning strategy is determined based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network, and includes: The layered feedback online autonomous learning strategy is determined based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network, and includes: The layered feedback online autonomous learning strategy is determined based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network, and includes: The method includes:

5. A vehicle control method based on vehicle artificial intelligence, characterized by, acquiring a video stream collected by a camera of a vehicle during vehicle driving; inputting the video stream into the online layered feedback self-learning vehicle artificial intelligence perception network model as claimed in any one of claims 1 to 4, so as to output a perception result through the online layered feedback self-learning vehicle artificial intelligence perception network model; controlling the vehicle based on the perception result. The device includes:

6. A layered feedback self-learning device for vehicle artificial intelligence, characterized by, a video stream acquisition module, configured to acquire a video stream collected by a camera of a vehicle during vehicle driving; a video stream input module, configured to input the video stream into a vehicle artificial intelligence perception network model to be online trained, and execute the following steps through the vehicle artificial intelligence perception network model: extracting feature information of multiple dimensions of each video frame in the video stream, wherein the vehicle artificial intelligence perception network model includes a two-dimensional feature information extraction sub-model, a three-dimensional feature information extraction sub-model, and a four-dimensional feature information extraction sub-model, and the feature information of the multiple dimensions includes two-dimensional feature information, three-dimensional feature information, and four-dimensional feature information; for feature information of each dimension of a current video frame, calculating first difference information between first feature information and second feature information; the first feature information is feature information of the dimension of the video frame, and the second feature information is feature information of the dimension of a next video frame of the video frame; the current video frame is any video frame of the video stream; generating motion space information of the vehicle based on the first difference information and motion information of the vehicle, and predicting the feature information of the dimension of the next video frame based on the motion space information; ​ determine first parameter adjustment gradient information corresponding to the two-dimensional feature information extraction sub-model based on second difference information between the predicted two-dimensional feature information of the next video frame and the extracted two-dimensional feature information of the next video frame, and construct a two-dimensional feature feedback network based on the first parameter adjustment gradient information; determine second parameter adjustment gradient information corresponding to the three-dimensional feature information extraction sub-model based on second difference information between the predicted three-dimensional feature information of the next video frame and the extracted three-dimensional feature information of the next video frame, and construct a three-dimensional feature feedback network based on the second parameter adjustment gradient information; determine third parameter adjustment gradient information corresponding to the four-dimensional feature information extraction sub-model based on second difference information between the predicted four-dimensional feature information of the next video frame and the extracted four-dimensional feature information of the next video frame, and construct a four-dimensional feature feedback network based on the third parameter adjustment gradient information; construct an inter-layer feedback network based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, and the four-dimensional feature feedback network; determine a hierarchical feedback online self-learning strategy based on the two-dimensional feature feedback network, the three-dimensional feature feedback network, the four-dimensional feature feedback network, and the inter-layer feedback network; perform hierarchical feedback self-learning on the vehicle artificial intelligence perception network model based on the hierarchical feedback online self-learning strategy, to obtain an online hierarchical feedback self-learned vehicle artificial intelligence perception network model.

7. A vehicle control device based on vehicle artificial intelligence, characterized by, The apparatus comprises: a video stream acquisition module configured to acquire a video stream captured by a camera of a vehicle during driving of the vehicle; an online hierarchical feedback self-learning module configured to input the video stream into the online hierarchical feedback self-learned vehicle artificial intelligence perception network model of any one of claims 1 to 4, to output a perception result by the online hierarchical feedback self-learned vehicle artificial intelligence perception network model; a vehicle control module configured to perform corresponding control on the vehicle based on the perception result.

8. An electronic device, comprising: comprises: at least one processor; a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method of any one of claims 1 to 4 or claim 5.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method of any one of claims 1 to 4 or claim 5.

Images