Intelligent generation method and system for automobile AI journey video

Abstract

The present invention discloses a method and device for intelligently generating automobile AI journey videos, wherein the method comprises collecting in-car video stream data and out-car video stream data respectively, and collecting in-car voice data, and preprocessing each data; generating a journey video algorithm model through model training; importing the journey video algorithm model into an open neural network exchange model, and outputting an open neural network exchange file of the journey video algorithm model; predicting the preprocessed in-car video stream data, out-car video stream data and in-car voice data through the open neural network exchange file to generate multiple template videos; the present invention can help users to analyze and obtain pictures of historical wonderful moments that need to be recorded through AI intelligence; with travel pictures with identification marks, automatically generate journey videos of time memories for users to use and share.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted intelligent technology, and in particular to a method and system for intelligently generating an automobile AI journey video. Background Art

[0002] With the improvement of vehicle networking capabilities and vehicle intelligence, users have higher requirements for vehicle intelligence and have further requirements for sharing their journeys and preserving wonderful history. Simply saving pictures cannot meet users' usage needs. When driving, it is not convenient for users to take pictures in real time. We can use AI intelligence to help users analyze and obtain pictures of historical moments that need to be recorded. With travel pictures with identification marks, journey videos of time memories are automatically generated for users to use and share.

[0003] Most current cars have an album function that simply saves pictures to a specified folder, but does not intelligently recognize pictures and generate AI journey videos. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for intelligently generating automobile AI journey videos to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for intelligently generating an automobile AI journey video, comprising:

[0006] Collect in-vehicle video stream data and out-vehicle video stream data respectively, as well as in-vehicle voice data, and pre-process each data;

[0007] Generate a travel video algorithm model through model training, where the travel video algorithm model includes a face recognition algorithm model, a voice emotion recognition algorithm model, and a target detection algorithm model;

[0008] Import the journey video algorithm model into the open neural network exchange model, and output the open neural network exchange file of the journey video algorithm model;

[0009] The pre-processed in-vehicle video stream data, out-of-vehicle video stream data and in-vehicle voice data are predicted through the open neural network exchange file to generate multiple template videos.

[0010] Furthermore, the in-vehicle video stream data and the out-vehicle video stream data are collected separately, as well as the in-vehicle voice data, and each data is pre-processed, and further includes:

[0011] Set a specified time interval and divide the interval into segments according to the specified time interval;

[0012] Based on the in-vehicle video stream data, the cockpit picture of the corresponding section is generated, based on the out-vehicle video stream data, the outside scene picture data of the corresponding section is generated, and based on the in-vehicle voice data, the passenger voice of the corresponding section is generated.

[0013] Furthermore, the face recognition algorithm model also includes:

[0014] Build a face recognition neural network model for facial expression recognition;

[0015] The face recognition neural network model has three convolutional layers and three fully connected layers;

[0016] Insert a pooling layer into the adjacent convolutional layer.

[0017] Furthermore, the speech emotion recognition algorithm model also includes:

[0018] The speech emotion recognition algorithm model is based on DNN-HMM speech emotion recognition. The dimension of the DNN output vector corresponds to the number of states in the HMM, and each dimensional output corresponds to a bound three-phone state.

[0019] Furthermore, a pre-trained GMM-HMM recognition system is used to perform forced alignment on the training corpus to obtain the target output value of each frame of speech on the DNN.

[0020] Furthermore, before training a DNN-HMM acoustic model, a GMM-HMM acoustic model is first trained, and each speech frame is labeled with an HMM state label by a forced alignment method based on the Viterbi algorithm;

[0021] Based on the HMM state labels, train a DNN model based on the DNN training algorithm;

[0022] The GMM part of the HMM model that calculates the observation probability is replaced by the DNN model, while the transition probability and initial probability parts are retained.

[0023] Furthermore, the target detection algorithm model also includes:

[0024] Based on the deep learning template YOLO, a target detection algorithm model is built. A separate CNN model is used to feed the image data of the scene outside the vehicle into the CNN network, process the network prediction results, and realize end-to-end target detection.

[0025] Furthermore, the CNN network divides the image data of the scene outside the vehicle into a grid, detects each cell, detects the target whose center point falls within the cell, and predicts the target through the cell.

[0026] Furthermore, the preprocessed in-vehicle video stream data, the out-vehicle video stream data and the in-vehicle voice data are predicted by an open neural network exchange file to generate multiple template videos, and also include:

[0027] Predict facial expressions, voice emotions, and exterior scenes of the vehicle, label them, and form a labeled data set;

[0028] Build a video template. According to the requirements of the video template, select the marked pictures for picture synthesis, add music, text or transition effects, and generate multi-template videos through video editing.

[0029] On the other hand, a device for intelligently generating an AI journey video of a car is provided, which uses any of the above-mentioned methods for intelligently generating an AI journey video of a car, including:

[0030] The acquisition module is used to collect in-vehicle video stream data and out-vehicle video stream data, as well as in-vehicle voice data, and pre-process the data;

[0031] A construction module is used to generate a travel video algorithm model through model training, wherein the travel video algorithm model includes a face recognition algorithm model, a speech emotion recognition algorithm model, and a target detection algorithm model;

[0032] A processing module, used for importing the journey video algorithm model into the open neural network exchange model, and outputting the open neural network exchange file of the journey video algorithm model;

[0033] The analysis module is used to predict the pre-processed in-vehicle video stream data, the out-vehicle video stream data and the in-vehicle voice data through an open neural network exchange file to generate multiple template videos.

[0034] Compared with the prior art, the beneficial effects of the present invention are: the present invention can use AI intelligence to help users analyze and obtain pictures of historical wonderful moments that need to be recorded; with travel pictures with identification marks, travel videos of time memories are automatically generated for users to use and share. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a method for intelligently generating an AI automobile journey video in an embodiment of the present invention;

[0036] Figure 2 This is a block diagram of a device for intelligently generating an AI automobile journey video in an embodiment of the present invention;

[0037] Figure 3 This is a logic block diagram of a method for intelligently generating an AI automobile journey video in an embodiment of the present invention;

[0038] Figure 4: is a logic block diagram of the first convolutional layer in an embodiment of the present invention;

[0039] Figure 5 : is a logic block diagram of the second convolutional layer in an embodiment of the present invention;

[0040] Figure 6 This is a logic block diagram of the output layer in an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the logic structure of speech emotion recognition in an embodiment of the present invention;

[0042] Figure 8 Schematic diagram of target detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In the description of the present invention, unless otherwise specified, "plurality" means two or more than two; the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", "front end", "rear end", "head", "tail", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0045] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] Please refer to the attached drawings of the specification. The present invention provides a technical solution: a method for intelligently generating an AI automobile journey video, comprising the following steps:

[0047] S102, respectively collecting in-vehicle video stream data and out-vehicle video stream data, and collecting in-vehicle voice data, and pre-processing each data;

[0048] Specifically, a specified time interval is set, and the interval segments are divided according to the specified time interval;

[0049] Based on the in-vehicle video stream data, the cockpit picture of the corresponding section is generated, based on the out-vehicle video stream data, the outside scene picture data of the corresponding section is generated, and based on the in-vehicle voice data, the passenger voice of the corresponding section is generated.

[0050] Among them, in the production stage of AI smart travel video, we first perform data prediction processing on the DSM camera, forward camera and voice to generate the cockpit pictures, passenger voice and outdoor scenery pictures of the interval segment; data preprocessing of the picture and voice data includes converting the two video stream data of the DSM camera and the forward camera into the picture data of the interval segment of the specified time interval (2s), and at the same time, the voice text is cleaned and converted.

[0051] S104, generating a journey video algorithm model through model training, wherein the journey video algorithm model includes a face recognition algorithm model, a voice emotion recognition algorithm model, and a target detection algorithm model;

[0052] Among them, facial expression recognition, speech emotion recognition and target detection models are generated through model training;

[0053] Specifically, the face recognition algorithm model also includes:

[0054] Build a face recognition neural network model for facial expression recognition;

[0055] The face recognition neural network model has three convolutional layers and three fully connected layers;

[0056] Insert a pooling layer into the adjacent convolutional layer.

[0057] Among them, the face recognition algorithm model includes 3 convolutional layers and 3 fully connected layers, and the adjacent convolutional layers are inserted with pooling layers. The convolutional layer is used for feature extraction, the pooling layer is used for feature dimension reduction, and the fully connected layer is used for classification; each node of the neural network is fully connected, even for a 48*48 black and white picture, the feature dimension is 48*48*1=2304; the larger the feature dimension, the easier it is for the neural network to overfit; therefore, the image data is not suitable for ordinary fully connected neural networks, and convolutional neural networks are usually used for dimensionality reduction; the image data is three-dimensional, and the first and second dimensions represent the length and width of the image. The third dimension is the color channel, 3 for color and 1 for grayscale; the convolution kernel is a two-dimensional matrix, and in image processing, the size of the convolution kernel is generally 3*3 or 5*5. Dimensionality reduction is to use a numerical value to represent the data of a convolution kernel in the first and second dimensions. Maximum pooling (taking the maximum value of the data of a convolution kernel) or average pooling (taking the average value of the data of a convolution kernel) can be used; the convolution step is the distance moved each time the convolution is performed; the model includes 3 convolution layers and 3 fully connected layers, and the adjacent convolution layers are inserted with pooling layers; the convolution layer is used for feature extraction, the pooling layer is used for feature dimensionality reduction, and the fully connected layer is used for classification; Figure 4 As shown in the figure, the input image is first converted into a grayscale image. Each pixel is a piece of data according to brightness, with a value of 0-255. The brighter the image, the higher the pixel value. The image size is 48*48, and the color channel dimension of the grayscale image is 1, so the input dimension of the first convolution layer is 48*48*1. The convolution kernel is 5*5, followed by a maximum pooling layer, the convolution kernel is 3*3, and the convolution step size is 2*2; as shown in the figure, Figure 5 As shown in the figure, the input dimension of the second convolutional layer is 64, the convolution kernel is 3*3, followed by an average pooling layer, the convolution kernel is 3*3, and the convolution step is 2*2. The input dimension of the third convolutional layer is 128, and the rest is the same as the second convolutional layer; as shown in the figure Figure 6 As shown in the figure, the dimensions of the first two fully connected layers are 1024, the dropout is 0.2, and the output of the last fully connected layer is the number of categories, 7, and the predicted probabilities of these 7 categories are output through the softmax activation function. Figure 6 Except for the final output layer, the activation functions of other layers are all relu.

[0058] Specifically, the speech emotion recognition algorithm model also includes:

[0059] The speech emotion recognition algorithm model is based on DNN-HMM speech emotion recognition. The dimension of the DNN output vector corresponds to the number of states in the HMM, and each dimensional output corresponds to a bound three-phone state.

[0060] Among them, Figure 7As shown in the figure, for speech emotion recognition based on DNN-HMM, the dimension of the DNN output vector corresponds to the number of states in the HMM, and usually each dimension of the output corresponds to a bound triphone state; HMM is used to describe the dynamic changes of speech signals, and each output node of DNN is used to estimate the posterior probability of a state of the continuous density HMM. The overall input of the model is the fbank feature, and then DNN models the likelihood of all clustered states (such as triphones after state binding) to obtain the posterior probability. It is then decoded in combination with HMM.

[0061] Furthermore, a pre-trained GMM-HMM recognition system is used to perform forced alignment on the training corpus to obtain the target output value of each frame of speech on the DNN.

[0062] During training, in order to obtain the target output value (standard value) of each frame of speech on the DNN, it is necessary to perform forced alignment on the training corpus through the pre-trained GMM-HMM recognition system.

[0063] Furthermore, before training a DNN-HMM acoustic model, a GMM-HMM acoustic model is first trained, and each speech frame is labeled with an HMM state label by a forced alignment method based on the Viterbi algorithm;

[0064] Based on the HMM state labels, train a DNN model based on the DNN training algorithm;

[0065] The GMM part of the HMM model that calculates the observation probability is replaced by the DNN model, while the transition probability and initial probability parts are retained.

[0066] Among them, to train a DNN-HMM acoustic model, you first need to train a GMM-HMM acoustic model, and use the Viterbi algorithm-based forced alignment method to label each speech frame with an HMM state label, and then use this state label to train a DNN model based on the DNN training algorithm; finally, use the DNN model to replace the GMM part of the HMM model that calculates the observation probability, but retain other parts such as the transition probability and initial probability.

[0067] DNN modeling objects:

[0068] The speech recognition problem can be expressed as finding the sentence that maximizes the probability P(W|O)P(W|O):

[0069]

[0070] Among them, p(w) is the language model, p(o|w)p(o|w) is the acoustic model; it is expanded using Viterbi decoding:

[0071]

[0072] In the traditional GMM-HMM model, we use GMM to model the acoustic model. It is a generative model that can directly generate the likelihood probability p(ot|qt)p(ot|qt), which is the observation probability required by HMM. Now we want to use DNN, but DNN can only give the posterior probability p(qt|ot)p(qt|ot) at each node (state) after the observation value is input to the DNN output layer. Therefore, we convert it through Bayes' theorem:

[0073] p(o t |q t )=p(q t |o y )p(o t ) / p(q t )

[0074] Among them, p(ot)p(ot) remains unchanged, p(qt)p(qt) is the prior probability of the state, which is also trained in the training process, that is, aligning the observations (feature vectors) with the state. The proportion of observations aligned with a certain state is the prior probability of this state.

[0075] Specifically, the target detection algorithm model also includes:

[0076] Based on the deep learning template YOLO, a target detection algorithm model is built. A separate CNN model is used to feed the image data of the scene outside the vehicle into the CNN network, process the network prediction results, and realize end-to-end target detection.

[0077] Among them, Figure 8 As shown in Figure 2, the YOLO algorithm uses a single CNN model to achieve end-to-end target detection. The entire system is as follows: Figure 8 As shown in the figure: first resize the input image to 448x448, then send it to the CNN network, and finally process the network prediction results to get the detected target. Compared with the R-CNN algorithm, it is a unified framework, which is faster, and the training process of Yolo is also end-to-end.

[0078] Furthermore, the CNN network divides the image data of the scene outside the vehicle into a grid, detects each cell, detects the target whose center point falls within the cell, and predicts the target through the cell.

[0079] Among them, Yolo's CNN network divides the input image into an S×S grid, and then each cell is responsible for detecting the targets whose center points fall within the grid, such as Figure 8 As shown in the figure, we can see that the center of the dog object falls within a cell in the lower left corner, so this cell is responsible for predicting the dog; each cell predicts B bounding boxes and the confidence score of the bounding box; the so-called confidence score actually includes two aspects, one is the possibility that the bounding box contains the object, and the other is the accuracy of the bounding box; the former is recorded as Pr(object), when the bounding box is the background (i.e., does not contain the object), Pr(object) = 0; and when the bounding box contains the object, Pr(object) = 1; the accuracy of the bounding box can be expressed by the IOU (intersection over union) between the predicted box and the actual box (ground truth) union, intersection-over-union ratio), recorded as IOUpredtruth; therefore, the confidence can be defined as Pr(object)*IOUpredtruth; Yolo's confidence is the product of two factors, and the accuracy of the predicted box is also reflected in it; the size and position of the bounding box can be represented by 4 values: (x, y, w, h), where (x, y) is the center coordinate of the bounding box, and w and h are the width and height of the bounding box; another point to note is that the predicted value of the center coordinate (x, y) is the offset value relative to the coordinate point of the upper left corner of each cell, and the unit is relative to the cell size. The coordinate definition of the cell is as follows Figure 6 As shown; the predicted values ​​of w and h of the bounding box are the ratio of the width and height of the entire image, so theoretically the size of the four elements should be in the range of [0,1]; thus, the predicted value of each bounding box actually contains five elements: (x, y, w, h, c), of which the first four represent the size and position of the bounding box, and the last value is the confidence; for each cell, it is also necessary to give the predicted C category probability values, which represent the probability that the target of the bounding box predicted by the cell belongs to each category; these probability values ​​are the conditional probabilities under the confidence of each bounding box, that is, Pr(classi|object); calculate the class-specific confidence scores of each bounding box:

[0080]

[0081] The confidence of the bounding box category represents the possibility that the object in the bounding box belongs to each category and how well the bounding box matches the object. Generally, the prediction box of the network is filtered according to the category confidence; therefore, each cell needs to predict (B*5+C) values; if the input image is divided into an S×S grid, then the final prediction value is a tensor of size S×S×(B*5+C); for PASCAL VOC data, there are 20 categories in total. If S=7,B=2 is used, then the final prediction result is a tensor of size 7×7×30. This is the principle of the entire algorithm; the target detection algorithm model preferably uses the deep learning template PaddleDetection, which is an end-to-end target detection suite based on PaddlePaddle.

[0082] S106, importing the journey video algorithm model into the open neural network exchange model, and outputting the open neural network exchange file of the journey video algorithm model;

[0083] Among them, the Open Neural Network Exchange (ONNX) file based on the deep learning model predicts the image data, predicts the facial expressions, voice emotions and external scenery, and marks them.

[0084] S108. Predict the preprocessed in-vehicle video stream data, out-of-vehicle video stream data, and in-vehicle voice data by opening a neural network exchange file to generate multiple template videos.

[0085] Specifically, the pre-processed in-car video stream data, out-of-car video stream data and in-car voice data are predicted through an open neural network exchange file to generate multiple template videos, and also include:

[0086] Predict facial expressions, voice emotions, and exterior scenes of the vehicle, label them, and form a labeled data set;

[0087] Build a video template. According to the requirements of the video template, select the marked pictures for picture synthesis, add music, text or transition effects, and generate multi-template videos through video editing.

[0088] Among them, by converting the video stream data into image data in data preprocessing, the image is given to the ONNX model service for prediction, and a labeled data set consisting of facial expressions and scenes outside the car is generated.

[0089] On the other hand, a device for intelligently generating an AI journey video of a car is provided, which uses any of the above-mentioned methods for intelligently generating an AI journey video of a car, including:

[0090] The acquisition module 10 is used to acquire in-vehicle video stream data and out-vehicle video stream data, as well as in-vehicle voice data, and pre-process the data;

[0091] Among them, a specified time interval is set, and the interval is divided according to the specified time interval; the cockpit picture of the corresponding interval is generated based on the in-vehicle video stream data, the outside scene picture data of the corresponding interval is generated based on the outside video stream data, and the passenger voice of the corresponding interval is generated based on the in-vehicle voice data.

[0092] A construction module 20 is used to generate a journey video algorithm model through model training, wherein the journey video algorithm model includes a face recognition algorithm model, a voice emotion recognition algorithm model and a target detection algorithm model;

[0093] Among them, facial expression recognition, speech emotion recognition and target detection models are generated through model training; a face recognition neural network model is established for facial expression recognition; the face recognition neural network model sets three convolutional layers and three fully connected layers; the pooling layer is inserted into the adjacent convolutional layer; the speech emotion recognition algorithm model is based on DNN-HMM speech emotion recognition, the dimension of the DNN output vector corresponds to the number of states in the HMM, and each dimensional output corresponds to a bound three-phoneme state; the pre-trained GMM-HMM recognition system is forced to align on the training corpus to obtain the target output of each frame of speech on the DNN Output value; before training a DNN-HMM acoustic model, first train a GMM-HMM acoustic model, and label each speech frame with an HMM state label by forcing it based on the Viterbi algorithm; based on the HMM state label, train a DNN model based on the DNN training algorithm; replace the GMM part of the HMM model that calculates the observation probability with the DNN model, and retain the transition probability and initial probability parts; the CNN network divides the image data of the scene outside the car into a grid, detects each cell, detects the target whose center point falls within the cell, and predicts the target through the cell.

[0094] The processing module 30 is used to import the journey video algorithm model into the open neural network exchange model and output the open neural network exchange file of the journey video algorithm model;

[0095] Among them, the Open Neural Network Exchange (ONNX) file based on the deep learning model predicts the image data, predicts the facial expressions, voice emotions and external scenery, and marks them.

[0096] The analysis module 40 is used to predict the pre-processed in-vehicle video stream data, the out-vehicle video stream data and the in-vehicle voice data through an open neural network exchange file to generate multiple template videos.

[0097] Among them, video editing selects marked images for image synthesis, adds music, text or transition effects and generates multi-template videos according to the requirements of video templates, and recommends them to customers for selection and sharing; predicts facial expressions, voice emotions and external scenery, marks them, and forms a marked data set; constructs video templates, selects marked images for image synthesis, adds music, text or transition effects according to the requirements of video templates, and generates multi-template videos through video editing.

[0098] The following is a specific application scenario in an embodiment of the present invention:

[0099] After starting the car, the driver navigates. The system uses rules to determine whether it is necessary to turn on the AI ​​road video intelligent generation function, and prompts the user to turn on the photo and video recording functions.

[0100] During driving, when the user's facial expression changes and the voice recognition shows the user's emotions change, pictures are collected and the picture algorithm is used to identify and mark the beautiful scenery.

[0101] When you are about 2 kilometers away from the destination, synthesize the pictures, add text, background music, etc. according to the video template, and edit and generate the video.

[0102] After the driver parks the car, multiple travel video templates are recommended for the driver to select and share.

[0103] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for intelligently generating automobile AI journey videos, characterized in that: include: Collect in-vehicle video stream data and out-vehicle video stream data respectively, as well as in-vehicle voice data, and pre-process each data; Generate a travel video algorithm model through model training, where the travel video algorithm model includes a face recognition algorithm model, a voice emotion recognition algorithm model, and a target detection algorithm model; Import the journey video algorithm model into the open neural network exchange model, and output the open neural network exchange file of the journey video algorithm model; The pre-processed in-vehicle video stream data, out-of-vehicle video stream data and in-vehicle voice data are predicted through the open neural network exchange file to generate multiple template videos.

2. The method for intelligently generating an automobile AI journey video according to claim 1, characterized in that: Collecting in-car video stream data and out-car video stream data respectively, as well as collecting in-car voice data, and pre-processing each data, also includes: Set a specified time interval and divide the interval into segments according to the specified time interval; Based on the in-vehicle video stream data, the cockpit picture of the corresponding section is generated, based on the out-vehicle video stream data, the outside scene picture data of the corresponding section is generated, and based on the in-vehicle voice data, the passenger voice of the corresponding section is generated.

3. The method for intelligently generating an automobile AI journey video according to claim 1, characterized in that: The face recognition algorithm model also includes: Build a face recognition neural network model for facial expression recognition; The face recognition neural network model has three convolutional layers and three fully connected layers; Insert a pooling layer into the adjacent convolutional layer.

4. The method for intelligently generating an automobile AI journey video according to claim 1, characterized in that: The speech emotion recognition algorithm model also includes: The speech emotion recognition algorithm model is based on DNN-HMM speech emotion recognition. The dimension of the DNN output vector corresponds to the number of states in the HMM, and each dimensional output corresponds to a bound three-phone state.

5. The method for intelligently generating automobile AI journey videos according to claim 4, characterized in that: include: The pre-trained GMM-HMM recognition system is forced to align the training corpus to obtain the target output value of each frame of speech on the DNN.

6. The method for intelligently generating an automobile AI journey video according to claim 5, characterized in that: include: Before training a DNN-HMM acoustic model, first train a GMM-HMM acoustic model and assign an HMM state label to each speech frame using the Viterbi algorithm-based forced alignment method. Based on the HMM state labels, train a DNN model based on the DNN training algorithm; The GMM part of the HMM model that calculates the observation probability is replaced by the DNN model, while the transition probability and initial probability parts are retained.

7. The method for intelligently generating an automobile AI journey video according to claim 1, characterized in that: The target detection algorithm model also includes: Based on the deep learning template YOLO, a target detection algorithm model is built. A separate CNN model is used to feed the image data of the scene outside the vehicle into the CNN network, process the network prediction results, and realize end-to-end target detection.

8. The method for intelligently generating automobile AI journey videos according to claim 7, characterized in that: include: The CNN network divides the image data of the scenery outside the car into grids, detects each cell, detects the target whose center point falls within the cell, and predicts the target through the cell.

9. The method for intelligently generating an automobile AI journey video according to claim 1, characterized in that: The pre-processed in-car video stream data, out-of-car video stream data and in-car voice data are predicted through the open neural network exchange file to generate multiple template videos, including: Predict facial expressions, voice emotions, and exterior scenes of the vehicle, label them, and form a labeled data set; Build a video template. According to the requirements of the video template, select the marked pictures for picture synthesis, add music, text or transition effects, and generate multi-template videos through video editing.

10. A vehicle AI journey video intelligent generation device, using a vehicle AI journey video intelligent generation method according to any one of claims 1 to 9, characterized in that: include: The acquisition module is used to collect in-vehicle video stream data and out-vehicle video stream data, as well as in-vehicle voice data, and pre-process the data; A construction module is used to generate a travel video algorithm model through model training, wherein the travel video algorithm model includes a face recognition algorithm model, a speech emotion recognition algorithm model, and a target detection algorithm model; A processing module, used for importing the journey video algorithm model into the open neural network exchange model, and outputting the open neural network exchange file of the journey video algorithm model; The analysis module is used to predict the pre-processed in-vehicle video stream data, the out-vehicle video stream data and the in-vehicle voice data through an open neural network exchange file to generate multiple template videos.

Images