A vehicle data processing method and apparatus

Abstract

Embodiments of the present application disclose a vehicle data processing method and device, the method comprising: collecting vehicle end data; training and reasoning a pre-created vehicle end intelligent driving model according to the vehicle end data to obtain vehicle end intelligent driving model parameters; collecting processed data obtained by processing the vehicle end data during training and reasoning of the vehicle end intelligent driving model; compressing the processed data to obtain compressed data, and sending the compressed data to a cloud end to train a pre-created cloud end intelligent driving model through the processed data, optimize the cloud end intelligent driving model, and obtain cloud end intelligent driving model adjustment parameters. Through the embodiment scheme, the data transmitted to the cloud end does not involve desensitization processing, can avoid being limited by a third party, and reduces cloud end data transmission volume, broadband cost and storage cost.

Description

Technical Field

[0001] This application relates to vehicle data processing technology, and more particularly to a vehicle data processing method and apparatus. Background Technology

[0002] Currently, training for intelligent driving scenarios relies on a large amount of collected data, which is transmitted back to the cloud via the network. The cloud then desensitizes and labels the data before training and simulating using a large deep learning model. This process raises legal compliance issues, as well as network bandwidth and data transmission costs associated with the massive data transmission. Summary of the Invention

[0003] This application provides a vehicle data processing method and apparatus that does not involve de-identification processing, avoids reliance on third-party organizations, and reduces cloud data transmission volume, broadband costs, and storage costs.

[0004] This application provides a vehicle data processing method applied to a vehicle, the method including:

[0005] Collect vehicle-side data; the vehicle-side data includes driving data and video data;

[0006] The vehicle-side intelligent driving model is trained and inferred based on the vehicle-side data to obtain the vehicle-side intelligent driving model parameters.

[0007] The data obtained by processing the vehicle-side data during the training and inference process of the vehicle-side intelligent driving model includes any one or more of the following: labeled data, extracted video frames, extracted keyframes, and feature vectors of the video frames and keyframes.

[0008] The processed data is compressed to obtain compressed data, which is then sent to the cloud to train a pre-created cloud-based intelligent driving model, optimize the cloud-based intelligent driving model, and obtain the cloud-based intelligent driving model adjustment parameters.

[0009] In an exemplary embodiment of this application, the method may further include:

[0010] The system receives the cloud-based intelligent driving model adjustment parameters returned from the cloud, updates the parameters of the vehicle-side intelligent driving model based on the cloud-based intelligent driving model adjustment parameters, and optimizes the vehicle-side intelligent driving model.

[0011] In an exemplary embodiment of this application, when the vehicle-side data is video data, data processing of the vehicle-side data may include:

[0012] The video data is segmented according to a preset time interval, and the segmented video data is then frame-extracted to obtain video frames.

[0013] The depth features of the video frames are extracted using a preset feature extraction strategy;

[0014] Extract the content features of the video frames;

[0015] Keyframes in the video frame are obtained based on the depth features and the content features;

[0016] The keyframes are subjected to feature extraction using the preset feature extraction strategy to obtain keyframe feature values.

[0017] In an exemplary embodiment of this application, the preset feature extraction strategy may include:

[0018] Continuous frames are obtained as input to a 3D convolutional neural network. After multiple convolution kernels and downsampling, the continuous input frames are converted into feature vector representations.

[0019] Select the feature map of the last convolutional layer as the multi-frame feature vector to be extracted.

[0020] In an exemplary embodiment of this application, extracting the content features of the video frame may include:

[0021] The video frame is divided into multiple units;

[0022] Statistical analysis is performed on each pixel in each unit according to the gradient direction to obtain a histogram with the gradient direction as the coordinate axis, thus obtaining a multi-dimensional feature vector.

[0023] Combine multiple multidimensional feature vectors within each unit block;

[0024] A preset number of units are grouped into a unit block and normalized within the block. The feature vectors of all unit blocks are combined to obtain the content feature vector of the video frame.

[0025] In an exemplary embodiment of this application, obtaining keyframes from the video frame based on the depth features and the content features may include:

[0026] The similarity between the depth features and the content features is weighted according to their respective weights.

[0027] The similarity between the depth features and the content features after linear fusion and weighting;

[0028] The keyframes are extracted based on the similarity obtained after fusion, according to a threshold.

[0029] In an exemplary embodiment of this application, sending the compressed data to the cloud may include:

[0030] The obtained compressed data is sent to the Message Queuing Telemetry Transport Protocol (MQTT) to obtain the compressed data via message subscription within the MQTT in the cloud; or...

[0031] The compressed data is sent directly to the cloud via network transmission.

[0032] This application also provides a vehicle data processing device that can be applied to a vehicle and may include a processor and a computer-readable storage medium. The computer-readable storage medium stores instructions, and when the instructions are executed by the processor, the vehicle data processing method is implemented.

[0033] This application also provides a vehicle data processing method applied in the cloud, the method including:

[0034] The system receives compressed data sent from the vehicle terminal; the compressed data is the data processed by the vehicle terminal after collecting vehicle terminal data; the vehicle terminal data includes driving data and video data.

[0035] The compressed data is decompressed to obtain the processed data.

[0036] The pre-created cloud-based intelligent driving model is trained using the processed data to obtain the cloud-based intelligent driving model adjustment parameters required by the vehicle.

[0037] Send the cloud-based intelligent driving model adjustment parameters required by the vehicle to the vehicle.

[0038] In an exemplary embodiment of this application, before training a pre-created cloud-based intelligent driving model using the processed data, the method may further include:

[0039] The processed data is divided into different scenario data;

[0040] According to different scenarios, the corresponding scenario data is input into the cloud-based intelligent driving model to achieve scenario training of the cloud-based intelligent driving model.

[0041] Compared with related technologies, the embodiments of this application may include: collecting vehicle-side data; the vehicle-side data includes driving data and video data; training and inferring a pre-created vehicle-side intelligent driving model based on the vehicle-side data to obtain vehicle-side intelligent driving model parameters; collecting processed data obtained by processing the vehicle-side data during the training and inference process of the vehicle-side intelligent driving model; compressing the processed data to obtain compressed data, and sending the compressed data to the cloud to train a pre-created cloud-based intelligent driving model using the processed data, optimize the cloud-based intelligent driving model, and obtain cloud-based intelligent driving model adjustment parameters. Through this embodiment, the data transmitted to the cloud does not involve anonymization processing, avoiding reliance on third-party institutions, reducing cloud data transmission volume, and reducing bandwidth and storage costs.

[0042] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0043] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0044] Figure 1 This is a flowchart of a vehicle data processing method applied to the vehicle side according to an embodiment of this application;

[0045] Figure 2 This is a flowchart illustrating a method for processing vehicle-side data according to an embodiment of this application.

[0046] Figure 3 This is a block diagram illustrating data transmission between the vehicle and the cloud in an embodiment of this application.

[0047] Figure 4 This is a schematic diagram comparing data processing between the vehicle and the cloud in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram illustrating communication between the vehicle and the cloud via MQTT, according to an embodiment of this application.

[0049] Figure 6 This is a block diagram of the vehicle data processing device according to an embodiment of this application;

[0050] Figure 7 This is a flowchart of a vehicle data processing method applied to the cloud, according to an embodiment of this application. Detailed Implementation

[0051] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0052] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0053] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0054] This application provides a vehicle data processing method, applied to the vehicle end, such as... Figure 1 As shown, the method may include steps S101-S104:

[0055] S101. Collect vehicle-side data; the vehicle-side data includes driving data and video data;

[0056] S102. Train and infer the pre-created vehicle-side intelligent driving model based on the vehicle-side data to obtain vehicle-side intelligent driving model parameters.

[0057] S103. Collect processed data obtained by processing the vehicle-side data during the training and inference process of the vehicle-side intelligent driving model; the processed data includes any one or more of the following: labeled data, extracted video frames, extracted keyframes, and feature vectors of the video frames and keyframes.

[0058] S104. Compress the processed data to obtain compressed data, and send the compressed data to the cloud to train the pre-created cloud-based intelligent driving model, optimize the cloud-based intelligent driving model, and obtain the cloud-based intelligent driving model adjustment parameters.

[0059] In the exemplary embodiments of this application, the current approach for training intelligent driving scenarios primarily involves uploading data collected from the vehicle to the cloud after it has been anonymized by a qualified third-party organization. A data annotation platform model built in the cloud then annotates specific objects, and model training and simulation are performed in the cloud data center. This approach presents several problems:

[0060] 1. It requires the assistance of a qualified third-party organization for de-identification processing, which limits its accessibility to external resources;

[0061] 2. Transmission from the vehicle to the cloud involves a large amount of audio and video data, driving data, etc., which consumes a lot of bandwidth and incurs bandwidth costs.

[0062] 3. Large amounts of data also need to be stored, which will consume a lot of storage resources and costs.

[0063] In an exemplary embodiment of this application, after the vehicle collects vehicle-side data (which may include, but is not limited to, driving data and various video data collected in real time during driving), operations that originally needed to be processed in the cloud can be performed on the vehicle side. This includes processing video data, extracting video frames, data annotation, extracting feature values ​​of video frames for intelligent driving training, etc., thereby making full use of the vehicle's idle computing resources and saving the computing power required by the cloud. The original data (e.g., may include, but is not limited to, facial data, license plate information, in-vehicle scene data, military restricted area scene data, etc.) does not leave the vehicle, meeting the legal and regulatory requirements of intelligent driving scenarios. The cloud does not store original data containing sensitive information, avoiding security and compliance issues.

[0064] In an exemplary embodiment of this application, the vehicle can first collect vehicle data during intelligent driving. It can perceive and identify environmental information and collect audio and video data through sensing devices such as cameras and radar (e.g., including but not limited to lidar, millimeter-wave radar, etc.). During intelligent driving, it can also collect driving data (e.g., including but not limited to driving speed, direction, position, start and stop information, etc.) through various on-board sensors (e.g., including but not limited to speed sensors, acceleration sensors, gyroscopes, angle sensors, positioning devices, etc.).

[0065] In an exemplary embodiment of this application, the acquired vehicle-side data can be input into a pre-created vehicle-side intelligent driving model (which may be referred to as a vehicle-side front-end small model) to perform inference and training on the vehicle-side intelligent driving model, and the vehicle-side data can be processed during the inference and training process.

[0066] In exemplary embodiments of this application, data processing may include, but is not limited to, any one or more of the following: data annotation, video frame extraction, key frame extraction, and feature vector extraction of the video frames and key frames.

[0067] In an exemplary embodiment of this application, when the vehicle-side data is video data, such as Figure 2 As shown, data processing of the vehicle-side data may include steps S201-S205:

[0068] S201. The video data is cut into segments according to a preset time interval, and the segments are then extracted to obtain video frames.

[0069] In an exemplary embodiment of this application, the collected video data can be segmented at certain time intervals. The segmented video data can be divided according to different scenes and categories. For each scene and each type of video data, video frames are extracted to form video frames.

[0070] S202. Extract the depth features of the video frame using a preset feature extraction strategy.

[0071] In an exemplary embodiment of this application, the preset feature extraction strategy may include:

[0072] Continuous frames are obtained as input to a 3D convolutional neural network. After multiple convolution kernels and downsampling, the continuous input frames are converted into feature vector representations.

[0073] Select the feature map of the last convolutional layer as the multi-frame feature vector to be extracted.

[0074] In an exemplary embodiment of this application, the vehicle-side intelligent driving model can be created based on a 3D (three-dimensional) convolutional neural network. The 3D convolutional neural network can be used to extract deep features from video frames: continuous frames of video frames can be input into the 3D convolutional neural network. After multiple convolution kernels and downsampling of the input continuous video frames in the 3D convolutional neural network, the input continuous video frames are converted into feature vector representations. The last fully connected layer in the original 3D convolutional neural network structure is removed, and the feature map of the last convolutional layer is selected as the multi-frame feature vector to be extracted.

[0075] S203. Extract the content features of the video frame.

[0076] In an exemplary embodiment of this application, extracting the content features of the video frame may include:

[0077] The video frame is divided into multiple units;

[0078] Statistical analysis is performed on each pixel in each unit according to the gradient direction to obtain a histogram with the gradient direction as the coordinate axis, thus obtaining a multi-dimensional feature vector.

[0079] Combine multiple multidimensional feature vectors within each unit block;

[0080] A preset number of units are grouped into a unit block and normalized within the block. The feature vectors of all unit blocks are combined to obtain the content feature vector of the video frame.

[0081] S204. Obtain keyframes from the video frame based on the depth features and the content features.

[0082] In an exemplary embodiment of this application, obtaining keyframes from the video frame based on the depth features and the content features may include:

[0083] The similarity between the depth features and the content features is weighted according to their respective weights.

[0084] The similarity between the depth features and the content features after linear fusion and weighting;

[0085] The keyframes are extracted based on the similarity obtained after fusion, according to a threshold.

[0086] S205. Use the preset feature extraction strategy to extract features from the key frame and obtain key frame feature values.

[0087] In an exemplary embodiment of this application, a 3D convolutional neural network can also be used to extract deep features from keyframes: continuous frames of keyframes can be input into the 3D convolutional neural network. After multiple convolution kernels and downsampling of the input continuous keyframes within the 3D convolutional neural network, the input continuous keyframes are converted into feature vector representations. The last fully connected layer in the original 3D convolutional neural network structure is removed, and the feature map of the last convolutional layer is selected as the multi-frame feature vector to be extracted.

[0088] In an exemplary embodiment of this application, after feature extraction of the keyframes, the feature vector calculated by the 3D convolutional neural network is obtained, and the data is sent to the vehicle-side data transmission module.

[0089] In an exemplary embodiment of this application, the vehicle-side data sending module can encrypt processed data such as driving data and feature vector data through a Tbox (telematics processor) and then send it to the cloud receiving module.

[0090] In an exemplary embodiment of this application, a schematic diagram of data transmission between the vehicle and the cloud is shown below. Figure 3 As shown in the diagram, this illustrates the comparison of data processing between the vehicle and the cloud. Figure 4 As shown.

[0091] In an exemplary embodiment of this application, sending the compressed data to the cloud may include:

[0092] The obtained compressed data is sent to the Message Queuing Telemetry Transport Protocol (MQTT) to obtain the compressed data via message subscription within the MQTT in the cloud; or...

[0093] The compressed data is sent directly to the cloud via network transmission.

[0094] In exemplary embodiments of this application, as Figure 5 As shown, the vehicle-side data sending module can package and compress the processed data based on a certain time frequency and sending strategy, and use the vehicle's built-in network to send the data to MQTT (Message Queuing Telemetry Transport) via 4G / 5G or APN (Access Point) channel, or send it to the cloud receiving module via HTTP (Hypertext Transfer Protocol).

[0095] In an exemplary embodiment of this application, the cloud consumes messages in MQTT or receives HTTP requests; the received data is decompressed and transmitted to the intelligent driving data center; based on the received processed data, the cloud-based intelligent driving model is trained with a large amount of data, and the inference optimization parameters that need to be adjusted for the vehicle-side front-end small model can be extracted, and the data is sent to MQTT to realize OTA (over the air) upgrade.

[0096] In an exemplary embodiment of this application, the cloud can divide the processed data into different scene data; according to different scenes, the corresponding scene data is input into the cloud-based intelligent driving model to achieve scene training of the cloud-based intelligent driving model. This cloud-based intelligent driving model is a large-scale deep learning neural network model built in the cloud (which can be referred to as the cloud-based large model). The corresponding scene data is input into this large-scale deep learning neural network model, and multiple rounds of training are performed based on a large number of feature vectors and driving data sent from the vehicle until the loss value converges. The cloud-based model parameters are then adjusted to obtain cloud-based intelligent driving model adjustment parameters, which can be used to adjust the vehicle-side intelligent driving model parameters.

[0097] In an exemplary embodiment of this application, the cloud also encrypts the adjusted cloud-based intelligent driving model parameters before feeding them back to the vehicle.

[0098] In an exemplary embodiment of this application, the method may further include:

[0099] The vehicle receives the cloud-based intelligent driving model adjustment parameters returned from the cloud, and updates the parameters of the vehicle-side intelligent driving model according to the cloud-based intelligent driving model adjustment parameters, thereby optimizing the vehicle-side intelligent driving model.

[0100] The exemplary embodiments of this application include at least the following advantages:

[0101] 1. Previously, the collected raw audio and video data and driving data needed to be directly transmitted to the cloud-based large model. In this embodiment, the data is first input into the vehicle-side front-end small model, processed on the vehicle side to generate feature value vectors, and then transmitted to the cloud, thus improving data transmission efficiency.

[0102] 2. It can reduce the bandwidth consumption problem caused by large data transmission in the original data transmission strategy; the original transmission of a 60-second, 10-frame, 4096*2176 video was 30MB, but now the text vector is only 200KB after lossless compression. Compared with the original, the bandwidth usage is reduced to 1 / 150 of the original.

[0103] 3. The cloud does not store the raw data, but only the processed feature vector data. The amount of raw data collected by intelligent driving is huge. If all the raw data were to be stored on disk, it would consume a lot of storage resources. Now the cloud only needs to store the processed feature vectors. This data is in text format and can be compressed to a great extent, which greatly reduces storage space resources.

[0104] 4. Make full use of idle resources on the vehicle end, decompose some inference and parsing to be processed on the vehicle end, reduce the amount of data transmission work, and reduce the amount of data transmission per vehicle by nearly 800M per day.

[0105] 5. By transmitting feature vectors required for intelligent driving training, sensitive data such as faces, license plates, and military restricted areas are not sent out, thus meeting the requirements of safety regulations and reducing cloud computing overhead and latency caused by desensitization.

[0106] 6. Enhance the accuracy and timeliness of model training.

[0107] This application also provides a vehicle data processing device 1, such as... Figure 6 As shown, it can be applied to the vehicle end and may include a processor 11 and a computer-readable storage medium 12. The computer-readable storage medium 12 stores instructions, and when the instructions are executed by the processor 11, the vehicle data processing method is implemented.

[0108] In the exemplary embodiments of this application, any of the embodiments in the foregoing vehicle data processing method embodiments are applicable to the device embodiments, and will not be described in detail here.

[0109] This application also provides a vehicle data processing method, applied in the cloud, such as... Figure 7 As shown, the method may include steps S301-S304:

[0110] S301. Receive compressed data sent by the vehicle terminal; the compressed data is the data after the vehicle terminal processes the collected vehicle terminal data; the vehicle terminal data includes driving data and video data;

[0111] S302. After decompressing the compressed data, obtain the processed data.

[0112] S303. Train the pre-created cloud-based intelligent driving model using the processed data to obtain the cloud-based intelligent driving model adjustment parameters required by the vehicle.

[0113] S304. Send the cloud-based intelligent driving model adjustment parameters required by the vehicle to the vehicle.

[0114] In an exemplary embodiment of this application, before training a pre-created cloud-based intelligent driving model using the processed data, the method may further include:

[0115] The processed data is divided into different scenario data;

[0116] According to different scenarios, the corresponding scenario data is input into the cloud-based intelligent driving model to achieve scenario training of the cloud-based intelligent driving model.

[0117] In the exemplary embodiments of this application, the cloud data processing schemes involved in the aforementioned vehicle-side data processing method embodiments are all applicable to this embodiment, and will not be described in detail here.

[0118] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A vehicle data processing method, characterized in that, Applied to the vehicle end, the method includes: Collect vehicle-side data; the vehicle-side data includes driving data and video data; The vehicle-side intelligent driving model is trained and inferred based on the vehicle-side data to obtain the vehicle-side intelligent driving model parameters. The processed data obtained by processing the vehicle-side data during the training and inference process of the vehicle-side intelligent driving model includes any one or more of the following: labeled data, extracted video frames, extracted keyframes, and feature vectors of the video frames and keyframes. The processed data is compressed to obtain compressed data, and the compressed data is sent to the cloud to divide the processed data into different scenario data; according to different scenarios, the corresponding scenario data is input into a pre-created cloud intelligent driving model for training, optimizing the cloud intelligent driving model, and obtaining the cloud intelligent driving model adjustment parameters; The parameters of the vehicle-side intelligent driving model are updated based on the parameters of the cloud-based intelligent driving model to optimize the vehicle-side intelligent driving model.

2. The vehicle data processing method according to claim 1, characterized in that, When the vehicle-side data is video data, data processing is performed on the vehicle-side data, including: The video data is segmented according to a preset time interval, and the segmented video data is then frame-extracted to obtain video frames. The depth features of the video frames are extracted using a preset feature extraction strategy; Extract the content features of the video frames; Keyframes in the video frame are obtained based on the depth features and the content features; The keyframes are subjected to feature extraction using the preset feature extraction strategy to obtain keyframe feature values.

3. The vehicle data processing method according to claim 2, characterized in that, The preset feature extraction strategy includes: Continuous frames are obtained as input to a 3D convolutional neural network. After multiple convolution kernels and downsampling, the continuous input frames are converted into feature vector representations. Select the feature map of the last convolutional layer as the multi-frame feature vector to be extracted.

4. The vehicle data processing method according to claim 2, characterized in that, The extraction of content features from the video frames includes: The video frame is divided into multiple units; Statistical analysis is performed on each pixel in each unit according to the gradient direction to obtain a histogram with the gradient direction as the coordinate axis, thus obtaining a multi-dimensional feature vector. Combine multiple multidimensional feature vectors within each unit block; A preset number of units are grouped into a unit block and normalized within the block. The feature vectors of all unit blocks are combined to obtain the content feature vector of the video frame.

5. The vehicle data processing method according to claim 2, characterized in that, The step of obtaining keyframes from the video frame based on the depth features and the content features includes: The similarity between the depth features and the content features is weighted according to their respective weights. The similarity between the depth features and the content features after linear fusion and weighting; The keyframes are extracted based on the similarity obtained after fusion, according to a threshold.

6. The vehicle data processing method according to claim 1, characterized in that, Sending the compressed data to the cloud includes: The obtained compressed data is sent to the Message Queuing Telemetry Transport Protocol (MQTT) to obtain the compressed data via message subscription within the MQTT in the cloud; or... The compressed data is sent directly to the cloud via network transmission.

7. A vehicle data processing device, characterized in that, The method is applied to a vehicle and includes a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed by the processor, implement the vehicle data processing method as described in any one of claims 1-6.

8. A vehicle data processing method, characterized in that, Applied to the cloud, the method includes: The system receives compressed data sent from the vehicle terminal; the compressed data is the data after the vehicle terminal processes the collected vehicle terminal data; the vehicle terminal data includes driving data and video data, and the processed data includes any one or more of the following: labeled data, extracted video frames, extracted keyframes, and feature vectors of the video frames and keyframes. The compressed data is decompressed to obtain the processed data. The pre-created cloud-based intelligent driving model is trained using the processed data to obtain the cloud-based intelligent driving model adjustment parameters required by the vehicle. The cloud-based intelligent driving model adjustment parameters required by the vehicle are sent to the vehicle, so that the vehicle can update the parameters of the vehicle's intelligent driving model according to the cloud-based intelligent driving model adjustment parameters, thereby optimizing the vehicle's intelligent driving model. Before training a pre-created cloud-based intelligent driving model using the processed data, the method further includes: The processed data is divided into different scenario data; According to different scenarios, the corresponding scenario data is input into the cloud-based intelligent driving model to achieve scenario training of the cloud-based intelligent driving model.

Images