Vehicle video data learning method and device based on YTS engine AI algorithm

Abstract

The present application provides a vehicle video data learning method and device based on the YTS engine AI algorithm, which relates to the field of vehicle data technology and solves the technical problem of low overall execution efficiency of learning tasks. The method includes: analyzing the corresponding features to be learned and the difficulty of feature learning in each dimension to be learned of the vehicle video sample through the AI ​​algorithm; according to the corresponding features to be learned and the difficulty of feature learning in each dimension to be learned, the learning tasks corresponding to the vehicle video sample are allocated to GPUs of different computing power for learning through the AI ​​system; the current learning degree and learning duration of each learning task are monitored through the AI ​​system; if the first current learning degree of the first learning task executed by the first GPU is greater than the first preset learning degree and the first learning duration of the first learning task is less than the first preset duration, the first learning task is transferred to the second GPU to continue to execute the learning task.

Description

Technical Field

[0001] The present application relates to the field of vehicle-mounted data technology, and in particular to a vehicle-mounted video data learning method and device based on the YTS engine AI algorithm. Background Art

[0002] Graphics Processing Unit (GPU), also known as display core, display chip, or video processor, is a coprocessor used to process images and graphics operations. It is widely used in personal computers, workstations, and some mobile devices, such as smartphones and tablets.

[0003] At present, due to limited GPU computing resources and different learning tasks of various models, the learning time of distributed learning varies greatly. Some tasks are completed with delays, while others are completed ahead of time. It is impossible to achieve the overall unified completion of all learning tasks, resulting in low overall execution efficiency of learning tasks. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for learning vehicle video data based on the YTS engine AI algorithm to solve the technical problem of low overall execution efficiency of learning tasks.

[0005] In a first aspect, the present application provides a vehicle video data learning method based on the YTS engine AI algorithm, the method comprising:

[0006] Obtaining an in-vehicle video sample, wherein the in-vehicle video sample corresponds to a plurality of dimensions to be learned;

[0007] Analyzing the features to be learned and the difficulty level of feature learning corresponding to each dimension to be learned of the vehicle video sample by an artificial intelligence (AI) algorithm; wherein the AI ​​algorithm is run by an AI system;

[0008] According to the features to be learned corresponding to each dimension to be learned and the difficulty of learning the features, the AI ​​system allocates the learning tasks corresponding to the vehicle video samples to GPUs with different computing powers for learning; the more difficult the feature learning is, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it;

[0009] Monitoring the current learning level and learning duration of each learning task through the AI ​​system;

[0010] If a first current learning degree of a first learning task executed by the first GPU is greater than a first preset learning degree and a first learning duration of the first learning task is less than a first preset duration, the first learning task is transferred to a second GPU to continue to execute the learning task; wherein the computing power of the second GPU is less than the computing power of the first GPU;

[0011] If the third current learning level of the third learning task executed by the third GPU is less than the second preset learning level and the third learning duration of the third learning task is greater than the second preset duration, the third learning task is transferred to the fourth GPU to continue executing the learning task; wherein the computing power of the third GPU is smaller than the computing power of the fourth GPU; the first preset learning level is greater than the second preset learning level, and the first preset duration is less than the second preset duration.

[0012] In a possible implementation, the vehicle-mounted video sample is driving video data;

[0013] The obtaining of the vehicle-mounted video sample comprises:

[0014] The vehicle-mounted video data is collected through the vehicle-mounted terminal; wherein the vehicle-mounted video data includes early video data and late video data, and the early video data corresponds to a driving video time earlier than the driving video time corresponding to the late video data;

[0015] At least part of the driving video data is transmitted to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal, so that the AI ​​system performs video data processing based on the early video data and the late video data.

[0016] In a possible implementation, the vehicle-mounted terminal corresponds to a specified vehicle-mounted computing power threshold;

[0017] After collecting the driving video data through the vehicle-mounted terminal, the method further includes:

[0018] Analyzing the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned of the driving video data by the vehicle-mounted terminal, and determining the required feature processing computing power according to the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned;

[0019] If the feature processing computing power is less than the specified vehicle computing power threshold, feature extraction is performed on the driving video data through the vehicle-mounted terminal to obtain a feature extraction result, and the extracted features are preprocessed based on the feature extraction result to obtain a feature preprocessing result; wherein the data volume of the feature preprocessing result is less than the data volume of the driving video data;

[0020] The feature preprocessing result is transmitted to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal, so that the AI ​​system continues to process data based on the feature preprocessing result.

[0021] In a possible implementation, there are multiple vehicle-mounted terminals, and the multiple vehicle-mounted terminals include a first vehicle-mounted terminal and a second vehicle-mounted terminal;

[0022] After determining the required feature processing computing power according to the feature extraction amount and the feature preprocessing amount corresponding to each dimension to be learned, the method further includes:

[0023] Monitoring the feature processing computing power corresponding to each of the plurality of vehicle-mounted terminals through the AI ​​system;

[0024] If the first feature processing computing power corresponding to the first vehicle-mounted terminal is greater than or equal to the first designated vehicle-mounted computing power threshold corresponding to the first vehicle-mounted terminal, the feature processing computing power corresponding to the other vehicle-mounted terminals except the first vehicle-mounted terminal among the plurality of vehicle-mounted terminals is compared with the designated vehicle-mounted computing power threshold by the AI ​​system;

[0025] If the second feature processing computing power corresponding to the second vehicle-mounted terminal is less than the second designated vehicle-mounted computing power threshold corresponding to the second vehicle-mounted terminal and the difference between the second feature processing computing power and the second designated vehicle-mounted computing power threshold is greater than the preset difference value, the first driving video data corresponding to the first vehicle-mounted terminal is adjusted to be transmitted to the second vehicle-mounted terminal, so that the second vehicle-mounted terminal performs feature extraction on the first driving video data to obtain a first feature extraction result, and pre-processes the extracted first feature based on the first feature extraction result to obtain a first feature preprocessing result.

[0026] In a possible implementation, transmitting at least part of the driving video data to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal includes:

[0027] Determine the duration of the vehicle video processed by the AI ​​server; the duration of the vehicle video processed is greater than the duration of the previous video corresponding to the previous video data;

[0028] Selecting target late-stage video data from the late-stage video data according to the difference between the duration of the early-stage video and the duration of the processed vehicle-mounted video, so that the duration of the late-stage video corresponding to the target late-stage video data is less than or equal to the difference;

[0029] The target late video data and all the early video data are transmitted to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal.

[0030] In a possible implementation, the AI ​​system allocates the learning tasks corresponding to the vehicle video samples to GPUs with different computing powers for learning according to the features to be learned corresponding to each dimension to be learned and the difficulty of learning the features, including:

[0031] Determine the required target computing power through the AI ​​system according to the features to be learned corresponding to each of the dimensions to be learned and the difficulty of learning the features;

[0032] For each dimension to be learned, matching the target computing power with the GPU computing power corresponding to each GPU through the AI ​​system, and determining a target GPU matching the target computing power from the multiple GPUs;

[0033] The AI ​​system allocates the learning tasks corresponding to the vehicle-mounted video samples to the target GPU for learning.

[0034] In a possible implementation, the multiple dimensions to be learned include any one or more of the following:

[0035] The time dimension, target object dimension, dynamic physical dimension, ray tracing dimension, shape dimension, color dimension and conversation content corresponding intention dimension in the in-vehicle video sample.

[0036] In the second aspect, the present application provides a vehicle-mounted video data learning device based on the YTS engine AI algorithm, comprising:

[0037] An acquisition module, used for acquiring an in-vehicle video sample, wherein the in-vehicle video sample corresponds to a plurality of dimensions to be learned;

[0038] An analysis module, used for analyzing the features to be learned and the difficulty of feature learning corresponding to each dimension to be learned of the vehicle video sample through an AI algorithm; the AI ​​algorithm is run through an AI system;

[0039] an allocation module, for allocating the learning tasks corresponding to the vehicle video samples to GPUs of different computing powers for learning through the AI ​​system according to the features to be learned corresponding to each dimension to be learned and the difficulty of learning the features; the more difficult the feature learning is, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it;

[0040] A monitoring module, used to monitor the current learning level and learning time of each learning task through the AI ​​system;

[0041] A first conversion module is configured to convert the first learning task executed by the first GPU to the second GPU to continue executing the learning task if the first current learning degree of the first learning task executed by the first GPU is greater than the first preset learning degree and the first learning duration of the first learning task is less than the first preset duration; wherein the computing power of the second GPU is less than the computing power of the first GPU;

[0042] A second conversion module is used to convert the third learning task executed by the third GPU to the fourth GPU to continue executing the learning task if the third current learning level of the third learning task executed by the third GPU is less than the second preset learning level and the third learning duration of the third learning task is greater than the second preset duration; wherein the computing power of the third GPU is smaller than the computing power of the fourth GPU; the first preset learning level is greater than the second preset learning level, and the first preset duration is less than the second preset duration.

[0043] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method described in the first aspect.

[0045] This application brings the following beneficial effects:

[0046] The present application provides a vehicle video data learning method and device based on the YTS engine AI algorithm, which can obtain vehicle video samples, wherein the vehicle video samples correspond to multiple dimensions to be learned; the AI ​​algorithm is used to analyze the corresponding features to be learned and the difficulty of feature learning in each dimension to be learned of the vehicle video samples; the AI ​​algorithm is run through the AI ​​system; according to the corresponding features to be learned in each dimension to be learned and the difficulty of feature learning, the learning tasks corresponding to the vehicle video samples are allocated to GPUs with different computing powers for learning through the AI ​​system; the more difficult the feature learning difficulty is, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it; the current learning of each learning task is monitored by the AI ​​system degree and learning duration; if the first current learning degree of the first learning task executed by the first GPU is greater than the first preset learning degree and the first learning duration of the first learning task is less than the first preset duration, the first learning task is transferred to the second GPU to continue to execute the learning task; wherein the computing power of the second GPU is smaller than the computing power of the first GPU; if the third current learning degree of the third learning task executed by the third GPU is less than the second preset learning degree and the third learning duration of the third learning task is greater than the second preset duration, the third learning task is transferred to the fourth GPU to continue to execute the learning task; wherein the computing power of the third GPU is smaller than the computing power of the fourth GPU; the first preset learning degree is greater than the second preset learning degree, and the first preset duration is less than the second preset duration. In this solution, a learning task is only executed by one GPU graphics card at a time, realizing container-based learning, that is, a set of computing power of a GPU graphics card corresponds to one container server, and multiple learning tasks can be switched and executed between different GPU graphics cards according to the task completion status and the computing power of the GPU graphics card, so as to realize the allocation and adjustment of computing resources on demand at any time, reduce the learning time difference of distributed learning, ensure that the learning time of all learning dimensions is close to or even completes the learning tasks at the same time, realize the overall unified completion of learning tasks of all dimensions, and improve the overall execution efficiency of learning tasks.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A flowchart of a vehicle video data learning method based on the YTS engine AI algorithm provided in an embodiment of the present application;

[0050] Figure 2 Another flowchart of the vehicle video data learning method based on the YTS engine AI algorithm provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of a vehicle-mounted video data learning device based on the YTS engine AI algorithm provided in an embodiment of the present application;

[0052] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0054] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0055] At present, due to the limited GPU computing resources and the different learning tasks of each model, the learning time of distributed learning varies greatly. Some are completed with delay, while others are completed in advance, and it is impossible to achieve the overall unified completion of all learning tasks, resulting in low overall execution efficiency of learning tasks. Based on this, the embodiment of the present application provides a vehicle video data learning method and device based on the YTS engine AI algorithm, which can solve the technical problem of low overall execution efficiency of learning tasks.

[0056] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0057] Figure 1 The flowchart of a vehicle video data learning method based on the YTS engine AI algorithm provided in the embodiment of the present application is as follows. Figure 1 As shown, the method includes:

[0058] Step S110, obtaining vehicle-mounted video samples.

[0059] The vehicle video samples correspond to multiple dimensions to be learned, which include any one or more of the following: time dimension, target object dimension, dynamic physics dimension, ray tracing (light effect tracking) dimension, shape dimension, color dimension, and intention dimension corresponding to the conversation content in the vehicle video samples.

[0060] As an optional implementation, the original video may be edited, formatted, etc., to meet the requirements of further analysis. Then, specialized software tools are used to analyze the video content, for example, to identify specific behaviors or events through machine learning algorithms.

[0061] It should be noted that the YTS (Unity TV Service) engine in the embodiment of the present application represents the Unity visual rendering service engine. Unity is a real-time 3D interactive content creation and operation platform. All creators, including game development, art, architecture, car design, and film and television, use Unity to turn their ideas into reality. The platform provides a complete set of software solutions that can be used to create, operate, and realize any real-time interactive 2D and 3D content. Supported platforms include mobile phones, tablets, PCs, game consoles, augmented reality, and virtual reality devices.

[0062] Step S120, analyzing the features to be learned and the difficulty level of feature learning corresponding to each dimension to be learned in the vehicle video sample through an AI algorithm.

[0063] In the embodiment of the present application, the AI ​​algorithm is run by the AI ​​system. In this step, the AI ​​system analyzes the features to be learned and the difficulty of feature learning in each dimension to be learned by the model.

[0064] In one possible implementation, the collected vehicle video samples are first preprocessed, including cropping irrelevant parts, adjusting resolution and frame rate, removing noise, etc. Then, key frames or specific events in the video (such as vehicle lane change, pedestrian crossing the road) are annotated, which will serve as the basis for supervised learning. After that, feature definition is performed. For example, in driving behavior analysis, possible dimensions include safety, efficiency, etc., while specific features may involve vehicle speed changes, following distance, steering angle, etc.

[0065] Then, select the appropriate machine learning or deep learning model based on the nature of the features to be learned. For example, for image recognition tasks, select a convolutional neural network (CNN); for time series analysis (such as speed changes over time), select a long short-term memory network (LSTM). The selected model is then used to extract features from the video sample, including: preliminary training, that is, training the model with annotated data sets so that it can recognize and extract specified features; feature representation, that is, the model outputs a feature vector that represents the performance of the input video clip in various learning dimensions.

[0066] The learning difficulty of each feature can be evaluated in the following ways: Accuracy and error analysis: Evaluate the performance of the model on different features through methods such as cross-validation. High accuracy and low error mean that the feature is easier to learn. Computational resource consumption: Observe the time and computing resources consumed by the model when learning different features. Higher resource consumption may indicate that the feature is more complex and difficult to learn.

[0067] Step S130, according to the corresponding features to be learned in each dimension to be learned and the difficulty of feature learning, the AI ​​system allocates the learning tasks corresponding to the vehicle video samples to GPUs with different computing powers for learning.

[0068] It should be noted that the more difficult the feature learning is in the embodiment of the present application, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it. In this step, according to the features to be learned and the difficulty of feature learning in each dimension, the learning tasks corresponding to the above video samples are allocated to GPU graphics cards with different computing powers for model training through the AI ​​system; the more difficult the feature learning is, the higher the computing power of the GPU graphics card allocated to it; the more features to be learned, the higher the computing power of the GPU graphics card allocated to it.

[0069] For each feature to be learned Fi , its complexity can be defined C ( Fi ). The complexity can be determined based on factors such as the dimension of the data, the degree of nonlinearity, and the noise level: C ( Fi )= f (dimension, nonlinearity, noise level, ...). GPU computing power representation, assuming there is n Different GPUs, the computing power of each GPU can be represented by an indicator Pj It means that this indicator can be a comprehensive reflection of hardware performance indicators such as floating-point operations per second (FLOPS) and memory bandwidth: Pj = g (FLOPS, memory bandwidth, ...)

[0070] For the task time prediction model, given a specific feature Fi and a GPU j , a model can be built to predict the time required to process this feature Tij . This model may rely on empirical data or simulation results: Tij = h ( C ( Fi ), Pj );here h is a function, which may be a complex mathematical model or machine learning model, used to predict the task completion time based on feature complexity and GPU computing power.

[0071] In the embodiment of the present application, the goal of optimizing the allocation of computing resources is to find an allocation scheme that minimizes the total completion time of all tasks or maximizes the utilization of system resources. xij Indicates whether to include the feature i Assign to GPU j (1 for yes, 0 for no), the optimization problem can be formalized as: min∑ Tijxij. Subject to the constraint that each feature can only be assigned to one GPU: ∑ xij =1,∀ i ; The load on each GPU does not exceed its maximum capacity:∑ C ( Fi ) xij ≤ ,∀ j。

[0072] Step S140, monitoring the current learning level and learning time of each learning task through the AI ​​system.

[0073] The AI ​​system monitors the current learning level and learning time of learning tasks in various dimensions. It is a monitoring mechanism for the machine learning or deep learning training process. This mechanism is mainly used to track and evaluate the performance and development progress of different models when performing specific learning tasks.

[0074] As an optional implementation, first prepare the corresponding training data set for the model and ensure that the data has been properly cleaned, converted and annotated. Then, divide the data into training set, validation set and test set to facilitate subsequent model training and performance evaluation. Then select the appropriate model architecture according to the task requirements (such as CNN for image recognition, RNN for sequence data processing, etc.). Then set the initial values ​​of the model parameters, usually using random initialization method. Then use the training data to train the model, and record the learning rate, loss function value and other information of each iteration during the process.

[0075] For the monitoring process, use monitoring tools to track key indicators in the model training process, including but not limited to learning rate, loss value changes, accuracy improvement, etc. Based on the above indicators, calculate the progress of the model in completing a specific learning task. For example, the degree of completion can be estimated by comparing the current loss value with the expected target. Record the training time for each stage, including the time required for a single iteration and the total training time required to achieve the predetermined accuracy. Then, regularly analyze the performance of the model to identify problems such as overfitting or underfitting. Adjust model parameters (such as learning rate), optimize algorithms, or even change the model structure to improve performance based on the analysis results. Through the above steps, the training process of machine learning or deep learning models can be effectively monitored and optimized, thereby accelerating the model development cycle and improving the quality and applicability of the final model.

[0076] Step S150: if the first current learning degree of the first learning task executed by the first GPU is greater than the first preset learning degree and the first learning duration of the first learning task is less than the first preset duration, the first learning task is transferred to the second GPU to continue executing the learning task.

[0077] The computing power of the second GPU is smaller than that of the first GPU. In this step, if the first current learning level of the first learning task executed by the first GPU graphics card is greater than the first preset learning level (such as 99% completed) and the first learning duration of the first learning task is less than the first preset duration, the first learning task is transferred to the second GPU graphics card to continue the learning task; the computing power of the second GPU graphics card is smaller than that of the first GPU graphics card.

[0078] Step S160: If the third current learning level of the third learning task executed by the third GPU is less than the second preset learning level and the third learning duration of the third learning task is greater than the second preset duration, the third learning task is transferred to the fourth GPU to continue executing the learning task.

[0079] The computing power of the third GPU is smaller than that of the fourth GPU. The first preset learning degree is greater than the second preset learning degree, and the first preset duration is less than the second preset duration. In this step, if the third current learning degree of the third learning task executed by the third GPU graphics card is less than the second preset learning degree and the third learning duration of the third learning task is greater than the second preset duration, the third learning task is transferred to the fourth GPU graphics card to continue the learning task; wherein the computing power of the third GPU graphics card is smaller than that of the fourth GPU graphics card.

[0080] In the embodiment of the present application, a learning task is executed by only one GPU graphics card at a time, realizing container-based learning, that is, a set of computing power of a GPU graphics card corresponds to a container server, and multiple learning tasks can be switched and executed between different GPU graphics cards according to the task completion status and the computing power of the GPU graphics card, so as to realize the allocation and adjustment of computing resources on demand at any time, reduce the learning time difference of distributed learning, ensure that the learning time of all learning dimensions is close to or even completes the learning tasks at the same time, realize the overall unified completion of learning tasks of all dimensions, and improve the overall execution efficiency of learning tasks.

[0081] The above steps are described in detail below.

[0082] In some embodiments, the vehicle video sample is driving video data; the above step S110 may specifically include the following steps: collecting driving video data through the vehicle terminal; wherein the vehicle video data includes early video data and late video data, and the driving video time corresponding to the early video data is earlier than the driving video time corresponding to the late video data; transmitting at least part of the driving video data to the AI ​​server corresponding to the AI ​​system through the vehicle terminal, so that the AI ​​system performs video data processing based on the early video data and the late video data. In this way, the efficiency of obtaining vehicle video samples can be improved.

[0083] In some embodiments, the vehicle-mounted terminal corresponds to a specified vehicle-mounted computing power threshold; after the vehicle-mounted terminal collects the driving video data, such as Figure 2 As shown, the method may also include the following steps:

[0084] Step S210, analyzing the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned of the driving video data by the vehicle-mounted terminal, and determining the required feature processing computing power according to the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned;

[0085] Step S220: if the feature processing computing power is less than the specified vehicle computing power threshold, feature extraction is performed on the driving video data through the vehicle terminal to obtain a feature extraction result, and the extracted features are preprocessed based on the feature extraction result to obtain a feature preprocessing result; wherein the data volume of the feature preprocessing result is less than the data volume of the driving video data;

[0086] Step S230, transmitting the feature preprocessing result to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal, so that the AI ​​system continues to process data based on the feature preprocessing result.

[0087] By extracting and preprocessing the driving video data on the vehicle side, the amount of data that needs to be transmitted can be reduced, thereby reducing the pressure and delay of data transmission. At the same time, this method also ensures that only the most important information is sent to the AI ​​server, improving the computing efficiency of the overall system. Moreover, the required feature processing computing power is determined according to each dimension to be learned and compared with the computing power threshold of the vehicle side to decide whether to process on the vehicle side, so as to realize the dynamic allocation of tasks according to computing power demand. In this way, the allocation of computing tasks can be dynamically adjusted according to actual needs, effectively avoiding resource waste, while also ensuring processing speed and efficiency. Furthermore, the amount of data of the feature preprocessing result is smaller than the amount of data of the original driving video data, which means that less data needs to be transmitted from the vehicle side to the AI ​​server side, reducing the amount of data transmission, which not only speeds up the data transmission speed, but also enables the AI ​​system to receive the processed data faster, enhancing the real-time response capability of the system.

[0088] In some embodiments, there are multiple vehicle-mounted terminals, and the multiple vehicle-mounted terminals include a first vehicle-mounted terminal and a second vehicle-mounted terminal; after determining the required feature processing computing power according to the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned, the method may further include the following steps:

[0089] The feature processing computing power corresponding to each of the multiple vehicle-mounted terminals is monitored by the AI ​​system; if the first feature processing computing power corresponding to the first vehicle-mounted terminal is greater than or equal to the first designated vehicle-mounted computing power threshold corresponding to the first vehicle-mounted terminal, the feature processing computing power corresponding to the other vehicle-mounted terminals among the multiple vehicle-mounted terminals except the first vehicle-mounted terminal is compared with the designated vehicle-mounted computing power threshold by the AI ​​system;

[0090] If the second feature processing computing power corresponding to the second vehicle-mounted terminal is less than the second designated vehicle-mounted computing power threshold corresponding to the second vehicle-mounted terminal and the difference between the second feature processing computing power and the second designated vehicle-mounted computing power threshold is greater than the preset phase difference value, the first driving video data corresponding to the first vehicle-mounted terminal is adjusted to be transmitted to the second vehicle-mounted terminal, so that the second vehicle-mounted terminal performs feature extraction on the first driving video data to obtain a first feature extraction result, and pre-processes the extracted first feature based on the first feature extraction result to obtain a first feature preprocessing result.

[0091] In an embodiment of the present application, when the feature processing computing power of a certain vehicle-mounted terminal (such as the first vehicle-mounted terminal) approaches or exceeds its computing power threshold, the system can automatically identify whether other vehicle-mounted terminals (such as the second vehicle-mounted terminal) have sufficient remaining computing power to take over additional tasks. This mechanism can ensure that the computing resources of all vehicle-mounted terminals are fully utilized, avoid the situation where some devices are overloaded while other devices are idle, and achieve dynamic load balancing.

[0092] By redirecting the driving video data of the first vehicle-mounted terminal to the second vehicle-mounted terminal with sufficient computing power for processing, not only can the data processing burden of the first vehicle-mounted terminal be reduced, but also the data processing speed of the entire system can be accelerated, thus ensuring that applications with high real-time requirements (such as autonomous driving, real-time road condition analysis, etc.) can obtain faster response speeds, optimize data transmission and processing processes, and enhance real-time data processing capabilities.

[0093] Moreover, by adjusting task allocation based on actual needs, the task allocation strategy can be flexibly adjusted according to the difference between the current feature processing computing power of each vehicle-mounted terminal and the specified vehicle-mounted computing power threshold. This method not only improves the adaptability of the system, but also better copes with changes in computing needs in different scenarios, thereby improving the system's adaptability and flexibility.

[0094] Furthermore, since preliminary data processing (feature extraction and preprocessing) is performed on the vehicle side, the amount of data that needs to be transmitted to the AI ​​server is reduced, unnecessary data transmission is reduced, and communication costs are reduced. This not only reduces the demand for network bandwidth, but also reduces the load pressure on the back-end server, thereby indirectly reducing the operating cost of the system.

[0095] Through the above processing method, efficient resource management and task scheduling are achieved. By intelligently redistributing computing tasks, the system can improve real-time data processing capabilities while maintaining high efficiency, enhance the adaptability and flexibility of the system, and reduce communication costs. This method is particularly important for building more intelligent and efficient Internet of Vehicles and autonomous driving systems.

[0096] In some embodiments, the above-mentioned transmission of at least part of the driving video data through the vehicle-mounted terminal to the AI ​​server corresponding to the AI ​​system may specifically include the following steps: determining the processing vehicle video duration corresponding to the AI ​​server; the processing vehicle video duration is greater than the early video duration corresponding to the early video data; selecting target late video data from the late video data according to the difference between the early video duration and the processed vehicle video duration, so that the late video duration corresponding to the target late video data is less than or equal to the difference; transmitting the target late video data and all the early video data to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal.

[0097] By selecting and transmitting only the target post-recording data and all the pre-recording data, it is avoided that all the recording data is sent to the AI ​​server indiscriminately, thereby reducing the amount of unnecessary data transmission. This not only saves network bandwidth, but also reduces the communication cost between the vehicle-mounted end and the AI ​​server. Moreover, according to the video duration that the AI ​​server can process (i.e., the processing duration of the vehicle-mounted video), the amount of data to be transmitted is accurately calculated, and the selected data is ensured to meet this limit, so that the processing capacity of the AI ​​server can be maximized, while avoiding processing delays or overload problems caused by excessive data. Furthermore, if the processing capacity of the AI ​​server changes, or the characteristics of the video data on the vehicle-mounted end change, the standards for the pre-recording duration, the processing duration of the vehicle-mounted video, and the selection of the target post-recording data can be adjusted accordingly to adapt to the new needs, so as to achieve flexible adjustment of the transmission strategy according to the actual situation and enhance the flexibility of the system. Furthermore, the transmission of pre-recording data with high real-time requirements is given priority to ensure that these data can be processed in time. This is particularly important for some application scenarios that require rapid response (such as autonomous driving, emergency warning, etc.), which helps to make accurate decisions faster.

[0098] Through intelligent data screening and transmission mechanisms, more efficient data management and processing are achieved. Not only does it significantly reduce the amount of data transmission, it also improves the processing efficiency of the AI ​​server and enhances the overall flexibility of the system and its ability to support real-time analysis. This approach is particularly valuable for building efficient Internet of Vehicles and autonomous driving systems.

[0099] In some embodiments, the above step S130 may specifically include the following steps: determining the required target computing power through the AI ​​system according to the corresponding features to be learned in each dimension to be learned and the difficulty of feature learning; for each dimension to be learned, matching the target computing power with the GPU computing power corresponding to each GPU through the AI ​​system, and determining the target GPU that matches the target computing power from multiple GPUs; and assigning the learning tasks corresponding to the on-board video samples to the target GPU for learning through the AI ​​system.

[0100] In an embodiment of the present application, the required target computing power is determined based on the features in each dimension to be learned and the difficulty of feature learning, and the target computing power is matched with the computing power of each GPU, so as to accurately select the most suitable GPU for each learning task and optimize resource utilization. This method ensures the best utilization of resources and avoids the problem of resource waste or shortage.

[0101] By accurately allocating suitable GPUs for different learning tasks, the model training speed can be significantly accelerated. In particular, for features with high complexity and high learning difficulty, allocating more powerful computing resources can effectively shorten the training time, achieve accurate matching of computing resources, and improve overall learning efficiency.

[0102] Moreover, this mechanism of dynamically adjusting resource allocation allows dynamic adjustment of resource allocation according to different learning task requirements, enhancing the adaptability and flexibility of the system. Faced with tasks of different types and scales, the system can respond flexibly, enhancing the adaptability and flexibility of the system.

[0103] By properly allocating learning tasks to multiple target GPUs, this method supports large-scale parallel processing, helps to process learning tasks of multiple vehicle video samples at the same time, greatly improves data processing capabilities, and achieves efficient parallel computing. Due to the precise matching and efficient use of computing resources, this method not only improves work efficiency, but also helps to reduce unnecessary hardware investment and energy consumption.

[0104] In the embodiment of the present application, the allocation of computing resources is optimized by intelligently analyzing the characteristics of each dimension to be learned and its learning difficulty, thereby improving learning efficiency, enhancing the flexibility and adaptability of the system, and supporting efficient parallel processing. This can provide a more optimized solution for application scenarios that require a large amount of data analysis and model training, such as autonomous driving technology development, large-scale video surveillance analysis, etc.

[0105] Figure 3 A structural diagram of a vehicle-mounted video data learning device based on the YTS engine AI algorithm is provided. Figure 3 As shown, the vehicle-mounted video data learning device 300 based on the YTS engine AI algorithm includes:

[0106] An acquisition module 301 is used to acquire a vehicle video sample, wherein the vehicle video sample corresponds to a plurality of dimensions to be learned;

[0107] An analysis module 302 is used to analyze the features to be learned and the difficulty level of feature learning corresponding to each dimension to be learned of the vehicle-mounted video sample by an AI algorithm; the AI ​​algorithm is run by an AI system;

[0108] The allocation module 303 is used to allocate the learning tasks corresponding to the vehicle video samples to GPUs with different computing powers for learning through the AI ​​system according to the features to be learned corresponding to each dimension to be learned and the difficulty of learning the features; the more difficult the feature learning is, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it;

[0109] A monitoring module 304 is used to monitor the current learning level and learning time of each learning task through the AI ​​system;

[0110] A first conversion module 305 is configured to convert the first learning task executed by the first GPU to the second GPU to continue executing the learning task if the first current learning degree of the first learning task executed by the first GPU is greater than the first preset learning degree and the first learning duration of the first learning task is less than the first preset duration; wherein the computing power of the second GPU is less than the computing power of the first GPU;

[0111] The second conversion module 306 is used to convert the third learning task executed by the third GPU to the fourth GPU to continue executing the learning task if the third current learning level of the third learning task executed by the third GPU is less than the second preset learning level and the third learning duration of the third learning task is greater than the second preset duration; wherein the computing power of the third GPU is smaller than the computing power of the fourth GPU; the first preset learning level is greater than the second preset learning level, and the first preset duration is less than the second preset duration.

[0112] The in-vehicle video data learning device based on the YTS engine AI algorithm provided in the embodiment of the present application has the same technical features as the in-vehicle video data learning method based on the YTS engine AI algorithm provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0113] An electronic device provided in an embodiment of the present application is Figure 4 As shown, the electronic device 400 includes a processor 402 and a memory 401, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method provided in the above embodiment when executing the computer program.

[0114] See also Figure 4 The electronic device further includes: a bus 403 and a communication interface 404, a processor 402, a communication interface 404 and a memory 401 are connected via the bus 403; the processor 402 is used to execute an executable module stored in the memory 401, such as a computer program.

[0115] The memory 401 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 404 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0116] The bus 403 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0117] Among them, the memory 401 is used to store programs, and the processor 402 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 402 or implemented by the processor 402.

[0118] The processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 402. The above processor 402 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 401, and the processor 402 reads the information in the memory 401 and completes the steps of the above method in combination with its hardware.

[0119] Corresponding to the above-mentioned in-vehicle video data learning method based on the YTS engine AI algorithm, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned in-vehicle video data learning method based on the YTS engine AI algorithm.

[0120] The vehicle-mounted video data learning device based on the YTS engine AI algorithm provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, for the parts not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Technical personnel in the relevant field can clearly understand that for the convenience and conciseness of the description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0121] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0122] For another example, the flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0125] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the vehicle-mounted video data learning method based on the YTS engine AI algorithm described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0126] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0127] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A vehicle video data learning method based on the YTS engine AI algorithm, characterized in that: The method comprises: Obtaining an in-vehicle video sample, wherein the in-vehicle video sample corresponds to a plurality of dimensions to be learned; the in-vehicle video sample includes target late-stage video data and all early-stage video data, wherein the target late-stage video data is a portion of late-stage video data selected from all late-stage video data according to a difference between the early-stage video duration corresponding to the early-stage video data and the processed in-vehicle video duration corresponding to the AI ​​system; Analyzing the features to be learned and the difficulty of feature learning corresponding to each dimension to be learned of the vehicle video sample by an AI algorithm; wherein the AI ​​algorithm is run by an AI system; According to the features to be learned corresponding to each dimension to be learned and the difficulty of learning the features, the AI ​​system allocates the learning tasks corresponding to the vehicle video samples to GPUs with different computing powers for learning; the more difficult the feature learning is, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it; Monitoring the current learning level and learning duration of each learning task through the AI ​​system; If a first current learning degree of a first learning task executed by the first GPU is greater than a first preset learning degree and a first learning duration of the first learning task is less than a first preset duration, the first learning task is transferred to a second GPU to continue to execute the learning task; wherein the computing power of the second GPU is less than the computing power of the first GPU; If the third current learning level of the third learning task executed by the third GPU is less than the second preset learning level and the third learning duration of the third learning task is greater than the second preset duration, the third learning task is transferred to the fourth GPU to continue executing the learning task; wherein the computing power of the third GPU is smaller than the computing power of the fourth GPU; the first preset learning level is greater than the second preset learning level, and the first preset duration is less than the second preset duration.

2. The method according to claim 1, characterized in that The obtaining of the vehicle-mounted video sample comprises: The driving video data is collected through the vehicle-mounted terminal; the driving video time corresponding to the early video data is earlier than the driving video time corresponding to the late video data; At least part of the driving video data is transmitted to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal, so that the AI ​​system performs video data processing based on the early video data and the late video data.

3. The method according to claim 2, characterized in that The vehicle-mounted terminal corresponds to a specified vehicle-mounted computing power threshold; After collecting the driving video data through the vehicle-mounted terminal, the method further includes: Analyzing the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned of the driving video data by the vehicle-mounted terminal, and determining the required feature processing computing power according to the feature extraction amount and feature preprocessing amount corresponding to each dimension to be learned; If the feature processing computing power is less than the specified vehicle computing power threshold, feature extraction is performed on the driving video data through the vehicle-mounted terminal to obtain a feature extraction result, and the extracted features are preprocessed based on the feature extraction result to obtain a feature preprocessing result; wherein the data volume of the feature preprocessing result is less than the data volume of the driving video data; The feature preprocessing result is transmitted to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal, so that the AI ​​system continues to process data based on the feature preprocessing result.

4. The method according to claim 3, characterized in that The number of the vehicle-mounted terminals is multiple, and the multiple vehicle-mounted terminals include a first vehicle-mounted terminal and a second vehicle-mounted terminal; After determining the required feature processing computing power according to the feature extraction amount and the feature preprocessing amount corresponding to each dimension to be learned, the method further includes: Monitoring the feature processing computing power corresponding to each of the plurality of vehicle-mounted terminals through the AI ​​system; If the first feature processing computing power corresponding to the first vehicle-mounted terminal is greater than or equal to the first designated vehicle-mounted computing power threshold corresponding to the first vehicle-mounted terminal, the feature processing computing power corresponding to the other vehicle-mounted terminals except the first vehicle-mounted terminal among the plurality of vehicle-mounted terminals is compared with the designated vehicle-mounted computing power threshold by the AI ​​system; If the second feature processing computing power corresponding to the second vehicle-mounted terminal is less than the second designated vehicle-mounted computing power threshold corresponding to the second vehicle-mounted terminal and the difference between the second feature processing computing power and the second designated vehicle-mounted computing power threshold is greater than the preset difference value, the first driving video data corresponding to the first vehicle-mounted terminal is adjusted to be transmitted to the second vehicle-mounted terminal, so that the second vehicle-mounted terminal performs feature extraction on the first driving video data to obtain a first feature extraction result, and pre-processes the extracted first feature based on the first feature extraction result to obtain a first feature preprocessing result.

5. The method according to claim 2, characterized in that: The transmitting at least part of the driving video data through the vehicle-mounted terminal to the AI ​​server corresponding to the AI ​​system includes: Determine the duration of the vehicle video processed by the AI ​​server; the duration of the vehicle video processed is greater than the duration of the previous video corresponding to the previous video data; Selecting target late-stage video data from the late-stage video data according to the difference between the duration of the early-stage video and the duration of the processed vehicle-mounted video, so that the duration of the late-stage video corresponding to the target late-stage video data is less than or equal to the difference; The target late video data and all the early video data are transmitted to the AI ​​server corresponding to the AI ​​system through the vehicle-mounted terminal.

6. The method according to claim 1, characterized in that The method of allocating the learning tasks corresponding to the vehicle-mounted video samples to GPUs with different computing powers for learning by the AI ​​system according to the features to be learned corresponding to each of the dimensions to be learned and the difficulty of learning the features, includes: Determine the required target computing power through the AI ​​system according to the features to be learned corresponding to each of the dimensions to be learned and the difficulty of learning the features; For each dimension to be learned, matching the target computing power with the GPU computing power corresponding to each GPU through the AI ​​system, and determining a target GPU matching the target computing power from the multiple GPUs; The AI ​​system allocates the learning tasks corresponding to the vehicle-mounted video samples to the target GPU for learning.

7. The method according to claim 1, characterized in that The multiple dimensions to be learned include any one or more of the following: The time dimension, target object dimension, dynamic physical dimension, ray tracing dimension, shape dimension, color dimension and conversation content corresponding intention dimension in the in-vehicle video sample.

8. A vehicle video data learning device based on the YTS engine AI algorithm, characterized in that: include: An acquisition module is used to acquire an in-vehicle video sample, wherein the in-vehicle video sample corresponds to a plurality of dimensions to be learned; the in-vehicle video sample includes target late-stage video data and all early-stage video data, and the target late-stage video data is a portion of late-stage video data selected from all late-stage video data according to the difference between the early-stage video duration corresponding to the early-stage video data and the processed in-vehicle video duration corresponding to the AI ​​system; An analysis module, used for analyzing the features to be learned and the difficulty of feature learning corresponding to each dimension to be learned of the vehicle video sample through an AI algorithm; wherein the AI ​​algorithm is run through an AI system; an allocation module, for allocating the learning tasks corresponding to the vehicle video samples to GPUs of different computing powers for learning through the AI ​​system according to the features to be learned corresponding to each dimension to be learned and the difficulty of learning the features; the more difficult the feature learning is, the higher the computing power of the GPU allocated to it; the more features to be learned, the higher the computing power of the GPU allocated to it; A monitoring module, used to monitor the current learning level and learning time of each learning task through the AI ​​system; A first conversion module is configured to convert the first learning task executed by the first GPU to the second GPU to continue executing the learning task if the first current learning degree of the first learning task executed by the first GPU is greater than the first preset learning degree and the first learning duration of the first learning task is less than the first preset duration; wherein the computing power of the second GPU is less than the computing power of the first GPU; A second conversion module is used to convert the third learning task executed by the third GPU to the fourth GPU to continue executing the learning task if the third current learning level of the third learning task executed by the third GPU is less than the second preset learning level and the third learning duration of the third learning task is greater than the second preset duration; wherein the computing power of the third GPU is smaller than the computing power of the fourth GPU; the first preset learning level is greater than the second preset learning level, and the first preset duration is less than the second preset duration.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 7.

Images