On-board video quality evaluation method and apparatus, electronic device and storage medium

Abstract

The present application relates to an on-board video quality evaluation method and apparatus, an electronic device and a storage medium. The on-board video quality evaluation method comprises: acquiring on-board video data to be evaluated; using a preset intelligent driving vision algorithm encoder to encode the on-board video data to obtain intelligent driving vision algorithm data, the intelligent driving vision algorithm encoder comprising one or more intelligent driving vision algorithms; and determining a quality evaluation result of the on-board video data on the basis of the intelligent driving vision algorithm data and a preset quality evaluation model. In embodiments of the present application, the intelligent driving vision algorithm can be used as a basis for evaluating the quality of an on-board video, so that the quality evaluation result of the on-board video data is made on the basis of the intelligent driving vision algorithm, and the image quality of the on-board video data is evaluated from the perspective of performance when the intelligent driving vision algorithm is used, such that on-board video data meeting quality requirements can be screened for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm, and improving the safety of intelligent driving.

Description

Methods, devices, electronic equipment and storage media for evaluating in-vehicle video quality

[0001] This application claims priority to Chinese Patent Application No. 2025100351631, filed on January 9, 2025, entitled "Method, Apparatus, Electronic Device and Storage Medium for Evaluating In-Vehicle Video Quality", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of intelligent driving, and in particular to a method, apparatus, electronic device and storage medium for evaluating the quality of in-vehicle video. Background Technology

[0003] As we all know, in-vehicle video has been integrated into daily driving and entertainment systems. The quality of in-vehicle video greatly affects users' perception of the vehicle itself while driving or resting.

[0004] Currently, there are two main methods for evaluating the quality of in-vehicle video. One is to evaluate it directly based on relevant video attribute indicators, including but not limited to clarity, frame rate, color accuracy, contrast, brightness, stability, compression distortion, dynamic range, noise, and smoothness. The other is to use artificial intelligence neural network algorithms to perform subjective evaluation score regression, aiming to approximate the expression calculation of different people's subjective visual feelings as closely as possible.

[0005] However, neither of the above two methods can be linked to the relevant intelligent driving vision algorithm. In other words, regardless of the results of the in-vehicle video quality evaluation using the above two methods, there may still be a discrepancy between the actual in-vehicle video quality and the video quality required by the intelligent driving vision algorithm. This will affect the robustness of the algorithm and have a certain negative impact on driving safety. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a method, apparatus, electronic device and storage medium for evaluating in-vehicle video quality.

[0007] Firstly, this application provides a method for evaluating the quality of in-vehicle video, including:

[0008] Obtain the in-vehicle video data to be evaluated;

[0009] The in-vehicle video data is encoded using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data. The intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms.

[0010] The quality evaluation result of the in-vehicle video data is determined based on the intelligent driving vision algorithm data and the preset quality evaluation model.

[0011] Optionally, the in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; the in-vehicle video data is encoded using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, including:

[0012] The first vehicle video is input into a preset visual algorithm encoder to obtain the first encoded feature;

[0013] The second vehicle-mounted video is input into a preset visual algorithm encoder to obtain the second encoded feature;

[0014] The first encoded feature and the second encoded feature are determined as the intelligent driving vision algorithm data.

[0015] Optionally, the quality evaluation result of the in-vehicle video data is determined based on the intelligent driving vision algorithm data and a preset quality evaluation model, including:

[0016] Obtain the deep encoded data of the in-vehicle video data;

[0017] Based on the deep encoded data and the intelligent driving vision algorithm data, fused encoded data is determined;

[0018] The fused coded data is input into the preset quality evaluation model to obtain the quality evaluation result.

[0019] Optionally, the vehicle video data includes: a first vehicle video and a second vehicle video; obtaining the deep encoded data of the vehicle video data includes:

[0020] The first vehicle video is input into a pre-trained VIT encoder to obtain the third encoded feature;

[0021] The first vehicle-mounted video is input into a pre-trained VIT encoder to obtain the fourth encoding feature;

[0022] The third and fourth coding features are obtained as the deep coding data.

[0023] Optionally, determining fused coded data based on the deep coded data and the intelligent driving vision algorithm data includes:

[0024] Get evaluation prompts;

[0025] The evaluation prompt information is encoded using a text encoder to obtain text encoding features.

[0026] The text encoding features, the deep encoding data, and the intelligent driving vision algorithm data are cross-fused to obtain the fused encoding data.

[0027] Optionally, the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data are cross-fused to obtain the fused encoding data, including:

[0028] The text encoding features, the third and fourth encoding features in the deep encoding data, and the first and second encoding features in the intelligent driving vision algorithm data are input into the cross-fusion module so that the cross-fusion module outputs the fused encoding data.

[0029] Optionally, the cross-fusion module is used for:

[0030] Based on the feature dimension of the text encoding feature, the first encoding feature is subjected to a first compression and dimensionality reduction process to obtain a first compressed feature, and the second encoding feature is subjected to a first compression and dimensionality reduction process to obtain a second compressed feature;

[0031] Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to a second compression and dimensionality reduction process to obtain the third compressed feature, and the fourth encoding feature is subjected to a second compression and dimensionality reduction process to obtain the fourth compressed feature;

[0032] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data.

[0033] Optionally, based on the feature dimension of the text encoding features, the first encoding features are subjected to a first compression and dimensionality reduction process to obtain a first compressed feature, and the second encoding features are subjected to a first compression and dimensionality reduction process to obtain a second compressed feature, including:

[0034] Based on the feature dimensions of the text encoding features, the first encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the first compressed features;

[0035] Based on the feature dimensions of the text encoding features, the second encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the second compressed features.

[0036] Optionally, based on the feature dimension of the text encoding feature, a second compression and dimensionality reduction process is performed on the third encoding feature to obtain a third compressed feature, and a second compression and dimensionality reduction process is performed on the fourth encoding feature to obtain a fourth compressed feature, including:

[0037] Based on the feature dimensions of the text encoding features, feature flattening and fully connected processing are performed on the third encoding features to obtain the third compressed features;

[0038] Based on the feature dimensions of the text encoding features, the third encoding features are subjected to feature flattening and fully connected processing to obtain the fourth compressed feature.

[0039] Optionally, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data, including:

[0040] Get the splicing order;

[0041] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are concatenated according to the concatenation order to obtain the fused encoded data.

[0042] Optionally, before acquiring the in-vehicle video data to be evaluated, the method further includes:

[0043] Acquire the first vehicle-mounted training video and the first evaluation data of the first vehicle-mounted training video;

[0044] Acquire the second vehicle-mounted training video and the second evaluation data of the second vehicle-mounted training video;

[0045] Obtain the evaluation prompt training data corresponding to the first vehicle training video and the second vehicle training video;

[0046] The cross-fusion module and the initial quality evaluation model are trained using the first vehicle-mounted training video, the first evaluation data, the second vehicle-mounted training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, resulting in the trained cross-fusion module and the preset quality evaluation model.

[0047] Optionally, the cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, resulting in a trained cross-fusion module and the preset quality evaluation model, including:

[0048] The first vehicle training video is input into the intelligent driving vision algorithm encoder to obtain the first algorithm encoding feature, and the second vehicle training video is input into the intelligent driving vision algorithm encoder to obtain the second algorithm encoding feature.

[0049] The first vehicle training video is input into the pre-trained VIT encoder to obtain the first deep coding feature, and the second vehicle training video is input into the pre-trained VIT encoder to obtain the second deep coding feature.

[0050] The evaluation prompt training data is input into the text encoder to obtain the text encoding training features;

[0051] The first algorithm encoding features, the second algorithm encoding features, the first deep encoding features, the second deep encoding features, and the text encoding training features are input into the cascaded cross-fusion module and the quality evaluation model to obtain the first evaluation prediction data of the first vehicle training video and the second evaluation prediction data of the second vehicle training video.

[0052] Based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, the model parameters of the cross-fusion module and the quality evaluation model are adjusted until the cross-fusion module and the quality evaluation model converge, thus obtaining the trained cross-fusion module and the preset quality evaluation model.

[0053] Secondly, this application provides an in-vehicle video quality evaluation device, comprising:

[0054] The first acquisition module is used to acquire the in-vehicle video data to be evaluated;

[0055] The encoding module is used to encode the vehicle video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data. The intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms.

[0056] The determination module is used to determine the quality evaluation result of the in-vehicle video data based on the intelligent driving vision algorithm data and the preset quality evaluation model.

[0057] Optionally, the vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video; the encoding module includes:

[0058] The first encoding submodule is used to input the first vehicle-mounted video into a preset visual algorithm encoder to obtain the first encoded feature;

[0059] The second encoding submodule is used to input the second vehicle-mounted video into a preset visual algorithm encoder to obtain the second encoded feature;

[0060] The first determining submodule is used to determine the first encoded feature and the second encoded feature as the intelligent driving vision algorithm data.

[0061] Optionally, the determining module includes:

[0062] The first acquisition submodule is used to acquire the deep encoded data of the vehicle video data;

[0063] The second determining submodule is used to determine the fused coding data based on the deep coding data and the intelligent driving vision algorithm data;

[0064] The first input submodule is used to input the fused coded data into the preset quality evaluation model to obtain the quality evaluation result.

[0065] Optionally, the vehicle video data includes: a first vehicle video and a second vehicle video; the first acquisition submodule includes:

[0066] The first input unit is used to input the first vehicle-mounted video into a pre-trained VIT encoder to obtain the third encoding feature;

[0067] The second input unit is used to input the first vehicle-mounted video into a pre-trained VIT encoder to obtain the fourth encoding feature.

[0068] The first acquisition unit is used to acquire the third coding feature and the fourth coding feature as the deep coding data.

[0069] Optionally, the second determining submodule includes:

[0070] The second acquisition unit is used to acquire evaluation prompt information;

[0071] The first encoding unit is used to perform text encoding processing on the evaluation prompt information using a text encoder to obtain text encoding features;

[0072] The cross-fusion unit is used to cross-fuse the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data to obtain the fused encoding data.

[0073] Optionally, the cross-fusion unit includes:

[0074] The cross-fusion subunit is used to input the text encoding features, the third and fourth encoding features in the deep encoding data, and the first and second encoding features in the intelligent driving vision algorithm data into the cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

[0075] Optionally, the cross-fusion subunit includes:

[0076] Based on the feature dimension of the text encoding feature, the first encoding feature is subjected to a first compression and dimensionality reduction process to obtain a first compressed feature, and the second encoding feature is subjected to a first compression and dimensionality reduction process to obtain a second compressed feature;

[0077] Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to a second compression and dimensionality reduction process to obtain the third compressed feature, and the fourth encoding feature is subjected to a second compression and dimensionality reduction process to obtain the fourth compressed feature;

[0078] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data.

[0079] Optionally, the cross-fusion subunit is further configured to:

[0080] Based on the feature dimensions of the text encoding features, the first encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the first compressed features;

[0081] Based on the feature dimensions of the text encoding features, the second encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the second compressed features.

[0082] Optionally, the cross-fusion subunit is further configured to:

[0083] Based on the feature dimensions of the text encoding features, feature flattening and fully connected processing are performed on the third encoding features to obtain the third compressed features;

[0084] Based on the feature dimensions of the text encoding features, the third encoding features are subjected to feature flattening and fully connected processing to obtain the fourth compressed feature.

[0085] Optionally, the cross-fusion subunit is further configured to:

[0086] Get the splicing order;

[0087] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are concatenated according to the concatenation order to obtain the fused encoded data.

[0088] Optionally, the device further includes:

[0089] The second acquisition module is used to acquire the first vehicle-mounted training video and the first evaluation data of the first vehicle-mounted training video;

[0090] The third acquisition module is used to acquire the second vehicle-mounted training video and the second evaluation data of the second vehicle-mounted training video;

[0091] The fourth acquisition module is used to acquire the evaluation prompt training data corresponding to the first vehicle training video and the second vehicle training video;

[0092] The training module is used to train the cross-fusion module and the initial quality evaluation model using the first vehicle-mounted training video, the first evaluation data, the second vehicle-mounted training video, the second evaluation data, and the evaluation prompt training data, until the cross-fusion module and the quality evaluation model converge, thus obtaining the trained cross-fusion module and the preset quality evaluation model.

[0093] Optionally, the training module includes:

[0094] The second input submodule is used to input the first vehicle training video into the intelligent driving vision algorithm encoder to obtain the first algorithm encoding feature, and to input the second vehicle training video into the intelligent driving vision algorithm encoder to obtain the second algorithm encoding feature;

[0095] The third input submodule is used to input the first vehicle training video into the pre-trained VIT encoder to obtain the first deep coding feature, and to input the second vehicle training video into the pre-trained VIT encoder to obtain the second deep coding feature.

[0096] The fourth input submodule is used to input the evaluation prompt training data into the text encoder to obtain text encoding training features;

[0097] The fifth input submodule is used to input the first algorithm encoding features, the second algorithm encoding features, the first deep encoding features, the second deep encoding features, and the text encoding training features into the cascaded cross-fusion module and the quality evaluation model to obtain the first evaluation prediction data of the first vehicle training video and the second evaluation prediction data of the second vehicle training video.

[0098] The parameter adjustment submodule is used to adjust the model parameters of the cross-fusion module and the quality evaluation model based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, until the cross-fusion module and the quality evaluation model converge, thereby obtaining the trained cross-fusion module and the preset quality evaluation model.

[0099] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0100] Memory, used to store computer programs;

[0101] The processor, when executing a program stored in memory, implements the vehicle video quality evaluation method described in any of the first aspects.

[0102] Fourthly, this application provides a computer-readable storage medium storing a program for an in-vehicle video quality evaluation method, wherein when the program for the in-vehicle video quality evaluation method is executed by a processor, it implements the steps of the in-vehicle video quality evaluation method described in any of the first aspects.

[0103] The beneficial effects of this application are:

[0104] This application embodiment can use an intelligent driving vision algorithm encoder to encode in-vehicle video data, and then use a preset quality evaluation model to evaluate the quality of the encoded intelligent driving vision algorithm data. The quality of the in-vehicle video data and the intelligent driving vision algorithm are strongly correlated through encoding. The intelligent driving vision algorithm serves as the basis for evaluating the quality of in-vehicle video, so that the evaluation results of the in-vehicle video data quality are based on the intelligent driving vision algorithm. The image quality of the in-vehicle video data is evaluated from the perspective of its performance when the intelligent driving vision algorithm is used. This facilitates the selection of in-vehicle video data that meets the quality requirements for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm and enhancing the safety of intelligent driving. Attached Figure Description

[0105] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0106] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0107] Figure 1 is a flowchart of a vehicle video quality evaluation method provided in an embodiment of this application;

[0108] Figure 2 is a schematic diagram of the principle of an in-vehicle video quality evaluation method provided in an embodiment of this application;

[0109] Figure 3 is a flowchart of step S102 in Figure 1;

[0110] Figure 4 is a flowchart of step S103 in Figure 1;

[0111] Figure 5 is a flowchart of step S301 in Figure 4;

[0112] Figure 6 is a flowchart of step S302 in Figure 4;

[0113] Figure 7 is a schematic diagram of the principle of a cross-fusion module provided in an embodiment of this application;

[0114] Figure 8 is a flowchart of another vehicle-mounted video quality evaluation method provided in an embodiment of this application;

[0115] Figure 9 is a structural diagram of an in-vehicle video quality evaluation device provided in an embodiment of this application;

[0116] Figure 10 is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0117] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0118] To address the technical problem in existing technologies where the discrepancy between the actual quality of in-vehicle video and the video quality required by intelligent driving vision algorithms affects the robustness of these algorithms and negatively impacts driving safety, this application provides an in-vehicle video quality evaluation method, apparatus, electronic device, and storage medium. This method enables a strong correlation between the quality of in-vehicle video data and the intelligent driving vision algorithm through encoding. The intelligent driving vision algorithm serves as the basis for evaluating the quality of in-vehicle video, ensuring that the evaluation results are derived from the performance of the intelligent driving vision algorithm. By evaluating the image quality of the in-vehicle video data from the perspective of its performance during algorithm use, this method facilitates the selection of in-vehicle video data that meets the quality requirements of the intelligent driving vision algorithm, thereby improving the robustness of the algorithm and enhancing the safety of intelligent driving.

[0119] This application provides a method for evaluating the quality of in-vehicle video, as shown in Figure 1, including:

[0120] Step S101: Obtain the in-vehicle video data to be evaluated;

[0121] In this embodiment of the application, the vehicle video data includes at least one vehicle video. The vehicle video can be captured by a camera installed on the vehicle. The vehicle video can be composed of multiple consecutive frames of images captured by the camera, or it can be composed of multiple frames of images extracted at the same or different time intervals based on the original video captured by the camera.

[0122] In one embodiment of this application, the vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video. The first vehicle-mounted video and the second vehicle-mounted video may be videos captured at different times from the same camera, or videos captured by the same camera with different settings parameters, or videos taken from the same angle by different cameras, etc.

[0123] Step S102: Encode the vehicle video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data;

[0124] In this embodiment of the application, the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms. For example, the intelligent driving vision algorithm can be a pedestrian detection algorithm, a traffic light recognition algorithm, a lane line detection algorithm, a traffic sign recognition algorithm, etc.

[0125] In this step, the vehicle video data is input into the intelligent driving vision algorithm encoder, which encodes the vehicle video data to obtain the intelligent driving vision algorithm data.

[0126] In one embodiment of this application, when the vehicle video data includes a first vehicle video and a second vehicle video, the intelligent driving vision algorithm can be used to encode the first vehicle video and the second vehicle video respectively, and the encoded results are used as intelligent driving vision algorithm data.

[0127] Step S103: Determine the quality evaluation result of the in-vehicle video data based on the intelligent driving vision algorithm data and the preset quality evaluation model.

[0128] The preset quality evaluation model in this application embodiment is a large language model, such as the qwen2 model. The transformer encoder of this model only has an encoder structure, which includes a multi-head self-attention structure, a feedforward neural network, residual connections, etc., while the output layer consists of a linear layer and a softmax layer.

[0129] In this step, intelligent driving vision algorithm data can be input into a preset quality evaluation model so that the preset quality evaluation model outputs the quality evaluation result of the in-vehicle video data.

[0130] This application embodiment can use an intelligent driving vision algorithm encoder to encode in-vehicle video data, and then use a preset quality evaluation model to evaluate the quality of the encoded intelligent driving vision algorithm data. The quality of the in-vehicle video data and the intelligent driving vision algorithm are strongly correlated through encoding. The intelligent driving vision algorithm serves as the basis for evaluating the quality of in-vehicle video, so that the evaluation results of the in-vehicle video data quality are based on the intelligent driving vision algorithm. The image quality of the in-vehicle video data is evaluated from the perspective of its performance when the intelligent driving vision algorithm is used. This facilitates the selection of in-vehicle video data that meets the quality requirements for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm and enhancing the safety of intelligent driving.

[0131] When designing new automotive products, there are often many in-vehicle cameras to choose from, each with its own characteristics. Selection criteria must consider not only cost but also the cost of matching and optimizing current visual algorithms. This often requires testing each camera individually, collecting data and testing it in relevant visual scenarios within the currently trained algorithm's operating environment. However, due to the complexity of these scenarios, this process is often time-consuming and requires significant human and financial resources. Furthermore, in long-term research on a vehicle, the diversity of real-world driving environments necessitates extensive incremental adaptation work for new scenarios. This involves adjusting the parameters of the in-vehicle cameras for optimization. Conventional solutions often require adjusting the parameters to a level where human vision is relatively clear before collecting data to incrementally learn the intelligent driving vision algorithm model to ensure stability. This work often involves multiple rounds of debugging and training, still requiring substantial human and financial resources. Therefore, in the scenarios of evaluating and optimizing in-vehicle cameras, a method is needed to quickly evaluate the quality of different in-vehicle videos. In another embodiment of this application, the in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; step S102 uses a preset intelligent driving vision algorithm encoder to encode the in-vehicle video data to obtain intelligent driving vision algorithm data, as shown in Figures 2 and 3, including:

[0132] Step S201: Input the first vehicle video into a preset visual algorithm encoder to obtain the first encoded feature;

[0133] Step S202: Input the second vehicle video into a preset visual algorithm encoder to obtain the second encoded feature;

[0134] Step S203: The first encoded feature and the second encoded feature are determined as the intelligent driving vision algorithm data.

[0135] This application embodiment can encode the first vehicle video and the second vehicle video using a preset visual algorithm encoder, respectively. Based on a preset quality evaluation model and intelligent driving vision algorithm data containing first and second encoded features, the quality of the first and second vehicle videos can be evaluated. The quality of the first and second vehicle videos is strongly correlated with the intelligent driving vision algorithm through encoding. The intelligent driving vision algorithm serves as the basis for evaluating the quality of the first and second vehicle videos, ensuring that the evaluation results of the vehicle video data quality are based on the intelligent driving vision algorithm. The image quality of the first and second vehicle videos is evaluated from the perspective of their performance when used by the intelligent driving vision algorithm. This facilitates the intelligent driving vision algorithm to quickly select the first or second vehicle video that meets quality requirements in a short time, eliminating the need for multiple rounds of evaluation and improving the efficiency of vehicle video quality evaluation. This, in turn, improves the efficiency of vehicle camera product evaluation and vehicle camera optimization.

[0136] In another embodiment of this application, step S103 determines the quality evaluation result of the in-vehicle video data based on the intelligent driving vision algorithm data and the preset quality evaluation model, as shown in Figure 4, including:

[0137] Step S301: Obtain the deep encoded data of the vehicle video data;

[0138] In this embodiment, the deep encoded data is obtained by encoding in-vehicle video data using a pre-trained VIT (Vision Transformer) encoder. The VIT encoder combines novel representation learning, feature optimization, and feature enhancement techniques, resulting in superior encoding efficiency. The VIT encoder integrates in-vehicle video data across four dimensions: time, image width, image height, and number of image channels. It uses a uniform sampling frequency, selecting only n frames at a time, then dividing them into fixed-size blocks. These blocks are then processed through an encoder structure consisting of multiple transformer-blocks containing positional encodings, thus obtaining the deep encoded data.

[0139] Step S302: Determine fused coded data based on the deep coded data and the intelligent driving vision algorithm data;

[0140] Because the encoding methods for vehicle video data into deep encoded data and intelligent driving vision algorithm data are different, the feature dimensions of the encoded deep encoded data and intelligent driving vision algorithm data are different. The feature dimensions of the deep encoded data output by the VIT encoding algorithm are (X1,Y1) and (X2,Y2), while the feature dimensions of the intelligent driving vision algorithm data output by the intelligent driving vision algorithm encoder are (N1,C1,H1,W1) and (N2,C2,H2,W2). In order to facilitate subsequent quality evaluation using a preset quality evaluation model, it is necessary to fuse the deep encoded data and the intelligent driving vision algorithm data to obtain fused encoded data.

[0141] Step S303: Input the fused coded data into the preset quality evaluation model to obtain the quality evaluation result.

[0142] In this step, the fused coded data can be used as input to a preset quality evaluation model, and the preset quality evaluation model outputs the quality evaluation results.

[0143] This application embodiment fuses deep encoded data with intelligent driving vision algorithm data, thereby fusing the deep and mid-level representation features of in-vehicle video data. The fused encoded data is then used as input to a preset quality evaluation model, enabling the preset quality evaluation model to evaluate the quality of in-vehicle video data based on more multidimensional features, thus improving the accuracy of in-vehicle video quality evaluation.

[0144] Based on the foregoing embodiments, in vehicle camera product evaluation and vehicle camera optimization scenarios, a method is needed to quickly evaluate the quality of different vehicle videos. Therefore, in another embodiment of this application, when the vehicle video data includes a first vehicle video and a second vehicle video; step S301 obtains the deep encoded data of the vehicle video data, as shown in Figures 5 and 2, including:

[0145] Step S401: Input the first vehicle video into the pre-trained VIT encoder to obtain the third encoded feature;

[0146] Step S402: Input the first vehicle video into the pre-trained VIT encoder to obtain the fourth encoding feature;

[0147] Step S403: Obtain the third coding feature and the fourth coding feature as the deep coding data.

[0148] This application embodiment can encode the first vehicle video and the second vehicle video using a VIT encoder respectively, so that the quality of the first vehicle video and the second vehicle video can be evaluated based on intelligent driving vision algorithm data containing the first and second encoded features according to a preset quality evaluation model. By obtaining deep encoded data containing the deep representation features of the first vehicle video and the second vehicle video, it is convenient to fuse the deep representation features and mid-level representation features of the first vehicle video and the second vehicle video, and use the fused encoded data as the input of the preset quality evaluation model, so that the preset quality evaluation model can quickly evaluate the quality of the first vehicle video and the second vehicle video based on more multi-dimensional features, thereby improving the accuracy and efficiency of vehicle video quality evaluation.

[0149] In another embodiment of this application, step S302 determines fused coded data based on the deep coded data and the intelligent driving vision algorithm data, as shown in Figures 6 and 2, including:

[0150] Step S501: Obtain evaluation prompt information;

[0151] In this embodiment of the application, the evaluation prompt information may refer to the prompt information when the preset quality evaluation model performs quality evaluation. In other words, the preset quality evaluation model can perform quality evaluation on the vehicle video data according to the evaluation prompt information.

[0152] Evaluation prompts can be entered by the user, such as: "Which in-vehicle video has higher quality from the perspective of the accuracy of the intelligent driving vision algorithm?"

[0153] Step S502: Use a text encoder to perform text encoding processing on the evaluation prompt information to obtain text encoding features;

[0154] In this embodiment of the application, the text encoder includes a word segmentation step based on the Byte Pair Encoding (BPE) algorithm and a hybrid encoding step based on Embedding. The feature dimension output by the text encoding module is (D1, K1).

[0155] Step S503: Cross-fuse the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data to obtain the fused encoding data.

[0156] In this step, text encoding features, deep encoding data, and intelligent driving vision algorithm data can be cross-stitched and fused to obtain fused encoding data.

[0157] In one embodiment of this application, step S503 cross-fuses the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data to obtain the fused encoding data, including:

[0158] The text encoding features, the third and fourth encoding features in the deep encoding data, and the first and second encoding features in the intelligent driving vision algorithm data are input into the cross-fusion module so that the cross-fusion module outputs the fused encoding data.

[0159] After obtaining the fused encoded data, specific results can be analyzed in real-world scenarios. For example, if the focus of the intelligent driving vision algorithm is pedestrian detection, the evaluation prompt can be changed to "From the perspective of pedestrian detection algorithms, which video is better, and what score should it be given?" Similarly, for other intelligent driving vision algorithms, the preset quality evaluation model output text would be "The first video is given 2 points due to *****, indicating high quality; the second video is also given 2 points due to *****, indicating high quality, but the second video is better in detail." In this way, a more detailed evaluation of the quality of in-vehicle video can be quickly achieved in a shorter time.

[0160] This application embodiment uses a cross-fusion module to fuse text coding features, third and fourth coding features in deep coding data, and first and second coding features in intelligent driving vision algorithm data. This fuses shallow text coding features, deep representation features, and mid-level representation features of vehicle video data. The fused coded data is then used as input to a preset quality evaluation model, enabling the model to quickly evaluate the quality of vehicle video data based on more multidimensional features, thereby improving the accuracy and efficiency of vehicle video quality evaluation.

[0161] In another embodiment of this application, as shown in FIG7, the cross-fusion module is used for:

[0162] 1. Based on the feature dimension of the text encoding feature, perform a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and perform a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature;

[0163] Specifically, based on the feature dimensions of the text encoding features, the first encoding features can be processed by convolution with a small kernel, feature flattening, and fully connected to obtain the first compressed feature; based on the feature dimensions of the text encoding features, the second encoding features can be processed by convolution with a small kernel, feature flattening, and fully connected to obtain the second compressed feature.

[0164] Since the feature dimensions of the first encoding feature are (N1, C1, H1, W1), the feature dimensions of the second encoding feature are (N2, C2, H2, W2), and the feature dimensions of the text encoding feature are (D1, K1), the first encoding feature and the second encoding feature can be compressed and reduced according to the feature dimensions of the text encoding feature. The goal of compression is to transform the first two features to (D1, K1). Furthermore, convolution processing with small kernel convolution, feature flattening processing, and fully connected processing are performed respectively to obtain the compressed and reduced first and second compressed features.

[0165] 2. Based on the feature dimension of the text encoding feature, perform a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and perform a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature;

[0166] Specifically, based on the feature dimensions of the text encoding features, the third encoding features can be subjected to feature flattening and fully connected processing to obtain the third compressed features; based on the feature dimensions of the text encoding features, the third encoding features can be subjected to feature flattening and fully connected processing to obtain the fourth compressed features.

[0167] Since the feature dimension of the third encoding feature is (X1, Y1), the feature dimension of the second encoding feature is (X2, Y2), and the feature dimension of the text encoding feature is (D1, K1), the third and fourth encoding features can be compressed and reduced according to the feature dimension of the text encoding feature. The goal of compression is to transform the first two features to (D1, K1). Furthermore, feature flattening and fully connected processing are performed respectively to obtain the compressed and reduced third and fourth compressed features.

[0168] Third, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data.

[0169] Specifically, the splicing order can be obtained; according to the splicing order, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are spliced ​​together to obtain the fused encoded data.

[0170] For example, as shown in Figure 7, the splicing order can be: third compression feature + first compression feature + text encoding feature + second compression feature + fourth compression feature.

[0171] The cross-fusion module in this embodiment can cross-fuse the first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature to obtain the fused encoded data. This fuses the shallow text encoding features, the deep representation features, and the mid-level representation features of the vehicle video data. The fused encoded data is then used as input to a preset quality evaluation model, enabling the model to evaluate the quality of the vehicle video data based on more multidimensional features, thereby improving the accuracy of the vehicle video quality evaluation.

[0172] In another embodiment of this application, as shown in FIG8, before acquiring the vehicle-mounted video data to be evaluated, the method further includes:

[0173] Step S601: Obtain the first vehicle-mounted training video and the first evaluation data of the first vehicle-mounted training video;

[0174] Step S602: Obtain the second vehicle-mounted training video and the second evaluation data of the second vehicle-mounted training video;

[0175] Step S603: Obtain the evaluation prompt training data corresponding to the first vehicle training video and the second vehicle training video;

[0176] Step S604: Use the first vehicle-mounted training video, the first evaluation data, the second vehicle-mounted training video, the second evaluation data, and the evaluation prompt training data to train the cross-fusion module and the initial quality evaluation model until the cross-fusion module and the quality evaluation model converge, and obtain the trained cross-fusion module and the preset quality evaluation model.

[0177] In this embodiment of the application, the cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, resulting in a trained cross-fusion module and the preset quality evaluation model, including:

[0178] The first vehicle training video is input into the intelligent driving vision algorithm encoder to obtain the first algorithm encoding feature, and the second vehicle training video is input into the intelligent driving vision algorithm encoder to obtain the second algorithm encoding feature.

[0179] The first vehicle training video is input into the pre-trained VIT encoder to obtain the first deep coding feature, and the second vehicle training video is input into the pre-trained VIT encoder to obtain the second deep coding feature.

[0180] The evaluation prompt training data is input into the text encoder to obtain the text encoding training features;

[0181] The first algorithm encoding features, the second algorithm encoding features, the first deep encoding features, the second deep encoding features, and the text encoding training features are input into the cascaded cross-fusion module and the quality evaluation model to obtain the first evaluation prediction data of the first vehicle training video and the second evaluation prediction data of the second vehicle training video.

[0182] Based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, the model parameters of the cross-fusion module and the quality evaluation model are adjusted until the cross-fusion module and the quality evaluation model converge, thus obtaining the trained cross-fusion module and the preset quality evaluation model.

[0183] In the foregoing embodiments of this application, the intelligent driving vision algorithm, VIT algorithm, and large language model (preset quality evaluation model) are all pre-trained models. Only the cross-fusion module needs to be actually trained. Therefore, for the entire algorithm scheme, only minor parameter fine-tuning is required. For ease of understanding, the following provides a method for fine-tuning the parameters of the cross-fusion module and the preset quality evaluation model in practical applications:

[0184] 1. Data Production

[0185] First, sufficient in-vehicle videos from different scenarios are collected, each video no longer than 10 seconds. Next, videos are pre-evaluated manually and scored according to five levels. Then, videos with the same score are randomly paired and evaluated using visual algorithms. For example, if it's a pedestrian detection algorithm, the recall and precision of the pedestrian detection results are used as reference standards to generate comparative descriptive data. Finally, a certain number of pairs with different scores are randomly selected, and comprehensive comparative descriptive data is generated based on both the visual algorithm and the score.

[0186] 2. Loss Function

[0187] In this section, since the difference between the calculated output text description and the actual text evaluation description is still the focus, the cross-entropy loss function is still used, as shown in the formula below:

[0188] Where y is the real label. It is the probability value predicted by the model.

[0189] 3. Model fine-tuning

[0190] With the data, loss function, and algorithm network shown in Figure 2, fine-tuning of the model can begin. Only the weights of the large language model (i.e., the pre-defined quality assessment model) and the cross-fusion module are updated; the rest remain unchanged. To enhance training stability, the model is initially trained on simpler data with varying subjective human evaluation scores using a larger learning rate. Later, the more difficult data is trained using a smaller learning rate.

[0191] In another embodiment of this application, an in-vehicle video quality evaluation device is also provided, as shown in FIG9, comprising:

[0192] The first acquisition module 11 is used to acquire the vehicle video data to be evaluated;

[0193] Encoding module 12 is used to encode the vehicle video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data. The intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms.

[0194] The determination module 13 is used to determine the quality evaluation result of the in-vehicle video data based on the intelligent driving vision algorithm data and the preset quality evaluation model.

[0195] Optionally, the vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video; the encoding module includes:

[0196] The first encoding submodule is used to input the first vehicle-mounted video into a preset visual algorithm encoder to obtain the first encoded feature;

[0197] The second encoding submodule is used to input the second vehicle-mounted video into a preset visual algorithm encoder to obtain the second encoded feature;

[0198] The first determining submodule is used to determine the first encoded feature and the second encoded feature as the intelligent driving vision algorithm data.

[0199] Optionally, the determining module includes:

[0200] The first acquisition submodule is used to acquire the deep encoded data of the vehicle video data;

[0201] The second determining submodule is used to determine the fused coding data based on the deep coding data and the intelligent driving vision algorithm data;

[0202] The first input submodule is used to input the fused coded data into the preset quality evaluation model to obtain the quality evaluation result.

[0203] Optionally, the vehicle video data includes: a first vehicle video and a second vehicle video; the first acquisition submodule includes:

[0204] The first input unit is used to input the first vehicle-mounted video into a pre-trained VIT encoder to obtain the third encoding feature;

[0205] The second input unit is used to input the first vehicle-mounted video into a pre-trained VIT encoder to obtain the fourth encoding feature.

[0206] The first acquisition unit is used to acquire the third coding feature and the fourth coding feature as the deep coding data.

[0207] Optionally, the second determining submodule includes:

[0208] The second acquisition unit is used to acquire evaluation prompt information;

[0209] The first encoding unit is used to perform text encoding processing on the evaluation prompt information using a text encoder to obtain text encoding features;

[0210] The cross-fusion unit is used to cross-fuse the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data to obtain the fused encoding data.

[0211] Optionally, the cross-fusion unit includes:

[0212] The cross-fusion subunit is used to input the text encoding features, the third and fourth encoding features in the deep encoding data, and the first and second encoding features in the intelligent driving vision algorithm data into the cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

[0213] Optionally, the cross-fusion subunit includes:

[0214] Based on the feature dimension of the text encoding feature, the first encoding feature is subjected to a first compression and dimensionality reduction process to obtain a first compressed feature, and the second encoding feature is subjected to a first compression and dimensionality reduction process to obtain a second compressed feature;

[0215] Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to a second compression and dimensionality reduction process to obtain the third compressed feature, and the fourth encoding feature is subjected to a second compression and dimensionality reduction process to obtain the fourth compressed feature;

[0216] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data.

[0217] Optionally, the cross-fusion subunit is further configured to:

[0218] Based on the feature dimensions of the text encoding features, the first encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the first compressed features;

[0219] Based on the feature dimensions of the text encoding features, the second encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the second compressed features.

[0220] Optionally, the cross-fusion subunit is further configured to:

[0221] Based on the feature dimensions of the text encoding features, feature flattening and fully connected processing are performed on the third encoding features to obtain the third compressed features;

[0222] Based on the feature dimensions of the text encoding features, the third encoding features are subjected to feature flattening and fully connected processing to obtain the fourth compressed feature.

[0223] Optionally, the cross-fusion subunit is further configured to:

[0224] Get the splicing order;

[0225] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are concatenated according to the concatenation order to obtain the fused encoded data.

[0226] Optionally, the device further includes:

[0227] The second acquisition module is used to acquire the first vehicle-mounted training video and the first evaluation data of the first vehicle-mounted training video;

[0228] The third acquisition module is used to acquire the second vehicle-mounted training video and the second evaluation data of the second vehicle-mounted training video;

[0229] The fourth acquisition module is used to acquire the evaluation prompt training data corresponding to the first vehicle training video and the second vehicle training video;

[0230] The training module is used to train the cross-fusion module and the initial quality evaluation model using the first vehicle-mounted training video, the first evaluation data, the second vehicle-mounted training video, the second evaluation data, and the evaluation prompt training data, until the cross-fusion module and the quality evaluation model converge, thus obtaining the trained cross-fusion module and the preset quality evaluation model.

[0231] Optionally, the training module includes:

[0232] The second input submodule is used to input the first vehicle training video into the intelligent driving vision algorithm encoder to obtain the first algorithm encoding feature, and to input the second vehicle training video into the intelligent driving vision algorithm encoder to obtain the second algorithm encoding feature;

[0233] The third input submodule is used to input the first vehicle training video into the pre-trained VIT encoder to obtain the first deep coding feature, and to input the second vehicle training video into the pre-trained VIT encoder to obtain the second deep coding feature.

[0234] The fourth input submodule is used to input the evaluation prompt training data into the text encoder to obtain text encoding training features;

[0235] The fifth input submodule is used to input the first algorithm encoding features, the second algorithm encoding features, the first deep encoding features, the second deep encoding features, and the text encoding training features into the cascaded cross-fusion module and the quality evaluation model to obtain the first evaluation prediction data of the first vehicle training video and the second evaluation prediction data of the second vehicle training video.

[0236] The parameter adjustment submodule is used to adjust the model parameters of the cross-fusion module and the quality evaluation model based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, until the cross-fusion module and the quality evaluation model converge, thereby obtaining the trained cross-fusion module and the preset quality evaluation model.

[0237] In another embodiment of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0238] Memory, used to store computer programs;

[0239] When the processor executes the program stored in the memory, it implements the vehicle video quality evaluation method described in any of the foregoing method embodiments.

[0240] The electronic device provided in this application embodiment enables the processor to encode in-vehicle video data using an intelligent driving vision algorithm encoder by executing a program stored in the memory. Then, it uses a preset quality evaluation model to evaluate the quality of the encoded intelligent driving vision algorithm data. This strongly correlates the quality of the in-vehicle video data with the intelligent driving vision algorithm through encoding, using the intelligent driving vision algorithm as the basis for evaluating the quality of the in-vehicle video. The evaluation result of the in-vehicle video data quality is based on the intelligent driving vision algorithm, evaluating the image quality of the in-vehicle video data from the perspective of its performance when used. This facilitates the selection of in-vehicle video data that meets the quality requirements for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm and enhancing the safety of intelligent driving.

[0241] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in Figure 10, but this does not indicate that there is only one bus or one type of bus.

[0242] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.

[0243] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0244] The processor 1110 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0245] In another embodiment of this application, a computer-readable storage medium is provided, on which a program for an in-vehicle video quality evaluation method is stored. When the program for the in-vehicle video quality evaluation method is executed by a processor, it implements the steps of the in-vehicle video quality evaluation method described in any of the foregoing method embodiments.

[0246] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0247] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for evaluating the quality of in-vehicle video, characterized in that, include: Obtain the in-vehicle video data to be evaluated; The in-vehicle video data is encoded using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data. The intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms. The quality evaluation result of the in-vehicle video data is determined based on the intelligent driving vision algorithm data and the preset quality evaluation model.

2. The vehicle-mounted video quality evaluation method according to claim 1, characterized in that, The in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; the in-vehicle video data is encoded using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, including: The first vehicle video is input into a preset visual algorithm encoder to obtain the first encoded feature; The second vehicle-mounted video is input into a preset visual algorithm encoder to obtain the second encoded feature; The first encoded feature and the second encoded feature are determined as the intelligent driving vision algorithm data.

3. The method for evaluating the quality of in-vehicle video according to claim 1 or 2, characterized in that, The quality evaluation result of the in-vehicle video data is determined based on the intelligent driving vision algorithm data and the preset quality evaluation model, including: Obtain the deep encoded data of the in-vehicle video data; Based on the deep encoded data and the intelligent driving vision algorithm data, fused encoded data is determined; The fused coded data is input into the preset quality evaluation model to obtain the quality evaluation result.

4. The vehicle-mounted video quality evaluation method according to claim 3, characterized in that, The in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; obtaining the deep encoded data of the in-vehicle video data includes: The first vehicle video is input into a pre-trained VIT encoder to obtain the third encoded feature; The first vehicle-mounted video is input into a pre-trained VIT encoder to obtain the fourth encoding feature; The third and fourth coding features are obtained as the deep coding data.

5. The vehicle-mounted video quality evaluation method according to claim 3 or 4, characterized in that, Based on the deep encoded data and the intelligent driving vision algorithm data, fused encoded data is determined, including: Get evaluation prompts; The evaluation prompt information is encoded using a text encoder to obtain text encoding features. The text encoding features, the deep encoding data, and the intelligent driving vision algorithm data are cross-fused to obtain the fused encoding data.

6. The vehicle-mounted video quality evaluation method according to claim 5, characterized in that, The text encoding features, the deep encoding data, and the intelligent driving vision algorithm data are cross-fused to obtain the fused encoding data, including: The text encoding features, the third and fourth encoding features in the deep encoding data, and the first and second encoding features in the intelligent driving vision algorithm data are input into the cross-fusion module so that the cross-fusion module outputs the fused encoding data.

7. The vehicle-mounted video quality evaluation method according to claim 6, characterized in that, The cross-fusion module is used for: Based on the feature dimension of the text encoding feature, the first encoding feature is subjected to a first compression and dimensionality reduction process to obtain a first compressed feature, and the second encoding feature is subjected to a first compression and dimensionality reduction process to obtain a second compressed feature; Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to a second compression and dimensionality reduction process to obtain the third compressed feature, and the fourth encoding feature is subjected to a second compression and dimensionality reduction process to obtain the fourth compressed feature; The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data.

8. The vehicle-mounted video quality evaluation method according to claim 7, characterized in that, Based on the feature dimension of the text encoding features, the first encoding features are subjected to a first compression and dimensionality reduction process to obtain a first compressed feature, and the second encoding features are subjected to a first compression and dimensionality reduction process to obtain a second compressed feature, including: Based on the feature dimensions of the text encoding features, the first encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the first compressed features; Based on the feature dimensions of the text encoding features, the second encoding features are subjected to convolution processing with small kernel convolution, feature flattening, and fully connected processing to obtain the second compressed features.

9. The vehicle-mounted video quality evaluation method according to claim 7, characterized in that, Based on the feature dimensions of the text encoding features, the third encoding features are subjected to a second compression and dimensionality reduction process to obtain a third compressed feature. Similarly, the fourth encoding features are subjected to a second compression and dimensionality reduction process to obtain a fourth compressed feature, including: Based on the feature dimensions of the text encoding features, feature flattening and fully connected processing are performed on the third encoding features to obtain the third compressed features; Based on the feature dimensions of the text encoding features, the third encoding features are subjected to feature flattening and fully connected processing to obtain the fourth compressed feature.

10. The vehicle-mounted video quality evaluation method according to claim 7, characterized in that, The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are cross-fused to obtain the fused encoded data, including: Get the splicing order; The first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature are concatenated according to the concatenation order to obtain the fused encoded data.

11. The method for evaluating the quality of in-vehicle video according to claim 1 or 2, characterized in that, Before acquiring the in-vehicle video data to be evaluated, the method further includes: Acquire the first vehicle-mounted training video and the first evaluation data of the first vehicle-mounted training video; Acquire the second vehicle-mounted training video and the second evaluation data of the second vehicle-mounted training video; Obtain the evaluation prompt training data corresponding to the first vehicle training video and the second vehicle training video; The cross-fusion module and the initial quality evaluation model are trained using the first vehicle-mounted training video, the first evaluation data, the second vehicle-mounted training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, resulting in the trained cross-fusion module and the preset quality evaluation model.

12. The vehicle-mounted video quality evaluation method according to claim 11, characterized in that, The cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, resulting in a trained cross-fusion module and the preset quality evaluation model, including: The first vehicle training video is input into the intelligent driving vision algorithm encoder to obtain the first algorithm encoding feature, and the second vehicle training video is input into the intelligent driving vision algorithm encoder to obtain the second algorithm encoding feature. The first vehicle training video is input into the pre-trained VIT encoder to obtain the first deep coding feature, and the second vehicle training video is input into the pre-trained VIT encoder to obtain the second deep coding feature. The evaluation prompt training data is input into the text encoder to obtain the text encoding training features; The first algorithm encoding features, the second algorithm encoding features, the first deep encoding features, the second deep encoding features, and the text encoding training features are input into the cascaded cross-fusion module and the quality evaluation model to obtain the first evaluation prediction data of the first vehicle training video and the second evaluation prediction data of the second vehicle training video. Based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, the model parameters of the cross-fusion module and the quality evaluation model are adjusted until the cross-fusion module and the quality evaluation model converge, thus obtaining the trained cross-fusion module and the preset quality evaluation model.

13. A vehicle-mounted video quality evaluation device, characterized in that, include: The first acquisition module is used to acquire the in-vehicle video data to be evaluated; The encoding module is used to encode the vehicle video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data. The intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms. The determination module is used to determine the quality evaluation result of the in-vehicle video data based on the intelligent driving vision algorithm data and the preset quality evaluation model.

14. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the vehicle video quality evaluation method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for an in-vehicle video quality evaluation method, which, when executed by a processor, implements the steps of the in-vehicle video quality evaluation method according to any one of claims 1 to 12.

Images