Vehicle-mounted video quality evaluation method and device, electronic equipment and storage medium

Abstract

The invention relates to a vehicle-mounted video quality evaluation method and device, electronic equipment and a storage medium. The vehicle-mounted video quality evaluation method comprises the following steps: acquiring to-be-evaluated vehicle-mounted video data; a preset intelligent driving vision algorithm encoder is used for encoding the vehicle-mounted video data to obtain intelligent driving vision algorithm data, and the intelligent driving vision algorithm encoder comprises one or more intelligent driving vision algorithms; and determining a quality evaluation result of the vehicle-mounted video data based on the intelligent driving visual algorithm data and a preset quality evaluation model. According to the embodiment of the invention, the intelligent driving vision algorithm can be used as the basis for evaluating the quality of the vehicle-mounted video, so that the evaluation result of the quality of the vehicle-mounted video data is made on the basis of the intelligent driving vision algorithm, and the image quality of the vehicle-mounted video data is evaluated from the perspective of performance when the intelligent driving vision algorithm is used. And the vehicle-mounted video data meeting the quality requirement can be conveniently screened for the intelligent driving vision algorithm, so that the robustness of the intelligent driving vision algorithm is improved, and the safety of intelligent driving is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and in particular to a vehicle-mounted video quality evaluation method, device, electronic device and storage medium. Background Art

[0002] As we all know, in-car videos have been integrated into daily driving and entertainment systems. The quality of in-car videos will largely affect users' recognition of the vehicle itself when driving or resting.

[0003] At present, there are two main methods for evaluating the quality of in-vehicle videos. The first is to directly evaluate based on relevant attribute indicators of the video, including but not limited to clarity, frame rate, color accuracy, contrast, brightness, stability, compression distortion, dynamic range, noise, smoothness, etc.; the second is to use an artificial intelligence neural network algorithm to regress the subjective evaluation score, aiming to approximate the expression calculation of different people's subjective visual perception to the greatest extent.

[0004] However, the above two methods cannot be associated with the relevant intelligent driving vision algorithm. That is to say, no matter what the results of the vehicle video quality evaluation according to the above two methods are, there may still be a deviation between the actual vehicle video quality and the video quality required by the intelligent driving vision algorithm, which will affect the robustness of the algorithm and have a certain negative impact on driving safety. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a vehicle-mounted video quality evaluation method, device, electronic device and storage medium.

[0006] In a first aspect, the present application provides a method for evaluating vehicle-mounted video quality, comprising:

[0007] Obtain vehicle video data to be evaluated;

[0008] Encoding the vehicle-mounted video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, wherein the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms;

[0009] The quality evaluation result of the on-board video data is determined based on the intelligent driving vision algorithm data and a preset quality evaluation model.

[0010] Optionally, the vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video; the vehicle-mounted video data is encoded using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, including:

[0011] Inputting the first vehicle-mounted video into a preset visual algorithm encoder to obtain a first encoding feature;

[0012] Inputting the second vehicle-mounted video into a preset visual algorithm encoder to obtain a second encoding feature;

[0013] The first coding feature and the second coding feature are determined as the intelligent driving vision algorithm data.

[0014] Optionally, determining a quality evaluation result of the vehicle-mounted video data based on the intelligent driving vision algorithm data and a preset quality evaluation model includes:

[0015] Obtaining deep encoding data of the vehicle-mounted video data;

[0016] Determine fused coding data based on the deep coding data and the intelligent driving vision algorithm data;

[0017] The fused coded data is input into the preset quality evaluation model to obtain the quality evaluation result.

[0018] Optionally, the vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video; obtaining deep encoding data of the vehicle-mounted video data includes:

[0019] Inputting the first vehicle video into a pre-trained VIT encoder to obtain a third encoding feature;

[0020] Inputting a pre-trained VIT encoder into the first vehicle-mounted video to obtain a fourth encoding feature;

[0021] The third encoding feature and the fourth encoding feature are obtained as the deep encoding data.

[0022] Optionally, determining fused coding data based on the deep coding data and the intelligent driving vision algorithm data includes:

[0023] Get evaluation prompt information;

[0024] Using a text encoder to perform text encoding processing on the evaluation prompt information to obtain text encoding features;

[0025] The text encoding features, the deep encoding data and the intelligent driving vision algorithm data are cross-fused to obtain the fused encoding data.

[0026] Optionally, cross-fusion of the text encoding features, the deep encoding data and the intelligent driving vision algorithm data to obtain the fused encoding data includes:

[0027] The text encoding feature, the third encoding feature and the fourth encoding feature in the deep encoding data, and the first encoding feature and the second encoding feature in the intelligent driving vision algorithm data are input into a cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

[0028] Optionally, the cross-fusion module is used to:

[0029] Based on the feature dimension of the text encoding feature, performing a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and performing a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature;

[0030] Based on the feature dimension of the text encoding feature, performing a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and performing a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature;

[0031] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are cross-fused to obtain the fused encoding data.

[0032] Optionally, based on the feature dimension of the text encoding feature, performing a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and performing a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature, including:

[0033] Based on the feature dimension of the text encoding feature, performing convolution processing of a small-kernel convolution kernel, feature flattening processing, and full connection processing on the first encoding feature to obtain the first compressed feature;

[0034] Based on the feature dimension of the text encoding feature, the second encoding feature is subjected to convolution processing with a small-kernel convolution kernel, feature flattening processing, and full connection processing to obtain the second compressed feature.

[0035] Optionally, based on the feature dimension of the text encoding feature, performing a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and performing a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature, including:

[0036] Based on the feature dimension of the text encoding feature, performing feature flattening processing and full connection processing on the third encoding feature to obtain the third compressed feature;

[0037] Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to feature flattening processing and full connection processing to obtain the fourth compression feature.

[0038] Optionally, cross-merging the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature to obtain the fused encoding data includes:

[0039] Get the splicing order;

[0040] According to the splicing order, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are spliced ​​to obtain the fused encoding data.

[0041] Optionally, before obtaining the vehicle-mounted video data to be evaluated, the method further includes:

[0042] Acquire a first in-vehicle training video and first evaluation data of the first in-vehicle training video;

[0043] Acquire a second in-vehicle training video and second evaluation data of the second in-vehicle training video;

[0044] Acquire evaluation prompt training data corresponding to the first in-vehicle training video and the second in-vehicle training video;

[0045] The cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0046] Optionally, the cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, to obtain the trained cross-fusion module and the preset quality evaluation model, including:

[0047] Input the first vehicle-mounted training video into the intelligent driving vision algorithm encoder to obtain a first algorithm encoding feature, and input the second vehicle-mounted training video into the intelligent driving vision algorithm encoder to obtain a second algorithm encoding feature;

[0048] Input the first in-vehicle training video into a pre-trained VIT encoder to obtain a first deep coding feature, and input the second in-vehicle training video into a pre-trained VIT encoder to obtain a second deep coding feature;

[0049] Inputting the evaluation prompt training data into a text encoder to obtain text encoding training features;

[0050] Inputting the first algorithm coding feature, the second algorithm coding feature, the first deep coding feature, the second deep coding feature and the text coding training feature into the cascaded cross-fusion module and the quality evaluation model to obtain first evaluation prediction data of the first in-vehicle training video and second evaluation prediction data of the second in-vehicle training video;

[0051] Based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, the model parameters of the cross-fusion module and the quality evaluation model are adjusted until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0052] In a second aspect, the present application provides a vehicle-mounted video quality evaluation device, comprising:

[0053] A first acquisition module, used to acquire vehicle-mounted video data to be evaluated;

[0054] An encoding module, used to encode the vehicle-mounted video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, wherein the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms;

[0055] A determination module is used to determine the quality evaluation result of the vehicle-mounted video data based on the intelligent driving vision algorithm data and a preset quality evaluation model.

[0056] Optionally, the in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; and the encoding module includes:

[0057] A first encoding submodule, used for inputting the first vehicle-mounted video into a preset visual algorithm encoder to obtain a first encoding feature;

[0058] A second encoding submodule, used for inputting the second vehicle-mounted video into a preset visual algorithm encoder to obtain a second encoding feature;

[0059] The first determination submodule is used to determine the first coding feature and the second coding feature as the intelligent driving vision algorithm data.

[0060] Optionally, the determining module includes:

[0061] A first acquisition submodule is used to acquire deep coding data of the vehicle-mounted video data;

[0062] A second determination submodule is used to determine fused coding data based on the deep coding data and the intelligent driving vision algorithm data;

[0063] The first input submodule is used to input the fused coded data into the preset quality evaluation model to obtain the quality evaluation result.

[0064] Optionally, the in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; and the first acquisition submodule includes:

[0065] A first input unit, used for inputting the first vehicle video into a pre-trained VIT encoder to obtain a third encoding feature;

[0066] A second input unit is used to input a pre-trained VIT encoder into the first vehicle-mounted video to obtain a fourth encoding feature;

[0067] The first acquisition unit is used to acquire the third encoding feature and the fourth encoding feature as the deep encoding data.

[0068] Optionally, the second determining submodule includes:

[0069] A second acquisition unit, used to acquire evaluation prompt information;

[0070] A first encoding unit, configured to perform text encoding processing on the evaluation prompt information using a text encoder to obtain text encoding features;

[0071] A cross-fusion unit is used to cross-fuse the text encoding features, the deep encoding data and the intelligent driving vision algorithm data to obtain the fused encoding data.

[0072] Optionally, the cross-fusion unit includes:

[0073] A cross-fusion subunit is used to input the text encoding feature, the third encoding feature and the fourth encoding feature in the deep encoding data, and the first encoding feature and the second encoding feature in the intelligent driving vision algorithm data into a cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

[0074] Optionally, the cross-fusion subunit includes:

[0075] Based on the feature dimension of the text encoding feature, performing a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and performing a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature;

[0076] Based on the feature dimension of the text encoding feature, performing a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and performing a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature;

[0077] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are cross-fused to obtain the fused encoding data.

[0078] Optionally, the cross-fusion subunit is further used for:

[0079] Based on the feature dimension of the text encoding feature, performing convolution processing of a small-kernel convolution kernel, feature flattening processing, and full connection processing on the first encoding feature to obtain the first compressed feature;

[0080] Based on the feature dimension of the text encoding feature, the second encoding feature is subjected to convolution processing with a small-kernel convolution kernel, feature flattening processing, and full connection processing to obtain the second compressed feature.

[0081] Optionally, the cross-fusion subunit is further used for:

[0082] Based on the feature dimension of the text encoding feature, performing feature flattening processing and full connection processing on the third encoding feature to obtain the third compressed feature;

[0083] Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to feature flattening processing and full connection processing to obtain the fourth compression feature.

[0084] Optionally, the cross-fusion subunit is further used for:

[0085] Get the splicing order;

[0086] According to the splicing order, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are spliced ​​to obtain the fused encoding data.

[0087] Optionally, the device further comprises:

[0088] A second acquisition module is used to acquire a first in-vehicle training video and first evaluation data of the first in-vehicle training video;

[0089] A third acquisition module is used to acquire a second in-vehicle training video and second evaluation data of the second in-vehicle training video;

[0090] A fourth acquisition module, used to acquire evaluation prompt training data corresponding to the first in-vehicle training video and the second in-vehicle training video;

[0091] A training module is used to train a cross-fusion module and an initial quality evaluation model using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0092] Optionally, the training module includes:

[0093] A second input submodule is used to input the first vehicle training video into the intelligent driving vision algorithm encoder to obtain a first algorithm coding feature, and input the second vehicle training video into the intelligent driving vision algorithm encoder to obtain a second algorithm coding feature;

[0094] A third input submodule, used to input the first in-vehicle training video into a pre-trained VIT encoder to obtain a first deep coding feature, and input the second in-vehicle training video into the pre-trained VIT encoder to obtain a second deep coding feature;

[0095] A fourth input submodule, used for inputting the evaluation prompt training data into a text encoder to obtain text encoding training features;

[0096] a fifth input submodule, configured to input the first algorithm encoding feature, the second algorithm encoding feature, the first deep encoding feature, the second deep encoding feature, and the text encoding training feature into the cascaded cross-fusion module and the quality evaluation model to obtain first evaluation prediction data of the first in-vehicle training video and second evaluation prediction data of the second in-vehicle training video;

[0097] A parameter adjustment submodule is used to adjust the model parameters of the cross-fusion module and the quality evaluation model based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0098] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0099] Memory, used to store computer programs;

[0100] The processor is used to implement any of the vehicle-mounted video quality assessment methods described in the first aspect when executing the program stored in the memory.

[0101] In a fourth aspect, the present application provides a computer-readable storage medium, on which a program of a vehicle-mounted video quality assessment method is stored. When the program of the vehicle-mounted video quality assessment method is executed by a processor, the steps of the vehicle-mounted video quality assessment method described in any one of the first aspects are implemented.

[0102] Beneficial effects of the present invention:

[0103] The embodiments of the present application can use the intelligent driving vision algorithm encoder to encode the on-board video data, and then use a preset quality evaluation model to perform quality evaluation on the encoded intelligent driving vision algorithm data, and strongly associate the quality of the on-board video data with the intelligent driving vision algorithm through encoding. The intelligent driving vision algorithm is used as the basis for evaluating the quality of the on-board video, and the evaluation result of the on-board video data quality is made on the basis of the intelligent driving vision algorithm. The image quality of the on-board video data is evaluated from the perspective of the performance when the intelligent driving vision algorithm is used, which is convenient for screening the on-board video data that meets the quality requirements for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm and improving the safety of intelligent driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0105] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0106] Figure 1 A flowchart of a vehicle-mounted video quality evaluation method provided in an embodiment of the present application;

[0107] Figure 2 A schematic diagram of the principle of a vehicle-mounted video quality evaluation method provided in an embodiment of the present application;

[0108] Figure 3 for Figure 1 Flow chart of step S102;

[0109] Figure 4 for Figure 1 Flow chart of step S103;

[0110] Figure 5 for Figure 4 Flow chart of step S301 in FIG.

[0111] Figure 6 for Figure 4Flow chart of step S302;

[0112] Figure 7 A schematic diagram of the principle of a cross-fusion module provided in an embodiment of the present application;

[0113] Figure 8 A flowchart of another vehicle-mounted video quality evaluation method provided in an embodiment of the present application;

[0114] Fig. 9 A structural diagram of a vehicle-mounted video quality assessment device provided in an embodiment of the present application;

[0115] Fig.10 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0116] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0117] In order to solve the technical problem in the prior art that the deviation between the actual on-board video quality and the video quality required by the intelligent driving vision algorithm affects the robustness of the intelligent driving vision algorithm and has a certain adverse impact on the safety during driving, the embodiments of the present application provide a method, device, electronic device and storage medium for evaluating the quality of on-board video data, which can achieve a strong association between the quality of on-board video data and the intelligent driving vision algorithm through encoding, and use the intelligent driving vision algorithm as the basis for evaluating the quality of the on-board video, so that the evaluation result of the on-board video data quality is made on the basis of the intelligent driving vision algorithm, and the image quality of the on-board video data is evaluated from the perspective of the performance when the intelligent driving vision algorithm is used, so as to facilitate the screening of on-board video data that meets the quality requirements for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm and improving the safety of intelligent driving.

[0118] The present application embodiment provides a method for evaluating vehicle video quality. Figure 1 As shown, including:

[0119] Step S101, obtaining vehicle video data to be evaluated;

[0120] In an embodiment of the present application, the vehicle-mounted video data includes data of at least one vehicle-mounted video. The vehicle-mounted video may be collected by a camera set on the vehicle. The vehicle-mounted video may be composed of multiple continuous frames of images collected by the camera, or may be composed of multiple frames of images extracted at the same time interval or different time intervals based on the original video collected by the camera.

[0121] In one embodiment of the present application, the vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video. The first vehicle-mounted video and the second vehicle-mounted video may be videos collected at different times by the same camera, or may be videos collected under different setting parameters of the same camera, or may be videos of the same perspective taken by different cameras, etc.

[0122] Step S102, encoding the vehicle-mounted video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data;

[0123] In an embodiment of the present application, the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms. Exemplarily, the intelligent driving vision algorithm may be a pedestrian detection algorithm, a traffic light recognition algorithm, a lane line detection algorithm, a traffic sign recognition algorithm, and the like.

[0124] In this step, the vehicle-mounted video data is input into the intelligent driving vision algorithm encoder, and the intelligent driving vision algorithm encoder encodes the vehicle-mounted video data to obtain the intelligent driving vision algorithm data.

[0125] In one embodiment of the present application, when the in-vehicle video data includes a first in-vehicle video and a second in-vehicle video, the first in-vehicle video and the second in-vehicle video can be encoded respectively using the intelligent driving vision algorithm, and the encoding results of the two can be used as the intelligent driving vision algorithm data.

[0126] Step S103, determining a quality evaluation result of the vehicle-mounted video data based on the intelligent driving vision algorithm data and a preset quality evaluation model.

[0127] The preset quality evaluation model in the embodiment of the present application is a large language model, such as the qwen2 model. The transformer encoder of this model has only an Encoder structure. The Encoder structure includes a multi-head self-attention structure, a feedforward neural network, a residual connection, etc., and the output layer is composed of a linear layer and a Softmax layer.

[0128] In this step, the intelligent driving vision algorithm data can be input into a preset quality evaluation model so that the preset quality evaluation model outputs the quality evaluation result of the vehicle-mounted video data.

[0129] The embodiments of the present application can use the intelligent driving vision algorithm encoder to encode the on-board video data, and then use a preset quality evaluation model to perform quality evaluation on the encoded intelligent driving vision algorithm data, and strongly associate the quality of the on-board video data with the intelligent driving vision algorithm through encoding. The intelligent driving vision algorithm is used as the basis for evaluating the quality of the on-board video, and the evaluation result of the on-board video data quality is made on the basis of the intelligent driving vision algorithm. The image quality of the on-board video data is evaluated from the perspective of the performance when the intelligent driving vision algorithm is used, which is convenient for screening the on-board video data that meets the quality requirements for the intelligent driving vision algorithm, thereby improving the robustness of the intelligent driving vision algorithm and improving the safety of intelligent driving.

[0130] When designing new automotive products, there are often many products to choose from when it comes to on-board cameras. Each product has its own characteristics. The selection requirements must not only consider the cost, but also the matching and optimization costs of the current visual algorithm. In this regard, it is often necessary to test each model, and to collect data and test in the relevant visual scenes involved in the currently trained algorithm. However, due to the complexity of the scene, it often takes a long time, a lot of manpower and financial resources. In addition, in the long-term research of a car, due to the diversity of actual driving environment scenes, a lot of incremental adaptation work needs to be done on new scenes, and the optimization is achieved by adjusting the relevant parameters of the on-board camera. Conventional solutions often require data collection on the basis of adjusting to a clearer human vision to perform incremental learning on the intelligent driving vision algorithm model to ensure stability. Such work often requires multiple rounds of debugging and training, and still requires a lot of manpower and financial resources. Therefore, in the vehicle camera product evaluation scenario and the vehicle camera optimization scenario, a method is needed to quickly evaluate the quality of different vehicle videos. In another embodiment of the present application, the vehicle video data includes: a first vehicle video and a second vehicle video; step S102 uses a preset intelligent driving vision algorithm encoder to encode the vehicle video data to obtain intelligent driving vision algorithm data, such as Figure 2 and Figure 3 As shown, including:

[0131] Step S201, inputting the first vehicle-mounted video into a preset visual algorithm encoder to obtain a first encoding feature;

[0132] Step S202, inputting the second vehicle-mounted video into a preset visual algorithm encoder to obtain a second encoding feature;

[0133] Step S203: determine the first coding feature and the second coding feature as the intelligent driving vision algorithm data.

[0134] The embodiments of the present application can encode the first vehicle video and the second vehicle video respectively using a preset visual algorithm encoder, so as to perform quality evaluation on the first vehicle video and the second vehicle video based on a preset quality evaluation model based on the intelligent driving visual algorithm data containing the first encoding feature and the second encoding feature, and strongly associate the quality of the first vehicle video and the second vehicle video with the intelligent driving visual algorithm through encoding, and use the intelligent driving visual algorithm as the basis for quality evaluation of the first vehicle video and the second vehicle video, so that the evaluation result of the vehicle video data quality is made on the basis of the intelligent driving visual algorithm, and the image quality of the first vehicle video and the second vehicle video is evaluated from the perspective of the performance when the intelligent driving visual algorithm is used, so as to facilitate the intelligent driving visual algorithm to quickly screen the first vehicle video or the second vehicle video that meets the quality requirements in a short time, without the need for multiple rounds of evaluation, thereby improving the efficiency of vehicle video quality evaluation, and further improving the efficiency of vehicle camera product evaluation and vehicle camera optimization.

[0135] In another embodiment of the present application, step S103 determines the quality evaluation result of the vehicle video data based on the intelligent driving vision algorithm data and a preset quality evaluation model, such as Figure 4 As shown, including:

[0136] Step S301, obtaining deep coding data of the vehicle-mounted video data;

[0137] In the embodiment of the present application, the deep coding data is obtained by encoding the vehicle video data using a pre-trained VIT (Vision Transformer) encoder. The VIT encoder combines new representation learning, feature optimization and feature enhancement technologies, and has better performance in coding efficiency. The VIT encoder integrates the vehicle video data in four dimensions: time, image width, image height, and number of image channels. Here, the sampling frequency is unified and only n frames are selected each time, and then divided into small blocks of fixed size, and then passed through an encoder structure consisting of multiple transformer-blocks containing position encodings, thereby obtaining deep coding data.

[0138] Step S302, determining fused coding data based on the deep coding data and the intelligent driving vision algorithm data;

[0139] Since the encoding methods of encoding the on-board video data into deep encoding data and intelligent driving vision algorithm data are different, the feature dimensions of the encoded deep encoding data and the intelligent driving vision algorithm data are different. The feature dimensions of the deep encoding data output by the VIT encoding algorithm are (X1, Y1) and (X2, Y2), and the feature dimensions of the intelligent driving vision algorithm data output by the intelligent driving vision algorithm encoder are (N1, C1, H1, W1) and (N2, C2, H2, W2). In order to facilitate the subsequent quality evaluation using the preset quality evaluation model, it is necessary to fuse the deep encoding data with the intelligent driving vision algorithm data to obtain fused encoding data.

[0140] Step S303: input the fused coded data into the preset quality evaluation model to obtain the quality evaluation result.

[0141] In this step, the fused coded data may be used as an input of a preset quality assessment model, and the preset quality assessment model outputs a quality assessment result.

[0142] The embodiment of the present application fuses the deep coding data with the intelligent driving vision algorithm data to fuse the deep representation features and the mid-level representation features of the vehicle video data, and uses the fused fused coding data as the input of a preset quality evaluation model, so that the preset quality evaluation model can evaluate the quality of the vehicle video data based on the more dimensional features of the vehicle video data, thereby improving the accuracy of the vehicle video quality evaluation.

[0143] On the basis of the foregoing embodiments, in the vehicle camera product evaluation scenario and the vehicle camera optimization scenario, a method for quickly evaluating the quality of different vehicle videos is needed. To this end, in another embodiment of the present application, when the vehicle video data includes: a first vehicle video and a second vehicle video; step S301 obtains deep encoding data of the vehicle video data, such as Figure 5 and Figure 2 As shown, including:

[0144] Step S401, inputting the first vehicle video into a pre-trained VIT encoder to obtain a third encoding feature;

[0145] Step S402, inputting a pre-trained VIT encoder into the first vehicle-mounted video to obtain a fourth encoding feature;

[0146] Step S403: Acquire the third coding feature and the fourth coding feature as the deep coding data.

[0147] The embodiment of the present application can encode the first vehicle video and the second vehicle video respectively using the VIT encoder, so as to perform quality evaluation on the first vehicle video and the second vehicle video based on a preset quality evaluation model based on the intelligent driving vision algorithm data including the first encoding feature and the second encoding feature. By obtaining deep encoding data including the deep representation features of the first vehicle video and the second vehicle video, it is convenient to fuse the deep representation features and the middle-level representation features of the first vehicle video and the second vehicle video, and use the fused fused encoding data as the input of the preset quality evaluation model, so that the preset quality evaluation model can quickly perform quality evaluation on the first vehicle video and the second vehicle video based on the more dimensional features of the first vehicle video and the second vehicle video, thereby improving the accuracy and efficiency of the vehicle video quality evaluation.

[0148] In another embodiment of the present application, step S302 determines the fused coding data based on the deep coding data and the intelligent driving vision algorithm data, such as Figure 6 and Figure 2 As shown, including:

[0149] Step S501, obtaining evaluation prompt information;

[0150] In the embodiment of the present application, the evaluation prompt information may refer to prompt information when a preset quality evaluation model performs quality evaluation, that is, the preset quality evaluation model may perform quality evaluation on the vehicle-mounted video data according to the evaluation prompt information.

[0151] Evaluation prompt information can be input by the user, for example: "From the perspective of the accuracy of the intelligent driving vision algorithm, which in-vehicle video has higher quality?"

[0152] Step S502, using a text encoder to perform text encoding processing on the evaluation prompt information to obtain text encoding features;

[0153] In an embodiment of the present application, the text encoder includes a word segmentation step based on a Byte Pair Encoding (BPE) algorithm and a hybrid encoding step based on Embedding, and the feature dimension output by the text encoding module is (D1, K1).

[0154] Step S503: cross-fuse the text coding features, the deep coding data and the intelligent driving vision algorithm data to obtain the fused coding data.

[0155] In this step, the text encoding features, deep encoding data and intelligent driving vision algorithm data can be cross-spliced ​​and fused to obtain fused encoding data.

[0156] In one implementation of the present application, step S503 cross-fuses the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data to obtain the fused encoding data, including:

[0157] The text encoding feature, the third encoding feature and the fourth encoding feature in the deep encoding data, and the first encoding feature and the second encoding feature in the intelligent driving vision algorithm data are input into a cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

[0158] After obtaining the fused coded data, in actual scenarios, specific results can be analyzed based on specific situations. For example, if the intelligent driving vision algorithm of interest is the pedestrian detection algorithm, the evaluation prompt information can be changed to "From the perspective of the pedestrian detection algorithm, which video is better and how many points should be given to it?" Similarly, for other intelligent driving vision algorithms, the text output by the preset quality evaluation model will be "The first video is scored 2 points because of *****, and the quality is higher. The second video is also scored 2 points because of *****, and the quality is higher, but the second video is better in details." In this way, a more detailed evaluation of the quality of the in-vehicle video can be made quickly in a shorter time.

[0159] The embodiment of the present application uses a cross-fusion module to fuse the text coding features, the third coding features and the fourth coding features in the deep coding data, and the first coding features and the second coding features in the intelligent driving vision algorithm data, so as to fuse the shallow text coding features, the deep representation features and the middle-level representation features of the vehicle video data, and use the fused fused coding data as the input of the preset quality evaluation model, so that the preset quality evaluation model can quickly evaluate the quality of the vehicle video data based on the more dimensional features of the vehicle video data, thereby improving the accuracy and efficiency of the vehicle video quality evaluation.

[0160] In another embodiment of the present application, Figure 7 As shown, the cross-fusion module is used for:

[0161] 1. Based on the feature dimension of the text encoding feature, perform a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and perform a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature;

[0162] Specifically, based on the feature dimension of the text encoding feature, the first encoding feature can be convolved with a small kernel convolution kernel, feature flattened and fully connected to obtain the first compressed feature; based on the feature dimension of the text encoding feature, the second encoding feature can be convolved with a small kernel convolution kernel, feature flattened and fully connected to obtain the second compressed feature.

[0163] Since the feature dimension of the first coding feature is (N1, C1, H1, W1), the feature dimension of the second coding feature is (N2, C2, H2, W2), and the feature dimension of the text coding feature is (D1, K1), the first coding feature and the second coding feature can be subjected to a first compression and dimensionality reduction process respectively according to the feature dimension of the text coding feature. The goal of compression is to convert the first two features to (D1, K1). Furthermore, convolution processing with a small kernel convolution kernel, feature flattening processing and full connection processing are performed respectively to obtain the first compressed feature and the second compressed feature after compression and dimensionality reduction.

[0164] 2. Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to a second compression and dimensionality reduction process to obtain a third compressed feature, and the fourth encoding feature is subjected to a second compression and dimensionality reduction process to obtain a fourth compressed feature;

[0165] Specifically, based on the feature dimension of the text encoding feature, the third encoding feature can be subjected to feature flattening processing and full connection processing to obtain the third compressed feature; based on the feature dimension of the text encoding feature, the third encoding feature can be subjected to feature flattening processing and full connection processing to obtain the fourth compressed feature.

[0166] Since the feature dimension of the third coding feature is (X1, Y1), the feature dimension of the second coding feature is (X2, Y2), and the feature dimension of the text coding feature is (D1, K1), the third coding feature and the fourth coding feature can be subjected to a second compression and dimensionality reduction process respectively according to the feature dimension of the text coding feature. The goal of the compression is to convert the first two features to (D1, K1). Furthermore, feature flattening and full connection processes are performed respectively to obtain the third compressed feature and the fourth compressed feature after compression and dimensionality reduction.

[0167] 3. Cross-fusing the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature to obtain the fused encoding data.

[0168] Specifically, a splicing order can be obtained; according to the splicing order, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are spliced ​​to obtain the fused encoding data.

[0169] For example, Figure 7 As shown, the concatenation order may be: the third compression feature + the first compression feature + the text encoding feature + the second compression feature + the fourth compression feature.

[0170] The cross-fusion module of the embodiment of the present application can cross-fuse the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature to obtain the fused encoding data, so as to fuse the shallow text encoding features, the deep representation features and the middle representation features of the vehicle video data, and use the fused fused encoding data as the input of the preset quality evaluation model, so that the preset quality evaluation model can perform quality evaluation on the vehicle video data based on the more dimensional features of the vehicle video data, thereby improving the accuracy of the vehicle video quality evaluation.

[0171] In another embodiment of the present application, Figure 8 As shown, before obtaining the vehicle-mounted video data to be evaluated, the method further includes:

[0172] Step S601, obtaining a first in-vehicle training video and first evaluation data of the first in-vehicle training video;

[0173] Step S602, obtaining a second in-vehicle training video and second evaluation data of the second in-vehicle training video;

[0174] Step S603, obtaining evaluation prompt training data corresponding to the first in-vehicle training video and the second in-vehicle training video;

[0175] Step S604, using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data and the evaluation prompt training data to train the cross-fusion module and the initial quality evaluation model until the cross-fusion module and the quality evaluation model converge, to obtain a trained cross-fusion module and the preset quality evaluation model.

[0176] In an embodiment of the present application, the cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, and the trained cross-fusion module and the preset quality evaluation model are obtained, including:

[0177] Input the first vehicle-mounted training video into the intelligent driving vision algorithm encoder to obtain a first algorithm encoding feature, and input the second vehicle-mounted training video into the intelligent driving vision algorithm encoder to obtain a second algorithm encoding feature;

[0178] Input the first in-vehicle training video into a pre-trained VIT encoder to obtain a first deep coding feature, and input the second in-vehicle training video into a pre-trained VIT encoder to obtain a second deep coding feature;

[0179] Inputting the evaluation prompt training data into a text encoder to obtain text encoding training features;

[0180] Inputting the first algorithm coding feature, the second algorithm coding feature, the first deep coding feature, the second deep coding feature and the text coding training feature into the cascaded cross-fusion module and the quality evaluation model to obtain first evaluation prediction data of the first in-vehicle training video and second evaluation prediction data of the second in-vehicle training video;

[0181] Based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, the model parameters of the cross-fusion module and the quality evaluation model are adjusted until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0182] In the aforementioned embodiments of the present application, the intelligent driving vision algorithm, VIT algorithm, and large language model (preset quality evaluation model) are all adopted trained models, and only the cross-fusion module requires actual training. Therefore, only small parameter fine-tuning is required for the entire algorithm. For ease of understanding, the following provides a method for fine-tuning parameters of the cross-fusion module and the preset quality evaluation model in an actual application:

[0183] 1. Data production

[0184] First, we collected enough in-vehicle videos in different scenarios, with each video lasting no longer than 10 seconds. Then we pre-evaluated them manually and scored them according to five levels. Then we randomly grouped the videos with the same scores into two groups and evaluated the visual algorithm. For example, if it is a pedestrian detection algorithm, we used the recall rate and precision of the pedestrian detection results as reference standards to generate comparative descriptive data. Finally, we randomly selected a certain number of groups with different scores and generated comprehensive comparative descriptive data based on the visual algorithm and the scores.

[0185] 2. Loss Function

[0186] In this part, since the difference between the output text description and the actual text evaluation description is still calculated, the cross entropy loss function is still used. The formula is as follows:

[0187]

[0188] Among them, y is the true label, is the probability value predicted by the model.

[0189] 3. Model fine-tuning

[0190] With data and loss function plus Figure 2 After the algorithm network in the model is built, the model can be fine-tuned, where only the weights of the large language model (i.e. the preset quality evaluation model) and the algorithm of the cross-fusion module are updated, and the rest remain unchanged. In order to enhance the stability of training, the simpler data with different scores of manual subjective evaluation are trained in the early stage, using a larger learning rate, and the remaining more difficult data are trained with a smaller learning rate in the later stage.

[0191] In another embodiment of the present application, a vehicle-mounted video quality evaluation device is also provided, such as Fig. 9 As shown, including:

[0192] A first acquisition module 11 is used to acquire vehicle-mounted video data to be evaluated;

[0193] An encoding module 12, configured to encode the vehicle-mounted video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, wherein the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms;

[0194] The determination module 13 is used to determine the quality evaluation result of the vehicle video data based on the intelligent driving vision algorithm data and a preset quality evaluation model.

[0195] Optionally, the in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; and the encoding module includes:

[0196] A first encoding submodule, used for inputting the first vehicle-mounted video into a preset visual algorithm encoder to obtain a first encoding feature;

[0197] A second encoding submodule, used for inputting the second vehicle-mounted video into a preset visual algorithm encoder to obtain a second encoding feature;

[0198] The first determination submodule is used to determine the first coding feature and the second coding feature as the intelligent driving vision algorithm data.

[0199] Optionally, the determining module includes:

[0200] A first acquisition submodule is used to acquire deep coding data of the vehicle-mounted video data;

[0201] A second determination submodule is used to determine fused coding data based on the deep coding data and the intelligent driving vision algorithm data;

[0202] The first input submodule is used to input the fused coded data into the preset quality evaluation model to obtain the quality evaluation result.

[0203] Optionally, the in-vehicle video data includes: a first in-vehicle video and a second in-vehicle video; and the first acquisition submodule includes:

[0204] A first input unit, used for inputting the first vehicle video into a pre-trained VIT encoder to obtain a third encoding feature;

[0205] A second input unit is used to input a pre-trained VIT encoder into the first vehicle-mounted video to obtain a fourth encoding feature;

[0206] The first acquisition unit is used to acquire the third encoding feature and the fourth encoding feature as the deep encoding data.

[0207] Optionally, the second determining submodule includes:

[0208] A second acquisition unit, used to acquire evaluation prompt information;

[0209] A first encoding unit, configured to perform text encoding processing on the evaluation prompt information using a text encoder to obtain text encoding features;

[0210] A cross-fusion unit is used to cross-fuse the text encoding features, the deep encoding data and the intelligent driving vision algorithm data to obtain the fused encoding data.

[0211] Optionally, the cross-fusion unit includes:

[0212] A cross-fusion subunit is used to input the text encoding feature, the third encoding feature and the fourth encoding feature in the deep encoding data, and the first encoding feature and the second encoding feature in the intelligent driving vision algorithm data into a cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

[0213] Optionally, the cross-fusion subunit includes:

[0214] Based on the feature dimension of the text encoding feature, performing a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and performing a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature;

[0215] Based on the feature dimension of the text encoding feature, performing a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and performing a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature;

[0216] The first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are cross-fused to obtain the fused encoding data.

[0217] Optionally, the cross-fusion subunit is further used for:

[0218] Based on the feature dimension of the text encoding feature, performing convolution processing of a small-kernel convolution kernel, feature flattening processing, and full connection processing on the first encoding feature to obtain the first compressed feature;

[0219] Based on the feature dimension of the text encoding feature, the second encoding feature is subjected to convolution processing with a small-kernel convolution kernel, feature flattening processing, and full connection processing to obtain the second compressed feature.

[0220] Optionally, the cross-fusion subunit is further used for:

[0221] Based on the feature dimension of the text encoding feature, performing feature flattening processing and full connection processing on the third encoding feature to obtain the third compressed feature;

[0222] Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to feature flattening processing and full connection processing to obtain the fourth compression feature.

[0223] Optionally, the cross-fusion subunit is further used for:

[0224] Get the splicing order;

[0225] According to the splicing order, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are spliced ​​to obtain the fused encoding data.

[0226] Optionally, the device further comprises:

[0227] A second acquisition module is used to acquire a first in-vehicle training video and first evaluation data of the first in-vehicle training video;

[0228] A third acquisition module is used to acquire a second in-vehicle training video and second evaluation data of the second in-vehicle training video;

[0229] A fourth acquisition module, used to acquire evaluation prompt training data corresponding to the first in-vehicle training video and the second in-vehicle training video;

[0230] A training module is used to train a cross-fusion module and an initial quality evaluation model using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0231] Optionally, the training module includes:

[0232] A second input submodule is used to input the first vehicle training video into the intelligent driving vision algorithm encoder to obtain a first algorithm coding feature, and input the second vehicle training video into the intelligent driving vision algorithm encoder to obtain a second algorithm coding feature;

[0233] A third input submodule, used to input the first in-vehicle training video into a pre-trained VIT encoder to obtain a first deep coding feature, and input the second in-vehicle training video into the pre-trained VIT encoder to obtain a second deep coding feature;

[0234] A fourth input submodule, used for inputting the evaluation prompt training data into a text encoder to obtain text encoding training features;

[0235] a fifth input submodule, configured to input the first algorithm encoding feature, the second algorithm encoding feature, the first deep encoding feature, the second deep encoding feature, and the text encoding training feature into the cascaded cross-fusion module and the quality evaluation model to obtain first evaluation prediction data of the first in-vehicle training video and second evaluation prediction data of the second in-vehicle training video;

[0236] A parameter adjustment submodule is used to adjust the model parameters of the cross-fusion module and the quality evaluation model based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

[0237] In another embodiment of the present application, an electronic device is provided, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0238] Memory, used to store computer programs;

[0239] The processor is used to implement the vehicle-mounted video quality assessment method described in any of the aforementioned method embodiments when executing the program stored in the memory.

[0240] In the electronic device provided by the embodiment of the present invention, the processor realizes that the intelligent driving vision algorithm encoder can be used to encode the vehicle-mounted video data by executing the program stored in the memory, and then the quality of the encoded intelligent driving vision algorithm data is evaluated by using a preset quality evaluation model, and the quality of the vehicle-mounted video data is strongly associated with the intelligent driving vision algorithm through encoding, and the intelligent driving vision algorithm is used as the basis for evaluating the quality of the vehicle-mounted video, so that the evaluation result of the vehicle-mounted video data quality is made on the basis of the intelligent driving vision algorithm, and the image quality of the vehicle-mounted video data is evaluated from the perspective of the performance when the intelligent driving vision algorithm is used, so as to facilitate the intelligent driving vision algorithm to screen the vehicle-mounted video data that meets the quality requirements, thereby improving the robustness of the intelligent driving vision algorithm and improving the safety of intelligent driving.

[0241] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0242] The communication interface 1120 is used for communication between the above electronic device and other devices.

[0243] The memory 1130 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0244] The above-mentioned processor 1110 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0245] In another embodiment of the present application, a computer-readable storage medium is also provided, on which a program of the in-vehicle video quality assessment method is stored. When the program of the in-vehicle video quality assessment method is executed by a processor, the steps of the in-vehicle video quality assessment method described in any of the aforementioned method embodiments are implemented.

[0246] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0247] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle-mounted video quality evaluation method, characterized in that: include: Obtain vehicle video data to be evaluated; Encoding the vehicle-mounted video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, wherein the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms; The quality evaluation result of the on-board video data is determined based on the intelligent driving vision algorithm data and a preset quality evaluation model.

2. The vehicle-mounted video quality evaluation method according to claim 1, characterized in that: The vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video; the vehicle-mounted video data is encoded using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, including: Inputting the first vehicle-mounted video into a preset visual algorithm encoder to obtain a first encoding feature; Inputting the second vehicle-mounted video into a preset visual algorithm encoder to obtain a second encoding feature; The first coding feature and the second coding feature are determined as the intelligent driving vision algorithm data.

3. The vehicle-mounted video quality evaluation method according to claim 1, characterized in that: Determining a quality evaluation result of the vehicle-mounted video data based on the intelligent driving vision algorithm data and a preset quality evaluation model includes: Obtaining deep encoding data of the vehicle-mounted video data; Determine fused coding data based on the deep coding data and the intelligent driving vision algorithm data; The fused coded data is input into the preset quality evaluation model to obtain the quality evaluation result.

4. The vehicle-mounted video quality evaluation method according to claim 3, characterized in that: The vehicle-mounted video data includes: a first vehicle-mounted video and a second vehicle-mounted video; obtaining deep encoding data of the vehicle-mounted video data includes: Inputting the first vehicle video into a pre-trained VIT encoder to obtain a third encoding feature; Inputting a pre-trained VIT encoder into the first vehicle-mounted video to obtain a fourth encoding feature; The third encoding feature and the fourth encoding feature are obtained as the deep encoding data.

5. The vehicle-mounted video quality evaluation method according to claim 3, characterized in that: Determining fused coding data based on the deep coding data and the intelligent driving vision algorithm data includes: Get evaluation prompt information; Using a text encoder to perform text encoding processing on the evaluation prompt information to obtain text encoding features; The text encoding features, the deep encoding data and the intelligent driving vision algorithm data are cross-fused to obtain the fused encoding data.

6. The vehicle-mounted video quality evaluation method according to claim 5, characterized in that: Cross-fusing the text encoding features, the deep encoding data, and the intelligent driving vision algorithm data to obtain the fused encoding data includes: The text encoding feature, the third encoding feature and the fourth encoding feature in the deep encoding data, and the first encoding feature and the second encoding feature in the intelligent driving vision algorithm data are input into a cross-fusion module, so that the cross-fusion module outputs the fused encoding data.

7. The vehicle-mounted video quality evaluation method according to claim 6, characterized in that: The cross-fusion module is used for: Based on the feature dimension of the text encoding feature, performing a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and performing a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature; Based on the feature dimension of the text encoding feature, performing a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and performing a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature; The first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are cross-fused to obtain the fused encoding data.

8. The vehicle-mounted video quality evaluation method according to claim 7, characterized in that: Based on the feature dimension of the text encoding feature, performing a first compression and dimensionality reduction process on the first encoding feature to obtain a first compressed feature, and performing a first compression and dimensionality reduction process on the second encoding feature to obtain a second compressed feature, including: Based on the feature dimension of the text encoding feature, performing convolution processing of a small-kernel convolution kernel, feature flattening processing, and full connection processing on the first encoding feature to obtain the first compressed feature; Based on the feature dimension of the text encoding feature, the second encoding feature is subjected to convolution processing with a small-kernel convolution kernel, feature flattening processing, and full connection processing to obtain the second compressed feature.

9. The vehicle-mounted video quality evaluation method according to claim 7, characterized in that: Based on the feature dimension of the text encoding feature, performing a second compression and dimensionality reduction process on the third encoding feature to obtain a third compressed feature, and performing a second compression and dimensionality reduction process on the fourth encoding feature to obtain a fourth compressed feature, including: Based on the feature dimension of the text encoding feature, performing feature flattening processing and full connection processing on the third encoding feature to obtain the third compressed feature; Based on the feature dimension of the text encoding feature, the third encoding feature is subjected to feature flattening processing and full connection processing to obtain the fourth compression feature.

10. The vehicle-mounted video quality evaluation method according to claim 7, characterized in that: Cross-fusing the first compression feature, the second compression feature, the third compression feature, the fourth compression feature, and the text encoding feature to obtain the fused encoding data, including: Get the splicing order; According to the splicing order, the first compression feature, the second compression feature, the third compression feature, the fourth compression feature and the text encoding feature are spliced ​​to obtain the fused encoding data.

11. The vehicle-mounted video quality evaluation method according to claim 1, characterized in that: Before obtaining the vehicle-mounted video data to be evaluated, the method further includes: Acquire a first in-vehicle training video and first evaluation data of the first in-vehicle training video; Acquire a second in-vehicle training video and second evaluation data of the second in-vehicle training video; Acquire evaluation prompt training data corresponding to the first in-vehicle training video and the second in-vehicle training video; The cross-fusion module and the initial quality evaluation model are trained using the first in-vehicle training video, the first evaluation data, the second in-vehicle training video, the second evaluation data and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

12. The vehicle-mounted video quality evaluation method according to claim 11, characterized in that: The cross-fusion module and the initial quality evaluation model are trained using the first vehicle-mounted training video, the first evaluation data, the second vehicle-mounted training video, the second evaluation data, and the evaluation prompt training data until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model, including: Input the first vehicle-mounted training video into the intelligent driving vision algorithm encoder to obtain a first algorithm encoding feature, and input the second vehicle-mounted training video into the intelligent driving vision algorithm encoder to obtain a second algorithm encoding feature; Input the first in-vehicle training video into a pre-trained VIT encoder to obtain a first deep coding feature, and input the second in-vehicle training video into a pre-trained VIT encoder to obtain a second deep coding feature; Inputting the evaluation prompt training data into a text encoder to obtain text encoding training features; Inputting the first algorithm coding feature, the second algorithm coding feature, the first deep coding feature, the second deep coding feature and the text coding training feature into the cascaded cross-fusion module and the quality evaluation model to obtain first evaluation prediction data of the first in-vehicle training video and second evaluation prediction data of the second in-vehicle training video; Based on the difference between the first evaluation prediction data and the first evaluation data, and the difference between the second evaluation prediction data and the second evaluation data, the model parameters of the cross-fusion module and the quality evaluation model are adjusted until the cross-fusion module and the quality evaluation model converge, thereby obtaining a trained cross-fusion module and the preset quality evaluation model.

13. A vehicle-mounted video quality evaluation device, characterized in that: include: A first acquisition module, used to acquire vehicle-mounted video data to be evaluated; An encoding module, used to encode the vehicle-mounted video data using a preset intelligent driving vision algorithm encoder to obtain intelligent driving vision algorithm data, wherein the intelligent driving vision algorithm encoder includes one or more intelligent driving vision algorithms; A determination module is used to determine the quality evaluation result of the vehicle-mounted video data based on the intelligent driving vision algorithm data and a preset quality evaluation model.

14. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the vehicle-mounted video quality assessment method described in any one of claims 1 to 12 when executing the program stored in the memory.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program of the vehicle-mounted video quality assessment method, and when the program of the vehicle-mounted video quality assessment method is executed by the processor, the steps of the vehicle-mounted video quality assessment method described in any one of claims 1 to 12 are implemented.

Images