Image quality inspection method, device, vehicle and storage medium for vehicle-mounted video transmission

Abstract

The embodiment of the present invention relates to the field of image processing technology, and discloses an image quality inspection method for vehicle-mounted video transmission, the method comprising: obtaining original video stream data from a transmitting end in real time, and extracting each original video frame image of the original video stream data; obtaining multiple second video frame images from a receiving end in real time; identifying the multiple second video frame images, obtaining second identification information corresponding to each second video frame image and multiple third video frame images after de-identification; determining the first image transmission quality according to the difference between each first identification information and the corresponding second identification information; inputting each original video frame image and each third video frame image into a preset image quality assessment model, respectively, to obtain the second image transmission quality; determining the target image transmission quality according to the first image transmission quality and the second image transmission quality. The embodiment of the present invention realizes accurate and efficient evaluation of image quality.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and specifically to an image quality inspection method, device, vehicle and storage medium for vehicle-mounted video transmission. Background Art

[0002] At present, the development of vehicle instruments and navigation has entered the era of intelligence. With the development of automobile technology, the transmission and display of images in vehicles are becoming more and more important. The cockpit host of the car transmits the display signal to the receiving end through the transmitter, and the receiving end transmits the display signal to the cockpit display for display.

[0003] However, the inventors of the present application have discovered that the process of sending and receiving video images in existing vehicles is susceptible to interference, thereby affecting the quality of images received by the receiving end and causing hidden dangers to vehicle driving safety.

[0004] Therefore, there is an urgent need for a quality detection method for vehicle-mounted video transmission to ensure driving safety. Summary of the invention

[0005] In view of the above problems, an embodiment of the present invention provides an image quality inspection method, device, vehicle and storage medium for in-vehicle video transmission, which is used to solve the problem in the prior art that the sending and receiving process of video images in existing vehicles is easily interfered with, thereby affecting the image quality received by the receiving end, causing hidden dangers to vehicle driving safety, and poor quality detection effect and low efficiency in the in-vehicle video transmission process.

[0006] According to one aspect of an embodiment of the present invention, there is provided an image quality inspection method for vehicle-mounted video transmission, the method comprising:

[0007] Acquire original video stream data from the transmitting end in real time, and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the transmitting end;

[0008] Acquire multiple second video frame images from the receiving end in real time; the second video frame images are obtained by the sending end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain multiple continuous first video frame images, and encapsulating the multiple first video frame images according to a preset image transmission protocol;

[0009] Performing identification on the plurality of second video frame images to obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being removed from the identification;

[0010] determining the first image transmission quality according to the difference between each of the first identification information and the corresponding each of the second identification information;

[0011] Input each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain a second image transmission quality; the image quality assessment model is obtained by pre-training a neural network model based on image quality assessment samples;

[0012] A target image transmission quality is determined according to the first image transmission quality and the second image transmission quality.

[0013] In an optional manner, before acquiring multiple second video frame images from the receiving end in real time, the method further includes: acquiring each of the first identification information from the sending end; and matching the first identification information with each of the original video frame images.

[0014] In an optional manner, each of the first identification information includes frame sequence information and time information corresponding to the respective original video frame;

[0015] The real-time acquisition of original video stream data from the transmitting end and extraction of each original video frame image of the original video stream data includes:

[0016] Extracting the original video frame image corresponding to the time and frame number from the original video stream data according to the frame number information and the time information;

[0017] The step of adding first identification information to each original video frame image of the original video stream data to obtain a plurality of continuous first video frame images includes:

[0018] Determine the original video frame image corresponding to the frame sequence number and time from the original video stream data;

[0019] The corresponding frame number information and time information are added to the original video frame image of the corresponding frame number and time.

[0020] In an optional manner, after the identification of the plurality of second video frame images is performed to obtain the second identification information corresponding to each second video frame image and a plurality of third video frame images after being de-identified, the method further includes:

[0021] According to the frame sequence number information and time information, the extracted second identification information is matched with the first identification information to obtain the second identification information corresponding to each of the first identification information and the third video frame image corresponding to each of the original video frame images.

[0022] In an optional manner, the method further includes: identifying key frames in the original video stream data;

[0023] The sequence number information and time information corresponding to the key frame are determined as the first identification information.

[0024] In an optional manner, before inputting each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain the second image transmission quality, the method further includes:

[0025] Acquire image quality assessment samples, wherein the image quality assessment samples include original image samples, corresponding multiple low-quality image samples and quality score labels, wherein the low-quality image samples are low-quality image samples obtained after performing data processing of various image types;

[0026] Inputting the image quality assessment sample into a convolutional neural network to obtain a predicted quality score; the convolutional neural network includes a feature extraction layer and a quality assessment module; the quality assessment module includes a PSNR algorithm, a mean square error, a SSIM algorithm, and a FSIM algorithm;

[0027] The loss is calculated according to the predicted quality score and the quality score label, and the parameters of the convolutional neural network are adjusted. The steps of inputting the image quality assessment sample into the convolutional neural network to obtain the predicted quality score, and calculating the loss according to the predicted quality score and the quality score label, and adjusting the parameters of the convolutional neural network are re-executed until the loss is less than a preset loss threshold, then the training is stopped to obtain the image quality assessment model.

[0028] In an optional manner, determining the target image transmission quality according to the first image transmission quality and the second image transmission quality includes: performing a weighted average on the first image transmission quality and the second image transmission quality to obtain the target image transmission quality.

[0029] According to another aspect of an embodiment of the present invention, there is provided an image quality inspection device for vehicle-mounted video transmission, comprising:

[0030] The first acquisition module is used to acquire original video stream data from the sending end in real time and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the sending end;

[0031] A second acquisition module is used to acquire multiple second video frame images from the receiving end in real time; the second video frame images are obtained by the sending end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain multiple continuous first video frame images, and encapsulating the multiple first video frame images according to a preset image transmission protocol;

[0032] An identification module, used for identifying the plurality of second video frame images, obtaining second identification information corresponding to each second video frame image and a plurality of third video frame images after being de-identified;

[0033] A first determining module, configured to determine a first image transmission quality according to a difference between each of the first identification information and each of the corresponding second identification information;

[0034] A model evaluation module, used to input each of the original video frame images and each of the third video frame images into a preset image quality evaluation model, respectively, to obtain the second image transmission quality; the image quality evaluation model is obtained by training a neural network model in advance according to image quality evaluation samples;

[0035] An output module is used to determine a target image transmission quality according to the first image transmission quality and the second image transmission quality.

[0036] According to another aspect of an embodiment of the present invention, a vehicle is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0037] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the image quality inspection method for vehicle-mounted video transmission.

[0038] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium. When the executable instruction is executed on a processor of a vehicle, the processor executes the operation of the image quality inspection method for in-vehicle video transmission.

[0039] The embodiment of the present invention obtains original video stream data from a transmitting end in real time, and obtains multiple second video frame images from a receiving end in real time; identifies the multiple second video frame images to obtain second identification information corresponding to each second video frame image and multiple de-identified third video frame images; determines the first image transmission quality according to the difference between each first identification information and the corresponding each second identification information, and then inputs each original video frame image and each third video frame image into a preset image quality assessment model, respectively, to obtain the second image transmission quality, and finally determines the target image transmission quality according to the first image transmission quality and the second image transmission quality, which can effectively improve the accuracy of image quality inspection of vehicle-mounted video transmission.

[0040] Furthermore, by setting the identification information as the frame number and time information of the key frame and adding it to the transmitted image, the image quality can be checked by comparing the identification information. On the other hand, since only the key frame images in the original video stream data are extracted, the corresponding image frames of the sending and receiving ends can be quickly obtained. During the inspection, only the key frame images are analyzed, which can further improve the efficiency of the image quality inspection.

[0041] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0043] Figure 1 A schematic flow chart of an image quality inspection method for vehicle-mounted video transmission provided by an embodiment of the present invention is shown;

[0044] Figure 2 A schematic diagram showing an application scenario of an image quality inspection method for vehicle video transmission provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of the structure of an image quality assessment model provided by an embodiment of the present invention is shown;

[0046] Figure 4 A schematic diagram showing the structure of an image quality inspection device for vehicle-mounted video transmission provided by an embodiment of the present invention is shown;

[0047] Figure 5A schematic structural diagram of a vehicle provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0048] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0049] Figure 1 The flowchart of the image quality inspection method for vehicle-mounted video transmission provided by an embodiment of the present invention is shown. The method is executed by a quality inspection module (detection end) in the vehicle, wherein the quality inspection module can be a part of the processor in the vehicle, or a cloud server connected to the transmitting end and the receiving end of the vehicle, and the embodiment of the present invention does not impose specific restrictions. Among them, the image quality inspection method for vehicle-mounted video transmission can be to detect the image of the vehicle-mounted video transmission according to a preset inspection cycle, or it can be real-time detection, and this application does not specifically limit it. Figure 1 As shown, the method comprises the following steps:

[0050] Step 110: Acquire original video stream data from the transmitting end in real time, and extract each original video frame image of the original video stream data.

[0051] The original video stream data is the vehicle video acquired by the vehicle camera device in real time. Specifically, the vehicle video acquired by the vehicle camera device in real time, the original video stream data is acquired by the sending end, wherein: Figure 2 , is a schematic diagram of an application scenario of an image quality inspection method for vehicle-mounted video transmission in one embodiment of the present invention. Since the original video data collected from the vehicle-mounted camera device is lossless, Figure 2 As shown, the sending end may be the vehicle-mounted camera device itself, that is, the processor of the vehicle-mounted camera device.

[0052] Step 120: Acquire multiple second video frame images from the receiving end in real time.

[0053] Among them, the second video frame image is obtained by the sending end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain multiple continuous first video frame images, and encapsulating multiple first video frame images according to the preset image transmission protocol.

[0054] In the embodiment of the utility model, the detection end obtains the original video stream data and time information from the sending end in real time. The sending end obtains each original video frame image in the original video stream and adds the first identification information to each original video frame image to obtain multiple continuous first video frame images. The sending end encapsulates the multiple first video frame images according to the preset image transmission protocol to obtain encapsulated image data and sends it to the receiving end. The receiving end decodes the encapsulated image data according to the image transmission protocol to obtain multiple decoded second video frame images. Figure 2 As shown, in an application scenario of an embodiment of the present invention, the vehicle-mounted camera device transmits the packaged video frame image to the GPS navigator. Specifically, the processor in the vehicle-mounted camera device can transmit the first video frame image to the processor of the GPS navigator by wireless (such as a wireless receiver) or wired mode, and then display it on the display screen of the navigator. In order to enhance the accuracy of image quality inspection, the embodiment of the present invention sets a plurality of first identification information, each of which includes frame sequence information and time information corresponding to the respective original video frame image. In which, the frame sequence information indicates the position of the original video frame image in the original video stream data, such as N, representing the original video frame image in the original video stream data The Nth frame; the time information indicates the time point of the original video frame image in the original video stream data, such as M time, representing the original video frame image in the protection time length of the original video stream data. For example, the frame sequence number of the first frame image is Z1, the second frame is Z2... The time sequence of the first frame image is 00:01, the second frame is 00:02,... In a specific implementation, when performing image quality inspection, in order to improve the efficiency of quality inspection, only key frames in the original video stream data can be extracted for quality inspection, so the first identification information can be the frame sequence number and time information corresponding to the key frame. Therefore, the method also includes: identifying the key frame in the original video stream data; determining the sequence number information and time information corresponding to the key frame as the first identification information.

[0055] On the one hand, the transmitting end adds the characteristic value of the first identification information to the original video frame image corresponding to each first identification information, and sends it to the receiving end after encapsulation through the image transmission protocol. Specifically, it includes: determining the original video frame image of the corresponding frame number and time from the original video stream data; adding the corresponding frame number information and time information to the original video frame image of the corresponding frame number and time. Thus, during the image transmission process, the image carries the first identification information. Among them, for the process of adding the first identification information to the original video frame image to obtain the first video frame image, it is obtained by adding the characteristic information corresponding to the first identification information to the original video frame image corresponding to it. The embodiment of the present invention does not specifically limit the expression form of the first identification information. In a specific implementation, the first identification information can be a first identification image. It can be that the first identification image is generated separately or together according to the frame number and time, and the first identification image is added to the original video frame image to obtain the first video frame image. For example, when adding a logo to an image, the first logo image can be added in the form of a watermark, and a watermark image is generated by generating a frame sequence and a time sequence according to a preset image generation rule, and then the watermark image is added to the specified position of the image according to the preset adding rule, for example, it can be added to the lower right position of the image, wherein the watermark image can be a semi-transparent watermark, and the first video frame image is obtained. Among them, in order to facilitate subsequent extraction, the first identification information can also be added to the original video frame image in a layer manner to obtain the first video frame image, such as adding a semi-transparent watermark to a set layer. Among them, the sending end encapsulates multiple first video frame images, and when obtaining the encapsulated image data, the original video frame image without the first identification information added to the first video frame image and the key frame with the first identification information added are encapsulated together and then sent to the receiving end. When the receiving end decodes, it decodes according to the preset image transmission protocol, and the obtained second video frame image includes a video frame image without the first identification information and a key frame carrying the first identification information, so that when the detection end detects that there is no problem with the image quality later, the receiving end can send the first video frame image without the first identification information to the display end in real time.

[0056] On the other hand, the quality detection module of the present application obtains the first identification information from the transmitting end in real time. Therefore, the method also includes: obtaining each of the first identification information from the transmitting end, and making the first identification information correspond to each of the original video frame images. Specifically, according to the frame sequence number information and time information, the original video frame image corresponding to the time and frame sequence number is extracted from the original video stream data. Among them, the frame sequence number information and time information can be directly obtained from the transmitting end, and the original video frame image corresponding to the time and frame sequence number is extracted from the original video stream data according to the frame sequence number information and time information, and then the first identification image is generated separately or together according to the frame sequence number and time. It is also possible to directly obtain the first identification image of the transmitting end, and the embodiment of the present invention does not make specific restrictions. After directly obtaining the first identification image of the transmitting end, the first identification image is subjected to OCR recognition to obtain the frame sequence number and time, and then the original video frame image corresponding to the time and frame sequence number is extracted from the original video stream data according to the frame sequence number and time.

[0057] Step 130: performing identification on the plurality of second video frame images to obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being removed from the identification.

[0058] After decoding at the receiving end, each second video frame image obtained contains its own first identification information. Therefore, by identifying the multiple second video frame images, the second identification information corresponding to each second video frame image can be obtained. The second identification information can be a second identification image. Specifically, the process of extracting the second identification information from the second video frame image is the inverse process of adding the first identification information to the original video frame image. For the form of the watermark, the watermark in the second video frame image can be extracted: the second video frame image containing the first identification information is binarized to obtain a binarized image containing the second video frame image content as a foreground mask image, and the watermark image containing the identification information is obtained by referring to the foreground mask image, that is, the second identification image, and the third video frame image after removing the identification. After obtaining the second video frame image, the receiving end removes the watermark image in the same way, and sends the video frame image after removing the watermark image to the display end for display.

[0059] After the second identification information is identified, the extracted second identification information is matched with the first identification information according to the frame sequence number information and the time information, so as to obtain the second identification information corresponding to each of the first identification information and the third video frame image corresponding to each of the original video frame images. The first identification image and the second identification image can be matched by performing OCR recognition on the first identification image and the second identification image to obtain the frame sequence number information and the time information.

[0060] Step 140: Determine the first image transmission quality according to the difference between each first identification information and the corresponding second identification information.

[0061] After decoding at the receiving end, each second video frame image obtained includes the respective first identification information. However, due to the compression algorithm, image transmission protocol, transmission medium and bandwidth limitations during the transmission process, the quality of the first video frame image may be affected to varying degrees, so the first identification information included in the second video frame image may also be affected to varying degrees. Therefore, the embodiment of the present invention sets the identification information in the video frame image, and can compare the changes in the identification information in the images of the sending end and the receiving end, so as to determine the transmission quality of the first image according to the difference between each first identification information and the corresponding each second identification information. The existing image feature calculation method can be used to compare the difference in feature parameters between the first identification image and the second identification image to determine the transmission quality of the first image. For example, the color, background color and brightness between the two are compared to determine the transmission quality of the first image. After obtaining the difference between each first identification image and each second identification image, the average value of the difference between all first identification images and second identification images is calculated to obtain the transmission quality of the first image. The image transmission quality evaluation can be classified or scored according to the level or threshold, and the embodiment of the present invention does not make specific restrictions.

[0062] Step 150: Input each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain a second image transmission quality.

[0063] The image quality assessment model is obtained by pre-training the neural network model based on the image quality assessment samples. The specific training process is as follows:

[0064] 001: Get image quality assessment samples.

[0065] The image quality assessment sample includes an original image sample, a plurality of corresponding low-quality image samples and a quality score label, wherein the low-quality image sample is a low-quality image sample obtained after data processing of various image types. Noise parameters such as white noise function parameters, blur function parameters, JPEG compression function parameters, JPEG compression function parameters, and JPEG2000 compression function parameters can be added to the original image sample to obtain low-quality image samples of various image noise types. The quality score label is calculated by combining the degree of addition of the above noise parameters to determine its quality score, thereby obtaining the corresponding quality score label.

[0066] 002: Input the image quality assessment sample into a convolutional neural network to obtain a predicted quality score.

[0067] Among them, Figure 3 As shown, the convolutional neural network includes a feature extraction layer and a quality assessment module; specifically, the feature extraction layer includes an input layer, a convolution layer, a pooling layer and a fully connected layer for feature extraction.

[0068] In the embodiment of the present invention, the quality assessment module includes a PSNR algorithm, a mean square error, a SSIM algorithm and a FSIM algorithm.

[0069] Among them, PSNR (Peak Signal-to-Noise Ratio) is an indicator used to evaluate image quality, especially in the fields of image compression and image processing. It is based on the ratio between the maximum possible image signal power and the noise power of the image, and is usually used to measure the effect of image restoration or image compression algorithms. The key to calculating PSNR is to calculate the average square error between the original image and the distorted image, that is, the mean square error (MSE), and then use the PSNR formula to calculate the PSNR value. The calculation formula of the PSNR algorithm is as follows: ；

[0070] Among them, is the maximum possible pixel value of the image data, ‌In the training stage, MSE is the mean square error between the original image sample and the corresponding low-quality type image sample. In the prediction stage, MSE is the mean square error between the original video frame image and the third video frame image. ‌

[0071] Among them, the mean square error (MSE) is used to measure the degree of change in pixel values ​​before and after image processing. The calculation formula of the mean square error can be expressed as:

[0072] Among them, in the training stage, ‌ and ‌ respectively represent the pixel values ​​of the original video frame image and the third video frame image at the corresponding positions. In the prediction stage, ‌ and ‌ respectively represent the pixel values ​​of the original video frame image and the third video frame image at the corresponding positions, and N is the total number of pixels. ‌This formula calculates the difference in pixel values ​​between each original video frame image and each third video frame image, and obtains an overall difference measure between the two by averaging. ‌

[0073] Among them, SSIM (Structural Similarity Index Measurement) is used to measure the similarity between the original video frame image and the third video frame image, mainly considering the brightness, contrast and structural information of the image. The SSIM algorithm evaluates the similarity or distortion of the images by comparing the similarities of the two images in these aspects.

[0074] The FSIM algorithm evaluates image quality by calculating the similarity index between the original image and the processed image. The specific implementation steps include: reading the original video frame image and the third video frame image, and converting them into grayscale images. Blurring the grayscale image with a Gaussian filter to simulate the human eye's perception of the image. Calculate the structural similarity index (SSIM) and brightness similarity index (LCS) between the original video frame image and the third video frame image. The FSIM index is obtained by weighted averaging the SSIM and LCS. The higher the value, the better the image quality.

[0075] By setting the PSNR algorithm, mean square error, SSIM algorithm and FSIM algorithm, the image quality can be evaluated from multiple angles. After evaluation by each algorithm, the comprehensive evaluation unit performs weighted averaging on the evaluation of each algorithm to obtain a predicted quality score.

[0076] 003: Calculate the loss according to the predicted quality score and quality score label, adjust the parameters of the convolutional neural network, re-execute the steps of inputting the image quality assessment sample into the convolutional neural network to obtain the predicted quality score, and calculating the loss according to the predicted quality score and quality score label, and adjusting the parameters of the convolutional neural network, until the loss is less than the preset loss threshold, stop training, and obtain the image quality assessment model. The loss function of the image quality assessment model can adopt an existing loss function, such as a logarithmic loss function, which is not specifically limited in the embodiment of the present invention.

[0077] After obtaining the trained image quality assessment model, each of the original video frame images and each of the third video frame images are respectively input into the preset image quality assessment model for feature extraction, and are calculated by the above-mentioned various algorithms to obtain an evaluation of the image transmission quality between each of the original video frame images and each of the third video frame images through comprehensive evaluation. Then, the second image transmission quality is obtained by calculating the mean of the image transmission quality between all the original video frame images and the third video frame images.

[0078] Step 160: Determine a target image transmission quality according to the first image transmission quality and the second image transmission quality.

[0079] After the first image transmission quality and the second image transmission quality are obtained, a weighted average of the two is performed to obtain a target image transmission quality.

[0080] After determining the target image transmission quality, if the target image transmission quality is lower than the preset quality threshold, it is determined that the current image quality is poor, and a prompt message is sent to the user. The inspection end sends the low-quality prompt message to the receiving end, and after receiving the prompt message, the receiving end sends the second video frame image including the video frame image that does not carry the first identification information to the display screen, and prompts the low-quality prompt message on the display screen. If the target image transmission quality is higher than the preset quality threshold, it is determined that the current image quality is good, and the second video frame image including the video frame image that does not carry the first identification information is directly sent to the display screen.

[0081] The embodiment of the present invention obtains original video stream data from a transmitting end in real time, and obtains multiple second video frame images from a receiving end in real time; identifies the multiple second video frame images to obtain second identification information corresponding to each second video frame image and multiple de-identified third video frame images; determines the first image transmission quality according to the difference between each first identification information and the corresponding each second identification information, and then inputs each original video frame image and each third video frame image into a preset image quality assessment model, respectively, to obtain the second image transmission quality, and finally determines the target image transmission quality according to the first image transmission quality and the second image transmission quality, which can effectively improve the accuracy of image quality inspection of vehicle-mounted video transmission.

[0082] Furthermore, by setting the identification information as the frame number and time information of the key frame and adding it to the transmitted image, the image quality can be checked by comparing the identification information. On the other hand, since only the key frame images in the original video stream data are extracted, the corresponding image frames of the sending and receiving ends can be quickly obtained. During the inspection, only the key frame images are analyzed, which can further improve the efficiency of the image quality inspection.

[0083] Figure 4 FIG. 1 is a schematic diagram showing the structure of an image quality inspection device for vehicle-mounted video transmission provided by an embodiment of the present invention. Figure 4 As shown, the device 300 includes:

[0084] The first acquisition module 310 is used to acquire original video stream data from the sending end in real time and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the sending end;

[0085] The second acquisition module 320 is used to acquire a plurality of second video frame images from the receiving end in real time; the second video frame images are obtained by the sending end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain a plurality of continuous first video frame images, and encapsulating the plurality of first video frame images according to a preset image transmission protocol;

[0086] The identification module 330 is used to identify the plurality of second video frame images, and obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being removed from the identification;

[0087] A first determination module 340, configured to determine a first image transmission quality according to a difference between each of the first identification information and each of the corresponding second identification information;

[0088] The model evaluation module 350 is used to input each of the original video frame images and each of the third video frame images into a preset image quality evaluation model to obtain the second image transmission quality; the image quality evaluation model is obtained by training a neural network model in advance according to image quality evaluation samples;

[0089] The output module 360 ​​is configured to determine a target image transmission quality according to the first image transmission quality and the second image transmission quality.

[0090] The embodiment of the present invention obtains original video stream data from a transmitting end in real time, and obtains multiple second video frame images from a receiving end in real time; identifies the multiple second video frame images to obtain second identification information corresponding to each second video frame image and multiple de-identified third video frame images; determines the first image transmission quality according to the difference between each first identification information and the corresponding each second identification information, and then inputs each original video frame image and each third video frame image into a preset image quality assessment model, respectively, to obtain the second image transmission quality, and finally determines the target image transmission quality according to the first image transmission quality and the second image transmission quality, which can effectively improve the accuracy of image quality inspection of vehicle-mounted video transmission.

[0091] Furthermore, by setting the identification information as the frame number and time information of the key frame and adding it to the transmitted image, the image quality can be checked by comparing the identification information. On the other hand, since only the key frame images in the video frame images are extracted, the corresponding image frames of the sending and receiving ends can be quickly obtained. During the inspection, only the key frame images are analyzed, which can further improve the efficiency of the image quality inspection.

[0092] Figure 5 A schematic diagram of the structure of a vehicle provided in an embodiment of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the vehicle.

[0093] like Figure 5 As shown, the vehicle includes: a processor 402 , a communications interface 404 , a memory 406 , and a communications bus 408 .

[0094] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned embodiment of the image quality inspection method for vehicle-mounted video transmission.

[0095] Specifically, the program 410 may include program code including computer executable instructions.

[0096] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the vehicle may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0097] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0098] Program 410 can be specifically called by processor 402 to perform the following operations:

[0099] Acquire original video stream data from the transmitting end in real time, and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the transmitting end;

[0100] Acquire multiple second video frame images from the receiving end in real time; the second video frame images are obtained by the sending end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain multiple continuous first video frame images, and encapsulating the multiple first video frame images according to a preset image transmission protocol;

[0101] Performing identification on the plurality of second video frame images to obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being removed from the identification;

[0102] determining the first image transmission quality according to the difference between each of the first identification information and the corresponding each of the second identification information;

[0103] Input each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain a second image transmission quality; the image quality assessment model is obtained by pre-training a neural network model based on image quality assessment samples;

[0104] A target image transmission quality is determined according to the first image transmission quality and the second image transmission quality.

[0105] In an optional manner, before acquiring multiple second video frame images from the receiving end in real time, the method further includes: acquiring each of the first identification information from the sending end; and matching the first identification information with each of the original video frame images.

[0106] In an optional manner, each of the first identification information includes frame sequence information and time information corresponding to the respective original video frame;

[0107] The real-time acquisition of original video stream data from the transmitting end and extraction of each original video frame image of the original video stream data includes:

[0108] Extracting the original video frame image corresponding to the time and frame number from the original video stream data according to the frame number information and the time information;

[0109] The step of adding first identification information to each original video frame image of the original video stream data to obtain a plurality of continuous first video frame images includes:

[0110] Determine the original video frame image corresponding to the frame sequence number and time from the original video stream data;

[0111] The corresponding frame number information and time information are added to the original video frame image of the corresponding frame number and time.

[0112] In an optional manner, after the identification of the plurality of second video frame images is performed to obtain the second identification information corresponding to each second video frame image and a plurality of third video frame images after being de-identified, the method further includes:

[0113] According to the frame sequence number information and time information, the extracted second identification information is matched with the first identification information to obtain the second identification information corresponding to each of the first identification information and the third video frame image corresponding to each of the original video frame images.

[0114] In an optional manner, the method further includes: identifying key frames in the original video stream data;

[0115] The sequence number information and time information corresponding to the key frame are determined as the first identification information.

[0116] In an optional manner, before inputting each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain the second image transmission quality, the method further includes:

[0117] Acquire image quality assessment samples, wherein the image quality assessment samples include original image samples, corresponding multiple low-quality image samples and quality score labels, wherein the low-quality image samples are low-quality image samples obtained after performing data processing of various image types;

[0118] Inputting the image quality assessment sample into a convolutional neural network to obtain a predicted quality score; the convolutional neural network includes a feature extraction layer and a quality assessment module; the quality assessment module includes a PSNR algorithm, a mean square error, a SSIM algorithm, and a FSIM algorithm;

[0119] The loss is calculated according to the predicted quality score and the quality score label, and the parameters of the convolutional neural network are adjusted. The steps of inputting the image quality assessment sample into the convolutional neural network to obtain the predicted quality score, and calculating the loss according to the predicted quality score and the quality score label, and adjusting the parameters of the convolutional neural network are re-executed until the loss is less than a preset loss threshold, then the training is stopped to obtain the image quality assessment model.

[0120] In an optional manner, determining the target image transmission quality according to the first image transmission quality and the second image transmission quality includes: performing a weighted average on the first image transmission quality and the second image transmission quality to obtain the target image transmission quality.

[0121] The embodiment of the present invention obtains original video stream data from a transmitting end in real time, and obtains multiple second video frame images from a receiving end in real time; identifies the multiple second video frame images to obtain second identification information corresponding to each second video frame image and multiple de-identified third video frame images; determines the first image transmission quality according to the difference between each first identification information and the corresponding each second identification information, and then inputs each original video frame image and each third video frame image into a preset image quality assessment model, respectively, to obtain the second image transmission quality, and finally determines the target image transmission quality according to the first image transmission quality and the second image transmission quality, which can effectively improve the accuracy of image quality inspection of vehicle-mounted video transmission.

[0122] Furthermore, by setting the identification information as the frame number and time information of the key frame and adding it to the transmitted image, the image quality can be checked by comparing the identification information. On the other hand, since only the key frame images in the original video stream data are extracted, the corresponding image frames of the sending and receiving ends can be quickly obtained. During the inspection, only the key frame images are analyzed, which can further improve the efficiency of the image quality inspection.

[0123] An embodiment of the present invention provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on a processor of a vehicle, the processor executes the operations of the above-mentioned image quality inspection method for vehicle-mounted video transmission.

[0124] The executable instructions may specifically be used to cause the vehicle's processor to perform the following operations:

[0125] Acquire original video stream data from the transmitting end in real time, and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the transmitting end;

[0126] Acquire multiple second video frame images from the receiving end in real time; the second video frame images are obtained by the sending end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain multiple continuous first video frame images, and encapsulating the multiple first video frame images according to a preset image transmission protocol;

[0127] Performing identification on the plurality of second video frame images to obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being removed from the identification;

[0128] determining the first image transmission quality according to the difference between each of the first identification information and the corresponding each of the second identification information;

[0129] Input each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain a second image transmission quality; the image quality assessment model is obtained by pre-training a neural network model based on image quality assessment samples;

[0130] A target image transmission quality is determined according to the first image transmission quality and the second image transmission quality.

[0131] In an optional manner, before acquiring multiple second video frame images from the receiving end in real time, the method further includes: acquiring each of the first identification information from the sending end; and matching the first identification information with each of the original video frame images.

[0132] In an optional manner, each of the first identification information includes frame sequence information and time information corresponding to the respective original video frame;

[0133] The real-time acquisition of original video stream data from the transmitting end and extraction of each original video frame image of the original video stream data includes:

[0134] Extracting the original video frame image corresponding to the time and frame number from the original video stream data according to the frame number information and the time information;

[0135] The step of adding first identification information to each original video frame image of the original video stream data to obtain a plurality of continuous first video frame images includes:

[0136] Determine the original video frame image corresponding to the frame sequence number and time from the original video stream data;

[0137] The corresponding frame number information and time information are added to the original video frame image of the corresponding frame number and time.

[0138] In an optional manner, after the identification of the plurality of second video frame images is performed to obtain the second identification information corresponding to each second video frame image and a plurality of third video frame images after being de-identified, the method further includes:

[0139] According to the frame sequence number information and time information, the extracted second identification information is matched with the first identification information to obtain the second identification information corresponding to each of the first identification information and the third video frame image corresponding to each of the original video frame images.

[0140] In an optional manner, the method further includes: identifying key frames in the original video stream data;

[0141] The sequence number information and time information corresponding to the key frame are determined as the first identification information.

[0142] In an optional manner, before inputting each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain the second image transmission quality, the method further includes:

[0143] Acquire image quality assessment samples, wherein the image quality assessment samples include original image samples, corresponding multiple low-quality image samples and quality score labels, wherein the low-quality image samples are low-quality image samples obtained after performing data processing of various image types;

[0144] Inputting the image quality assessment sample into a convolutional neural network to obtain a predicted quality score; the convolutional neural network includes a feature extraction layer and a quality assessment module; the quality assessment module includes a PSNR algorithm, a mean square error, a SSIM algorithm, and a FSIM algorithm;

[0145] The loss is calculated according to the predicted quality score and the quality score label, and the parameters of the convolutional neural network are adjusted. The steps of inputting the image quality assessment sample into the convolutional neural network to obtain the predicted quality score, and calculating the loss according to the predicted quality score and the quality score label, and adjusting the parameters of the convolutional neural network are re-executed until the loss is less than a preset loss threshold, then the training is stopped to obtain the image quality assessment model.

[0146] In an optional manner, determining the target image transmission quality according to the first image transmission quality and the second image transmission quality includes: performing a weighted average on the first image transmission quality and the second image transmission quality to obtain the target image transmission quality.

[0147] The embodiment of the present invention obtains original video stream data from a transmitting end in real time, and obtains multiple second video frame images from a receiving end in real time; identifies the multiple second video frame images to obtain second identification information corresponding to each second video frame image and multiple de-identified third video frame images; determines the first image transmission quality according to the difference between each first identification information and the corresponding each second identification information, and then inputs each original video frame image and each third video frame image into a preset image quality assessment model, respectively, to obtain the second image transmission quality, and finally determines the target image transmission quality according to the first image transmission quality and the second image transmission quality, which can effectively improve the accuracy of image quality inspection of vehicle-mounted video transmission.

[0148] Furthermore, by setting the identification information as the frame number and time information of the key frame and adding it to the transmitted image, the image quality can be checked by comparing the identification information. On the other hand, since only the key frame images in the video image are extracted, the corresponding image frames of the sending and receiving ends can be quickly obtained. During the inspection, only the key frame images are analyzed, which can further improve the efficiency of the image quality inspection.

[0149] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0150] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0151] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0152] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0153] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. An image quality inspection method for vehicle-mounted video transmission, characterized in that: The method comprises: Acquire original video stream data from the transmitting end in real time, and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the transmitting end; Acquire multiple second video frame images from a receiving end in real time; the second video frame images are obtained by the receiving end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain multiple continuous first video frame images, and encapsulating the multiple first video frame images according to a preset image transmission protocol; Performing identification on the plurality of second video frame images to obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being removed from the identification; determining the first image transmission quality according to the difference between each of the first identification information and the corresponding each of the second identification information; Input each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain a second image transmission quality; the image quality assessment model is obtained by pre-training a neural network model based on image quality assessment samples; A target image transmission quality is determined according to the first image transmission quality and the second image transmission quality.

2. The method according to claim 1, characterized in that Before acquiring a plurality of second video frame images from the receiving end in real time, the method further includes: Acquire each of the first identification information from the sending end; The first identification information is matched with each of the original video frame images.

3. The method according to claim 1, characterized in that Each of the first identification information includes frame sequence information and time information corresponding to the original video frame image; The real-time acquisition of original video stream data from the transmitting end and extraction of each original video frame image of the original video stream data includes: Extracting the original video frame image corresponding to the time and frame number from the original video stream data according to the frame number information and the time information; The step of adding first identification information to each original video frame image of the original video stream data to obtain a plurality of continuous first video frame images includes: Determine the original video frame image corresponding to the frame sequence number and time from the original video stream data; The corresponding frame number information and time information are added to the original video frame image of the corresponding frame number and time.

4. The method according to claim 3, characterized in that After the identification of the plurality of second video frame images is performed to obtain second identification information corresponding to each second video frame image and a plurality of third video frame images after being de-identified, the method further includes: According to the frame sequence number information and time information, the extracted second identification information is matched with the first identification information to obtain the second identification information corresponding to each of the first identification information and the third video frame image corresponding to each of the original video frame images.

5. The method according to any one of claims 1 to 4, characterized in that: Before inputting each of the original video frame images and each of the third video frame images into a preset image quality assessment model to obtain the second image transmission quality, the method further includes: Acquire image quality assessment samples, wherein the image quality assessment samples include original image samples, corresponding multiple low-quality image samples and quality score labels, wherein the low-quality image samples are low-quality image samples obtained after performing data processing of various image types; Inputting the image quality assessment sample into a convolutional neural network to obtain a predicted quality score; the convolutional neural network includes a feature extraction layer and a quality assessment module; the quality assessment module includes a PSNR algorithm, a mean square error, a SSIM algorithm, and a FSIM algorithm; The loss is calculated according to the predicted quality score and the quality score label, and the parameters of the convolutional neural network are adjusted. The steps of inputting the image quality assessment sample into the convolutional neural network to obtain the predicted quality score, and calculating the loss according to the predicted quality score and the quality score label, and adjusting the parameters of the convolutional neural network are re-executed until the loss is less than a preset loss threshold, then the training is stopped to obtain the image quality assessment model.

6. The method according to any one of claims 1 to 4, characterized in that: The determining the target image transmission quality according to the first image transmission quality and the second image transmission quality includes: A weighted average is performed on the first image transmission quality and the second image transmission quality to obtain a target image transmission quality.

7. An image quality inspection device for vehicle-mounted video transmission, characterized in that: The device comprises: The first acquisition module is used to acquire original video stream data from the sending end in real time and extract each original video frame image of the original video stream data; the original video stream data is the vehicle-mounted video acquired by the vehicle-mounted camera device in real time at the sending end; A second acquisition module is used to acquire a plurality of second video frame images from a receiving end in real time; the second video frame images are obtained by the receiving end decoding the encapsulated image data according to the image transmission protocol; the encapsulated image data is obtained by adding first identification information to each original video frame image of the original video stream data to obtain a plurality of continuous first video frame images, and encapsulating the plurality of first video frame images according to a preset image transmission protocol; An identification module, used for identifying the plurality of second video frame images, obtaining second identification information corresponding to each second video frame image and a plurality of third video frame images after being de-identified; A first determining module, configured to determine a first image transmission quality according to a difference between each of the first identification information and each of the corresponding second identification information; A model evaluation module, used to input each of the original video frame images and each of the third video frame images into a preset image quality evaluation model, respectively, to obtain the second image transmission quality; the image quality evaluation model is obtained by training a neural network model in advance according to image quality evaluation samples; An output module is used to determine a target image transmission quality according to the first image transmission quality and the second image transmission quality.

8. A vehicle, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the image quality inspection method for vehicle-mounted video transmission as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on a processor of a vehicle, the processor executes the operation of the image quality inspection method for in-vehicle video transmission as described in any one of claims 1 to 6.

Images