A High-Resolution Video Image Preprocessing Method for Image Processing Chips

Abstract

The present invention belongs to the technical field of chip image processing, and particularly relates to a high-resolution video image preprocessing method for an image processing chip. The present invention discloses building a sub-region image preprocessing image quality adjustment model after image preprocessing. The trained model automatically adopts different image quality adjustment methods according to the characteristics of the high-resolution video images divided by the preprocessing. By dividing the video images into non-key areas and key areas, personalized processing is carried out for the video images in the non-key areas and key areas respectively. Through the built image preprocessing image quality adjustment model, the present invention reduces the computational amount of image quality adjustment, reduces the error of manual adjustment, and improves the efficiency and accuracy of image quality adjustment.

Description

Technical Field

[0001] The present invention belongs to the technical field of chip image processing, and in particular relates to a high-resolution video image preprocessing method for an image processing chip. Background Art

[0002] At present, the image quality can be optimized by adjusting the picture quality (PQ) adjustment parameters of the local or overall image. Specifically, the image quality can be optimized by artificial means, solutions based on deep models, and solutions combining artificial means with deep models. In the artificial means, the picture quality adjustment parameters are manually set, and the picture quality is optimized according to the manually set picture quality adjustment parameters. In the solution based on the deep model, the video stream can be segmented by the deep model, and the scene recognition can be performed on each video segment after the scene segmentation. The picture quality adjustment parameters of the image in the video segment are determined according to the scene recognition results of each segmented video segment, and then the picture quality is optimized. In the solution combining artificial means with deep models, scene recognition is performed on the images in the video stream, and then the picture quality adjustment parameters are matched from the correspondence between the preset scene and the picture quality parameters according to the identified scene, and then the picture quality is optimized.

[0003] However, image quality optimization by manual means requires human intervention and has poor optimization effects. Image quality optimization based on deep models requires large amounts of computation and is not suitable for scenarios with limited computing power, such as terminal devices. In addition, the processing speed is slow, making it difficult to achieve real-time optimization. The solution that combines traditional solutions with deep models also has the problem of large amounts of computation and is not suitable for scenarios with limited computing power. In addition, this solution has poor optimization effects on images outside of preset scenarios. In summary, the above three existing solutions have the problem of not being able to take into account both the effect and efficiency of image quality optimization, as well as poor adaptability. Through the above analysis, the existing technologies have the following problems that need to be solved:

[0004] How to automatically adjust the image quality, reduce the manual adjustment rate, improve the quality of high-resolution images to the best, use learning models other than deep learning models to optimize image quality, reduce the amount of calculation, so as to be suitable for scenarios with limited computing power such as terminal devices, improve processing speed, and achieve real-time optimization.

[0005] Therefore, it is necessary to redesign a high-resolution video image preprocessing method for image processing chips to address existing problems. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a high-resolution video image preprocessing method for an image processing chip.

[0007] In the first aspect of the present invention, a high-resolution video image preprocessing method for an image processing chip is provided, including the following steps:

[0008] S1. The image processing chip acquires a first high-resolution video image;

[0009] S2. Based on the first high-resolution video image, preprocessing division of the non-key area and key area of the image is performed to obtain a first non-key area image and a first key area image, and a first non-key area image quality adjustment method and a first key area image quality adjustment method for the first non-key area and the first key area are respectively collected;

[0010] S3. A sub-region image preprocessing image quality adjustment model is constructed based on a first two-dimensional input vector composed of the first non-key area image and the first key area image and a first two-dimensional output vector composed of the corresponding first non-key area image quality adjustment method and the corresponding first key area image quality adjustment method;

[0011] S4. A second high-resolution video image is received, and the image processing chip respectively generates a second non-key area image quality adjustment method and a second key area image quality adjustment method according to the sub-region image preprocessing image quality adjustment model;

[0012] S5. Image quality adjustment of the second high-resolution video image is performed according to the second non-key area image quality adjustment method and the second key area image quality adjustment method.

[0013] Further, the sub-region image preprocessing image quality adjustment model is constructed by using machine learning or deep learning methods.

[0014] Further, the first high-resolution video image or the second high-resolution video image is preprocessed for the non-key area and key area of the image according to the color gamut, and the inter-domain variance T between the non-key area and key area of the image is defined as:

[0015] ;

[0016] In the formula, is the ratio of the average value of the color gamut R belonging to the non-key area of the image to the average value of the color gamut R of the first high-resolution video image or the second high-resolution video image, g is the average value of the three color gamuts RGB of the first high-resolution video image or the second high-resolution video image, is the ratio of the average value of the color gamut G belonging to the key area of the image to the average value of the color gamut G of the first high-resolution video image or the second high-resolution video image, is the ratio of the average value of the color gamut B belonging to the key area of the image to the average value of the color gamut B of the first high-resolution video image or the second high-resolution video image, is the average value of the R color gamut in the non-key area of the image, is the average value of the G color gamut in the non-key area of the image, is the average value of the B color gamut in the non-key area of the image.

[0017] Furthermore, the inter-domain variance T is used to distinguish between the key area and the non-key area. The area where the regional average values of the RGB three color gamuts of the high-resolution video image are greater than the inter-domain variance T is the non-key area feature vector N(T), and the area where the regional average values of the RGB three color gamuts of the high-resolution video image are less than the inter-domain variance T is the key area feature vector M(T), thereby obtaining a two-dimensional input vector .

[0018] Furthermore, the calculation formula of the classifier based on the Fisher criterion adopted by the machine learning method is as follows:

[0019] ;

[0020] A is the two-dimensional input vector, is the output vector value composed of the corresponding image quality adjustment method, W T is the normal vector perpendicular to the hyperplane, K is the adjustment coefficient, N(T) is the non-key area feature vector, is the key area feature vector, represents the determinant value.

[0021] A high-resolution video image preprocessing system for an image processing chip is also provided. The system includes:

[0022] High-resolution image acquisition module: Acquire the first high-resolution video image and also acquire the second high-resolution video image;

[0023] Key area preprocessing module: Connected to the high-resolution image acquisition module, acquire the first high-resolution video image to perform preprocessing division of the non-key area and key area of the image to obtain the first non-key area image and the first key area image, and respectively collect the first non-key area image quality adjustment method corresponding to the first non-key area and the first key area image quality adjustment method corresponding to the first key area; also acquire the second high-resolution video image to perform preprocessing division of the non-key area and key area of the image to obtain the second non-key area image and the second key area image;

[0024] Sub-region image preprocessing image quality adjustment module: It is used to receive the first non-key region image and the first key region image obtained by preprocessing and partitioning by the key region preprocessing module, the first non-key region image quality adjustment method and the first key region image quality adjustment method, and construct a sub-region image preprocessing image quality adjustment model according to the first two-dimensional input vector composed of the first non-key region image and the first key region image and the first two-dimensional output vector composed of the corresponding first non-key region image quality adjustment method and the corresponding first key region image quality adjustment method. It receives the second two-dimensional input vector composed of the second non-key region image and the second key region image of the key region preprocessing module, and the sub-region image preprocessing image quality adjustment model processes the second two-dimensional input vector to obtain a second output vector;

[0025] Chip image processing module: It receives the second output vector of the sub-region image preprocessing image quality adjustment module, and performs partitioned image quality adjustment on the second high-resolution video image according to the second non-key region image quality adjustment method and the second key region image quality adjustment method obtained from the second output vector.

[0026] Furthermore, the sub-region image preprocessing image quality adjustment model is constructed by using machine learning or deep learning methods.

[0027] Furthermore, the first high-resolution video image or the second high-resolution video image is preprocessed for the non-key region and the key region of the image according to the color gamut. The inter-domain variance T between the non-key region and the key region of the image is defined as:

[0028] ;

[0029] In the formula, is the ratio of the average value of the color gamut R belonging to the non-key region of the image to the average value of the color gamut R of the first high-resolution video image or the second high-resolution video image, g is the average value of the three color gamuts of RGB of the first high-resolution video image or the second high-resolution video image, is the ratio of the average value of the color gamut G belonging to the key region of the image to the average value of the color gamut G of the first high-resolution video image or the second high-resolution video image, is the ratio of the average value of the color gamut B belonging to the key region of the image to the average value of the color gamut B of the first high-resolution video image or the second high-resolution video image, is the average value of the color gamut R of the non-key region of the image, is the average value of the color gamut G of the non-key region of the image, is the average value of the color gamut B of the non-key region of the image.

[0030] Further, the inter-domain variance T is used to distinguish the key areas and non-key areas. The area with the regional mean of the three RGB color gamuts of the high-resolution video image greater than the inter-domain variance T is the non-key area feature vector N(T), and the area with the regional mean of the three RGB color gamuts of the high-resolution video image less than the inter-domain variance T is the key area feature vector M(T), and then the two-dimensional input vector is obtained. 。

[0031] Further, the calculation formula of the classifier based on the Fisher criterion adopted by the machine learning method is as follows:

[0032] ;

[0033] A is the two-dimensional input vector, is the output vector value composed of the corresponding image quality adjustment methods, W T is the normal vector perpendicular to the hyperplane, K is the adjustment coefficient, N(T) is the non-key area feature vector, is the key area feature vector, represents the determinant value.

[0034] The present invention discloses building a sub-region image preprocessing image quality adjustment model after image preprocessing. The trained model automatically adopts different image quality adjustment methods according to the characteristics of the preprocessed high-resolution video image. By dividing the video image into non-key areas and key areas, personalized processing is carried out for the video images of non-key areas and key areas respectively. The image preprocessing image quality adjustment model built by the present invention reduces the calculation amount of image quality adjustment, reduces the error of manual adjustment, and improves the efficiency and accuracy of image quality adjustment.

[0035] More embodiments and improvement effects of the present invention will be further introduced in combination with the drawings and specific embodiments. Brief Description of the Drawings

[0036] Figure 1 is a flowchart of a high-resolution video image preprocessing method for an image processing chip according to the present invention;

[0037] Figure 2 is a schematic diagram of a high-resolution video image preprocessing system for an image processing chip according to the present invention;

[0038] Figure 3 is a schematic diagram of the division of non-key areas and key areas in an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of the online image quality adjustment mode in an embodiment of the present invention;

[0040] Figure 5 This is a schematic structural diagram of an electronic device for implementing the method of the present invention in an embodiment of the present invention. Detailed implementation manners

[0041] Next, with reference to the accompanying drawings and specific implementation manners, the invention will be further described.

[0042] To solve the above technical problems, the present invention proposes a high-resolution video image preprocessing method for an image processing chip.

[0043] In a first aspect of the present invention, there is provided a high-resolution video image preprocessing method for an image processing chip, including the following steps:

[0044] S1. The image processing chip acquires a first high-resolution video image;

[0045] S2. According to the first high-resolution video image, preprocessing division of the non-key area and key area of the image is performed to obtain a first non-key area image and a first key area image, and a first non-key area image quality adjustment method and a first key area image quality adjustment method for the first non-key area and the first key area are respectively collected;

[0046] S3. A sub-region image preprocessing image quality adjustment model is constructed according to a first two-dimensional input vector composed of the first non-key area image and the first key area image and a first two-dimensional output vector composed of the corresponding first non-key area image quality adjustment method and the corresponding first key area image quality adjustment method;

[0047] S4. Receive a second high-resolution video image, and the image processing chip respectively generates a second non-key area image quality adjustment method and a second key area image quality adjustment method according to the sub-region image preprocessing image quality adjustment model;

[0048] S5. Perform image quality adjustment on the second high-resolution video image according to the second non-key area image quality adjustment method and the second key area image quality adjustment method.

[0049] Further, the sub-region image preprocessing image quality adjustment model is constructed by using machine learning or deep learning methods.

[0050] Further, preprocessing of the non-key area and key area of the image is performed on the first high-resolution video image or the second high-resolution video image according to the color gamut, and the inter-domain variance T between the non-key area and key area of the image is defined as:

[0051] ;

[0052] In the formula, R is the ratio of the average value of the color gamut R in the non-key area of the image to the average value of the color gamut R in the first high-resolution video image or the second high-resolution video image. g is the average value of the three color gamuts of RGB in the first high-resolution video image or the second high-resolution video image. G is the ratio of the average value of the color gamut G in the key area of the image to the average value of the color gamut G in the first high-resolution video image or the second high-resolution video image. B is the ratio of the average value of the color gamut B in the key area of the image to the average value of the color gamut B in the first high-resolution video image or the second high-resolution video image. R is the average value of the color gamut R in the non-key area of the image. G is the average value of the color gamut G in the non-key area of the image. B is the average value of the color gamut B in the non-key area of the image. In this actual example, the values of the RGB three color gamuts are 0 - 255.

[0053] Further, the inter-domain variance T is used to distinguish the key area and the non-key area. The area where the regional average values of the three color gamuts of RGB in the high-resolution video image are greater than the inter-domain variance T is the non-key area feature vector N(T). If the value of the inter-domain variance T is 99, then the areas where the values of the three color gamuts of RGB are greater than 99 are non-key areas. In this embodiment, a non-key area feature vector that exists is N(T) = |(155, 215, 87)| / 3 = 152. The area where the regional average values of the three color gamuts of RGB in the high-resolution video image are less than the inter-domain variance T is the key area feature vector M(T). Then, the areas where the values of the three color gamuts of RGB are less than 99 are key areas. In this embodiment, a key area feature vector that exists is M(T) = |(56, 32, 90)| / 3 = 59. In this embodiment, |X| represents rounding X downwards, and then a two-dimensional input vector is obtained. Then, the two-dimensional input vector in this embodiment is .

[0054] Further, the calculation formula of the classifier based on the Fisher criterion adopted by the machine learning method is as shown below:

[0055] ;

[0056] A is the two-dimensional input vector. is the output vector value composed of the corresponding image quality adjustment methods. W T is the normal vector perpendicular to the hyperplane. K is the adjustment coefficient. N(T) is the non-key area feature vector. is the key area feature vector. represents the determinant value.

[0057] The two-dimensional output vector in this embodiment is a method for adjusting the image quality of non-key areas and key areas. The adjustment methods for non-key areas and key areas include the values of features such as color temperature C, vividness V, sharpness value B, blurriness Q, and blue light E, etc., not limited to 5 features, that is When the output vector value is greater than or equal to 0, the corresponding two-dimensional output vector is F(N,M) = ( ), When the output vector value is less than 0, the corresponding two-dimensional output vector is F'(N,M) = ( ). In this embodiment, the value range of the adjustment coefficient is -0.01 - 0.015, and this coefficient only appropriately corrects the output vector value obtained by classification.

[0058] A high-resolution video image preprocessing system for an image processing chip is also provided. The system includes:

[0059] High-resolution image acquisition module: Acquire the first high-resolution video image and also acquire the second high-resolution video image;

[0060] Key area preprocessing module: Connected to the high-resolution image acquisition module, acquire the first high-resolution video image to perform preprocessing division of the non-key area and key area of the image to obtain the first non-key area image and the first key area image, and respectively collect the first non-key area image quality adjustment method corresponding to the first non-key area, and the first key area image quality adjustment method corresponding to the first key area; also acquire the second high-resolution video image to perform preprocessing division of the non-key area and key area of the image to obtain the second non-key area image and the second key area image;

[0061] Sub-region image preprocessing image quality adjustment module: Used to receive the first non-key area image and the first key area image obtained by the preprocessing division of the key area preprocessing module, the first non-key area image quality adjustment method and the first key area image quality adjustment method, construct a sub-region image preprocessing image quality adjustment model according to the first two-dimensional input vector composed of the first non-key area image and the first key area image and the first two-dimensional output vector composed of the corresponding first non-key area image quality adjustment method and the corresponding first key area image quality adjustment method, receive the second two-dimensional input vector composed of the second non-key area image and the second key area image of the key area preprocessing module, and the sub-region image preprocessing image quality adjustment model processes the second two-dimensional input vector to obtain a second output vector;

[0062] Chip image processing module: Receive the second output vector of the sub-region image preprocessing image quality adjustment module, and perform partitioned image quality adjustment on the second high-resolution video image according to the second output vector to obtain the second non-key region image quality adjustment method and the second key region image quality adjustment method.

[0063] Furthermore, the sub-region image preprocessing image quality adjustment model is constructed by using machine learning or deep learning methods.

[0064] Furthermore, preprocess the non-key region and key region of the first high-resolution video image or the second high-resolution video image according to the color gamut. The inter-domain variance T between the non-key region and key region of the image is defined as:

[0065] ;

[0066] In the formula, is the ratio of the average value of the color gamut R belonging to the non-key region of the image to the average value of the color gamut R of the first high-resolution video image or the second high-resolution video image. g is the average value of the three color gamuts RGB of the first high-resolution video image or the second high-resolution video image. is the ratio of the average value of the color gamut G belonging to the key region of the image to the average value of the color gamut G of the first high-resolution video image or the second high-resolution video image. is the ratio of the average value of the color gamut B belonging to the key region of the image to the average value of the color gamut B of the first high-resolution video image or the second high-resolution video image. is the average value of the color gamut R of the non-key region of the image. is the average value of the color gamut G of the non-key region of the image. is the average value of the color gamut B of the non-key region of the image.

[0067] Furthermore, use the inter-domain variance T to distinguish the key region and the non-key region. The region where the regional average value of the three color gamuts RGB of the high-resolution video image is greater than the inter-domain variance T is the non-key region feature vector N(T), and the region where the regional average value of the three color gamuts RGB of the high-resolution video image is less than the inter-domain variance T is the key region feature vector M(T), and then obtain the two-dimensional input vector .

[0068] Furthermore, the calculation formula of the classifier based on the Fisher criterion used in the machine learning method is as follows:

[0069] ;

[0070] A is the two-dimensional input vector. is the output vector value composed of the corresponding image quality adjustment method. WT is the normal vector perpendicular to the hyperplane, K is the adjustment coefficient, and N(T) is the feature vector of the non-key area. is the feature vector of the key area, represents the determinant value.

[0071] The present invention discloses a method for building a sub-region image preprocessing image quality adjustment model after image preprocessing. The trained model automatically adopts different image quality adjustment methods according to the features of the preprocessed high-resolution video image. By dividing the video image into non-key areas and key areas, personalized processing is performed on the video images of the non-key areas and key areas respectively. Through the built image preprocessing image quality adjustment model, the present invention reduces the computational amount of image quality adjustment, reduces the error of manual adjustment, and improves the efficiency and accuracy of image quality adjustment.

[0072] For the partial module structures not specifically defined in the present invention, the content recorded in the prior art shall prevail. The prior art mentioned in the foregoing background art part and specific embodiment part of the present invention can be used as a part of the present invention to understand the meaning of some technical features or parameters.

Claims

1. A high-resolution video image preprocessing method for an image processing chip, characterized in that, It includes the following steps: S1. The image processing chip acquires the first high-resolution video image; S2. Based on the first high-resolution video image, preprocessing division of the non-key area and key area of the image is performed to obtain the first non-key area image and the first key area image, and the first non-key area image quality adjustment method and the first key area image quality adjustment method of the first non-key area and the first key area are respectively collected; The non-key area and key area of the image are divided by using the inter-domain variance T. The definition of the inter-domain variance T is: ； In the formula, is the ratio of the average value of color gamut R in the non-key area of the image to the average value of color gamut R in the high-resolution video image, and g is the average value of the three color gamuts RGB in the high-resolution video image. is the ratio of the average value of color gamut G in the key area of the image to the average value of color gamut G in the high-resolution video image. is the ratio of the average value of color gamut B in the key area of the image to the average value of color gamut B in the high-resolution video image. is the average value of color gamut R in the non-key area of the image. is the average value of color gamut G in the non-key area of the image. is the average value of color gamut B in the non-key area of the image. The region with the average value of the RGB three color gamuts of the high-resolution video image greater than the inter-domain variance T is the non-key area feature vector N(T), and the region with the average value of the RGB three color gamuts of the high-resolution video image less than the inter-domain variance T is the key area feature vector M(T), thereby obtaining a two-dimensional input vector ; S3. Based on the first two-dimensional input vector composed of the first non-key area image and the first key area image and the first two-dimensional output vector composed of the corresponding first non-key area image quality adjustment method and the corresponding first key area image quality adjustment method, a sub-region image preprocessing image quality adjustment model is constructed; S4. Receive the second high-resolution video image. The image processing chip generates the second non-key area image quality adjustment method and the second key area image quality adjustment method respectively according to the sub-region image preprocessing image quality adjustment model; S5. Perform image quality adjustment on the second high-resolution video image according to the second non-key area image quality adjustment method and the second key area image quality adjustment method; The sub-region image preprocessing image quality adjustment model is constructed by using machine learning; The calculation formula of the classifier based on the Fisher criterion is adopted for the machine learning method as shown below: ； A is a two-dimensional input vector, is the output vector value composed of the corresponding image quality adjustment method, W T is the normal vector perpendicular to the hyperplane, K is the adjustment coefficient, and the value range of the adjustment coefficient K is -0.01 - 0.

015. N(T) is the non-key area feature vector, is the key area feature vector, represents the determinant value.

2. A high-resolution video image preprocessing system for an image processing chip, characterized in that, The system includes: High-resolution image acquisition module: Acquire the first high-resolution video image and also acquire the second high-resolution video image; Key area preprocessing module: Connected to the high-resolution image acquisition module, acquire the first high-resolution video image to perform preprocessing division of the non-key area and key area of the image to obtain the first non-key area image and the first key area image, and respectively collect the first non-key area image quality adjustment method corresponding to the first non-key area and the first key area image quality adjustment method corresponding to the first key area; also acquire the second high-resolution video image to perform preprocessing division of the non-key area and key area of the image to obtain the second non-key area image and the second key area image; The non-key area and key area of the image are divided by using the inter-domain variance T. The definition of the inter-domain variance T is: ； In the formula, is the ratio of the average value of gamut R in the non-key area of the image to the average value of gamut R in the high-resolution video image. g is the average value of the three gamuts of RGB in the high-resolution video image. is the ratio of the average value of gamut G in the key area of the image to the average value of gamut G in the high-resolution video image. is the ratio of the average value of gamut B in the key area of the image to the average value of gamut B in the high-resolution video image. is the average value of gamut R in the non-key area of the image. is the average value of gamut G in the non-key area of the image. is the average value of gamut B in the non-key area of the image. Taking the regions where the regional means of the RGB three color gamuts of the high-resolution video image are greater than the inter-domain variance T as the non-key area feature vector N(T), and taking the regions where the regional means of the RGB three color gamuts of the high-resolution video image are less than the inter-domain variance T as the key area feature vector M(T), and then obtaining a two-dimensional input vector ; Sub-region image preprocessing image quality adjustment module: Used to receive the first non-key area image and the first key area image obtained by the preprocessing division of the key area preprocessing module, the first non-key area image quality adjustment method and the first key area image quality adjustment method, construct a sub-region image preprocessing image quality adjustment model based on the first two-dimensional input vector composed of the first non-key area image and the first key area image and the first two-dimensional output vector composed of the corresponding first non-key area image quality adjustment method and the corresponding first key area image quality adjustment method, receive the second two-dimensional input vector composed of the second non-key area image and the second key area image of the key area preprocessing module, and the sub-region image preprocessing image quality adjustment model processes the second two-dimensional input vector to obtain a second output vector; Chip image processing module: Receive the second output vector of the sub-region image preprocessing image quality adjustment module, and obtain the second non-key region image quality adjustment method and the second key region image quality adjustment method according to the second output vector to perform partitioned image quality adjustment on the second high-resolution video image; The sub-region image preprocessing image quality adjustment model is constructed using machine learning; The calculation formula of the classifier based on the Fisher criterion is used for the machine learning method as shown below: ； A is a two-dimensional input vector, is the output vector value composed of the corresponding image quality adjustment methods, W T is the normal vector perpendicular to the hyperplane, K is the adjustment coefficient, the value range of the adjustment coefficient K is -0.01 - 0.015, N(T) is the non-key area feature vector, is the key area feature vector, represents the determinant value.

Images