Method for measuring subjective feeling crosstalk of naked eye 3D display system based on image features

Abstract

The invention discloses a naked eye 3D display system subjective feeling crosstalk measurement method based on image features, and the method comprises the steps: firstly constructing a score crosstalk test image library and a feature score test image library, designing a subjective feeling score table, obtaining a subjective score watched by human eyes, and building a function relation between the subjective score and an image crosstalk value through function fitting; image feature values are extracted through camera shooting, a subjective score prediction model is trained in combination with subjective scores obtained through calculation, and finally subjective feeling crosstalk prediction based on the image feature values is achieved. The method overcomes the defect that the prior art only depends on objective parameter measurement and cannot reflect the subjective feeling of a viewer, can quantitatively measure the subjective feeling crosstalk of a specific image on a specific device, and has high precision (fitting precision 97.8% and relevancy 97.37%) and high efficiency (measurement precision 94.6%). And reliable technical support is provided for optimizing the performance of the naked eye 3D display system.

Description

Technical Field

[0001] The present invention relates to naked-eye 3D display testing technology, and in particular to a method for measuring subjective crosstalk of a naked-eye 3D display system based on image features. Background Art

[0002] Vision plays a vital role in human perception, and 80% of the information in life is obtained through vision. In order to meet people's growing needs, display technology has gradually evolved. Since the 21st century, 3D display technology has become a striking cutting-edge technology. Among them, naked-eye 3D display technology has attracted much attention due to its advantages such as no need for visual aids, simple structure, low cost, and superior performance. It has been widely used in outdoor advertising screens, product exhibitions and other fields. Naked-eye 3D display screens have crosstalk problems, which affects the viewing experience.

[0003] In order to evaluate the display effect of naked-eye 3D display system, researchers have proposed a variety of methods to measure crosstalk, but the current test method for crosstalk is mainly through objective parameter measurement. The crosstalk value is related to the performance of the display device and the image content, and cannot reflect the impact of the crosstalk degree on the subjective feelings of the viewer. In addition, the existing evaluation system is relatively complex and diverse, and it is difficult to promote and apply. Therefore, a fast evaluation method for crosstalk of naked-eye 3D display system combined with the subjective feelings of the viewer needs to be studied urgently.

[0004] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0005] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide a method for measuring subjective crosstalk of a naked-eye 3D display system based on image features.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for measuring subjective crosstalk of a naked-eye 3D display system based on image features comprises the following steps:

[0008] S1: construct a naked-eye 3D display device scoring crosstalk test library and a feature scoring test library respectively, wherein the scoring crosstalk test library contains multiple groups of stereo image pairs with increasing image crosstalk rates, and the feature scoring test library contains stereo image pairs without crosstalk addition and stereo image pairs with crosstalk addition;

[0009] S2: designing a subjective perception rating scale, dividing the ratings into multiple levels based on subjective evaluation indicators of visual perception characteristics; loading the rating crosstalk test gallery on a naked-eye 3D display device to obtain a subjective rating of a naked-eye 3D image viewed by a human eye;

[0010] S3: performing function fitting using the image crosstalk value of the score crosstalk test gallery and the subjective score of the gallery obtained in step S2 to establish a functional relationship between the subjective score of the image and the image crosstalk value;

[0011] S4: obtaining left and right monocular views of the naked-eye 3D display device after the camera shoots the feature scoring test library, and fusing them into a binocular image; extracting image feature values, and performing feature cascading on all feature values;

[0012] S5: Select several observers, load the feature scoring test gallery on the naked-eye 3D display device to conduct a subjective experiment, obtain the observers' subjective scores on the images in the feature scoring test gallery as labels for the training data; use the subjective scores obtained in the experiment and the cascaded feature value data obtained in step S4 to train a subjective scoring prediction model through a regression prediction or time series prediction algorithm, until the error is reduced to within a predetermined range, and complete the training, and establish a subjective feeling scoring prediction model based on the measured feature values ​​obtained from the images taken by the camera; predict the subjective feeling score based on the image feature value, and determine the subjective feeling crosstalk based on the score and the functional relationship obtained in step S3.

[0013] Furthermore, in step S1, the scoring crosstalk test library includes a plurality of stereoscopic image pairs with increasing image crosstalk rate gradients, and the crosstalk rate gradient range covers the maximum objective crosstalk rate range that the human eye can perceive the image stereoscopic effect.

[0014] Furthermore, in step S2, the subjective evaluation index based on visual perception characteristics includes the severity of ghosting, the degree of image blur and the depth of image entering and exiting the screen.

[0015] Further, in step S2, the process of obtaining the subjective score of the crosstalk test gallery of naked eye 3D display device viewed by human eyes includes:

[0016] The descriptions in the subjective rating scale were used as a reference for comparison;

[0017] Observers watched the images displayed by the naked-eye 3D display system and rated the severity of ghosting, image blur, and the depth of the image entering and exiting the screen;

[0018] The weighted average of the scores of the three indicators, namely, the severity of ghosting, the degree of image blur and the depth of image entering and exiting the screen, is calculated to obtain the subjective perception score of the image on the naked-eye 3D display system.

[0019] Furthermore, in step S3, the functional relationship established by performing function fitting between the image crosstalk value and the subjective score includes but is not limited to an exponential function, a logarithmic function, a linear function and a quadratic function.

[0020] Furthermore, in step S3, the image crosstalk rate is fitted with the subjective feeling score value of each observer respectively, and then the crosstalk-subjective feeling score fitting curves of multiple observers are averaged to obtain an average crosstalk-subjective feeling score fitting function.

[0021] Furthermore, in step S4, the acquired left and right monocular views are first preprocessed, including image cropping, removing the screen background and edges, etc., and the image with the screen background and edges removed is subjected to noise reduction filtering and grayscale processing; then the processed monocular images are fused into binocular images, and feature values ​​are extracted from the monocular images and the binocular images.

[0022] Further, in step S4, the image feature values ​​include disparity feature values ​​between the stereo image pair and HOG feature values ​​of the fused binocular image, and the feature cascade includes merging the HOG feature values ​​of the binocular image with the global disparity difference and local disparity difference feature values ​​of the stereo image pair into a feature matrix.

[0023] Furthermore, in step S5, the training process of the subjective rating prediction model includes:

[0024] Use regression prediction or time series prediction algorithms to train models based on subjective scores of feature scoring test images and cascaded feature value data;

[0025] By adjusting one or more parameters including sampling ratio, regression parameter and number of cycles, the error is reduced to a predetermined range and the model training is completed.

[0026] Furthermore, in step S5, the subjective rating prediction model is trained using a regression prediction algorithm or a time series prediction algorithm, and the mean square error MSE is used as an indicator to measure the prediction effect of the model.

[0027] Further, in step S5, predicting the subjective crosstalk score according to the image feature value includes: loading the test image into the display device to be tested, extracting the display features of the image to be tested using the method in step S4, and inputting the display features of the image to be tested as test set data into the subjective score prediction model to obtain a subjective score output;

[0028] Furthermore, the subjective scoring output is input as an independent variable into the scoring crosstalk fitting model based on the functional relationship in step S3 to obtain the image subjective perception crosstalk output; preferably, the subjective perception crosstalk outputs of different test images are weighted averaged to obtain the subjective perception crosstalk of the display system.

[0029] A system for measuring subjective crosstalk of a naked-eye 3D display system comprises a computer-readable storage medium and a processor. The computer-readable storage medium stores an executable program. When the executable program is executed by the processor, the method for measuring subjective crosstalk of a naked-eye 3D display system is implemented.

[0030] The present invention has the following beneficial effects:

[0031] The present invention provides a method for measuring subjective crosstalk of a naked-eye 3D display system based on image features. By constructing a scoring crosstalk test library and a feature scoring test library, designing a subjective scoring scale, and establishing a functional relationship between the subjective scoring of the image and the crosstalk value, high-precision detection of the subjective crosstalk of the naked-eye 3D display system is achieved. The method uses the image feature values ​​captured by the camera, trains a subjective scoring prediction model through regression prediction or time series prediction algorithm, and can quantitatively measure the subjective crosstalk of a specific image on a specific naked-eye 3D display device. The fitting accuracy of the subjective feeling-crosstalk fitting scoring function relationship of the present invention is as high as 97.8%, the correlation is 97.37%, and the measurement accuracy of the subjective feeling of the feature image library reaches 94.6%, which significantly improves the accuracy and reliability of the crosstalk measurement. In addition, by establishing a subjective feeling crosstalk measurement model based on feature-scoring-crosstalk, the method can analyze the influence of system parameters on subjective feeling crosstalk, significantly improve the crosstalk evaluation effect of the naked-eye 3D display system, and has high practical value and good application prospects.

[0032] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the overall solution of the subjective perception crosstalk testing system according to an embodiment of the present invention.

[0034] Figure 2 4 is a flow chart of a subjective scoring experiment according to an embodiment of the present invention.

[0035] Figure 3 It is a scoring crosstalk function fitting flow chart of an embodiment of the present invention.

[0036] Figure 4 4 is a flowchart of image preprocessing according to an embodiment of the present invention.

[0037] Figure 5 4 is a flowchart of a feature score prediction model algorithm according to an embodiment of the present invention.

[0038] Figure 6 It is a flow chart of a method for measuring subjective crosstalk of a naked-eye 3D display system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0040] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0041] See also Figures 1 to 6 The embodiment of the present invention provides a method for measuring subjective crosstalk of a naked-eye 3D display system based on image features, comprising the following steps:

[0042] Step S1: construct a naked-eye 3D display device scoring crosstalk test library and a feature scoring test library respectively, wherein the scoring crosstalk test library contains multiple groups of stereo image pairs with increasing image crosstalk rates, and the feature scoring test library contains stereo image pairs without added crosstalk and stereo image pairs with added crosstalk.

[0043] In a preferred embodiment, in step S1, the scoring crosstalk test library includes multiple groups of stereoscopic image pairs with increasing image crosstalk rate gradients, wherein the crosstalk rate gradient range covers the maximum objective crosstalk rate range that the human eye can perceive the stereoscopic effect of the image.

[0044] As an example, the naked-eye 3D display device scoring crosstalk test gallery may include five groups of images, and the crosstalk rate of each stereo image pair is divided into 20 gradients from 0% to 50%, wherein the crosstalk of the first 16 stereo image pairs increases from 0% to 30% with a gradient of 2%, and the crosstalk of the 17th to 20th stereo image pairs increases from 30% to 50% with a gradient of 5%. The naked-eye 3D display device feature scoring test gallery may include 235 stereo image pairs without added crosstalk and 100 stereo image pairs with added crosstalk.

[0045] Step S2: designing a subjective perception rating scale, dividing the ratings into multiple levels based on subjective evaluation indicators of visual perception characteristics; loading the rating crosstalk test gallery on a naked-eye 3D display device to obtain a subjective rating of a naked-eye 3D image viewed by a human eye.

[0046] In a preferred embodiment, in step S2, the subjective evaluation index based on visual perception characteristics includes the severity of ghosting, the degree of image blur, and the depth of image entry and exit from the screen. As an example, the score can be divided into 5 levels, where 5 points represent the best image effect and 1 point represents the worst image effect.

[0047] In a preferred embodiment, in step S2, the process of obtaining the subjective score of the crosstalk test gallery of naked eye 3D display device viewed by human eyes includes:

[0048] The descriptions in the subjective rating scale were used as a reference for comparison;

[0049] Observers watched the images displayed by the naked-eye 3D display system and rated the severity of ghosting, image blur, and the depth of the image entering and exiting the screen;

[0050] The weighted average of the scores of the three indicators, namely, the severity of ghosting, the degree of image blur and the depth of image entering and exiting the screen, is calculated to obtain the subjective perception score of the image on the naked-eye 3D display system.

[0051] For example, the weighted average calculation formula of the scores of the three indicators of ghosting, blurring and screen depth is as follows:

[0052] Subjective perception score = ghosting score × 35% + blur × 35% + screen depth × 30%.

[0053] The subjective feeling score value can be obtained according to the measurement results of multiple observers watching the display content of the naked eye 3D display system respectively; preferably, the subjective feeling score value is obtained according to the measurement results of each observer watching the display content of the naked eye 3D display system according to different image sequences.

[0054] Step S3: Perform function fitting using the image crosstalk value of the score crosstalk test gallery and the subjective score of the gallery obtained in step S2 to establish a functional relationship between the subjective score of the image and the image crosstalk value.

[0055] In a preferred embodiment, in step S3, the functional relationship established by performing function fitting between the image crosstalk value and the subjective score includes but is not limited to exponential function, logarithmic function, linear function, quadratic function and other functional relationships.

[0056] Preferably, the image crosstalk rate and the subjective feeling score value of each observer are functionally combined. Number fitting, then The crosstalk-subjective feeling score fitting curves of multiple observers were averaged to obtain the average Crosstalk-subjective perception score fitting function.

[0057] Step S4: obtaining the left and right monocular views of the naked-eye 3D display device after the camera shoots the feature scoring test library, and fusing them into a binocular image; extracting image feature values, and performing feature cascading on all feature values.

[0058] The camera can be a SLR camera or a binocular CCD. When using a SLR camera, the left and right monocular images can be taken at two shooting positions respectively according to the visual characteristics of the human eye. The camera is built based on the visual characteristics of the human eye, wherein the effective acquisition distance and lens viewing angle are determined according to the optimal viewing distance and visual angle range of the naked eye 3D display device, and the camera spacing and height are set according to the average eye spacing and average height of the observer; and camera calibration and distortion correction are performed.

[0059] In a preferred embodiment, in step S4, the acquired left and right monocular views are first preprocessed, including image cropping, removing the screen background and edges, etc., and the images without the screen background and edges are subjected to noise reduction filtering and grayscale processing; the processed monocular images are then fused into binocular images, and feature values ​​are extracted from the monocular images and the binocular images.

[0060] In a preferred embodiment, in step S4, the image feature values ​​include disparity feature values ​​between the stereo image pair and HOG feature values ​​of the fused binocular image, and the feature cascade includes merging the HOG feature values ​​of the binocular image with the global disparity difference and local disparity difference feature values ​​of the stereo image pair into a feature matrix.

[0061] Step S5: select several observers, load the feature scoring test library on the naked-eye 3D display device to conduct a subjective experiment, and obtain the observers' subjective scores on the images in the feature scoring test library as labels for training data; use the subjective scores obtained in the experiment and the cascaded feature value data obtained in step S4 to train the subjective scoring prediction model through regression prediction or time series prediction algorithm, until the error is reduced to a predetermined range, the training is completed, and a subjective feeling scoring prediction model based on the measured feature values ​​obtained from the camera images is established; predict the subjective feeling score according to the image feature value, and determine the subjective feeling crosstalk according to the score and the functional relationship described in step S3.

[0062] In a preferred embodiment, in step S5, the training process of the subjective rating prediction model includes:

[0063] Use regression prediction or time series prediction algorithms to train models based on subjective scores of feature scoring test images and cascaded feature value data; randomly select a portion of the data as a training set to build a model, and use the remaining portion as a test set;

[0064] By adjusting one or more parameters including sampling ratio, regression parameter and number of cycles, the error is reduced to a predetermined range and the model training is completed.

[0065] In a preferred embodiment, in step S5, the subjective rating prediction model is trained using an SVR (support vector regression) algorithm, and the mean square error MSE is used as an indicator to measure the subjective feeling rating measurement value results predicted by the model. Preferably, the SVR algorithm uses a grid search method with five-fold cross validation as a parameter optimization method to obtain the best result.

[0066] In some embodiments, the process of using the obtained prediction model of subjective perceived crosstalk to measure the subjective perceived crosstalk of a naked-eye 3D display device includes: loading a test image into the display device to be tested, and using the method in step S4 to extract the display features of the image to be tested. The display features of the image to be tested are input as test set data into the subjective scoring prediction model to obtain a subjective scoring output. Further, the subjective scoring output is input as an independent variable into the scoring crosstalk fitting model of the functional relationship in step S3 to obtain the image subjective perceived crosstalk output. Preferably, the subjective perceived crosstalk outputs of different test images are weighted averaged to obtain the subjective perceived crosstalk of the display system.

[0067] An embodiment of the present invention further provides a system for measuring subjective crosstalk of a naked-eye 3D display system, comprising a computer-readable storage medium and a processor, wherein the computer-readable storage medium stores an executable program, and when the executable program is executed by the processor, the method for measuring subjective crosstalk of a naked-eye 3D display system of any of the aforementioned embodiments is implemented.

[0068] The present invention realizes high-precision detection of crosstalk of naked-eye 3D display system by establishing a subjective crosstalk measurement method of naked-eye 3D display system based on image features, and overcomes the defect of relying only on objective parameter measurement in the prior art and failing to reflect the subjective feelings of viewers. The method constructs a scoring crosstalk test library and a feature scoring test library, designs a subjective feeling scoring scale, obtains subjective scores in combination with the visual perception characteristics of the human eye, and uses function fitting to establish a functional relationship between subjective scores and image crosstalk values, further extracts image feature values ​​through camera shooting, trains a subjective scoring prediction model, and finally realizes the prediction of subjective feeling crosstalk based on image feature values. The present invention can not only quantitatively measure the subjective feeling crosstalk of a specific image on a specific naked-eye 3D display device, but also directly evaluate the crosstalk performance of the display device through model prediction, with high precision (the fitting accuracy of the subjective feeling-crosstalk fitting scoring function relationship is 97.8%, and the correlation is 97.37%) and high efficiency (the subjective feeling measurement accuracy of the feature image library is 94.6%), which significantly improves the crosstalk evaluation effect of the naked-eye 3D display system and provides reliable technical support for optimizing the performance of display devices.

[0069] The specific embodiments and experimental verification of the present invention are further described below.

[0070] See also Figures 1 to 6A method for measuring the perceived crosstalk of a naked-eye 3D display system includes establishing a crosstalk scoring function relationship of the naked-eye 3D display system and a feature scoring prediction model, and the main process includes:

[0071] 1. According to the definition and principle of crosstalk, establish a scoring crosstalk test library and a feature scoring test library;

[0072] 2. Combine the visual characteristics of the human eye and the characteristics of the 3D gallery images to obtain the subjective perception score of the gallery;

[0073] 3. According to the crosstalk value added to the gallery, the corresponding relationship between the score and the crosstalk is obtained through function fitting;

[0074] 4. According to the theory of geometric optics and diffraction optics, a binocular perception model is established, and a series of image processing methods are used to obtain the parallax and binocular perception feature values, which specifically include the following steps:

[0075] The features of global disparity difference and local disparity difference are extracted for the preprocessed left image and right image respectively, and 4 feature values ​​are obtained for each pair of images;

[0076] The feature scoring test image library was extracted one by one, and 335 × 4 matrix, stored in Excel;

[0077] Gabor texture feature extraction is performed on the left image and the right image respectively to obtain the energy response map of the image;

[0078] Use SIFT matching operator to get the distance between the same-name points in the left and right images;

[0079] Using binocular fusion theory, the energy response maps of the left image and the right image are fused into a binocular image to facilitate HOG feature extraction;

[0080] Use HOG directional gradient algorithm to extract the gradient features of the binocular image;

[0081] Use principal component analysis to reduce the dimension of HOG features and improve the running speed of subsequent models;

[0082] The feature scoring test image library was extracted one by one, and 335 × 30 matrices, stored in Excel;

[0083] Cascading the disparity feature and the HOG feature, we get 335 × 34 feature matrix, stored in Excel.

[0084] 5. Compare the obtained perceptual crosstalk characteristic values ​​with the subjective scores, and analyze the factors that affect the deviation between the subjective scores and the characteristic values.

[0085] Preferably, in step 1, 5 pictures meeting the experimental requirements are selected, and crosstalk is added at a gradient of 2%. When the crosstalk is 30%, crosstalk is added at a gradient of 5% to 50%.

[0086] Preferably, in step 2, 20 observers are found to watch the pictures one by one, and each observer's continuous watching time shall not exceed 25 minutes, and the rest time shall not be less than 8 minutes. The process of obtaining the subjective rating of the picture library includes the following steps: Figure 2 As shown:

[0087] Preferably, a subjective pre-experiment is conducted, including:

[0088] Select the pre-experimental 3D images;

[0089] The observers made subjective rating evaluations according to the given rating scale in turn;

[0090] Table 1

[0091]

[0092] Each observer's subjective rating of the pictures was recorded;

[0093] Using statistical principles, we calculated the 95% confidence intervals for different images, and screened the candidates for the formal experiment based on the results that fell within the confidence intervals.

[0094] Randomly shuffle the image gallery for formal experiments;

[0095] Following the principle of maximum viewing time and minimum rest time, the screened observers were formally scored for the experiment;

[0096] Each observer is at the optimal viewing distance of the naked-eye 3D display device;

[0097] Each image in the gallery is subjectively rated according to a rating scale;

[0098] When subjectively scoring the crosstalk test gallery, observers can compare before and after images to improve the reliability of the scoring;

[0099] The subjective score of each observer is recorded and weighted to obtain the subjective score of the gallery images.

[0100] Preferably, in step 3, a suitable function form is selected to fit the score value and the crosstalk, including the following steps: Figure 3 As shown:

[0101] The obtained scoring data and crosstalk data were pre-tested for function fitting goodness, and the nonlinear regression method of iterative least squares estimation was used to substitute various function models for fitting crosstalk and scoring; the fitting function models included linear models, polynomial models, logarithmic models, exponential models and other forms;

[0102] Comparing fit coefficients R2 and root mean square error RMSE, select the best fitting relationship;

[0103] Use the best fitting relationship to fit the data of each formal experimental observer, and obtain multiple fitting curves, fitting equations and root mean square error (RMSE).

[0104] The fitting relationship is averaged to obtain the final fitting curve relationship and root mean square error RMSE.

[0105] Preferably, in step 4, a stereoscopic image pair is obtained using a binocular camera based on the constructed feature scoring test image library. The images in the image library are displayed on a naked eye 3D display screen, and then they are photographed. Before feature extraction, it is necessary to obtain and preprocess the images, such as Figure 4 As shown, it specifically includes:

[0106] Take images and build an image acquisition device based on the visual characteristics of the human eye. First, determine the effective acquisition distance to be 5m and the lens angle of view to be 60°. Then set the binocular camera based on the distance between human eyes and the height of an average person;

[0107] Preferably, in step 4, after preprocessing the captured image, the disparity features and HOG features are extracted, such as Figure 5 As shown in the feature extraction unit, the following steps are specifically included:

[0108] The features of global disparity difference and local disparity difference are extracted for the preprocessed left and right images, and 4 feature values ​​are obtained for each pair of images;

[0109] The feature scoring test image library was extracted one by one, and 335 × 4 matrix, stored in Excel;

[0110] Gabor texture feature extraction is performed on the left image and the right image respectively to obtain the energy response map of the image;

[0111] Use SIFT matching operator to get the distance between the same-name points in the left and right images;

[0112] Using binocular fusion theory, the energy response maps of the left image and the right image are fused into a binocular image to facilitate HOG feature extraction;

[0113] Use HOG directional gradient algorithm to extract the gradient features of the binocular image;

[0114] Use principal component analysis to reduce the dimension of HOG features and improve the running speed of subsequent models;

[0115] The feature scoring test image library was extracted one by one, and 335 × 30, stored in Excel;

[0116] Cascading the disparity feature and the HOG feature, we get 335 × 34 feature matrix, stored in Excel.

[0117] Preferably, in step 5, the feature value after the above cascade is used as input, and the subjective score value corresponding to the image is used as a label to perform regression prediction model training. In this embodiment, SVR (support vector regression) is used to evaluate the deviation between the subjective score and the feature value. The specific steps are as follows:

[0118] Taking the feature scoring test image library as an example, the image library has a total of 335 image pair samples, and the 335 samples are randomly sampled in different proportions.

[0119] Some samples are extracted as training sets (the feature value data is the input of the training set, and the subjective rating value is the label of the training set), and a subjective rating prediction model with the feature value as input is obtained.

[0120] The remaining samples are used as test sets to evaluate and test the performance of the model obtained from the above training sets.

[0121] By adjusting relevant parameters (such as sampling ratio, regression parameters, cross-fold number, etc.), the MSE (mean square error) can be reduced as much as possible. When the MSE is small enough, it can be considered that the prediction result of the feature score prediction model is close enough to the subjective score value.

[0122] Therefore, it can be divided into the following steps: Figure 5 The rating prediction unit is shown as follows:

[0123] Input the cascaded feature value data, run the SVR algorithm, and use deep learning methods to obtain the score prediction value.

[0124] Specifically, in the process of repeatedly running the algorithm, the MSE and RMSE of the test set are used to measure the quality of the score prediction value. In order to obtain the smallest possible MSE and RMSE values, the optimization parameter algorithm should be selected for iteration;

[0125] Preferably, in this embodiment, a grid search algorithm is selected for parameter optimization of the SVR algorithm to calculate the best hyperparameters c and g, and perform parameter optimization to avoid falling into a local optimal solution;

[0126] Preferably, in this embodiment, five-fold cross validation is used simultaneously to improve the randomness and trustworthiness of the training set and the test set;

[0127] Application Example 1

[0128] The present invention is applied to the prediction of high accuracy of perceptual crosstalk. The measurement system performs feature score prediction and score crosstalk function fitting on the constructed image library. The accuracy, MSE, etc. of the two models are shown in Table 2:

[0129] Table 2

[0130]

[0131] The error comparison of the feature score prediction model is subjective scoring, with a 5-point scale, an MSE of 0.2695 and an accuracy of 94.6%;

[0132] The correlation of the scoring crosstalk function fitting function is 97.37%, the error comparison is the crosstalk rate, the crosstalk rate is the percentage, the RMSE is 2.1631%, and the accuracy is 97.8%.

[0133] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0134] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0135] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0136] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0137] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0138] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0139] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0140] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0141] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0142] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0143] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0144] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0145] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for measuring subjective crosstalk of a naked-eye 3D display system based on image features, characterized in that: The following steps are involved: S1: construct a naked-eye 3D display device scoring crosstalk test library and a feature scoring test library respectively, wherein the scoring crosstalk test library contains multiple groups of stereo image pairs with increasing image crosstalk rates, and the feature scoring test library contains stereo image pairs without crosstalk addition and stereo image pairs with crosstalk addition; S2: designing a subjective perception rating scale, dividing the ratings into multiple levels based on subjective evaluation indicators of visual perception characteristics; loading the rating crosstalk test gallery on a naked-eye 3D display device to obtain a subjective rating of a naked-eye 3D image viewed by a human eye; S3: performing function fitting using the image crosstalk value of the score crosstalk test gallery and the subjective score of the gallery obtained in step S2 to establish a functional relationship between the subjective score of the image and the image crosstalk value; S4: obtaining left and right monocular views of the naked-eye 3D display device after the camera shoots the feature scoring test library, and fusing them into a binocular image; extracting image feature values, and performing feature cascading on all feature values; S5: Select several observers, load the feature scoring test gallery on the naked-eye 3D display device to conduct a subjective experiment, obtain the observers' subjective scores on the images in the feature scoring test gallery as labels for training data; use the subjective scores obtained in the experiment and the cascaded feature value data obtained in step S4 to train a subjective scoring prediction model through regression prediction or time series prediction algorithm, until the error is reduced to within a predetermined range, and complete the training, and establish a subjective feeling scoring prediction model based on the measured feature values ​​obtained from the images taken by the camera; predict the subjective feeling score based on the image feature value, and determine the subjective feeling crosstalk based on the score and the functional relationship described in step S3.

2. The method for measuring subjective crosstalk of a naked-eye 3D display system according to claim 1, characterized in that: In step S1, the scoring crosstalk test library includes multiple groups of stereoscopic image pairs with increasing image crosstalk rate gradients, and the crosstalk rate gradient range covers the maximum objective crosstalk rate range that the human eye can feel the image stereoscopic effect.

3. The method for measuring subjective crosstalk of a naked-eye 3D display system according to claim 1 or 2, characterized in that: In step S2, the subjective evaluation index based on visual perception characteristics includes the severity of ghosting, the degree of image blur and the depth of image entering and exiting the screen.

4. The method for measuring subjective crosstalk of a naked-eye 3D display system according to claim 3, characterized in that: In step S2, the process of obtaining the subjective score of the crosstalk test gallery of naked eye 3D display device viewed by human eyes includes: The descriptions in the subjective rating scale were used as a reference for comparison; Observers watched the images displayed by the naked-eye 3D display system and rated the severity of ghosting, image blur, and the depth of the image entering and exiting the screen; The weighted average of the scores of the three indicators, namely, the severity of ghosting, the degree of image blur and the depth of image entering and exiting the screen, is calculated to obtain the subjective perception score of the image on the naked-eye 3D display system.

5. The method for measuring subjective crosstalk of a naked-eye 3D display system according to any one of claims 1 to 4, characterized in that: In step S3, the functional relationship established by function fitting between the image crosstalk value and the subjective score includes but is not limited to an exponential function, a logarithmic function, a linear function and a quadratic function; preferably, the image crosstalk rate is fitted with the subjective feeling score value of each observer respectively, and then the crosstalk-subjective feeling score fitting curves of multiple observers are averaged to obtain an average crosstalk-subjective feeling score fitting function.

6. The method for measuring subjective crosstalk of a naked-eye 3D display system according to any one of claims 1 to 5, characterized in that: In step S4, the acquired left and right monocular views are preprocessed, including image cropping, removing the screen background and edges, etc., and the images without the screen background and edges are subjected to noise reduction filtering and grayscale processing; The processed monocular image is then fused into a binocular image, and feature values ​​are extracted from the monocular image and the binocular image.

7. The method for measuring subjective crosstalk of a naked-eye 3D display system according to any one of claims 1 to 6, characterized in that: In step S4, the image feature values ​​include disparity feature values ​​between the stereo image pair and HOG feature values ​​of the fused binocular image, and the feature cascade includes merging the HOG feature values ​​of the binocular image with the global disparity difference and local disparity difference feature values ​​of the stereo image pair into a feature matrix.

8. The method for measuring subjective crosstalk of a naked-eye 3D display system according to any one of claims 1 to 7, characterized in that: In step S5, the training process of the subjective rating prediction model includes: Use regression prediction or time series prediction algorithms to train models based on subjective scores of feature scoring test images and cascaded feature value data; randomly select a portion of the data as a training set to build a model, and use the remaining portion as a test set; By adjusting one or more parameters including sampling ratio, regression parameter and number of cycles, the error is reduced to a predetermined range and the model training is completed.

9. The method for measuring subjective crosstalk of a naked-eye 3D display system according to claim 8, characterized in that: In step S5, the subjective rating prediction model is trained using a regression prediction algorithm or a time series prediction algorithm, and the mean square error (MSE) is used as an indicator to measure the prediction effect of the model.

10. The method for measuring subjective crosstalk of a naked-eye 3D display system according to claim 9, characterized in that: In step S5, the subjective rating prediction model is optimized by using an optimization algorithm, and the optimization algorithms used include cross validation, grid search algorithm, sparrow optimization algorithm, and genetic optimization algorithm.

11. The method for measuring subjective crosstalk of a naked-eye 3D display system according to any one of claims 1 to 10, characterized in that: In step S5, predicting the subjective crosstalk score according to the image feature value includes: loading the test image into the display device to be tested, extracting the display features of the image to be tested using the method in step S4, and inputting the display features of the image to be tested as test set data into the subjective score prediction model to obtain a subjective score output; Furthermore, the subjective scoring output is input as an independent variable into the scoring crosstalk fitting model based on the functional relationship in step S3 to obtain the image subjective perception crosstalk output; preferably, the subjective perception crosstalk outputs of different test images in a preferred gallery are weighted averaged to obtain the subjective perception crosstalk of the display system.

Images