Image quality assessment method based on image content and image distortion perception order

Abstract

The present invention discloses an image quality assessment method based on the order of image content and image distortion perception, comprising the following steps: Step 1: Image selection and distortion processing; obtaining a set of original landscape images, a set of original character images, a set of distorted landscape images, and a set of distorted character images as image stimuli for subjects to view; Step 2: EEG signal acquisition experiment; collecting EEG signals generated by subjects when viewing image stimuli with different contents and different distortion conditions; Step 3: EEG signal preprocessing; processing the EEG signals collected in Step 2 to achieve data cleaning and data transformation. Step 4: Constructing and training an EEG signal quality assessment network. The present invention can construct and train an EEG signal quality assessment network to obtain quality prediction scores for images corresponding to the EEG signals.

Description

Technical Field

[0001] The present invention belongs to the technical field of image quality evaluation, and in particular relates to an image quality evaluation method based on the order of image content and image distortion perception. Background Art

[0002] Images are important visual carriers in our daily lives. Quality loss is inevitable during the acquisition, transmission, and processing processes, affecting people's perception of images. Therefore, designing effective quality evaluation methods is of great significance for the transmission of visual information in images.

[0003] Image quality assessment methods are primarily categorized as subjective and objective. Subjective methods rely on statistically extensive data sets and are therefore very expensive to perform. Objective methods primarily analyze images through the construction of algorithmic models, which can lead to discrepancies between quality scores and subjective evaluations. Electroencephalography (EEG) holds great promise in image quality assessment. In image quality assessment, EEG responses are non-invasively recorded while viewing images of varying quality levels, allowing analysis of the impact of image quality on perception and cognition.

[0004] Existing EEG research has demonstrated that high-quality images have a more positive impact on the brain's attention and cognition. By analyzing EEG signals over time and applying different processing methods to segments with different representational meanings, this provides prior knowledge for building quality assessment algorithms. Compared to objective quality assessment, EEG-based image quality assessment can better reflect the human eye's quality perception mechanism and overcome the high experimental costs of subjective quality assessment, making it of great research value.

[0005] The patent document “Image quality evaluation method and device based on EEG features” with patent application number CN202110700519.0 and application publication number CN 113554597 A discloses an image quality evaluation method based on EEG features. The disadvantage of this method is that the visual processing time of the human eye for image content and image distortion is at the millisecond level, and the order of perception of the two cannot be distinguished by the naked eye alone. Exploring the laws of perception order is conducive to designing an image quality evaluation model with better performance. Therefore, when performing image quality evaluation, this method does not take into account the perception order of the human visual system when perceiving image content and image distortion, resulting in low accuracy of the model's prediction results. Summary of the Invention

[0006] To overcome the problems of the above-mentioned prior art, the present invention aims to provide an image quality assessment method based on the perceptual order of image content and image distortion. Two groups of images with significant differences in image content are selected and distorted. A perceptual order experimental group experiment and a perceptual order control group experiment are conducted to obtain the corresponding behavioral responses of the subjects to the original and distorted images of different contents. An EEG signal acquisition experiment is conducted, and the EEG signals obtained from the perceptual order experimental group experiment and the perceptual order control group experiment are analyzed to obtain the perceptual order pattern between image content and image distortion. The EEG signals are preprocessed, and an EEG signal quality assessment network is constructed and trained to obtain the quality prediction score of the image corresponding to the EEG signal.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] An image quality assessment method based on image content and image distortion perception order comprises the following steps;

[0009] Step 1: Image selection and distortion processing; get the original landscape image set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 , as the image stimulus viewed by the subjects;

[0010] Step 2: EEG signal acquisition experiment: collect EEG signals generated by subjects when viewing image stimuli with different contents and different distortion conditions;

[0011] Step 3: EEG signal preprocessing; used to process the EEG signals collected in step 2 to achieve data cleaning and data transformation.

[0012] Step 4: Build and train the EEG signal quality evaluation network.

[0013] The step 1 is specifically as follows:

[0014] Step (1) image selection;

[0015] Select landscape images and person images from a high-definition image website to form an image set P. For example, select 3 landscape images and 3 person images.

[0016] Step (2) image distortion processing;

[0017] All images in the image set P are distorted to obtain the original landscape image set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22; Set the image quality parameter to be higher than the distortion perception threshold without affecting the subject's perception of the image content; obtain a result of the distortion processing of an image, namely P 11 With P 21 The images in are one-to-one corresponding, and similarly P 12 With P 22 It also corresponds one to one.

[0018] The step 2 is specifically as follows:

[0019] Step 2.1: Single trial process of the perceptual sequence experimental group experiment;

[0020] The perceptual sequence experiment required the subjects to select corresponding behavioral responses as feedback based on the image content and image distortion. The specific content is as follows:

[0021] (1) When the subject observed that the image content was a landscape, he or she chose to use the left hand to prepare for the key press; when the subject observed that the image content was a person, he or she chose to use the right hand to prepare for the key press;

[0022] (2) The subject presses the button when the subject observes the image as a distorted image, and does not press the button when the subject observes the image as an undistorted image.

[0023] Furthermore, a single trial process includes three parts: the attention point presentation stage, the experimental stimulus presentation stage, and the judgment stage;

[0024] During the fixation point presentation phase, a white plus sign was presented to the subjects for 1 second to make them focus on the experimental stimuli. At the same time, the fixation point presentation phase separated the two single trials, making the subjects aware that another single trial had begun.

[0025] Next, in the experimental stimulus presentation phase, the subjects were randomly presented with experimental stimuli containing two cues: image content and image distortion. The presentation time was 2 seconds.

[0026] Finally, in the judgment stage, the subjects were asked to select the corresponding behavioral response as feedback based on the observed experimental stimuli.

[0027] Step 2.2: Single trial process of the perceptual order control group experiment;

[0028] In the perceptual sequence experiment, the experimental group experiment assumes that image content perception precedes image distortion perception, and the control group experiment proposes the opposite hypothesis to eliminate the influence of the behavioral task set based on the experimental hypothesis on the experimental results;

[0029] The perceptual order control group experiment required the subjects to select corresponding behavioral responses as feedback based on the image content and image distortion. The specific content is as follows:

[0030] (1) When the subject observed the image as distorted, he / she chose the left hand to prepare for the key press; when the subject observed the image as undistorted, he / she chose the right hand to prepare for the key press;

[0031] (2) When the subject observes that the image content is a landscape, the subject presses the button; when the subject observes that the image content is a person, the subject does not press the button.

[0032] The single experimental process of the control group is as follows;

[0033] A single trial process includes three parts: the fixation point presentation stage, the experimental stimulus presentation stage and the judgment stage.

[0034] The 0.5-second white plus sign enables the subjects to focus their attention on the experimental stimulus. At the same time, the fixation point presentation phase can separate the two single trials, allowing the subjects to understand that another single trial has begun;

[0035] Next, in the experimental stimulus presentation phase, the subjects were randomly presented with experimental stimuli containing two cues: image content and image distortion. The presentation time was 4 seconds.

[0036] Finally, in the judgment stage, the subjects were asked to select the corresponding behavioral response as feedback based on the observed experimental stimuli.

[0037] Step 2.3: The subject views the distorted image;

[0038] Like the subject randomly plays the original landscape image set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 The images in were used as experimental stimuli.

[0039] Step 2.4: EEG signal acquisition;

[0040] During each experimental stimulation, EEG signals were collected using the NeuroScan system, which includes a Quik-Cap 64-conductor electrode cap, a SynAmps 2 64-conductor EEG signal amplifier, and Curry 7, a professional brain signal acquisition and processing software. The Quik-Cap 64-conductor electrode cap contacts the human scalp and receives human EEG signals. The SynAmps 2 64-conductor EEG signal amplifier amplifies the EEG signals received by the Quik-Cap 64-conductor electrode cap and sends them to Curry 7 software for data processing.

[0041] Furthermore, during the EEG signal acquisition process, the following acquisition conditions need to be met:

[0042] (1) Subjects were required to remain energetic and avoid inattention, frequent blinking, and drowsiness during the collection process;

[0043] (2) Ensure that the resistance of all electrodes of the conductive cap is less than 20 kilo-ohms (this resistance value indicates the degree of contact between the conductive cap and the human scalp, the smaller the better), and avoid adhesion between the electrodes;

[0044] (3) The experimental site has sufficient light, suitable temperature, and no noise;

[0045] (4) Ensure that the distance between the subject's eyes and the image stimulus is about 4 times the display height of the image stimulus.

[0046] Step 2.5: EEG signal analysis;

[0047] Subjects showed the same behavioral responses to the perception of image content and image distortion based on the cognitive processing of image perception.

[0048] In summary, when image distortion is higher than the level of merely noticeable distortion but lower than the level that affects content perception, the human visual system has a definite perceptual order for image content and image distortion, and the human visual system's perception of image content takes precedence over the perception of image distortion.

[0049] The preprocessing steps in step 3 include reference transfer, baseline correction, filtering, and artifact removal;

[0050] Step 3.1: Transfer reference;

[0051] The M1 and M2 electrodes located at the mastoid processes behind the ears on both sides are used as reference electrodes. The average value of the EEG signals collected by the M1 and M2 electrodes is used as the reference value of the EEG signals. Based on the reference value, the EEG signals of all electrodes are recalculated, and the voltage difference between M1 and M2 is subtracted from the voltage difference between other electrodes.

[0052] Step 3.2: Baseline correction;

[0053] From the referenced EEG signal, select the EEG signal segment from 200 milliseconds before the image stimulus is presented to the start of the image stimulus (the distorted image sequence viewed by the subject in step 2.3 consists of multiple image stimuli. Each time an image stimulus is viewed, an EEG signal is generated. That is, each image stimulus corresponds to an EEG signal. Therefore, the image stimulus here refers to the image stimulus corresponding to the EEG signal being preprocessed). Calculate the mean of this EEG signal segment, and subtract the mean value from the entire EEG signal segment from the mean value from 200 milliseconds before the image stimulus is presented to the start of the image stimulus.

[0054] Step: 3.3 Bandpass filtering;

[0055] The EEG signal after baseline correction was subjected to a band-pass filter to cut off the parts below 0.1 Hz and above 20 Hz, and retain the part between 0.1 and 20 Hz;

[0056] Step 3.4: Remove artifacts;

[0057] Use the independent component analysis function in the processing software Curry7 to remove artifacts from the EEG signal that has been bandpass filtered in step 3.3.

[0058] The step 4 is specifically as follows:

[0059] Step 4.1: Divide the training set and test set;

[0060] Select a portion of the preprocessed EEG samples in step 3 as a training set and another portion as a test set. The dimension of each EEG signal data sample is C×T.

[0061] Step 4.2: truncating the EEG signal preprocessed in step 3 to obtain an EEG signal segment N representing the image content and an EEG signal segment D representing the image distortion;

[0062] Step 4.3: Filter the EEG signal segment N representing the image content obtained in step 4.2 to obtain the 4-7 Hz theta wave N θ ;

[0063] Step 4.4: Construct the EEG signal quality feature extraction network S;

[0064] Construct an EEG signal quality feature extraction network S; where module 1 is a spatiotemporal convolution-average pooling module, module 2 is a separable convolution module, and the EEG signal input dimension is C×T, where C represents the number of EEG signal channels and T represents the number of sampling points;

[0065] Step 4.5: Combine the EEG signal segment D representing the image distortion obtained in step 4.2 and the θ wave N representing the image content obtained in step 4.3 θ Input into the EEG signal quality feature extraction network S to obtain the quality evaluation classification result.

[0066] Step 4.6: Train the EEG signal quality evaluation network;

[0067] During the training phase, the regularization coefficient is set to suppress overfitting. All training processes use the Adam optimization algorithm. The formula for calculating the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample is:

[0068]

[0069] b represents the number of training samples randomly selected from the training sample set B without replacement during iterative training of the EEG signal quality evaluation network S, q g represents the quality score label corresponding to the g-th training sample, Represents the quality prediction score corresponding to the g-th training sample.

[0070] The step 4.4 is specifically as follows:

[0071] In module 1, the first layer is a temporal convolution layer, which uses eight convolution kernels of equal size to extract the temporal features of the EEG signal. The second layer is a spatial convolution layer, which uses four deep convolution kernels of size C×1 to learn spatial filters. Batch normalization is then applied along the feature map dimensions, using an exponential linear unit as the activation function. Finally, a 1x4 average pooling layer is used for downsampling. In addition, the dropout technique is used to prevent overfitting, and the weights of each spatial filter are regularized by applying a maximum norm constraint of 1.

[0072] In module 2, four convolution kernels of equal size are first used for separable convolution, followed by eight convolution kernels of equal size for pointwise convolution. An exponential linear unit is then used as the activation function, and finally an average pooling layer is used for dimensionality reduction. Dropout is used to prevent overfitting. This step compresses the feature dimension to 1*10 to obtain the quality assessment classification results.

[0073] Beneficial effects of the present invention:

[0074] The key point of this invention is to utilize the different reactions of the brain's attention and cognitive systems caused by viewing distorted images in the human visual perception mechanism. By collecting the EEG signals generated when the subjects view image stimuli with different contents and different degrees of distortion, an image quality evaluation method based on image content and image distortion perception order is designed.

[0075] This study experimentally confirmed that subjects' perceptions of image content and distortion differ over time when viewing image stimuli. The study also determined the start and end times of the distortion- and content-representing EEG signal segments within the EEG signals generated by these stimuli. These distortion- and content-representing EEG signal segments were fed into an EEG signal quality assessment network to generate image quality scores that outperformed classical image quality assessment methods used in recent years. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of the process of the present invention.

[0077] Figure 2 This is a schematic diagram of a single experiment flow of the perception sequence experimental group of the present invention.

[0078] Figure 3 This is a schematic diagram of a single experimental process for the perception sequence control group of the present invention.

[0079] Figure 4 Schematic diagram of the perception sequence experiment flow of the present invention.

[0080] Figure 5 Schematic diagram of the EEG signal quality feature extraction network.

[0081] Figure 6 Schematic diagram of the preparatory laterality potential results for eight subjects. DETAILED DESCRIPTION

[0082] The present invention will be described in further detail below with reference to the accompanying drawings.

[0083] like Figure 1 As shown: Step 1 image selection and distortion processing;

[0084] (1) Image selection

[0085] Three landscape images and three person images are selected from a high-definition image website to form an image set P containing six images.

[0086] (2) Image distortion processing

[0087] The above images were distorted. Excessive distortion can affect the subject's perception of image content, while low distortion can render the subject unable to discern the distortion. Therefore, the image quality parameter was set to a value above the distortion perception threshold without affecting the subject's perception of image content.

[0088] In this embodiment, the VideoWriter tool of MATLAB software is used to adjust the quality parameter (QP) of the image, and the six images in the image set P are distorted to obtain the original landscape image set P. 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 .

[0089] The QP range is 0 to 100. A QP of 100 indicates an undistorted image, while a QP of 0 indicates the most severe image distortion. A larger QP value reduces distortion, making it less noticeable to the subject. Experience shows that image distortion is very noticeable when the QP is between 0 and 10, while it is imperceptible when the QP is between 80 and 100. In this example, the OP value for distorted images is uniformly set to 40.

[0090] Step 2: EEG signal acquisition experiment;

[0091] 2.1: Single process of the perceptual sequence experimental group experiment;

[0092] The perceptual sequence experiment required the subjects to select corresponding behavioral responses as feedback based on the image content and image distortion. The specific content is as follows:

[0093] The original landscape image set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 Played randomly to the subjects.

[0094] (1) When the subject observes that the image content is a landscape, the subject chooses the left hand to prepare for the key press; when the subject observes that the image content is a person, the subject chooses the right hand to prepare for the key press.

[0095] (2) The subject presses the button when the subject observes the image as a distorted image, and does not press the button when the subject observes the image as an undistorted image.

[0096] like Figure 2 As shown in the figure, a single trial consists of three phases: the attention point presentation phase, the experimental stimulus presentation phase, and the judgment phase. During the attention point presentation phase, a white plus sign is displayed to the subject for one second, allowing them to focus their attention on the experimental stimulus. This phase also serves to separate the two single trials, ensuring that the subject is aware that another single trial has begun. Next, during the experimental stimulus presentation phase, the subject is randomly presented with experimental stimuli containing two cues, image content and image distortion, for two seconds. Finally, during the judgment phase, the subject is asked to select a behavioral response based on the observed experimental stimulus as feedback.

[0097] 2.2: Single process of the perceptual order control group experiment;

[0098] In the perceptual sequence experiment, the experimental group experiment assumes that image content perception precedes image distortion perception, and the control group experiment proposes the opposite hypothesis to eliminate the influence of the behavioral tasks set based on the experimental hypothesis on the experimental results.

[0099] The perceptual order control group experiment required the subjects to select corresponding behavioral responses as feedback based on the image content and image distortion. The specific content is as follows:

[0100] (1) When the subject observed the image as distorted, he / she chose the left hand to prepare for the key press; when the subject observed the image as undistorted, he / she chose the right hand to prepare for the key press;

[0101] (2) When the subject observes that the image content is a landscape, the subject presses the button; when the subject observes that the image content is a person, the subject does not press the button.

[0102] The single process of the control group experiment is as follows Figure 3 As shown in the figure, a single trial process consists of three parts: the fixation point presentation phase, the experimental stimulus presentation phase, and the judgment phase. During the fixation point presentation phase, a white plus sign is presented to the subject for 0.5 seconds, allowing the subject to focus on the experimental stimulus. At the same time, the fixation point presentation phase separates the two single trials, allowing the subject to understand that another single trial has begun. Next, during the experimental stimulus presentation phase, the subject is randomly presented with experimental stimuli containing two cues, image content and image distortion, for 4 seconds. Finally, in the judgment phase, the subject is required to select a corresponding behavioral response as feedback based on the observed experimental stimulus.

[0103] 2.3: Subjects viewed distorted images;

[0104] Original Landscape Image Set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 There are 12 images in total.

[0105] In this experiment, 8 subjects (4 males and 4 females) were randomly presented with the above 12 images as stimuli, and each stimulus was repeated 50 times. Each subject was required to complete 200 trials in total. The specific experimental process is as follows: Figure 4 As shown:

[0106] 2.4: EEG signal acquisition;

[0107] EEG signals were collected during each trial of the subjects using the NeuroScan EEG signal acquisition system, which includes a Quik-Cap 64-conductor electrode cap, a SynAmps2 64-conductor EEG signal amplifier, and the professional brain signal acquisition and processing software Curry7.

[0108] During the EEG data collection process, the following collection conditions need to be met:

[0109] (1) Subjects were required to remain energetic and avoid inattention, frequent blinking, and drowsiness during the collection process;

[0110] (2) Ensure that the resistance of all electrodes of the conductive cap is less than 20 kilo-ohms and avoid adhesion between the electrodes;

[0111] (3) The experimental site has sufficient light, suitable temperature, and no noise;

[0112] (4) Ensure that the distance between the subject's eyes and the image stimulus is about 4 times the display height of the image stimulus.

[0113] In this embodiment, a total of 4800 EEG signal samples were collected from 8 subjects. The dimension of each EEG signal data sample is C×T, where C represents the number of channels of the EEG signal and T represents the number of sampling points.

[0114] In this embodiment, C is 64 and T is 1000.

[0115] 2.5: EEG signal analysis;

[0116] In this embodiment, the preparatory laterality potentials of 8 subjects are shown in the following table. Figure 6 As can be seen from the results, the preparatory lateralization potentials of all eight subjects in the experimental group met the hypothesized expectations for the experimental group, while the preparatory lateralization potentials of all eight subjects in the control group did not meet the hypothesized expectations for the control group. All eight subjects exhibited similar behavioral responses to image content and image distortion, consistent with the cognitive processing of image perception. Therefore, this experiment suggests that there is a definite perceptual order for image content and image distortion, and the perceptual order explored in this experiment is not a false proposition.

[0117] In summary, when image distortion is higher than the level of merely noticeable distortion but lower than the level that affects content perception, the human visual system has a definite perceptual order for image content and image distortion, and the human visual system's perception of image content takes precedence over the perception of image distortion.

[0118] Step 3: EEG signal preprocessing;

[0119] To more accurately analyze the characteristics and changes of EEG signals, raw EEG data needs to be preprocessed to achieve data cleaning and data transformation. Preprocessing steps include reference transfer, baseline correction, filtering, and artifact removal.

[0120] 3.1: For reference;

[0121] The NeuroScan acquisition system used in this experiment defaults to using the Ref electrode in the Quik-Cap as the reference electrode. Because the Ref electrode often varies in location among subjects due to varying head sizes, the M1 and M2 electrodes, located bilaterally behind the ears on the mastoid process, were selected as reference electrodes. The average value of the signals collected by the M1 and M2 electrodes was used as the reference value for the EEG signals, and the EEG signals of all electrodes were recalculated based on this reference value.

[0122] 3.2: Baseline correction;

[0123] Baseline correction can avoid EEG drift caused by noise interference and imbalance between different electrodes. Therefore, we selected the EEG signal segment from 200 milliseconds before the image stimulus was presented to the beginning of the image stimulus, and calculated the mean of this EEG signal segment as the baseline for correction of the entire EEG signal.

[0124] 3.3: Bandpass filtering;

[0125] During the EEG acquisition process, signals contain a significant amount of both physiological and non-physiological noise. Physiological noise, such as myoelectric and oculoscopic noise, is high-frequency, while non-physiological noise, such as DC offset, is low-frequency. Therefore, in this experiment, a bandpass filter was used to intercept noise signals below 0.1 Hz and above 20 Hz, retaining the EEG signal between 0.1 and 20 Hz.

[0126] 3.4: Remove artifacts;

[0127] During EEG signal acquisition, interference from sources such as blinking, muscle movement, and heartbeats can leave artifacts similar to those in the EEG signal. These artifacts can interfere with EEG signal interpretation and analysis, and cannot be completely removed using bandpass filtering. Therefore, in this experiment, the independent component analysis function in the processing software Curry7 was used to remove these artifacts.

[0128] Step 4: Build and train the EEG signal quality evaluation network;

[0129] 4.1: Divide the training set and test set;

[0130] There are 4800 EEG samples in total, 80% of which are selected as the training set (3840) and 20% as the test set (960). The dimension of each EEG signal data sample is C×T. In this embodiment, C is 64 and T is 1000.

[0131] 4.2: Truncate the EEG signal to obtain the EEG signal segment N that perceives the image content and the EEG signal segment D that represents the image distortion.

[0132] Based on the latency of P300 in the EEG signal, this experiment first scanned the positive peak of the EEG signal from 300 milliseconds to 600 milliseconds, cut off the segment of the EEG signal from 100 milliseconds before the positive peak to 100 milliseconds after the positive peak as the EEG signal segment N representing the image content perceived by the human eye, and cut off the segment of the EEG signal from the beginning to the positive peak as the EEG signal segment D representing the image distortion.

[0133] 4.3: Filter the EEG signal segment N representing the image content to obtain the 4-7 Hz theta wave N θ .

[0134] 4.4: Construct EEG signal quality feature extraction network S.

[0135] Construct the EEG signal quality feature extraction network S. The network structure is as follows Figure 5 As shown in Figure 1, module 1 is the spatiotemporal convolution-average pooling module, and module 2 is the separable convolution module. The input dimension of the EEG signal is C×T, where C represents the number of EEG signal channels and T represents the number of sampling points.

[0136] In module 1, the first layer is a temporal convolution layer, using eight 1×64 convolution kernels to extract temporal features of the EEG signal. The second layer is a spatial convolution layer, using four C×1 depthwise convolution kernels to learn spatial filters. Batch normalization is then applied along the feature map dimensions, using an exponential linear unit as the activation function. Finally, a 1×4 average pooling layer is used for downsampling. Dropout is also used to prevent overfitting, regularizing each spatial filter weight by applying a maximum norm constraint of 1.

[0137] In module 2, four 1x16 kernels are used for separable convolution, followed by eight 1x1 kernels for pointwise convolution. An exponential linear unit is used as the activation function, and finally a 1x8 average pooling layer is used for dimensionality reduction. Dropout is used to prevent overfitting.

[0138] 4.5: The EEG signal θ wave N representing the image content θ The EEG signal D representing the image distortion is input into the EEG signal quality feature extraction network S to obtain the quality evaluation classification result.

[0139] 4.6: Training EEG signal quality evaluation network;

[0140] During the training phase, the regularization coefficient is set to suppress overfitting, the batch size is set to 64, and the initial learning rate Ir = 2×10 -3 After every 50 training iterations, the learning rate is reduced to 1 / 10 of the previous stage, and a total of 200 training iterations are run. All training processes use the Adam optimization algorithm. The formula for calculating the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample is:

[0141]

[0142] b represents the number of training samples randomly selected from the training sample set B without replacement during iterative training of the EEG signal quality evaluation network S, q g represents the quality score label corresponding to the g-th training sample, Represents the quality prediction score corresponding to the g-th training sample.

[0143] Step 5: Model classification results;

[0144] The method proposed in this application was selected to compare the classification accuracy with ShallowConvNet (S-Conv), OCCLN, BN3, and CNN-3. The results are shown in the table.

[0145] From the results in the table, we can see that the classification accuracy of the model proposed in this chapter is 81.54, which is better than other deep learning algorithms.

[0146] Multi-model comparison results

[0147] Model of this chapter (%) S-Conv(%) OCCLN (%) BN3(%) CNN-3 (%) 81.54 76.02 74.92 71.31 72.06

[0148] The above content is only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. An image quality assessment method based on image content and image distortion perception order, characterized in that: The following steps are involved: Step 1: Image selection and distortion processing; Get the original landscape image set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 , as the image stimulus viewed by the subjects; Step 2: EEG signal acquisition experiment: collect EEG signals generated by subjects when viewing image stimuli with different contents and different distortion conditions; Step 3: EEG signal preprocessing: used to process the EEG signals collected in step 2 to achieve data cleaning and data transformation; Step 4: Build and train the EEG signal quality evaluation network; The step 2 is specifically as follows: Step 2.1: Single trial process of the perceptual sequence experimental group experiment; The perceptual sequence experiment required the subjects to select corresponding behavioral responses as feedback based on the image content and image distortion. The specific content is as follows: (1) When the subject observed that the image content was a landscape, he or she chose to use the left hand to prepare for the key press; when the subject observed that the image content was a person, he or she chose to use the right hand to prepare for the key press; (2) The subject presses the button when the subject observes the image as a distorted image, and does not press the button when the subject observes the image as an undistorted image; Step 2.2: Single trial process of the perceptual order control group experiment; In the perceptual sequence experiment, the experimental group experiment assumes that image content perception precedes image distortion perception, and the control group experiment proposes the opposite hypothesis to eliminate the influence of the behavioral task set based on the experimental hypothesis on the experimental results; The perceptual order control group experiment required the subjects to select corresponding behavioral responses as feedback based on the image content and image distortion. The specific content is as follows: (1) When the subject observed the image as distorted, he / she chose the left hand to prepare for the key press; when the subject observed the image as undistorted, he / she chose the right hand to prepare for the key press; (2) The subject presses the button when the image content is a landscape, but does not press the button when the image content is a person; Step 2.3: The subject views the distorted image; The original landscape image set P is randomly played to the subjects. 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 The images in were used as experimental stimuli; Step 2.4: EEG signal acquisition; During each experimental stimulation, EEG signals were collected using a NeuroScan system, which includes a Quik-Cap 64-conductor electrode cap, a SynAmps 2 64-conductor EEG signal amplifier, and Curry 7, a professional brain signal acquisition and processing software. The Quik-Cap 64-conductor electrode cap contacts the human scalp to receive EEG signals. The SynAmps 2 64-conductor EEG signal amplifier amplifies the EEG signals received by the Quik-Cap 64-conductor electrode cap and then feeds them into Curry 7 software for data processing. Step 2.5: EEG signal analysis; Subjects exhibited similar behavioral responses to both image content and image distortion, based on the cognitive processing of image perception. When image distortion was above the level of merely noticeable distortion but below the level that affected content perception, the human visual system exhibited a definite perceptual order for image content and image distortion, with the human visual system prioritizing image content over distortion. The step 4 is specifically as follows: Step 4.1: Divide the training set and test set; Select a portion of the preprocessed EEG samples in step 3 as a training set and another portion as a test set. The dimension of each EEG signal data sample is C×T. Step 4.2: truncating the EEG signal preprocessed in step 3 to obtain an EEG signal segment N representing the image content and an EEG signal segment D representing the image distortion; Step 4.3: Filter the EEG signal segment N representing the image content obtained in step 4.2 to obtain the 4-7 Hz theta wave N θ ; Step 4.4: Construct an EEG signal quality evaluation network S; Construct an EEG signal quality evaluation network S; where module 1 is a spatiotemporal convolution-average pooling module, module 2 is a separable convolution module, and the EEG signal input dimension is C×T, where C represents the number of EEG signal channels and T represents the number of sampling points; Step 4.5: Combine the EEG signal segment D representing the image distortion obtained in step 4.2 and the θ wave N representing the image content obtained in step 4.3 θ Input into the EEG signal quality evaluation network S to obtain the quality evaluation classification result; Step 4.6: Train the EEG signal quality evaluation network S; The step 4.6 is specifically as follows: During the training phase, all training processes use the Adam optimization algorithm. The formula for calculating the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample is: b represents the number of training samples randomly selected from the training sample set B without replacement during iterative training of the EEG signal quality evaluation network S, q g represents the quality score label corresponding to the g-th training sample, Represents the quality prediction score corresponding to the g-th training sample.

2. The image quality assessment method based on image content and image distortion perception order according to claim 1, characterized in that: The step 1 is specifically as follows: Step (1) image selection; Select landscape images and person images from high-definition image websites to form an image set P; Step (2) image distortion processing; All images in the image set P are distorted to obtain the original landscape image set P 11 , original character image set P 12 , distorted landscape image set P 21 , distorted character image set P 22 ;Set the image quality parameter to be higher than the distortion perception threshold without affecting the subject's perception of the image content; The distortion processing of an image yields a result, namely P 11 With P 21 The images in are one-to-one corresponding, and similarly P 12 With P 22 It also corresponds one to one.

3. The image quality assessment method based on image content and image distortion perception order according to claim 1, characterized in that: In the step 2.1; A single trial process includes three parts: attention point presentation stage, experimental stimulus presentation stage and judgment stage; During the fixation point presentation phase, a white plus sign was presented to the subjects for 1 second to make them focus on the experimental stimuli. At the same time, the fixation point presentation phase separated the two single trials, making the subjects aware that another single trial had begun. Next, in the experimental stimulus presentation phase, the subjects were randomly presented with experimental stimuli containing two cues: image content and image distortion. The presentation time was 2 seconds. Finally, in the judgment stage, the subjects were asked to select the corresponding behavioral response as feedback based on the observed experimental stimuli.

4. The image quality assessment method based on image content and image distortion perception order according to claim 1, characterized in that: The step 2.2 is specifically as follows: The single experimental process of the control group is as follows; A single trial process consists of three parts: the fixation point presentation phase, the experimental stimulus presentation phase, and the judgment phase. During the fixation point presentation phase, a white plus sign is presented to the subject for 0.5 seconds to encourage the subject to focus on the experimental stimulus. The fixation point presentation phase also separates the two single trials, allowing the subject to understand that the other single trial has begun. Next, in the experimental stimulus presentation phase, the subjects were randomly presented with experimental stimuli containing two cues: image content and image distortion. The presentation time was 4 seconds. Finally, in the judgment stage, the subjects were asked to select the corresponding behavioral response as feedback based on the observed experimental stimuli.

5. The image quality assessment method based on image content and image distortion perception order according to claim 1, characterized in that: During the EEG signal acquisition process in step 2.4, the following acquisition conditions need to be met: (1) Subjects were required to remain energetic and avoid inattention, frequent blinking, and drowsiness during the collection process; (2) Ensure that the resistance of all electrodes of the conductive cap is less than 20 kilo-ohms and avoid adhesion between the electrodes; (3) The experimental site has sufficient light, suitable temperature, and no noise; (4) Ensure that the distance between the subject's eyes and the image stimulus is 4 times the display height of the image stimulus.

6. The image quality assessment method based on image content and image distortion perception order according to claim 1, characterized in that: The preprocessing steps in step 3 include reference transfer, baseline correction, filtering, and artifact removal; Step 3.1: Transfer reference; The M1 and M2 electrodes located at the bilateral mastoid processes behind the ears are used as reference electrodes. The average value of the EEG signals collected by the M1 and M2 electrodes is used as the reference value of the EEG signals. The EEG signals of all electrodes are recalculated based on the reference value, and the voltage difference between M1 and M2 is subtracted from the voltage difference between other electrodes. Step 3.2: Baseline correction; The EEG signal segment from 200 milliseconds before the image stimulus was presented to the beginning of the image stimulus was selected from the reference EEG signal, the mean of the EEG signal segment was calculated, and the mean of the segment from 200 milliseconds before the image stimulus was presented to the beginning of the image stimulus was subtracted from the entire EEG signal segment; Step: 3.3 Bandpass filtering; The EEG signal after baseline correction was subjected to a band-pass filter to cut off the parts below 0.1 Hz and above 20 Hz, and retain the part between 0.1 and 20 Hz; Step 3.4: Remove artifacts; Use the independent component analysis function in the processing software Curry7 to remove artifacts from the EEG signal that has been bandpass filtered in step 3.

3.

7. The image quality assessment method based on image content and image distortion perception order according to claim 1, characterized in that: The step 4.4 is specifically as follows: In module 1, the first layer is the temporal convolution layer, which uses 8 convolution kernels of the same size to extract the temporal features of the EEG signal; the second layer is the spatial convolution layer, which uses 4 deep convolution kernels of size C×1 to learn the spatial filter; Then, batch normalization is applied along the feature map dimensions, using exponential linear units as the activation function. Finally, a 1x4 average pooling layer is used for downsampling. In addition, the Dropout technique is used to prevent overfitting, and the weights of each spatial filter are regularized by applying a maximum norm constraint of 1. In module 2, four convolution kernels of the same size are first used for separable convolution, followed by eight convolution kernels of the same size for point-wise convolution. Then, an exponential linear unit is used as the activation function, and finally an average pooling layer is used for dimensionality reduction. Dropout technology is used to prevent overfitting. This step is used to compress the feature dimension to 1*10 to obtain the quality evaluation classification result.

Images