A mobile phone photography effect comprehensive adjustment method and system

Abstract

The application provides a mobile phone photography effect comprehensive adjustment method and system, and belongs to the technical field of image processing, and comprises the following steps: obtaining a basic adjustment data set based on a quantitative reference, a comparison model number and mobile phone photography scene parameters; obtaining a data set label by weightedly averaging a subjective evaluation mechanism, a comparison machine evaluation score mechanism and an evaluation grade standard; adding the data set label to the basic adjustment data set to obtain a comprehensive adjustment data set; determining a mobile phone photography effect basic adjustment model architecture, training a mobile phone photography effect adjustment basic model by using a preset loss function and the comprehensive adjustment data set, and obtaining a mobile phone photography effect comprehensive adjustment model; and inputting an adjusted mobile phone photography picture into the mobile phone photography effect comprehensive adjustment model and outputting a mobile phone photography effect comprehensive adjustment result. The mobile phone photography comprehensive adjustment method provided by the application can effectively solve the problem that the adjustment indicators in the mobile phone photography field are inconsistent with the subjective feelings of the human eye, and the quality of automatically quantized mobile phone photographed pictures is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for comprehensive adjustment of mobile phone photography effects. Background Technology

[0002] As smartphones have become an indispensable part of people's lives, mobile phone photography has largely met people's daily shooting needs. However, due to the combined effects of factors such as phone positioning, cost, component supply, processing chips, and the subjective preferences of tuning personnel, the photographic effects of different mobile phones vary greatly, making it difficult to maintain a consistent user experience.

[0003] To address the aforementioned issues, the smartphone industry provides corresponding image output calibration interfaces and has established a comprehensive calibration quantitative benchmark with reference value ranges for use as quantitative indicators in the calibration process. However, since cameras lack human-like conditioned reflexes to factors such as color and environment, calibration based solely on quantitative indicators can result in final image quality that meets theoretical values ​​but fails to satisfy human visual perception and artistic expression needs. Meanwhile, some studies combining blind testing and subjective human evaluation aim to improve the subjective visual effect of high-quality photographs, but their objectivity is limited by the sample size and the professionalism of the test subjects, hindering effective implementation and in-depth exploration. In the field of image processing, deep learning technology learns the inherent patterns and representational layers of sample data. The information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved far greater results in speech and image recognition than previous related technologies. Subjective evaluation methods involve conducting image rating experiments similar to those in psychology or sociology, to evaluate images based on individual subjective assessments. This approach typically requires a relatively fixed set of steps: preparing an image set; inviting observers to conduct evaluations; and weighting the ratings to arrive at a final score for image quality evaluation.

[0004] It can be seen that the existing technical solutions have the following shortcomings:

[0005] (1) Quantitative indicators contradict the subjective feelings of the human eye;

[0006] (2) Subjective manual evaluation requires a lot of manpower and resources to take pictures of the sample photos and evaluate them, and also requires a high level of professionalism from the evaluators.

[0007] (3) There is a lack of subjective benchmark datasets to guide the design of comprehensive calibration quantitative indicators that conform to the subjective perception of the human eye.

[0008] Therefore, how to design an automatic comprehensive adjustment method that conforms to the subjective perception of the human eye has become an urgent problem to be solved in the comprehensive adjustment of mobile phone photography effects. Summary of the Invention

[0009] This invention provides a method and system for comprehensive adjustment of mobile phone photography effects, which solves the shortcomings of current mobile phone photography comprehensive adjustment that cannot meet the visual needs of the human eye, as well as the shortcomings of traditional subjective manual evaluation that consumes a lot of resources and requires a high degree of professionalism.

[0010] In a first aspect, the present invention provides a method for comprehensive adjustment of mobile phone photography effects, including:

[0011] Obtain the quantitative benchmark and reference camera model of the mobile phone, determine the mobile phone photography scene parameters, and obtain the basic calibration dataset based on the quantitative benchmark, the reference camera model and the mobile phone photography scene parameters;

[0012] Determine the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation personnel level standard; then, perform a weighted average of the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation level standard to obtain the dataset label.

[0013] Add the dataset labels to the basic calibration dataset to obtain the comprehensive calibration dataset;

[0014] The basic model architecture for mobile phone photography effect is determined, and the basic model architecture for mobile phone photography effect is trained using a preset loss function and the comprehensive tuning dataset to obtain a comprehensive tuning model for mobile phone photography effect.

[0015] Input the mobile phone photography image into the comprehensive mobile phone photography effect adjustment model, and output the comprehensive mobile phone photography effect adjustment result.

[0016] According to a comprehensive adjustment method for mobile phone photography effects provided by the present invention, a quantitative benchmark of the photography mobile phone and a reference mobile phone model are obtained, mobile phone photography scene parameters are determined, and a basic adjustment dataset is obtained based on the quantitative benchmark, the reference mobile phone model, and the mobile phone photography scene parameters, including:

[0017] Based on the key imaging characteristic parameters and parameter value ranges of the mobile phone, the quantitative benchmark of the photography mobile phone is determined, and the reference model is determined based on the quantitative indicators of the mobile phone calibration.

[0018] Determine the shooting location and the color temperature of the location, and construct the mobile phone photography location parameters based on the shooting location and the color temperature of the location;

[0019] The basic calibration dataset is obtained by combining the quantitative benchmark of the photography mobile phone, the control camera model, and the mobile phone photography site parameters.

[0020] According to the comprehensive adjustment method for mobile phone photography effects provided by the present invention, a subjective evaluation mechanism, a control machine evaluation score evaluation mechanism, and an evaluation personnel level standard are determined, including:

[0021] By comparing two photographs taken by different control cameras in any identical scene, the subjective evaluation mechanism, including a first comparison result, a second comparison result, and a third comparison result, is determined.

[0022] The subjective quality values ​​of the control camera sample images were used to evaluate two photos taken by different control cameras, resulting in the control camera evaluation score evaluation mechanism.

[0023] Based on the photographer's years of experience, multiple assessment personnel are categorized into different levels.

[0024] According to the comprehensive adjustment method for mobile phone photography effects provided by the present invention, after adding the dataset label to the basic adjustment dataset to obtain the comprehensive adjustment dataset, the method further includes:

[0025] The consistency of the subjective quality value of the control sample image with four objective indicators was tested using the Pearson linear correlation coefficient (PLCC), Kendall's rank correlation coefficient (KROCC), Spearman's rank correlation coefficient (SROCC), and root mean square error (RMSE). The difference between the subjective quality value of the control sample image and the existing quantitative benchmark was obtained to verify whether the existing quantitative benchmark is consistent with human subjective perception.

[0026] According to a comprehensive mobile phone photography effect adjustment method provided by the present invention, a basic adjustment model architecture for mobile phone photography effects is determined, and the basic adjustment model architecture for mobile phone photography effects is trained using a preset loss function and the comprehensive adjustment dataset to obtain a comprehensive adjustment model for mobile phone photography effects, including:

[0027] The basic tuning model architecture for mobile phone photography effects is constructed using Convolutional Neural Networks (CNN) and Transformers.

[0028] Determine the reference photo, and improve the dimensionality information of the control machine sample to obtain the dimensionally improved control machine sample;

[0029] The deep semantic information of the reference photo is extracted by a convolutional neural network (CNN) and a linear layer, as well as the deep semantic information of the comparison machine sample after the dimensionality is increased.

[0030] The differential features are obtained by subtracting the enhanced semantic information of the comparison machine sample from the deeper semantic information of the reference photo.

[0031] The deep semantic information of the improved dimensionality comparison machine sample and the differential features are used as the input of the encoder;

[0032] The differential features, the deep semantic information of the improved dimensionality control sample, and the deep semantic information of the reference photograph are used as inputs to the decoder to obtain the objective value of the control sample.

[0033] The mean square error of the subjective value and the objective value of any comparison camera sample image is used as the loss function to train and converge the basic model architecture for mobile phone photography effect adjustment, thereby obtaining the comprehensive adjustment model for mobile phone photography effect.

[0034] According to the present invention, a method for comprehensive adjustment of mobile phone photography effects is provided, which improves the dimensional information of a control camera sample to obtain a control camera sample with improved dimensionality, including:

[0035] The original control sample was divided into several pre-set square blocks;

[0036] The aforementioned preset square blocks are stacked along the channel dimension of the original control sample to obtain the improved control sample.

[0037] According to the present invention, a comprehensive adjustment method for mobile phone photography effects is provided, wherein the encoder includes a multi-head attention mechanism module, a residual connection and normalization layer, a feedforward neural network, and a residual connection and normalization layer;

[0038] The decoder includes a masked multi-head attention mechanism module, a residual connection and normalization layer, a multi-head attention mechanism module, a residual connection and normalization layer, a feedforward neural network, and a residual connection and normalization layer.

[0039] Secondly, the present invention also provides a comprehensive mobile phone photography effect adjustment system, comprising:

[0040] The data processing module is used to obtain the quantitative benchmark of the camera phone and the reference camera model, determine the camera photography scene parameters, and obtain the basic calibration dataset based on the quantitative benchmark, the reference camera model and the camera photography scene parameters.

[0041] The label processing module is used to determine the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation personnel level standard, and to perform a weighted average of the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation level standard to obtain the dataset labels.

[0042] The integrated data module is used to add the dataset labels to the basic calibration dataset to obtain the integrated calibration dataset.

[0043] The training module is used to determine the basic model architecture for adjusting mobile phone photography effects. The basic model architecture for adjusting mobile phone photography effects is trained using a preset loss function and the comprehensive adjustment dataset to obtain the comprehensive adjustment model for mobile phone photography effects.

[0044] The calibration module is used to input the mobile phone photography images into the comprehensive calibration model of mobile phone photography effects and output the comprehensive calibration results of mobile phone photography effects.

[0045] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the mobile phone photography effect comprehensive adjustment method as described above.

[0046] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the mobile phone photography effect comprehensive adjustment method as described above.

[0047] The mobile phone photography effect comprehensive adjustment method and system provided by this invention, by constructing an overall framework for the production of subjective benchmark datasets in the field of mobile phone photography, proposes an automatic intelligent evaluation method and an automatic mobile phone photo quantification mode, which can effectively solve the problem of inconsistency between adjustment indicators and human subjective perception in the field of mobile phone photography, and realize the automatic quantification of the quality of mobile phone photos. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 This is one of the flowcharts illustrating the comprehensive adjustment method for mobile phone photography effects provided by the present invention;

[0050] Figure 2 This is the second flowchart of the comprehensive adjustment method for mobile phone photography effects provided by the present invention;

[0051] Figure 3 This is a schematic diagram of the basic model for adjusting mobile phone photography effects provided by the present invention;

[0052] Figure 4 This is a schematic diagram of the structure of the mobile phone photography effect comprehensive adjustment system provided by the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] To address the limitations of existing mobile phone photography calibration methods that fail to meet human visual needs and the high resource consumption and professional requirements of traditional subjective manual evaluation, this invention proposes a learnable, human-subjective mobile phone photography effect calibration method. This method solves the problem of requiring manual sample shooting and evaluation in the field of mobile phone photography calibration, and provides a dataset to guide learning in this area. By introducing deep learning technology into this field, it can promote the development of mobile phone photography and even transform the mobile phone industry.

[0056] Figure 1 This is one of the flowcharts illustrating the comprehensive adjustment method for mobile phone photography effects provided in this embodiment of the invention, such as... Figure 1 As shown, it includes:

[0057] Step 100: Obtain the quantitative benchmark of the camera phone and the reference camera model, determine the camera photography scene parameters, and obtain the basic calibration dataset based on the quantitative benchmark, the reference camera model and the camera photography scene parameters;

[0058] Step 200: Determine the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation personnel level standard; and perform a weighted average of the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation level standard to obtain the dataset labels.

[0059] Step 300: Add the dataset labels to the basic calibration dataset to obtain the comprehensive calibration dataset;

[0060] Step 400: Determine the basic model architecture for mobile phone photography effect tuning, and train the basic model architecture for mobile phone photography effect tuning using a preset loss function and the comprehensive tuning dataset to obtain the comprehensive tuning model for mobile phone photography effect.

[0061] Step 500: Input the adjusted mobile phone photography image into the comprehensive mobile phone photography effect adjustment model, and output the comprehensive mobile phone photography effect adjustment result.

[0062] Specifically, on the one hand, embodiments of the present invention first construct a subjective benchmark dataset, such as Figure 2As shown, the benchmark data was obtained by selecting a quantitative benchmark and a mobile phone model, i.e., determining the relevant parameters of the control phone and the specific mobile phone photography scenario; then, the subjective evaluation method, scoring rules and evaluation personnel were determined to obtain data labels. The benchmark data and data labels were combined to obtain the subjective benchmark dataset.

[0063] On the other hand, by designing an automatic intelligent evaluation method, including designing the model architecture, determining the loss function and determining the training strategy, and finally using the constructed dataset to train the model, a quantitative expression index that conforms to the subjective perception of the human eye can be obtained.

[0064] By combining subjective benchmark datasets with automatic intelligent evaluation methods, an automatic intelligent evaluation system that conforms to human subjective perception, as proposed in this embodiment of the invention, is obtained.

[0065] This invention constructs an overall framework for creating a subjective benchmark dataset in the field of mobile phone photography, proposes an automatic intelligent evaluation method and an automatic mobile phone photo quantification mode, which can effectively solve the problem of inconsistency between calibration indicators and human subjective perception in the field of mobile phone photography, and realize the automatic quantification of the quality of mobile phone photos.

[0066] Based on the above embodiments, a quantitative benchmark for a camera phone and a reference phone model are obtained, and the camera photography scene parameters are determined. A basic calibration dataset is then obtained based on the quantitative benchmark, the reference phone model, and the camera photography scene parameters, including:

[0067] Based on the key imaging characteristic parameters and parameter value ranges of the mobile phone, the quantitative benchmark of the photography mobile phone is determined, and the model of the reference phone is determined based on the quantitative indicators of the mobile phone calibration.

[0068] Determine the shooting location and the color temperature of the location, and construct the mobile phone photography location parameters based on the shooting location and the color temperature of the location;

[0069] The basic calibration dataset is obtained by combining the quantitative benchmark of the photography mobile phone, the control camera model, and the mobile phone photography site parameters.

[0070] In this embodiment of the invention, the selection of various key imaging characteristics, such as imaging uniformity, focus accuracy, exposure, white balance, color, and sharpness, and their reference value ranges are included. These indicators are determined to serve as quantitative benchmarks in the calibration process. Simultaneously, several mainstream mobile phone models recognized in the industry as having the best camera performance are selected, and then one control phone is chosen from each of these mainstream models as the research object.

[0071] The selection of mobile photography scenes includes choosing the location and the color temperature of the location. Locations include outdoor, shopping mall, and indoor spaces. The color temperature is selected from the range of 1800 Kelvin to 16000 Kelvin, and a representative color temperature is combined with the location and background to form a specific mobile photography scene. Specific mobile photography scenes are shown in Table 1.

[0072] Table 1

[0073]

[0074] Based on the above embodiments, the subjective evaluation mechanism, the control machine evaluation score evaluation mechanism, and the evaluation personnel level standards are determined, including:

[0075] By comparing two photographs taken by different control cameras in any identical scene, the subjective evaluation mechanism, including a first comparison result, a second comparison result, and a third comparison result, is determined.

[0076] The subjective quality values ​​of the control camera sample images were used to evaluate two photos taken by different control cameras, resulting in the control camera evaluation score evaluation mechanism.

[0077] Based on the photographer's years of experience, multiple assessment personnel are categorized into different levels.

[0078] Specifically, in the subjective evaluation method of this invention, when the evaluator compares two photos taken by different control cameras in the same scene, there are 3 options. For the two photos taken by control camera 1 and control camera 2, the evaluator can choose that the effect of control camera 1 is better, the effect of control camera 2 is better, or both are equally good.

[0079] The scoring rule in this invention is named the Control Sample Quality Subjective Value (CSQSV). When the evaluator deems the two fusion methods identical, the subjective score is reduced. The CSQSV scoring formula is as follows:

[0080] CSQSV i =Σ j∈M θ(f i ,f j )

[0081] Where M is the set of control machines, f i This is a photo taken by the comparison camera i, f j These are photos taken by the comparison camera, when f i Better still, θ(f) i ,f j ) = 3; when f i and f j When they are equally good, θ(f) i ,f j ) = 1; when fj Better still, θ(f) i ,f j =0. The higher the score, the better the quantification result of the control machine.

[0082] To select evaluators, they were divided into four categories: experts, researchers, senior photography enthusiasts, and ordinary photography enthusiasts in the field of comprehensive mobile phone photography effect tuning. The level of the evaluator was determined by the number of years of research experience in the field. For example, experts needed more than 3 years of research experience, researchers needed 2 to 3 years, senior photography enthusiasts needed 1 to 2 years, and ordinary photography enthusiasts needed less than 1 year.

[0083] Based on the above embodiments, after adding the dataset labels to the basic calibration dataset to obtain the comprehensive calibration dataset, the method further includes:

[0084] The consistency of the subjective quality value of the control sample image with four objective indicators was tested using the Pearson linear correlation coefficient (PLCC), Kendall's rank correlation coefficient (KROCC), Spearman's rank correlation coefficient (SROCC), and root mean square error (RMSE). The difference between the subjective quality value of the control sample image and the existing quantitative benchmark was obtained to verify whether the existing quantitative benchmark is consistent with human subjective perception.

[0085] Specifically, in this embodiment of the invention, the weighted average of the scores given by the evaluators is used as the label for the dataset. After the dataset is constructed, the consistency between CSQSV and existing quantitative benchmarks is tested using four consistency indicators. A subjective dataset is used to verify whether the current quantitative benchmark is consistent with human subjective perception.

[0086] The four consistency indicators include the Pearson linear correlation coefficient (PLCC), the Kendall rank-order correlation coefficient (KROCC), the Spearman rank-order correlation coefficient (SROCC), and the root mean square error (RMSE).

[0087] The basic information of the constructed subjective benchmark dataset for mobile photography is shown in Table 2.

[0088] Table 2

[0089] site 3 Venue color temperature 10 background 6 control machine 11 Taking photos 1970 Evaluation group Experts, researchers, seasoned photography enthusiasts, and ordinary photography enthusiasts

[0090] Based on the above embodiments, a basic model architecture for mobile phone photography effect tuning is determined. The basic model architecture is trained using a preset loss function and the comprehensive tuning dataset to obtain a comprehensive mobile phone photography effect tuning model, including:

[0091] The basic tuning model architecture for mobile phone photography effects is constructed using Convolutional Neural Networks (CNN) and Transformers.

[0092] Determine the reference photo, and improve the dimensionality information of the control machine sample to obtain the dimensionally improved control machine sample;

[0093] The deep semantic information of the reference photo is extracted by a convolutional neural network (CNN) and a linear layer, as well as the deep semantic information of the comparison machine sample after the dimensionality is increased.

[0094] The differential features are obtained by subtracting the enhanced semantic information of the comparison machine sample from the deeper semantic information of the reference photo.

[0095] The deep semantic information of the improved dimensionality comparison machine sample and the differential features are used as the input of the encoder;

[0096] The differential features, the deep semantic information of the improved dimensionality control sample, and the deep semantic information of the reference photograph are used as inputs to the decoder to obtain the objective value of the control sample.

[0097] The mean square error of the subjective value and the objective value of any comparison camera sample image is used as the loss function to train and converge the basic model architecture for mobile phone photography effect adjustment, thereby obtaining the comprehensive adjustment model for mobile phone photography effect.

[0098] Among them, the dimensionality-enhanced control machine samples obtained by enhancing the dimensionality information of the control machine samples include:

[0099] The original control sample was divided into several pre-set square blocks;

[0100] The aforementioned preset square blocks are stacked along the channel dimension of the original control sample to obtain the improved control sample.

[0101] The encoder includes a multi-head attention mechanism module, a residual connection and normalization layer, a feedforward neural network, and a residual connection and normalization layer.

[0102] The decoder includes a masked multi-head attention mechanism module, a residual connection and normalization layer, a multi-head attention mechanism module, a residual connection and normalization layer, a feedforward neural network, and a residual connection and normalization layer.

[0103] Specifically, the design of the automatic intelligent evaluation method in this embodiment of the invention includes designing the model architecture, determining the loss function, and determining the training strategy.

[0104] The model architecture utilizes Convolutional Neural Networks (CNNs) and Transformers. CNNs extract features from the receptive field to obtain deeper semantic information, while the Transformer's self-attention mechanism uncovers potential correlations between objective quantitative indicators and human subjective perception. Finally, the model is trained using the constructed dataset to obtain an automatic intelligent evaluation that aligns with human subjective perception.

[0105] like Figure 3 As shown, the automatic intelligent evaluation method includes the following steps:

[0106] Determine a reference photo and select a suitable one. This means choosing a photo with the highest subjective value for a particular location, color temperature, and background as the reference photo for that specific scene.

[0107] To enhance the dimensionality of the control sample images and fully represent their features, a linear mapping is used to transform the original control sample images into higher-dimensional features. Specifically, each control sample image is divided into g×g blocks, and these blocks are then stacked along the channel dimension. For example, if the original control sample image has a shape of H×W×1, the mapped shape will be h×w×g. 2 ,in:

[0108]

[0109] Deep semantic information is extracted using conventional convolutional neural networks (CNNs) and linear layers. Only the structure before the global average pooling layer in the CNN model is preserved for subsequent semantic feature extraction.

[0110] Extract the semantic features of the reference photo and the control image, and then subtract the semantic features of the control image from the semantic features of the reference photo to obtain the difference features between the two.

[0111] The algorithm learns the potential relationship between control sample images and reference photos. The semantic features of the control sample images and the extracted difference features are used as input to the encoder. The encoder consists of a multi-head attention mechanism module, residual connections and normalization layers (Add&Norm), a feedforward neural network, and residual connections and normalization layers (Add&Norm).

[0112] The objective value of the photo is obtained by using the difference feature Q (query), the control sample feature K (key), and the reference photo feature V (value) as input to the subsequent decoder to obtain the objective value of the control sample. The decoder has a structure largely the same as the encoder, including a masked multi-head attention mechanism module, residual connections and normalization layers (Add&Norm), a feedforward neural network, and residual connections and normalization layers (Add&Norm).

[0113] Determine the loss function, and choose the Mean Square Error (MSE) loss function commonly used in regression tasks, as follows:

[0114]

[0115] Where B is the number of control samples in each batch, o i Let s be the objective value of the i-th control sample image. i Let be the subjective value of the i-th control sample image.

[0116] Finally, the training strategy was determined, and the determination of the training strategy is relatively flexible. Specifically, g×g can be set to 8×8, so the initial dimension of the control sample is 64; the number of decoder-encoder is set to 2, so that the model parameters are not too numerous while still being able to complete the regression task; the CNN model can adopt the classic ResNet50 and ResNet101 structures. At the same time, training strategies such as learning rate decay are adopted.

[0117] The mobile phone photography effect comprehensive adjustment system provided by the present invention is described below. The mobile phone photography effect comprehensive adjustment system described below and the mobile phone photography effect comprehensive adjustment method described above can be referred to in correspondence.

[0118] Figure 4 This is a schematic diagram of the structure of the mobile phone photography effect comprehensive adjustment system provided in the embodiment of the present invention, as shown below. Figure 4 As shown, it includes: a data processing module 41, a label processing module 42, a comprehensive data module 43, a training module 44, and a tuning module 45, wherein:

[0119] The data processing module 41 is used to obtain the quantitative benchmark of the photography phone and the reference phone model, determine the mobile phone photography scene parameters, and obtain the basic calibration dataset based on the quantitative benchmark, the reference phone model, and the mobile phone photography scene parameters; the label processing module 42 is used to determine the subjective evaluation mechanism, the reference phone evaluation score evaluation mechanism, and the evaluation personnel level standard, and to perform a weighted average of the subjective evaluation mechanism, the reference phone evaluation score mechanism, and the evaluation level standard to obtain the dataset label; the comprehensive data module 43 is used to add the dataset label to the basic calibration dataset to obtain the comprehensive calibration dataset; the training module 44 is used to determine the basic calibration model architecture of the mobile phone photography effect, and to train the basic calibration model architecture of the mobile phone photography effect using a preset loss function and the comprehensive calibration dataset to obtain the comprehensive calibration model of the mobile phone photography effect; the calibration module 45 is used to input the calibrated mobile phone photography images into the comprehensive calibration model of the mobile phone photography effect and output the comprehensive calibration result of the mobile phone photography effect.

[0120] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logic instructions in the memory 530 to execute a comprehensive mobile phone photography effect tuning method. This method includes: acquiring a quantitative benchmark for the photography phone and a reference phone model; determining mobile phone photography scene parameters; obtaining a basic tuning dataset based on the quantitative benchmark, the reference phone model, and the mobile phone photography scene parameters; determining a subjective evaluation mechanism, a reference phone evaluation score mechanism, and an evaluator level standard; performing a weighted average of the subjective evaluation mechanism, the reference phone evaluation score mechanism, and the evaluator level standard to obtain dataset labels; adding the dataset labels to the basic tuning dataset to obtain a comprehensive tuning dataset; determining a basic tuning model architecture for mobile phone photography effects; training the basic tuning model architecture for mobile phone photography effects using a preset loss function and the comprehensive tuning dataset to obtain a comprehensive tuning model for mobile phone photography effects; inputting tuned mobile phone photography images into the comprehensive tuning model for mobile phone photography effects; and outputting the comprehensive tuning result for mobile phone photography effects.

[0121] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the comprehensive adjustment method for mobile phone photography effects provided by the above methods. The method includes: obtaining a quantitative benchmark of the mobile phone and a reference phone model; determining mobile phone photography scene parameters; obtaining a basic adjustment dataset based on the quantitative benchmark, the reference phone model, and the mobile phone photography scene parameters; determining a subjective evaluation mechanism, a reference phone evaluation score mechanism, and an evaluation personnel level standard; performing a weighted average of the subjective evaluation mechanism, the reference phone evaluation score mechanism, and the evaluation level standard to obtain dataset labels; adding the dataset labels to the basic adjustment dataset to obtain a comprehensive adjustment dataset; determining a basic adjustment model architecture for mobile phone photography effects; training the basic adjustment model architecture for mobile phone photography effects using a preset loss function and the comprehensive adjustment dataset to obtain a comprehensive adjustment model for mobile phone photography effects; inputting adjusted mobile phone photography images into the comprehensive adjustment model for mobile phone photography effects, and outputting comprehensive adjustment results for mobile phone photography effects.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for comprehensive adjustment of mobile phone photography effects, characterized in that, include: Obtain the quantitative benchmark and reference camera model of the mobile phone, determine the mobile phone photography scene parameters, and obtain the basic calibration dataset based on the quantitative benchmark, the reference camera model and the mobile phone photography scene parameters; Determine the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation personnel level standard; then, perform a weighted average of the subjective evaluation mechanism, the control machine evaluation score mechanism, and the evaluation personnel level standard to obtain the dataset label. Add the dataset labels to the basic calibration dataset to obtain the comprehensive calibration dataset; The basic model architecture for mobile phone photography effect is determined, and the basic model architecture for mobile phone photography effect is trained using a preset loss function and the comprehensive tuning dataset to obtain a comprehensive tuning model for mobile phone photography effect. Input the mobile phone photography image into the comprehensive mobile phone photography effect adjustment model, and output the comprehensive mobile phone photography effect adjustment result; A basic model architecture for mobile phone photography effect tuning is determined. This basic model architecture is trained using a preset loss function and the comprehensive tuning dataset to obtain a comprehensive mobile phone photography effect tuning model, including: The basic tuning model architecture for mobile phone photography effects is constructed using Convolutional Neural Networks (CNN) and Transformers. Determine the reference photo, and improve the dimensionality information of the control machine sample to obtain the dimensionally improved control machine sample; The deep semantic information of the reference photo is extracted by a convolutional neural network (CNN) and a linear layer, as well as the deep semantic information of the comparison machine sample after the dimensionality is increased. The differential features are obtained by subtracting the enhanced semantic information of the comparison machine sample from the deeper semantic information of the reference photo. The deep semantic information of the improved dimensionality comparison machine sample and the differential features are used as the input of the encoder; The differential features, the deep semantic information of the improved dimensionality control sample, and the deep semantic information of the reference photograph are used as inputs to the decoder to obtain the objective value of the control sample. The mean square error of the subjective value and the objective value of any comparison camera sample image is used as the loss function to train and converge the basic model architecture for mobile phone photography effect adjustment, thereby obtaining the comprehensive adjustment model for mobile phone photography effect.

2. The method for comprehensive adjustment of mobile phone photography effects according to claim 1, characterized in that, Obtain the quantitative benchmark and reference camera model for photography, determine the mobile phone photography scene parameters, and obtain a basic calibration dataset based on the quantitative benchmark, the reference camera model, and the mobile phone photography scene parameters, including: Based on the key imaging characteristic parameters and parameter value ranges of the mobile phone, the quantitative benchmark of the photography mobile phone is determined, and the reference model is determined based on the quantitative indicators of the mobile phone calibration. Determine the shooting location and the color temperature of the location, and construct the mobile phone photography location parameters based on the shooting location and the color temperature of the location; The basic calibration dataset is obtained by combining the quantitative benchmark of the photography mobile phone, the control camera model, and the mobile phone photography site parameters.

3. The method for comprehensive adjustment of mobile phone photography effects according to claim 1, characterized in that, Establish subjective evaluation mechanisms, machine-controlled evaluation scoring mechanisms, and evaluation personnel grading standards, including: By comparing two photographs taken by different control cameras in any identical scene, the subjective evaluation mechanism, including a first comparison result, a second comparison result, and a third comparison result, is determined. The subjective quality values ​​of the control camera sample images were used to evaluate two photos taken by different control cameras, resulting in the control camera evaluation score evaluation mechanism. Based on the photographer's years of experience, multiple assessment personnel are categorized into different levels.

4. The mobile phone photography effect comprehensive adjustment method according to claim 1, characterized in that, After adding the dataset labels to the basic tuning dataset to obtain the comprehensive tuning dataset, the following steps are also included: The consistency of the subjective quality value of the control sample image with four objective indicators was tested using the Pearson linear correlation coefficient (PLCC), Kendall's rank correlation coefficient (KROCC), Spearman's rank correlation coefficient (SROCC), and root mean square error (RMSE). The difference between the subjective quality value of the control sample image and the existing quantitative benchmark was obtained to verify whether the existing quantitative benchmark is consistent with human subjective perception.

5. The method for comprehensive adjustment of mobile phone photography effects according to claim 1, characterized in that, The dimensionality of the control machine sample is increased to obtain the dimensionality-enhanced control machine sample, including: The original control sample was divided into several pre-set square blocks; The aforementioned preset square blocks are stacked along the channel dimension of the original control sample to obtain the improved control sample.

6. The method for comprehensive adjustment of mobile phone photography effects according to claim 1, characterized in that, The encoder includes a multi-head attention mechanism module, a residual connection and normalization layer, a feedforward neural network, and a residual connection and normalization layer; The decoder includes a masked multi-head attention mechanism module, a residual connection and normalization layer, a multi-head attention mechanism module, a residual connection and normalization layer, a feedforward neural network, and a residual connection and normalization layer.

7. A comprehensive mobile phone photography effect adjustment system, based on the comprehensive mobile phone photography effect adjustment method according to any one of claims 1 to 6, characterized in that, include: The data processing module is used to obtain the quantitative benchmark of the camera phone and the reference camera model, determine the camera photography scene parameters, and obtain the basic calibration dataset based on the quantitative benchmark, the reference camera model and the camera photography scene parameters. The label processing module is used to determine the subjective evaluation mechanism, the control machine evaluation score mechanism, and the assessment personnel level standard, and to perform a weighted average of the subjective evaluation mechanism, the control machine evaluation score mechanism, and the assessment personnel level standard to obtain the dataset labels; The integrated data module is used to add the dataset labels to the basic calibration dataset to obtain the integrated calibration dataset. The training module is used to determine the basic model architecture for adjusting mobile phone photography effects. The basic model architecture for adjusting mobile phone photography effects is trained using a preset loss function and the comprehensive adjustment dataset to obtain the comprehensive adjustment model for mobile phone photography effects. The calibration module is used to input the mobile phone photography images into the comprehensive calibration model of mobile phone photography effects and output the comprehensive calibration results of mobile phone photography effects.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the comprehensive adjustment method for mobile phone photography effects as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the comprehensive adjustment method for mobile phone photography effects as described in any one of claims 1 to 6.

Images