Identification method, terminal and computer storage medium

Abstract

The embodiment of the present application discloses a recognition method, which is applied in a terminal, including: obtaining image data of an image to be recognized and color temperature data of the image to be recognized, processing the image data of the image to be recognized and the color temperature data of the image to be recognized respectively, obtaining image features of the image to be recognized, inputting the image features of the image to be recognized into a pre-trained machine learning classification model, and recognizing the image to be recognized to recognize that the image to be recognized is an indoor image or an outdoor image. The embodiment of the present application also provides a terminal and a computer storage medium.

Description

Technical Field

[0001] The present application relates to image recognition technology, and more particularly to a recognition method, a terminal and a computer storage medium. Background Art

[0002] At present, in the indoor and outdoor detection of images, there are methods based on external devices, such as wireless fidelity (WIFI), light sensing and infrared, and there are methods based on the image itself. Among them, the method based on the image itself often adopts artificial intelligence (AI) algorithm. The AI ​​algorithm is used to distinguish indoor and other scenes (for example: green plants, sky, and portraits). The AI ​​algorithm can provide multi-classification results for indoor and outdoor. However, the accuracy of the recognition rate for indoor and outdoor is not high, which is about 85%. In addition, the YUV data used by the AI ​​algorithm makes the algorithm unsuitable for automatic white balance (AWB) / automatic exposure (AE). It can be seen that the accuracy of the existing indoor and outdoor recognition methods for images is low. Summary of the invention

[0003] The embodiments of the present application provide a recognition method, a terminal and a computer storage medium, which can improve the accuracy of indoor and outdoor recognition of images.

[0004] The technical solution of this application is implemented as follows:

[0005] An embodiment of the present application provides an identification method, which is applied in a terminal, and the method includes:

[0006] Acquire image data of an image to be identified and color temperature data of the image to be identified;

[0007] Processing the image data of the image to be identified and the color temperature data of the image to be identified respectively to obtain image features of the image to be identified;

[0008] The image features of the image to be identified are input into a pre-trained machine learning classification model to identify the image to be identified so as to identify whether the image to be identified is an indoor image or an outdoor image.

[0009] An embodiment of the present application provides a terminal, the terminal comprising:

[0010] An acquisition module, used to acquire image data of an image to be identified and color temperature data of the image to be identified;

[0011] A processing module, used to process the image data of the image to be identified and the color temperature data of the image to be identified respectively, so as to obtain the image features of the image to be identified;

[0012] The recognition module is used to input the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized so as to identify whether the image to be recognized is an indoor image or an outdoor image.

[0013] An embodiment of the present application also provides a terminal, which includes: a processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the identification method described in one or more of the above embodiments is executed.

[0014] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the identification method described in one or more of the above embodiments.

[0015] The embodiment of the present application provides a recognition method, a terminal and a computer storage medium, which is applied in a terminal, and includes: obtaining image data of an image to be recognized and color temperature data of the image to be recognized, processing the image data of the image to be recognized and the color temperature data of the image to be recognized respectively, obtaining image features of the image to be recognized, inputting the image features of the image to be recognized into a pre-trained machine learning classification model, and recognizing the image to be recognized to identify the image to be recognized as an indoor image or an outdoor image; that is, in the embodiment of the present application, after the image features of the image to be recognized are obtained by processing the acquired image data and color temperature data of the image to be recognized, the image features are input into the pre-trained machine learning classification model for recognition, so that the image to be recognized can be recognized as an indoor image or an outdoor image. In this way, the image features of the image to be recognized are recognized by the machine learning classification model, and the image to be recognized can be more accurately identified as an indoor image or an outdoor image, thereby improving the accuracy of image recognition and helping to obtain an image with a more realistic effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of an optional identification method provided in an embodiment of the present application;

[0017] Figure 2a A schematic diagram of the structure of an example of a terminal provided with a color temperature sensor;

[0018] Figure 2b A schematic diagram of the structure of another example of a terminal provided with a color temperature sensor;

[0019] Figure 3 is the spectral response curve obtained by the color temperature sensor;

[0020] Figure 4 A flowchart of image processing in an image signal processor;

[0021] Figure 5 A flowchart of an optional method for training a machine learning classifier provided in an embodiment of the present application;

[0022] Figure 6 An optional histogram provided for an embodiment of the present application;

[0023] Figure 7 A distribution curve of a hinge loss function provided in an embodiment of the present application;

[0024] Figure 8a An optional image to be identified provided in an embodiment of the present application;

[0025] Figure 8b The embodiments of the present application provide Figure 8a The corresponding gradient image;

[0026] Fig. 9 is the voltage curve of alternating current;

[0027] Fig.10a An optional curve of time domain information of FD1 provided in an embodiment of the present application;

[0028] Fig.10b For the embodiments of this application and Fig.10a The corresponding curve of frequency domain information;

[0029] Fig.11 A schematic diagram of a terminal structure provided in an embodiment of the present application Figure 1 ;

[0030] Fig.12 A second schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0032] Embodiment 1

[0033] The present application embodiment provides an identification method, which is applied in a terminal. Figure 1 A schematic diagram of an optional identification method provided in an embodiment of the present application is provided. Figure 1 As shown, the above identification method may include:

[0034] S101: Acquire image data of an image to be identified and color temperature data of the image to be identified;

[0035] Specifically, the camera of the terminal collects the image to be identified and can obtain image data of the image to be identified. The terminal is provided with a color temperature sensor, which can obtain color temperature data. Figure 2a is a schematic structural diagram of an example of a terminal provided with a color temperature sensor, Figure 2b FIG. 1 is a schematic diagram of another example of a terminal provided with a color temperature sensor. Figure 2a As shown, a color temperature sensor is set next to the front camera of the terminal. Figure 2a The black circle in the middle is the color temperature sensor. Figure 2b As shown, a color temperature sensor is set next to the rear camera of the terminal. Figure 2b The black circle in the middle is the color temperature sensor.

[0036] Here, it should be noted that the color temperature sensor set on the terminal can obtain image component R, image component G, image component B, visible light C, full spectrum (WB, Wide Band), correlated color temperature (CCT, Correlated ColourTemperature) and light flicker frequency (Flicker Frequency) and intensity of two channels (FD1 and FD2). Figure 3 is the spectral response curve obtained by the color temperature sensor, such as Figure 3 As shown, the spectral response curves of R, G, B, C, WB, FD1 and FD2 are shown. Figure 3 It can be seen that the FD1 channel can cover almost the full spectrum.

[0037] Figure 4 It is a flowchart of image signal processing in an image signal processor, such as Figure 4 As shown, the upper dotted box is the processing flow for RAW data, including AE, AWB / auto focus (AF, Auto Focus), ..., demosaicing DEMOSAIC, ..., image space color conversion CSC, and the lower dotted box is the processing flow for YUV data, including: denoising, AI, ...; and the recognition method provided in the embodiment of the present application is applied after AE and before AWB / AF, and mainly processes RAW data.

[0038] It can be seen that when using an image signal processor, feature extraction needs to rely on full-size photos (such as 4000*3000), and a large number of structural features are extracted using multi-scale filtering methods. However, the image signal processor (ISP) of a mobile phone can usually only provide a small-size image after reduction, such as (120*90). The feature accuracy obtained by using the filtering method based on the full-size image is greatly reduced. The number of high-dimensional structural related features extracted from the image often exceeds 3000, which is difficult to do real-time processing when used in a mobile phone. In addition, from the actual effect, the error rate of complex structural features is very high when facing the lack of sky reference, pure color, and indoor artificial buildings (the accuracy rate is less than 80%); and the scene recognition algorithm based on YUV data is Figure 4 It is located after DEMOSAIC and tends to be the final scene. Due to the deviation in the time domain, it cannot be well used by the front-end AE&AWB⁡ therefore, the accuracy of the existing recognition method is low, which affects the effect of the image captured by the terminal.

[0039] In order to improve the accuracy of identifying the image to be identified, in an embodiment of the present application, when capturing an image, the terminal first collects the image to be identified through the camera of the terminal, such as an image in JPEG format, so that image data of the image to be identified can be obtained, and the image data is of RAW data type, and the color temperature data of the image to be identified is collected through a color temperature sensor arranged next to the camera, and the color temperature data includes at least time domain information of FD1 and FD2. In this way, the terminal can obtain the image data and color temperature data of the image to be identified.

[0040] S102: processing the image data of the image to be identified and the color temperature data of the image to be identified respectively to obtain image features of the image to be identified;

[0041] In an optional embodiment, S102 may include:

[0042] Processing the image data of the image to be recognized to obtain processed image data;

[0043] Processing the color temperature data of the image to be recognized to obtain processed color temperature data;

[0044] The processed image data and the processed color temperature data are used to form image features of the image to be identified.

[0045] Specifically, in order to obtain the image features of the image to be identified, the image data of the image to be identified and the color temperature data of the image to be identified can be processed separately, and then the image features of the image to be identified can be formed by using the processed image data of the image to be identified and the color temperature data of the image to be identified. It should be noted that in actual applications, the number of features in the image features of the image to be identified is generally 12, of which 8 features are obtained from the processed image data and 4 features are obtained from the processed color temperature data.

[0046] In order to obtain the 12 features, in an optional embodiment, processing the image data of the image to be recognized to obtain the processed image data may include:

[0047] Calculating the horizontal gradient value of the image to be identified and the vertical gradient value of the image to be identified according to the image data of the image to be identified;

[0048] The sum of the horizontal gradient value of the image to be identified and the vertical gradient value of the image to be identified is determined as the actual gradient value of the image to be identified;

[0049] Selecting, from the actual gradient values ​​of the image to be identified, actual gradient values ​​that fall within a preset weak gradient value range;

[0050] The image components of the image to be identified with the selected actual gradient values ​​are grouped to obtain eight groups of image components, and the number of image components in each group is normalized to obtain processed image data.

[0051] Specifically, based on the image data of the image to be identified, the lateral gradient value and the longitudinal gradient value of each pixel in the image to be identified are calculated, and then the sum of the lateral gradient value and the longitudinal gradient value of each pixel is added to determine the actual gradient value of each pixel.

[0052] Since a weak gradient value range is pre-stored in the terminal, for example, the gradient value range is generally 0-512, the weak gradient value range can be set to 10-100 in actual applications. In this way, pixel points whose actual gradient values ​​fall within the weak gradient value range can be screened out, and then, the image components of the image to be identified with the selected actual gradient values, such as the G component, are divided into 8 groups.

[0053] In practical applications, in order to divide image components into 8 groups, the value range of the image components can be first divided into 8 equal parts to form 8 image component intervals. Then, the grouping interval into which each image component falls is determined for grouping. After the grouping is completed, the number of image components in each group is determined, and then the number is normalized to obtain 8 features. Here, the grouping can be performed in the form of constructing a histogram to obtain 8 features.

[0054] In order to obtain 12 features, in an optional embodiment, the color temperature data of the image to be identified is processed to obtain the processed color temperature data, which may include:

[0055] Performing time-frequency conversion on the time domain information of one channel in the color temperature data of the image to be identified to obtain frequency domain information;

[0056] The two largest frequency amplitudes and the frequencies corresponding to the two largest frequency amplitudes are selected from the frequency domain information, and are normalized respectively to obtain processed color temperature data.

[0057] Specifically, after collecting the data of FD1 and FD2 through the color temperature sensor, since FD1 almost covers the entire spectrum, FD1 in the color temperature data of the image to be identified is selected here, and the time domain information of FD1 is converted into the frequency domain information of FD1 by video conversion. The two largest frequency amplitudes FDM1 and FDM2, as well as the frequencies FDQ1 and FDQ2 corresponding to the two largest frequency amplitudes are selected from the frequency domain information of FD1, and FDM1 and FDM2 are normalized, and FDQ1 and FDQ2 are normalized, thereby obtaining four features.

[0058] In practical applications, 65535 can be used to normalize FDM1 and FDM2, and 200 Hz can be used to normalize FDQ1 and FDQ2 to determine the four features.

[0059] In this way, the image features of the image to be identified can be determined.

[0060] S103: Inputting image features of the image to be identified into a pre-trained machine learning classification model to identify the image to be identified, so as to identify whether the image to be identified is an indoor image or an outdoor image.

[0061] After the image features of the image to be identified are determined in S102, the determined image features are input into a pre-trained machine learning classification model for identification, thereby identifying whether the image to be identified is an outdoor image or an indoor image.

[0062] Among them, the above-mentioned machine learning classification model can be any one of the following: a support vector machine model, a Bayesian classifier, an integrated learning model and a decision tree. Here, the embodiment of the present application does not make any specific limitations on this.

[0063] In addition, the embodiment of the present application adopts a pre-trained machine learning classification model, which is obtained by training the machine learning classification model. In a specific implementation manner, the training can be obtained in the following manner:

[0064] Acquire image data of a to-be-trained image set and color temperature data of the to-be-trained image set;

[0065] The image data of the image set to be trained and the color temperature data of the image set to be trained are processed respectively to obtain the image features of the image set to be trained;

[0066] The machine learning classification model is trained using the image features of the image set to be trained to determine the model parameters when the loss function in the classification model takes the smallest value, thereby obtaining a pre-trained machine learning classification model.

[0067] Here, the image data of the image set to be trained is first obtained through a camera, and the color temperature data of the image set to be trained is obtained through a color temperature sensor.

[0068] After obtaining the image data and color temperature data of the image set to be trained, it is necessary to perform feature extraction on the brightness image component of each image in the image set to be trained. In practical applications, features can be obtained in the form of a histogram.

[0069] Specifically, Figure 5 A flowchart of an optional method for training a machine learning classifier provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, taking the support vector machine as an example for training, the training method may include:

[0070] S501: Acquire JPEG data;

[0071] Specifically: for each image in the image set to be trained, taking the image data in JPEG format as an example, the image data (stats data) written by ISP is parsed from exif. For example, each image in the image set to be trained is 120*90, and each point has its own RGB value.

[0072] S502: Segment the stats data;

[0073] Specifically, the stats data is segmented to obtain more images to be trained.

[0074] S503: Calculate the actual gradient value of each pixel;

[0075] Calculate the horizontal gradient value and vertical gradient value of each pixel for the 120*90 stats data, and determine the sum of the horizontal gradient value and the vertical gradient value as the actual gradient value of each pixel;

[0076] S504: Selecting pixel points whose actual gradient values ​​fall within a preset weak gradient value range;

[0077] Specifically, the actual gradient value range is generally 0-512, and the weak gradient value range is set to 10-100, so that the pixels corresponding to the actual gradient values ​​falling within 10-100 are screened out.

[0078] S505: constructing a histogram for the G component of the selected pixel points;

[0079] Specifically, Figure 6 An optional histogram provided in the embodiment of the present application is as follows: Figure 6 As shown in the figure, the horizontal axis of the histogram is the value range of the G component of the 8 groups, and the vertical axis is the distribution of the G component. Figure 6 An optional histogram of the G component provided in the embodiment of the present application is as follows: Figure 6 As shown, 1-8 in the horizontal axis indicates 8 value intervals of the G component, and the vertical axis indicates the number of G components included in each value interval.

[0080] S506: Obtaining time domain information of FD1;

[0081] Specifically, eight features are obtained by processing the image data. In the processing of the color temperature data, the time domain information of FD1 is obtained through the color temperature sensor provided on the terminal.

[0082] S507: Perform time-frequency conversion on the time domain information of FD1 to obtain frequency domain information of FD1;

[0083] S508: extracting features from frequency domain information;

[0084] Specifically, the two largest frequency amplitudes FDM1 and FDM2, and the frequencies FDQ1 and FDQ2 corresponding to the two largest frequency amplitudes are selected from the frequency domain information, and FDM1 and FDM2 are normalized, and FDQ1 and FDQ2 are normalized, so as to obtain four features;

[0085] S509: Select a loss function to perform training and update parameters;

[0086] Specifically, after obtaining the image features of each image in the image set to be trained, the image features of each image are input into the support vector machine to find the parameters in the support vector machine when the loss function is minimized, so as to update the parameters in the support vector machine.

[0087] At this point, the image features of each image can be obtained.

[0088] Among them, the general machine learning classification model adopts the support vector machine model, and the loss function used is the hinge loss function. The form of the hinge loss function is:

[0089]

[0090] in, Through model training, w and b in the loss function can be updated.

[0091] Figure 7 A distribution curve of a hinge loss function provided in an embodiment of the present application, wherein the step size in the model training parameters is 0.01, and λ=80000 in the loss function.

[0092] In other words, more optimized model parameters can be obtained through model training to update the original model to obtain a trained machine learning classification model.

[0093] It should be noted that for the 8 features extracted from the image data, since it can be known from the reflection model that the weak edge of a smooth object may reflect more light source colors, here, two spatial filters [-1, 0, 1] and [1, 0, -1] are used to convolve with the image in the horizontal and vertical directions respectively to obtain the horizontal and vertical gradient values ​​of the image. Because the stats image is 8 bits, the horizontal and vertical gradient ranges are gx = [0, 255], gy = [0, 255]. Ignoring the gradient direction, only considering the gradient intensity, the gradient intensity can be expressed by the sum of the horizontal and vertical gradients, g = gx + gy. Then the gradient range is g = [0, 510]. Here, g = [0, 100] is set as the weak gradient range to simulate the weak edge of the object with a weak gradient.

[0094] Figure 8a An optional image to be identified provided in an embodiment of the present application, Figure 8b The embodiments of the present application provide Figure 8a The corresponding gradient image; Figure 8a and Figure 8b As shown, Figure 8a Calculating each pixel in can get Figure 8a The horizontal gradient value and vertical gradient value of each pixel in the image are obtained. Figure 8b The gradient image shown.

[0095] The exposure strategies for indoor and outdoor scenes are usually different, which is reflected in the brightness histogram. Due to the huge difference in brightness between bright and dark areas outdoors, and the large difference in the area of ​​the light source, the brightness of the dark area indoors is usually lower. Here, a weak gradient (10-100) is also used to approximate weak edges. The brightness of the weak edges in the scene is extracted and accumulated and distributed into 8 bins. The brightness in each bin is normalized to form 8 features.

[0096] In practical applications, common light frequencies: outdoor natural light is almost equal to the DC component, while artificial light sources, AC power in North America and Japan is usually 60Hz, and AC power in China, the European Union, and Australia is generally 50Hz. In countries with underdeveloped power grid configurations, the AC frequency is not stable. Fig. 9 is the voltage curve of alternating current, such as Fig. 9 As shown, the voltage of 50Hz and 60Hz alternating current.

[0097] Then, the frequency value that can be obtained by the flicker channel of the color temperature sensor in an outdoor environment will be very low, while the frequency value of the artificial light source in an indoor environment will be higher.

[0098] Depend on Figure 3 It can be seen that the FD1 channel of the color temperature sensor can almost cover the entire spectrum. Therefore, the FD1 channel information is used for time-frequency conversion, and then the two frequencies with the strongest amplitudes in the spectrum are extracted as FD1Q1 and FD1Q2, and the corresponding intensities FD1M1 and FD1M2. FD1Q1 and FD1Q2 are normalized to 200 Hz, and FD1M1 and FD1M2 are normalized to 65535 to form four features.

[0099] Fig.10a An optional curve of the time domain information of FD1 provided in an embodiment of the present application, Fig.10b For the embodiments of this application and Fig.10a The corresponding frequency domain information curve is as follows: Fig.10a and Fig.10b As shown, after time-frequency conversion, the two strongest frequencies can be found from the frequency domain information as FD1Q1 and FD1Q2, and the corresponding intensities FD1M1 and FD1M2.

[0100] The characteristic values ​​of the four features extracted from the following four images by the above-mentioned method of processing the color temperature data are shown in Table 1:

[0101] Table 1

[0102] FDQ1 FDQ2 FDM1 FDM2 Image 1 0.255 0 0.981 0 Image 2 0.0001 0.0005 0.952 0.0002 Image 3 0.0013 0.232 0.743 0.231 Image 4 0.253 0.0001 0.954 0.00001

[0103] In order to improve the accuracy of recognition, the machine learning classification model can be tested after training. In an optional embodiment, after the machine learning classification model is trained using the image features of the image set to be trained to determine the model parameters when the loss function in the machine learning classification model takes the minimum value, and the pre-trained machine learning classification model is obtained, the method may further include:

[0104] Acquire image data of the image set to be tested and color temperature data of the image to be tested;

[0105] Processing the image data of the image to be tested and the color temperature data of the image to be tested respectively to obtain the image features of the image to be tested;

[0106] Input the image features of the image to be tested into the pre-trained machine learning classification model, identify the image to be tested, and obtain the test result;

[0107] Based on the test results, determine whether the pre-trained machine learning classification model has passed.

[0108] Specifically, after the machine learning classification model is trained using the image features of the image set to be trained, the trained machine learning classification model must be tested. The test results are used to determine whether the trained model has passed the test. Only the model that has passed the test can be used to recognize images. Models that have not passed the test need further training to pass the test and be used to recognize images.

[0109] Here, in testing the trained machine learning classification model, we must first obtain the image set to be tested. Here, for the image set to be tested, we must not only obtain the image data of each image in the image set to be tested, but also the color temperature data of each image. Similarly, the image data of each image in the image set to be tested and the color temperature data of each image are processed separately in the same manner as in the recognition method or the training method to obtain the image features of each image in the image set to be tested.

[0110] Then, the image features of each image in the image set to be tested are input into the trained machine learning classification model for testing to obtain a test result, which includes whether the image to be tested is an indoor image or an outdoor image.

[0111] In order to determine whether the trained machine learning classification model has passed the test by accuracy, in an optional embodiment, determining whether the pre-trained machine learning classification model has passed the test according to the test results includes:

[0112] Compare the test result with the test result of the preset image set to be tested to determine the accuracy of the test result;

[0113] When the accuracy of the test result is greater than or equal to the preset threshold, it is determined that the pre-trained machine learning classification model has passed;

[0114] When the accuracy of the test result is less than a preset threshold, the image data of the new image set to be trained and the color temperature data of the new image set to be trained are re-acquired, the image data of the image set to be trained is updated to the image data of the new image set to be trained, the color temperature data of the image set to be trained is updated to the color temperature data of the new image set to be trained, and the process is returned to process the image data of the image set to be trained and the color temperature data of the image set to be trained respectively to obtain the image features of the image set to be trained.

[0115] Specifically, the test result of each image is compared with the pre-stored test result of each image. If they are the same, it means the test is correct. If they are different, it means the test is wrong. In this way, the percentage of correct tests, that is, the accuracy of the test results, can be obtained.

[0116] Then compare the accuracy of the test results with a preset threshold, for example, the preset threshold is 95%. When the accuracy of the test results is greater than or equal to 95%, it is determined that the trained machine learning classification model has passed the test and can be used for image recognition; when the accuracy of the test results is less than 95%, it is determined that the trained machine learning classification model has not passed the test and needs to continue training, so the image data of the new image set to be trained and the color temperature data of the new image set to be trained are re-acquired, and the trained machine learning classification model continues to be trained until a trained machine learning model that passes the test is obtained and used for image recognition.

[0117] In addition, the recognition method provided by the present application is used to implement 1024 indoor pictures and 1134 outdoor pictures in the database, and the flicker frequency distribution ranges from 1Hz to 135Hz. Each time, 80% of the pictures are extracted as training data to train the support vector machine, and 20% of the pictures are used to test the trained support vector machine. The accuracy of the test results is shown in Table 2 below:

[0118] Table 2

[0119] Number of test rounds Indoor accuracy Outdoor accuracy Comprehensive accuracy 1 96.78％ 96.65％ 96.71％ 2 96.72％ 96.54％ 96.63％ 3 96.48％ 96.92％ 96.70％ 4 96.39％ 96.65％ 96.52％ 5 96.32％ 96.26％ 96.29％ average 96.53％ 96.60％ 96.56％

[0120] Through the above examples, it can be seen that accurate indoor and outdoor judgment is very important for the application of the AWB algorithm. When the scene is judged to be outdoor, the AWB algorithm can simply set the color temperature to 5000~5000k, and the color deviation value can be between 0.001 and 0.005 to obtain a relatively ideal white balance effect. For outdoor scenes with low brightness and lack of sky reference, as well as large areas of pure color scenes, the effect can far exceed that of competing models.

[0121] When the current scene is outdoors, the AE algorithm does not need to consider the impact of flicker when adjusting the brightness, so you can safely reduce the exposure time to suppress motion blur.

[0122] That is to say, the embodiment of the present application utilizes single-channel optical frequency information, and the frequency intensity is used as the feature of the support vector machine to participate in indoor and outdoor classification, and uses the brightness feature based on weak gradient as the auxiliary group feature for classification, which can improve the accuracy of indoor and outdoor recognition of images.

[0123] An embodiment of the present application provides a recognition method, which is applied to a terminal, and includes: obtaining image data of an image to be recognized and color temperature data of the image to be recognized, processing the image data of the image to be recognized and the color temperature data of the image to be recognized respectively, obtaining image features of the image to be recognized, inputting the image features of the image to be recognized into a pre-trained machine learning classification model, and recognizing the image to be recognized to identify the image to be recognized as an indoor image or an outdoor image; that is, in an embodiment of the present application, after the image features of the image to be recognized are obtained by processing the acquired image data and color temperature data of the image to be recognized, the image features are input into a pre-trained machine learning classification model for recognition, so that the image to be recognized can be recognized as an indoor image or an outdoor image. In this way, the image features of the image to be recognized are recognized by the machine learning classification model, and the image to be recognized can be more accurately identified as an indoor image or an outdoor image, thereby improving the accuracy of image recognition and helping to obtain an image with a more realistic effect.

[0124] Embodiment 2

[0125] Fig.11 A schematic diagram of a terminal structure provided in an embodiment of the present application Figure 1 ,like Fig.11 As shown, an embodiment of the present application provides a terminal, including:

[0126] An acquisition module 111 is used to acquire image data of an image to be identified and color temperature data of the image to be identified;

[0127] A processing module 112, used to process the image data of the image to be identified and the color temperature data of the image to be identified respectively, to obtain the image features of the image to be identified;

[0128] The recognition module 113 is used to input the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized so as to identify whether the image to be recognized is an indoor image or an outdoor image.

[0129] Optionally, the processing module 112 is specifically configured to:

[0130] Processing the image data of the image to be recognized to obtain processed image data;

[0131] Processing the color temperature data of the image to be recognized to obtain processed color temperature data;

[0132] The processed image data and the processed color temperature data are used to form image features of the image to be identified.

[0133] Optionally, the processing module 112 processes the image data of the image to be recognized, and obtains the processed image data, including:

[0134] Calculating the horizontal gradient value of the image to be identified and the vertical gradient value of the image to be identified according to the image data of the image to be identified;

[0135] The sum of the horizontal gradient value of the image to be identified and the vertical gradient value of the image to be identified is determined as the actual gradient value of the image to be identified;

[0136] Selecting, from the actual gradient values ​​of the image to be identified, actual gradient values ​​that fall within a preset weak gradient value range;

[0137] The image components of the image to be identified with the selected actual gradient values ​​are grouped to obtain eight groups of image components, and the number of image components in each group is normalized to obtain processed image data.

[0138] Optionally, the processing module 112 processes the color temperature data of the image to be recognized, and obtains the processed color temperature data, including:

[0139] Performing time-frequency conversion on the time domain information of one channel in the color temperature data of the image to be identified to obtain frequency domain information;

[0140] The two largest frequency amplitudes and the frequencies corresponding to the two largest frequency amplitudes are selected from the frequency domain information, and are normalized respectively to obtain processed color temperature data.

[0141] Optionally, the terminal is also used to:

[0142] Acquire image data of a to-be-trained image set and color temperature data of the to-be-trained image set;

[0143] The image data of the image set to be trained and the color temperature data of the image set to be trained are processed respectively to obtain the image features of the image set to be trained;

[0144] The machine learning classification model is trained using the image features of the image set to be trained to determine the model parameters when the loss function in the machine learning classification model takes the smallest value, thereby obtaining a pre-trained machine learning classification model.

[0145] Optionally, the terminal trains the machine learning classification model using the image features of the image set to be trained to determine the model parameters when the loss function in the machine learning classification model takes the minimum value, and after obtaining the pre-trained machine learning classification model, is further used for:

[0146] Acquire image data of the image set to be tested and color temperature data of the image set to be tested;

[0147] The image data of the image set to be tested and the color temperature data of the image set to be tested are processed respectively to obtain the image features of the image set to be tested;

[0148] Input the image features of the image set to be tested into the pre-trained machine learning classification model, identify the image set to be tested, and obtain the test results;

[0149] Based on the test results, determine whether the pre-trained machine learning classification model has passed.

[0150] Optionally, the terminal determines whether the pre-trained machine learning classification model has passed the test based on the test results, including:

[0151] Compare the test result with the test result of the preset image set to be tested to determine the accuracy of the test result;

[0152] When the accuracy of the test result is greater than or equal to the preset threshold, it is determined that the pre-trained machine learning classification model has passed;

[0153] When the accuracy of the test result is less than a preset threshold, the image data of the new image set to be trained and the color temperature data of the new image set to be trained are re-acquired, the image data of the image set to be trained is updated to the image data of the new image set to be trained, the color temperature data of the image set to be trained is updated to the color temperature data of the new image set to be trained, and the process is returned to process the image data of the image set to be trained and the color temperature data of the image set to be trained respectively to obtain the image features of the image set to be trained.

[0154] In practical applications, the acquisition module 111, processing module 112 and identification module 113 can be implemented by a processor located on the terminal, specifically a CPU, a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0155] Fig.12 A second schematic diagram of a terminal structure provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the embodiment of the present application provides a terminal 1200, including:

[0156] A processor 121 and a storage medium 122 storing executable instructions of the processor 121, wherein the storage medium 122 relies on the processor 121 to perform operations through a communication bus 123, and when the instructions are executed by the processor 121, the identification method described in the above embodiment 1 is executed.

[0157] It should be noted that in actual application, the various components in the terminal are coupled together through the communication bus 123. It is understandable that the communication bus 123 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 123 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.12 Various buses are labeled as communication buses 123.

[0158] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the identification method described in the first embodiment.

[0159] Among them, the computer-readable storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and other memories.

[0160] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0161] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0164] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A recognition method, characterized in that: The method is applied in a terminal, and the method includes: Acquire image data of an image to be identified and color temperature data of the image to be identified; The image data of the image to be identified and the color temperature data of the image to be identified are processed respectively to obtain the image features of the image to be identified; the image data of the image to be identified is of original RAW data type and is collected by the camera on the terminal; the color temperature data of the image to be identified is collected by the color temperature sensor on the terminal; Inputting the image features of the image to be identified into a pre-trained machine learning classification model to identify the image to be identified to determine whether the image to be identified is an indoor image or an outdoor image; The processing of the image data of the image to be identified and the color temperature data of the image to be identified respectively to obtain the image features of the image to be identified includes: Calculating a horizontal gradient value of the image to be identified and a vertical gradient value of the image to be identified according to the image data of the image to be identified; Determine the sum of the horizontal gradient value of the image to be identified and the vertical gradient value of the image to be identified as the actual gradient value of the image to be identified; Selecting, from the actual gradient values ​​of the image to be identified, actual gradient values ​​that fall within a preset weak gradient value range; The image components of the image to be identified of the selected actual gradient values ​​are grouped to obtain eight groups of image components, and the number of image components in each group is normalized to obtain processed image data; Performing time-frequency conversion on the time domain information of one channel in the color temperature data of the image to be identified to obtain frequency domain information; Select the two largest frequency amplitudes and the frequencies corresponding to the two largest frequency amplitudes from the frequency domain information, and normalize them respectively to obtain processed color temperature data; The processed image data and the processed color temperature data are used to form image features of the image to be identified.

2. The method according to claim 1, characterized in that The method further comprises: Acquire image data of a to-be-trained image set and color temperature data of the to-be-trained image set; Processing the image data of the to-be-trained image set and the color temperature data of the to-be-trained image set respectively to obtain image features of the to-be-trained image set; The machine learning classification model is trained using the image features of the image set to be trained to determine the model parameters when the loss function in the machine learning classification model takes the smallest value, thereby obtaining the pre-trained machine learning classification model.

3. The method according to claim 2, characterized in that After the machine learning classification model is trained using the image features of the image set to be trained to determine the model parameter at which the loss function in the machine learning classification model takes the minimum value and the pre-trained machine learning classification model is obtained, the method further includes: Acquire image data of a to-be-tested image set and color temperature data of the to-be-tested image set; Processing the image data of the image set to be tested and the color temperature data of the image set to be tested respectively to obtain image features of the image set to be tested; Inputting the image features of the image set to be tested into the pre-trained machine learning classification model, identifying the image set to be tested, and obtaining a test result; According to the test result, determine whether the pre-trained machine learning classification model passes.

4. The method according to claim 3, characterized in that The step of judging whether the pre-trained machine learning classification model passes the test according to the test result includes: Comparing the test result with the preset test result of the image set to be tested to determine the accuracy of the test result; When the accuracy of the test result is greater than or equal to a preset threshold, it is determined that the pre-trained machine learning classification model has passed; When the accuracy of the test result is less than a preset threshold, the image data of a new image set to be trained and the color temperature data of the new image set to be trained are reacquired, the image data of the image set to be trained is updated to the image data of the new image set to be trained, the color temperature data of the image set to be trained is updated to the color temperature data of the new image set to be trained, and the process of processing the image data of the image set to be trained and the color temperature data of the image set to be trained respectively and obtaining the image features of the image set to be trained is returned.

5. A terminal, characterized in that: The terminal comprises: An acquisition module, used to acquire image data of an image to be identified and color temperature data of the image to be identified; a processing module, used to process the image data of the image to be identified and the color temperature data of the image to be identified respectively, to obtain the image features of the image to be identified; the image data of the image to be identified is of original RAW data type, collected by the camera on the terminal; the color temperature data of the image to be identified is collected by the color temperature sensor on the terminal; A recognition module, used for inputting the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized so as to identify whether the image to be recognized is an indoor image or an outdoor image; The processing module is specifically used to calculate the horizontal gradient value and the vertical gradient value of the image to be identified according to the image data of the image to be identified; determine the sum of the horizontal gradient value and the vertical gradient value of the image to be identified as the actual gradient value of the image to be identified; select the actual gradient value that falls within the preset weak gradient value range from the actual gradient value of the image to be identified; group the image components of the image to be identified of the selected actual gradient value to obtain eight groups of image components, and normalize the number of image components in each group to obtain processed image data; Performing time-frequency conversion on the time domain information of one channel in the color temperature data of the image to be identified to obtain frequency domain information; selecting the two largest frequency amplitudes and the frequencies corresponding to the two largest frequency amplitudes from the frequency domain information, and normalizing them respectively to obtain processed color temperature data; The processed image data and the processed color temperature data are used to form image features of the image to be identified.

6. A terminal, characterized in that: The terminal includes: a processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations via a communication bus, and when the instructions are executed by the processor, the identification method described in any one of claims 1 to 4 is executed.

7. A computer storage medium, characterized in that: Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the identification method described in any one of claims 1 to 4.

Images