An identification method, a terminal, and a computer storage medium

Abstract

An embodiment of the present application discloses an identification method, which is applied to a terminal and includes: obtaining image data of an image to be identified, obtaining color temperature data of the environment where the image to be identified is taken through a color temperature sensor, respectively processing the image data of the image to be identified and the time-domain information of two channels of the color temperature data to obtain image features of the image to be identified, and inputting the image features of the image to be identified into a pre-trained machine learning classification model to identify the image to be identified, so as to identify whether the image to be identified is an indoor image or an outdoor image. An embodiment of the present application also provides a terminal and a computer storage medium at the same time.

Description

Technical Field

[0001] This application relates to the technology of image recognition, and particularly to a recognition method, a terminal and a computer storage medium. Background Art

[0002] In the indoor and outdoor detection of images, there are methods based on external devices, such as Wireless Fidelity (WIFI), light sensors and infrared, and there are also methods based on the images themselves. Among them, the methods based on the images themselves often use Artificial Intelligence (AI) algorithms. The AI algorithm is used to distinguish indoor and other scenes (such as: green plants & sky & portraits). The AI algorithm can provide multi-classification results for indoor and outdoor, but the accuracy of the recognition rate for indoor and outdoor is not high, about 85%, and moreover, the YUV data used by the AI algorithm makes this algorithm not suitable for being provided to Auto White Balance (AWB) / Automatic Exposure (AE) for use. Thus, it can be seen that the accuracy of the existing methods for indoor and outdoor recognition of images is relatively low. Summary of the Invention

[0003] Embodiments of this application provide a recognition method, a terminal and a computer storage medium, which can improve the accuracy of indoor and outdoor recognition of images.

[0004] The technical solution of this application is implemented as follows:

[0005] Embodiments of this application provide a recognition method, which is applied to a terminal. The method includes:

[0006] Obtain the image data of the image to be recognized;

[0007] Obtain the color temperature data of the environment where the image to be recognized is taken through a color temperature sensor;

[0008] Process the time-domain information of two channels of the image data of the image to be recognized and the color temperature data respectively to obtain the image features of the image to be recognized;

[0009] Input the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized, so as to recognize that the image to be recognized is an indoor image or an outdoor image.

[0010] Embodiments of this application provide a terminal, which includes:

[0011] A first acquisition module, configured to obtain the image data of the image to be recognized;

[0012] A second acquisition module, configured to acquire the color temperature data of the environment where the image to be recognized is captured through a color temperature sensor;

[0013] A processing module, configured to process the image data of the image to be recognized and the time-domain information of two channels of the color temperature data respectively, to obtain the image features of the image to be recognized;

[0014] An identification module, configured to input the image features of the image to be recognized into a pre-trained machine learning classification model, and identify the image to be recognized, so as to identify that the image to be recognized is an indoor image or an outdoor image.

[0015] An embodiment of the present application further provides a terminal, where the terminal includes: a processor and a storage medium storing executable instructions of the processor, and the storage medium depends on the processor to execute operations through a communication bus. When the instructions are executed by the processor, the above-mentioned identification method of one or more embodiments is executed.

[0016] An embodiment of the present application provides a computer storage medium, storing executable instructions. When the executable instructions are executed by one or more processors, the processor executes the above-mentioned identification method of one or more embodiments.

[0017] An embodiment of the present application provides an identification method, a terminal and a computer storage medium. The method is applied to a terminal, and the method includes: acquiring the image data of the image to be recognized, acquiring the color temperature data of the environment where the image to be recognized is captured through a color temperature sensor, processing the image data of the image to be recognized and the time-domain information of two channels of the color temperature data respectively, to obtain the image features of the image to be recognized, inputting the image features of the image to be recognized into a pre-trained machine learning classification model, and identifying the image to be recognized, so as to identify that the image to be recognized is an indoor image or an outdoor image; that is to say, in the embodiment of the present application, after processing the image data of the image to be recognized and the time-domain information of two channels of the color temperature data obtained, to obtain the image features of the image to be recognized, the image features are input into a pre-trained machine learning classification model for identification, so that it can be identified that the image to be recognized is an indoor image or an outdoor image. In this way, by identifying the image features of the image to be recognized through a machine learning classification model, it can be more accurately identified that the image to be recognized is an indoor image or an outdoor image, thereby improving the accuracy of image recognition and helping to obtain a more realistic image effect. Description of the Drawings

[0018] Figure 1 It is a schematic flowchart of an optional identification method provided by an embodiment of the present application;

[0019] Figure 2aSchematic structural diagram of an example of a terminal provided with a color temperature sensor;

[0020] Figure 2b Schematic structural diagram of another example of a terminal provided with a color temperature sensor;

[0021] Figure 3 Spectral response curve obtained by the color temperature sensor;

[0022] Figure 4 Flow chart of image signal processing in an image processor;

[0023] Figure 5a Typical fluorescent lamp and spectral energy distribution diagram;

[0024] Figure 5b Typical sunlight spectral energy distribution diagram;

[0025] Figure 5c Typical incandescent lamp spectral energy distribution diagram;

[0026] Figure 6 Flow schematic diagram of an optional method for training a machine learning classifier provided by an embodiment of the present application;

[0027] Figure 7 An optional histogram provided by an embodiment of the present application;

[0028] Figure 8 Distribution curve of a hinge loss function provided by an embodiment of the present application;

[0029] Figure 9a An optional image to be recognized provided by an embodiment of the present application;

[0030] Figure 9b Provided by an embodiment of the present application and Figure 9a Corresponding gradient image;

[0031] Figure 10 Voltage curve of alternating current;

[0032] Figure 11 Schematic structure of a terminal provided by an embodiment of the present application Figure One ;

[0033] Figure 12 Schematic diagram II of the structure of a terminal provided by an embodiment of the present application. Detailed implementation mode

[0034] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application.

[0035] Embodiment 1

[0036] An embodiment of the present application provides an identification method, which is applied to a terminal. Figure 1 As shown in the flowchart of an optional identification method provided by an embodiment of the present application, Figure 1 the above identification method may include:

[0037] S101: Obtain the image data of the image to be identified;

[0038] S102: Obtain the color temperature data of the environment where the image to be identified is taken through a color temperature sensor;

[0039] Specifically, the camera of the terminal captures the image to be identified, and the image data of the image to be identified can be obtained. A color temperature sensor is set on the terminal, and the color temperature sensor can obtain the color temperature data. Figure 2a As shown in the structural schematic diagram of an example of a terminal provided with a color temperature sensor, Figure 2b As shown in the structural schematic diagram of another example of a terminal provided with a color temperature sensor, Figure 2a as shown, a color temperature sensor is set beside the front camera of the terminal. Figure 2a The black circle in Figure 2b is the color temperature sensor. As shown in Figure 2b a color temperature sensor is set beside the rear camera of the terminal.

[0040] Here, it should be noted that the color temperature sensor set on the terminal can obtain the image components R, G, B, visible light C, full spectrum (WB, Wide Band), correlated color temperature (CCT, Correlated Colour Temperature), and the light flicker frequency and intensity of two channels (FD1 and FD2). Figure 3 As shown in the spectral response curve obtained by the color temperature sensor, Figure 3 as shown, it shows the spectral response curves of R, G, B, C, WB, FD1, and FD2. It can be seen from Figure 3 that the FD1 channel can almost cover the full spectrum.

[0041] Figure 4 As shown in the flowchart of image signal processing in the image signal processor, Figure 4As shown in the figure, the upper dashed box represents the processing flow for RAW data, including AE, AWB / Auto Focus (AF), …, Demosaicing, …, Image Space Color Conversion (CSC). The lower dashed box represents the processing flow for YUV data, including: Denoising, AI, …; The recognition method provided by the embodiments of the present application is applied after AE and before AWB / AF, and mainly processes RAW data.

[0042] It can be seen that when using an image signal processor, feature extraction relies on full-size photos (such as 4000*3000), and a multi-scale filtering method is applied to extract a large number of structural features. However, the image signal processor (ISP) of a mobile phone usually only provides a small-sized image after reduction, such as (120*90). The accuracy of the features obtained using the filtering method based on the full-size image is greatly reduced. High-dimensional structural-related features are extracted from the image, and the number of features often exceeds 3000. It is very difficult to perform real-time processing when used in a mobile phone, and from the actual effect, the complex structural features have a high misjudgment rate (the correct rate is less than 80%) when facing the lack of sky reference, solid colors, and indoor artificial buildings; moreover, the scene recognition algorithm based on YUV data is Figure 4 after Demosaicing and tends to the final seen scene. Due to the deviation in the time domain, it cannot be well used by the front-end AE&AWB⁡ Therefore, the accuracy of the existing recognition methods is relatively low, affecting the effect of the images captured by the terminal.

[0043] In order to improve the accuracy of recognizing the image to be recognized, in the embodiments of the present application, when the terminal captures an image, it first collects the image to be recognized through the camera of the terminal, such as an image in JPEG format, so as to obtain the image data of the image to be recognized. The image data is of the RAW data type, and the color temperature data of the image to be recognized is collected through the color temperature sensor set beside the camera. The color temperature data at least includes the time domain information of FD1 and FD2. In this way, the terminal can obtain the image data and color temperature data of the image to be recognized.

[0044] S103: Process the image data of the image to be recognized and the time domain information of the two channels of the color temperature data respectively to obtain the image features of the image to be recognized;

[0045] In an optional embodiment, S102 may include:

[0046] Process the image data of the image to be recognized to obtain the processed image data;

[0047] Process the time domain information of the two channels of the color temperature data to obtain the processed color temperature data;

[0048] Form image features of the image to be recognized by using the processed image data and the processed color temperature data.

[0049] Specifically, in order to obtain the image features of the image to be recognized, here, the time-domain information of the two channels of the image data and the color temperature data of the image to be recognized can be processed respectively, and then the image features of the image to be recognized can be formed by using the processed image data and the processed color temperature data of the image to be recognized. It should be noted here that in practical applications, the number of features in the formed image features of the image to be recognized is generally 10, among which, 8 features are obtained from the processed image data and 2 features are obtained from the processed color temperature data.

[0050] In order to obtain 10 features, in an optional embodiment, processing the image data of the image to be recognized to obtain the processed image data may include:

[0051] According to the image data of the image to be recognized, calculate the horizontal gradient value of the image to be recognized and the vertical gradient value of the image to be recognized;

[0052] Determine the sum of the horizontal gradient value and the vertical gradient value of the image to be recognized as the actual gradient value of the image to be recognized;

[0053] Select the actual gradient values falling within the preset weak gradient value range from the actual gradient values of the image to be recognized;

[0054] Group the image components corresponding to the actual gradient values falling within the weak gradient value range to obtain eight groups of image components, and normalize the number of each group of image components to obtain the processed image data.

[0055] Specifically, according to the image data of the image to be recognized, calculate the horizontal gradient value and the vertical gradient value of each pixel point in the image to be recognized, and then determine the sum obtained by adding the horizontal gradient value and the vertical gradient value of each pixel point as the actual gradient value of each pixel point.

[0056] Since the weak gradient value range is pre-stored in the terminal, for example, the gradient value range is generally 0 - 512, and in practical applications, the weak gradient value range can be set to 10 - 100. In this way, the pixel points with actual gradient values falling within the weak gradient value range can be screened out, and then, the image components of the image to be recognized with the selected actual gradient values, such as the G component, are divided into 8 groups.

[0057] In practical applications, in order to divide the image components into 8 groups, the value range of the image components can be first evenly divided into 8 equal parts to form 8 image component intervals. Then, the grouping interval into which each image component falls is determined for grouping. After the grouping is completed, the number of image components in each group is determined, and then the number is normalized to obtain 8 features. Among them, a histogram can be constructed for grouping to obtain 8 features.

[0058] In order to obtain 10 features, in an alternative embodiment, processing the color temperature data to obtain the processed color temperature data may include:

[0059] Performing time-frequency conversion on the time-domain information of the two channels in the color temperature data respectively to obtain the frequency-domain information of the two channels;

[0060] Respectively obtaining the amplitudes of the two channels when the frequency value is 0 from the frequency-domain information of the two channels;

[0061] According to the amplitudes of the two channels and the mean value of the visible light band of the color temperature data, calling a preset calculation formula for the infrared band to obtain the processed color temperature data.

[0062] Specifically, after collecting the time-domain information of FD1 and FD2 through a color temperature sensor, performing time-frequency conversion on the time-domain information of FD1 and FD2 respectively to obtain the frequency-domain information of FD1 and FD2, and selecting the intensities of FD1 and FD2 when the frequency value is 0 from the frequency-domain information of FD1 and FD2, that is, the amplitude DC(FD1) of FD1 and the amplitude DC(FD2) of FD2. Finally, calling the following calculation formula for the infrared band:

[0063]

[0064]

[0065] Among them, IR1 mainly measures the intensity of the infrared band from 800nm to 900nm, IR2 mainly measures the intensity of the infrared band from 950nm to 1000nm, the DC operator represents obtaining the DC component of the corresponding channel, that is, the intensity (also called amplitude) corresponding to the frequency value of 0, and C is the mean value of the visible light band of the color temperature data, so as to obtain 2 features.

[0066] For the 2 features constructed for IR1 and IR2, since the light in the spectral range of 380nm to 780nm can be perceived by the human eye, these are called the visible light band, and the region after 800nm is usually called the infrared band, which cannot be perceived by the human eye. Figure 5a Is a typical fluorescent lamp and spectral energy distribution diagram, Figure 5b Is a typical sunlight spectral energy distribution diagram, Figure 5cis a typical spectral energy distribution diagram of incandescent lamp, as shown in Figure 5a , Figure 5b and Figure 5c . It can be seen from the spectral energy distributions of various light sources that the energy in the infrared band of 800nm - 900nm is very weak in the indoor fluorescent lamp scenario, while there is still quite strong energy in the infrared band of 800nm - 900nm under sunlight, and it starts to decay rapidly after 950nm. The energy of the incandescent lamp in the infrared band of 800nm - 1000nm shows an increasing trend. Therefore, theoretically, differential features can be made based on the infrared band information fed back by the color temperature sensor. Then, two features can be obtained according to the above formulas (1) and (2).

[0067] In this way, the image features of the image to be recognized can be determined.

[0068] S104: Input the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized, so as to recognize whether the image to be recognized is an indoor image or an outdoor image.

[0069] After determining the image features of the image to be recognized through S103, the determined image features are input into a pre-trained machine learning classification model for recognition, so as to recognize whether the image to be recognized belongs to an outdoor image or an indoor image.

[0070] Among them, the above machine learning classification model can be any one of the following: support vector machine model, Bayesian classifier, ensemble learning model, and decision tree. Here, the embodiments of the present application do not make specific limitations on this.

[0071] In an optional embodiment, after S104, the method may include:

[0072] Perform automatic white balance processing and / or automatic focus processing on the image to be recognized according to whether the recognized image to be recognized is an indoor image or an outdoor image, and obtain the processed image to be recognized.

[0073] That is to say, after determining that the image to be recognized is an indoor image or an outdoor image, the recognition result can be provided for use in the AWB and AE processing flows to perform AWB and / or AE processing on the image, enhancing the visual effect of the image.

[0074] In addition, the machine learning classification model adopted in the embodiments of the present application is a pre-trained one, and this model is obtained by training the machine learning classification model. In the specific implementation, it can be obtained by training in the following manner:

[0075] Obtain the image data of the image set to be trained and the color temperature data of the image set to be trained;

[0076] Process the time-domain information of the two channels of the image data of the training image set and the color temperature data of the image set to be trained respectively to obtain the image features of the image set to be recognized;

[0077] Use the image features of the image set to be trained to train the machine learning classification model to determine the model parameters when the value of the loss function in the classification model is the smallest, and obtain the pre-trained machine learning classification model.

[0078] Here, first obtain the image data of the image set to be trained through the camera, and obtain the color temperature data of the environment of the image set to be trained through the color temperature sensor.

[0079] After obtaining the image data of the image set to be trained and the color temperature data of the image set to be trained, it is necessary to extract features for the luminance image component of each image in the image set to be trained. In practical applications, features can be obtained in the form of a histogram.

[0080] Specifically, Figure 6 is a schematic flowchart of an optional method for training a machine learning classifier provided by an embodiment of the present application. As Figure 6 shown, taking the support vector machine as an example for training, the training method may include:

[0081] S601: Obtain JEPG data;

[0082] Specifically: For each image in the obtained image set to be trained, taking the image data in JPEG format as an example, parse the image data (stats data) written by ISP from exif. For example, taking each image in the image set to be trained as 120*90, each point has its own RGB value.

[0083] S602: Perform segmentation processing on the stats data;

[0084] Specifically, performing segmentation processing on the stats data can obtain more images to be trained.

[0085] S603: Calculate the actual gradient value of each pixel point;

[0086] Calculate the horizontal gradient value and vertical gradient value of each pixel point for the 120*90 stats data, and determine the sum of the horizontal gradient value and the vertical gradient value as the actual gradient value of each pixel point;

[0087] S604: Select the pixel points whose actual gradient values fall within the preset weak gradient value range;

[0088] Specifically, the range of the actual gradient value is generally 0 - 512, and the range of the weak gradient value is set to 10 - 100. In this way, the pixel points corresponding to the actual gradient values falling within 10 - 100 are screened out.

[0089] S605: Construct a histogram for the G component of the selected pixel points;

[0090] Specifically, Figure 7 An optional histogram provided by an embodiment of the present application is as Figure 7 shown. The horizontal axis of the histogram is the value range of the G component of 8 groups, the vertical axis is the distribution of the G component, 1 - 8 in the abscissa identify 8 value intervals of the G component, and the vertical axis is the number of G components included in each value interval.

[0091] S606: Obtain the time - domain information of FD1 and FD2;

[0092] Specifically, 8 features are obtained through the processing of image data. In the processing of color temperature data, the time - domain information of FD1 and FD2 is obtained through the color temperature sensor set on the terminal.

[0093] S607: Perform time - frequency conversion on the time - domain information of FD1 and FD2 respectively to obtain the frequency - domain information of FD1 and the frequency - domain information of FD2;

[0094] S608: Extract features from the frequency - domain information of FD1 and FD2;

[0095] Specifically, from the frequency - domain information of the two channels, the amplitudes of the two channels when the frequency value is 0 are respectively obtained, and the mean C of the visible light band of the color temperature data of the image to be recognized is obtained. By calling formulas (1) and (2), IR1 and IR2 are calculated, thereby obtaining 2 features;

[0096] S609: Select a loss function for training to update the parameters;

[0097] Specifically, after obtaining the image features of each image in the image set to be trained, the image features of each image are input into the support vector machine, and the parameters in the support vector machine when the loss function is minimized are found to update the parameters in the support vector machine; thus, the image features of each image can be obtained.

[0098] Among them, generally, the machine - learning classification model uses the support vector machine model, and the loss function used is the hinge loss function. The form of the hinge loss function is:

[0099]

[0100] Among them, Through model training, w and b in the loss function can be updated.

[0101] Figure 8 This is the distribution curve of a hinge loss function provided by an embodiment of the present application. Among them, the step size in the model training parameters is 0.01, and λ = 80000 in the loss function.

[0102] That is to say, more optimized model parameters can be obtained through model training to update the original model to obtain a trained machine learning classification model.

[0103] Among them, it should be noted that for the 8 features extracted from the image data, since it can be understood from the reflection model that the weak edges of smooth objects may reflect more light source colors. Here, two spatial filters are used to convolve with the horizontal and vertical directions of the image respectively to obtain the horizontal and vertical gradient values of the image. Since the stats image is 8-bit, the gradient ranges in the horizontal and vertical directions are gx and gy. Ignoring the gradient direction and only considering the gradient intensity, the gradient intensity can be represented by the sum of the horizontal and vertical gradients: g = gx + gy. Then the gradient range g is the weak gradient range, and the weak gradient is used to simulate the weak edges of objects.

[0104] Figure 9a An optional image to be recognized provided by an embodiment of the present application Figure 9b Provided by an embodiment of the present application corresponding to Figure 9a The corresponding gradient image; as shown in Figure 9a And Figure 9b Shown, by calculating each pixel point in Figure 9a The horizontal gradient value and vertical gradient value of each pixel point in Figure 9a Can be obtained, so as to obtain the gradient image shown in Figure 9b

[0105] The exposure strategies in indoor and outdoor scenarios are usually different. Reflected on the luminance histogram, due to the extremely large luminance difference between the bright and dark areas outdoors and the large area difference of the light-emitting bodies, the luminance of the indoor dark area is usually lower. Here, the weak gradient (10 - 100) is also used to approximate the weak edge situation, and the luminance of the weak edges in the scene is extracted and accumulated and distributed into 8 bins. The luminance in each bin is normalized to form 8 features.

[0106] In practical applications, common light frequencies: natural light outdoors is almost equal to the DC component, while for artificial light sources, the alternating current in North America and Japan is usually 60 Hz, and the alternating current in China, the European Union, and Australia is generally 50 Hz. In countries with underdeveloped power grid configurations, the alternating current frequency is not stable. Figure 10 Is the voltage curve of the alternating current, as shown in Figure 10 Shown, the voltages of 50 Hz and 60 Hz alternating currents.

[0107] ​Then, the frequency values that can be obtained by the channels of the color temperature sensor in the outdoor environment will be very low, while the frequency values of the artificial light sources in the indoor environment will be higher.

[0108] To improve the recognition accuracy, testing can be carried out after the machine learning classification model training is completed. In an alternative embodiment, after training the machine learning classification model using the image features of the image set to be trained to determine the model parameters when the value of the loss function in the machine learning classification model is minimized, and obtaining the pre-trained machine learning classification model, the method may further include:

[0109] Obtain the image data of the image set to be tested and the color temperature data of the image to be tested;

[0110] Process the time-domain information of the two channels of the image data of the image to be tested and the color temperature data of the image to be tested respectively to obtain the image features of the image to be tested;

[0111] Input the image features of the image to be tested into the pre-trained machine learning classification model to identify the image to be tested and obtain the test result;

[0112] According to the test result, determine whether the pre-trained machine learning classification model passes.

[0113] Specifically, after training the machine learning classification model using the image features of the image set to be trained, the trained machine learning classification model needs to be tested. The trained model is judged whether it passes the test through the test result. Only the model that passes the test can be used to identify images, and the model that fails the test needs to be further trained to pass the test and be used to identify images.

[0114] Here, in the test of the trained machine learning classification model, first, the image set to be tested needs to be obtained. Here, for the image set to be tested, not only the image data of each image in the image set to be tested needs to be obtained, but also the color temperature data of the environment of each image. Similarly, in the same way as in the recognition method or the training method, the time-domain information of the two channels of the image data of each image in the image set to be tested and the color temperature data of each image are processed respectively to obtain the image features of each image in the image set to be tested.

[0115] Then, input the image features of each image in the image set to be tested into the trained machine learning classification model for testing to obtain the test result, and the test result includes that the image to be tested is an indoor image or an outdoor image.

[0116] In an alternative embodiment, to determine whether a trained machine learning classification model passes based on accuracy, determining whether a pre-trained machine learning classification model passes according to test results includes:

[0117] Comparing the test results with the test results of a preset set of images to be tested to determine the accuracy rate of the test results;

[0118] When the accuracy rate of the test results is greater than or equal to a preset threshold, it is determined that the pre-trained machine learning classification model passes;

[0119] When the accuracy rate of the test results is less than the preset threshold, re-obtain the image data of a new set of images to be trained and the color temperature data of the new set of images to be trained, and return to perform processing on the time-domain information of the two channels of the image data of the images to be trained and the color temperature data of the images to be trained respectively to obtain the image features of the images to be recognized.

[0120] Specifically, compare the test results of each image with the pre-stored test results of each image. If they are the same, it means the test is correct; if they are different, it means the test is incorrect. In this way, the percentage of correct tests, that is, the accuracy rate of the test results, can be obtained.

[0121] Then compare the accuracy rate of the test results with the preset threshold. For example, the preset threshold is 95%. When the accuracy rate of the test results is greater than or equal to 95%, it is determined that the trained machine learning classification model passes the test and can be used for image recognition; when the accuracy rate of the test results is less than 95%, it is determined that the trained machine learning classification model fails the test and needs to continue training. Therefore, re-obtain the image data of a new set of images to be trained and the color temperature data of the new set of images to be trained, and continue to train the trained machine learning classification model until a trained machine learning model that passes the test is obtained and is used for image recognition.

[0122] Accurate indoor and outdoor determination is very important for the application of the AWB algorithm. When the scene is determined to be outdoor, the AWB algorithm can relatively simply set the color temperature to 5000 - 5000k, and a color deviation value of 0.001 - 0.005 can obtain a relatively ideal white balance effect. A better effect can be obtained for scenes lacking sky reference outdoors at low brightness and for large-area solid-color scenes.

[0123] When the current scene is outdoor, the AE algorithm does not need to consider the influence of stroboscopic when adjusting the brightness, so it can safely reduce the exposure time to suppress motion blur.

[0124] That is to say, in the embodiment of the present application, the single-channel optical frequency information and the frequency intensity are used as the features of the support vector machine to participate in indoor and outdoor classification, and the brightness feature based on the weak gradient is used as the auxiliary feature for classification, which can improve the recognition accuracy of indoor and outdoor images.

[0125] An embodiment of the present application provides a recognition method, which is applied to a terminal. The method includes: obtaining the image data of the image to be recognized, obtaining the color temperature data of the environment where the image to be recognized is taken through a color temperature sensor, respectively processing the time-domain information of the two channels of the image data of the image to be recognized and the color temperature data to obtain the image features of the image to be recognized, and inputting the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized, so as to recognize that the image to be recognized is an indoor image or an outdoor image; that is to say, in the embodiment of the present application, after processing the time-domain information of the two channels of the image data and the color temperature data of the obtained image to be recognized to obtain the image features of the image to be recognized, the image features are input into a pre-trained machine learning classification model for recognition, so that it can be recognized that the image to be recognized is an indoor image or an outdoor image. In this way, recognizing the image features of the image to be recognized through the machine learning classification model can more accurately recognize that the image to be recognized is an indoor image or an outdoor image, thereby improving the accuracy of image recognition and helping to obtain a more realistic image.

[0126] Embodiment 2

[0127] Figure 11 Schematic diagram of the structure of a terminal provided by an embodiment of the present application Figure One , such as Figure 11 shown, an embodiment of the present application provides a terminal, including:

[0128] The first acquisition module 111 is used to acquire the image data of the image to be recognized;

[0129] The second acquisition module 112 acquires the color temperature data of the environment where the image to be recognized is taken through a color temperature sensor;

[0130] The processing module 113 is used to respectively process the time-domain information of the two channels of the image data of the image to be recognized and the color temperature data to obtain the image features of the image to be recognized;

[0131] The recognition module 114 is used to input the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized, so as to recognize that the image to be recognized is an indoor image or an outdoor image.

[0132] Optionally, the processing module 113 is specifically used for:

[0133] Process the image data of the image to be recognized to obtain the processed image data;

[0134] Process the time-domain information of the two channels of the color temperature data to obtain the processed color temperature data;

[0135] Use the processed image data and the processed color temperature data to form the image features of the image to be recognized.

[0136] Optionally, when the processing module 113 processes the image data of the image to be recognized to obtain the processed image data, it includes:

[0137] Calculate the horizontal gradient value of the image to be recognized and the vertical gradient value of the image to be recognized according to the image data of the image to be recognized;

[0138] Determine the sum of the horizontal gradient value and the vertical gradient value of the image to be recognized as the actual gradient value of the image to be recognized;

[0139] Select the actual gradient values that fall within the preset weak gradient value range from the actual gradient values of the image to be recognized;

[0140] Group the image components corresponding to the actual gradient values that fall within the weak gradient value range to obtain eight groups of image components, and normalize the number of each group of image components to obtain the processed image data.

[0141] Optionally, when the processing module 113 processes the time-domain information of the two channels of the color temperature data to obtain the processed color temperature data, it includes:

[0142] Perform time-frequency conversion on the time-domain information of the two channels in the color temperature data respectively to obtain the frequency-domain information of the two channels;

[0143] Obtain the amplitudes of the two channels when the frequency value is 0 from the frequency-domain information of the two channels respectively;

[0144] According to the amplitudes of the two channels and the mean value of the visible light band of the color temperature data, call the preset calculation formula for the infrared band to obtain the processed color temperature data.

[0145] Optionally, this terminal is further used for:

[0146] Obtain the image data of the image set to be trained and the color temperature data of the image set to be trained;

[0147] Process the image data of the image set to be trained and the time-domain information of the two channels of the color temperature data of the image set to be trained respectively to obtain the image features of the image set to be recognized;

[0148] Use the image features of the image set to be trained to train the machine learning classification model, so as to determine the model parameters when the value of the loss function in the machine learning classification model is the smallest, and obtain the pre-trained machine learning classification model.

[0149] Optionally, after the terminal uses the image features of the image set to be trained to train the machine learning classification model to determine the model parameters when the value of the loss function in the machine learning classification model is the smallest and obtains the pre-trained machine learning classification model, it is further used for:

[0150] Obtain the image data of the image set to be tested and the color temperature data of the image set to be tested;

[0151] Process the time-domain information of the two channels of the image data of the image set to be tested and the color temperature data of the image set to be tested respectively to obtain the image features of the image set to be tested;

[0152] Input the image features of the image set to be tested into the pre-trained machine learning classification model to identify the image set to be tested and obtain the test result;

[0153] According to the test result, judge whether the pre-trained machine learning classification model passes.

[0154] Optionally, when the terminal judges whether the pre-trained machine learning classification model passes according to the test result, it includes:

[0155] Compare the test result with the preset test result of the image set to be tested to determine the correct rate of the test result;

[0156] When the correct rate of the test result is greater than or equal to the preset threshold, determine that the pre-trained machine learning classification model passes;

[0157] When the correct rate of the test result is less than the preset threshold, re-obtain the image data of the new image set to be trained and the color temperature data of the new image set to be trained, and return to execute the processing of the time-domain information of the two channels of the image data of the image to be trained and the color temperature data of the image to be trained respectively to obtain the image features of the image to be recognized.

[0158] Optionally, the terminal is further used for:

[0159] After inputting the image features of the image to be recognized into the pre-trained machine learning model to recognize the image to be recognized to identify whether the image to be recognized is an indoor image or an outdoor image, perform automatic white balance processing and / or automatic focusing processing on the image to be recognized according to whether the recognized image to be recognized is an indoor image or an outdoor image to obtain the processed image to be recognized.

[0160] In practical applications, the above-mentioned first acquisition module 111, second acquisition module 112, processing module 113, and recognition module 114 can be implemented by a processor located on the terminal, specifically implemented by a CPU, a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing), or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0161] Figure 12 FIG. 2 is a second schematic structural diagram of a terminal provided by an embodiment of the present application. As Figure 12 shown, an embodiment of the present application provides a terminal 1200, including:

[0162] a processor 121 and a storage medium 122 storing executable instructions of the processor 121. The storage medium 122 depends on the processor 121 to perform operations through a communication bus 123. When the instructions are executed by the processor 121, the recognition method described in the first embodiment above is executed.

[0163] It should be noted that in practical applications, each component in the terminal is coupled together through the communication bus 123. It can be understood that the communication bus 123 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 123 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 12 all kinds of buses are labeled as the communication bus 123.

[0164] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processor executes the recognition method described in the first embodiment.

[0165] Among them, the computer-readable storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.

[0166] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure One one flow or multiple flows and / or blocks Figure One one block or multiple blocks.

[0168] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure One one flow or multiple flows and / or blocks Figure One one block or multiple blocks.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks Figure One one process or a plurality of processes and / or blocks Figure One and steps for implementing the functions specified in one block or a plurality of blocks.

[0170] As mentioned above, the foregoing are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application.

Claims

1. A recognition method, characterized in that, The method is applied to a terminal, and the method includes: Obtaining image data of an image to be recognized; Obtaining color temperature data of the environment where the image to be recognized is captured through a color temperature sensor; Processing the time-domain information of the two channels of the image data of the image to be recognized and the color temperature data respectively to obtain the image features of the image to be recognized; Inputting the image features of the image to be recognized into a pre-trained machine learning classification model to recognize the image to be recognized, so as to recognize whether the image to be recognized is an indoor image or an outdoor image; The processing the time-domain information of the two channels of the image data of the image to be recognized and the color temperature data respectively to obtain the image features of the image to be recognized includes: Processing the image data of the image to be recognized to obtain processed image data; Performing time-frequency conversion on the time-domain information of the two channels of the color temperature data respectively to obtain the frequency-domain information of the two channels; Respectively obtaining the amplitudes of the two channels when the frequency value is 0 from the frequency-domain information of the two channels; According to the amplitudes of the two channels and the mean value of the visible light band of the color temperature data, calling a preset calculation formula for the infrared band to obtain the processed color temperature data; the preset calculation formula for the infrared band is: Wherein, IR1 is used to measure the infrared band intensity of 800nm - 900nm, IR2 is used to measure the infrared band intensity of 950nm - 1000nm; FD1 and FD2 are the two channels of the color temperature data respectively; DC(FD1) is the amplitude of channel FD1 when the frequency value is 0 selected from the frequency-domain information of channel FD1; DC(FD2) is the amplitude of channel FD2 when the frequency value is 0 selected from the frequency-domain information of channel FD2; C is the mean value of the visible light band of the color temperature data; Forming the image features of the image to be recognized by using the processed image data and the processed color temperature data.

2. The method according to claim 1, characterized in that, The processing the image data of the image to be recognized to obtain processed image data includes: Calculating the horizontal gradient value of the image to be recognized and the vertical gradient value of the image to be recognized according to the image data of the image to be recognized; Determining the sum of the horizontal gradient value and the vertical gradient value of the image to be recognized as the actual gradient value of the image to be recognized; Selecting the actual gradient values that fall within a preset weak gradient value range from the actual gradient values of the image to be recognized; Grouping the image components corresponding to the actual gradient values that fall within the weak gradient value range to obtain eight groups of image components, and normalizing the number of each group of image components to obtain the processed image data.

3. The method according to claim 1, wherein The method further includes: Obtaining the image data of the image set to be trained and the color temperature data of the image set to be trained; Processing the time-domain information of the two channels of the image data of the image set to be trained and the color temperature data of the image set to be trained respectively to obtain the image features of the image set to be recognized; Training a machine learning classification model using the image features of the to-be-trained image set to determine the model parameters when the value of the loss function in the machine learning classification model is minimized, and obtaining the pre-trained machine learning classification model.

4. The method according to claim 3, wherein After training the machine learning classification model using the image features of the to-be-trained image set to determine the model parameters when the value of the loss function in the machine learning classification model is minimized, and obtaining the pre-trained machine learning classification model, the method further includes: Obtaining the image data of the to-be-tested image set and the color temperature data of the to-be-tested image set; Processing the time-domain information of two channels of the image data of the to-be-tested image set and the color temperature data of the to-be-tested image set respectively to obtain the image features of the to-be-tested image set; Inputting the image features of the to-be-tested image set into the pre-trained machine learning classification model to identify the to-be-tested image set and obtaining a test result; Judging whether the pre-trained machine learning classification model passes according to the test result.

5. The method according to claim 4, characterized in that, The judging whether the pre-trained machine learning classification model passes according to the test result includes: Comparing the test result with the preset test result of the to-be-tested image set to determine the accuracy rate of the test result; When the accuracy rate of the test result is greater than or equal to the preset threshold, determining that the pre-trained machine learning classification model passes; When the accuracy rate of the test result is less than the preset threshold, re-obtaining the image data of the new to-be-trained image set and the color temperature data of the new to-be-trained image set, and returning to execute the processing of the time-domain information of two channels of the image data of the to-be-trained image and the color temperature data of the to-be-trained image to obtain the image features of the to-be-identified image.

6. The method according to claim 1, wherein After inputting the image features of the to-be-identified image into the pre-trained machine learning model to identify the to-be-identified image as an indoor image or an outdoor image, the method further includes: Performing automatic white balance processing and / or automatic focusing processing on the to-be-identified image according to whether the to-be-identified image is identified as an indoor image or an outdoor image to obtain the processed to-be-identified image.

7. A terminal, characterized in that, The terminal includes: A first acquisition module for acquiring the image data of the to-be-identified image; A second acquisition module for acquiring the color temperature data of the environment where the to-be-identified image is taken through a color temperature sensor; A processing module for processing the time-domain information of two channels of the image data of the to-be-identified image and the color temperature data respectively to obtain the image features of the to-be-identified image; An identification module for inputting the image features of the to-be-identified image into the pre-trained machine learning classification model to identify the to-be-identified image as an indoor image or an outdoor image; The processing module is configured to process the image data of the image to be recognized to obtain the processed image data; perform time-frequency conversion on the time-domain information of the two channels of the color temperature data respectively to obtain the frequency-domain information of the two channels; respectively obtain the amplitudes of the two channels when the frequency value is 0 from the frequency-domain information of the two channels; according to the amplitudes of the two channels and the mean value of the visible light band of the color temperature data, call a preset calculation formula for the infrared band to obtain the processed color temperature data; the preset calculation formula for the infrared band is: wherein, IR1 is used to measure the infrared band intensity of 800 nm to 900 nm, and IR2 is used to measure the infrared band intensity of 950 nm to 1000 nm; FD1 and FD2 are respectively the two channels of the color temperature data; DC(FD1) is the amplitude of channel FD1 when the frequency value is 0 selected from the frequency-domain information of channel FD1; DC(FD2) is the amplitude of channel FD2 when the frequency value is 0 selected from the frequency-domain information of channel FD2; C is the mean value of the visible light band of the color temperature data; form the image feature of the image to be recognized by using the processed image data and the processed color temperature data; When the processing module processes the time-domain information of two channels of the color temperature data to obtain the processed color temperature data, it includes: Perform time-frequency conversion on the time-domain information of the two channels of the color temperature data respectively to obtain the frequency-domain information of the two channels; Respectively obtain the amplitudes of the two channels when the frequency value is 0 from the frequency-domain information of the two channels; According to the amplitudes of the two channels and the mean value of the visible light band of the color temperature data, call the preset calculation formula for the infrared band to obtain the processed color temperature data.

8. A terminal, characterized in that, The terminal includes: a processor and a storage medium storing instructions executable by the processor. The storage medium depends on the processor to perform operations through a communication bus. When the instructions are executed by the processor, the recognition method according to any one of claims 1 to 6 above is executed.

9. A computer storage medium, characterized in that, Store executable instructions. When the executable instructions are executed by one or more processors, the processor executes the recognition method according to any one of claims 1 to 6.

Images