Image processing method, image recognition method, control device, readable medium and equipment

Abstract

The invention relates to the technical field of images, particularly provides an image processing method, an image recognition method, a control device, a readable medium and equipment, and aims to solve the problem of realizing accurate AON recognition on the premise of low power consumption. In order to achieve the purpose, an image sensor of the terminal equipment obtains a current image, obtains current environment light brightness according to an environment light sensor, obtains a current environment scene according to the environment light brightness, and carries out image correction on the image according to the environment scene, so that a corrected image is obtained. Through the configuration mode, different environment scenes can be distinguished according to different environment light brightness, so that distinctive image correction is performed on the image according to the different environment scenes, the image quality of the corrected image in the different environment scenes can be effectively improved without application of rich ISP functions, and the user experience is improved. While the power consumption of the terminal equipment is reduced, the recognition effect of the AON function can be effectively ensured.

Description

Technical Field

[0001] This application relates to the field of image technology, and specifically provides an image processing method, an image recognition method, a control device, a readable medium, and a device. Background Art

[0002] Currently, there are two main solutions for implementing the AON function, namely always-on camera face recognition and face gaze, on mobile devices. The first solution is that the main processor directly calls the front camera to take pictures, and then recognizes faces or gazes in the security processor or the main processor. The second solution is to call the front camera in low-power mode to take pictures in the low-power processor, and recognize faces or gazes in the low-power processor. The first solution can process images by using rich ISP (Image Signal Processing) functions, so as to keep the images clear under different ambient light brightness. However, this solution requires a relatively large power consumption for AP wake-up. The second solution has a lower power consumption and only simple gamma processing, and cannot process images according to different ambient light. The image quality in dark, large backlight and other scenarios is relatively low, which affects face recognition and gaze recognition.

[0003] Correspondingly, there is a need for a new image processing solution in this field to solve the above problems. Summary of the Invention

[0004] In order to overcome the above defects, this application is proposed to provide a solution to solve or at least partially solve the problem of achieving accurate AON recognition under the premise of low power consumption.

[0005] In a first aspect, this application provides an image processing method, which is applied to a terminal device, and an image sensor and an ambient light sensor are arranged on the terminal device; the method includes:

[0006] Obtain a current image through the image sensor;

[0007] Obtain the ambient light brightness corresponding to the image through the ambient light sensor;

[0008] Obtain the ambient scene corresponding to the image according to the ambient light brightness;

[0009] Based on the ambient scene, perform image correction on the image to obtain a corrected image.

[0010] In a technical solution of the above image processing method, the performing image correction on the image based on the ambient scene to obtain a corrected image includes:

[0011] Obtain the gamma coefficient corresponding to the ambient scene according to the ambient scene;

[0012] Perform image correction on the image according to the gamma coefficient to obtain the corrected image.

[0013] In a technical solution of the above image processing method, the method further includes obtaining the gamma coefficient corresponding to the environmental scene according to the following steps:

[0014] Collect an image set for obtaining the gamma coefficient, and obtain the ambient light luminance corresponding to each image in the image set;

[0015] Obtain the environmental scene corresponding to the image according to the ambient light luminance;

[0016] For the images of each environmental scene, obtain multiple adjusted gamma coefficients according to the preset initial value of the gamma coefficient and the preset step;

[0017] Perform image correction on the image according to each adjusted gamma coefficient to obtain the corrected image corresponding to each adjusted gamma coefficient;

[0018] Perform image recognition on the corrected image to obtain an image recognition result;

[0019] Obtain the gamma coefficient corresponding to the environmental scene according to the image recognition result.

[0020] In a technical solution of the above image processing method, the obtaining the gamma coefficient corresponding to the environmental scene according to the image recognition result includes:

[0021] For each adjusted gamma coefficient, obtain the confidence mean of the image recognition results of the images under the same environmental scene;

[0022] Select the adjusted gamma coefficient with the largest confidence mean as the gamma coefficient corresponding to the environmental scene.

[0023] In a technical solution of the above image processing method, the ambient light luminance includes the front ambient light luminance and the rear ambient light luminance;

[0024] The obtaining the environmental scene corresponding to the image according to the ambient light luminance includes:

[0025] Obtain the environmental scene according to the front ambient light luminance and the rear ambient light luminance.

[0026] In a technical solution of the above image processing method, the image sensor is a front image sensor, and the environmental scene includes a dark scene, a face light scene, a backlight scene, and a normal light scene;

[0027] Obtaining the environmental scene according to the front ambient light brightness and the rear ambient light brightness includes:

[0028] When both the front ambient light brightness and the rear ambient light brightness are less than a first preset brightness, determining that the environmental scene is the dark scene;

[0029] When both the front ambient light brightness and the rear ambient light brightness are greater than a second preset brightness, and the difference obtained by subtracting the rear ambient light brightness from the front ambient light brightness is greater than a third preset brightness, determining that the environmental scene is the front light scene;

[0030] When both the front ambient light brightness and the rear ambient light brightness are greater than a second preset brightness, and the difference obtained by subtracting the front ambient light brightness from the rear ambient light brightness is greater than a third preset brightness, determining that the environmental scene is the backlight scene;

[0031] When both the front ambient light brightness and the rear ambient light brightness do not meet the above conditions, determining that the environmental scene is the normal light scene;

[0032] Wherein, the first preset brightness is less than the third preset brightness which is less than the second preset brightness, and the first preset brightness is greater than zero.

[0033] In a technical solution of the above image processing method, the ambient light sensor includes a front ambient light sensor and a rear ambient light sensor; the ambient light brightness includes the front ambient light brightness and the rear ambient light brightness;

[0034] Obtaining the ambient light brightness corresponding to the image through the ambient light sensor includes:

[0035] Obtaining the front ambient light brightness through the front ambient light sensor;

[0036] Obtaining the rear ambient light brightness through the rear ambient light sensor.

[0037] In a technical solution of the above image processing method, obtaining the current image through the image sensor includes:

[0038] Collecting a low-resolution image through the image sensor as the current image;

[0039] Wherein, the low-resolution image is an image with a resolution less than the VGA specification resolution.

[0040] In a technical solution of the above image processing method, the current image is an image with a QVGA specification resolution or a QQVGA specification resolution.

[0041] In a second aspect, the present application provides an image recognition method, which includes:

[0042] Obtain an image to be recognized, where the image is a corrected image obtained by the image processing method according to any one of the above image processing methods;

[0043] Perform image recognition based on the image to be recognized to obtain an image recognition result.

[0044] In a technical solution of the above image recognition method, the performing image recognition based on the image to be recognized to obtain an image recognition result includes:

[0045] Perform face recognition based on the image to be recognized to obtain a face recognition result.

[0046] In a technical solution of the above image recognition method, the performing image recognition based on the image to be recognized to obtain an image recognition result includes:

[0047] Perform gaze recognition based on the image to be recognized to obtain a gaze recognition result.

[0048] In a third aspect, a control device is provided. The control device includes at least one processor and at least one storage device. The storage device is adapted to store multiple program codes, and the program codes are adapted to be loaded and run by the processor to execute the image processing method according to any one of the technical solutions of the above image processing method or the image recognition method according to any one of the technical solutions of the above image recognition method.

[0049] In a technical solution of the above image recognition method, the processor is a low-power processor.

[0050] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores multiple program codes, and the program codes are adapted to be loaded and run by a processor to execute the image processing method according to any one of the technical solutions of the above image processing method or the image recognition method according to any one of the technical solutions of the above image recognition method.

[0051] In a fifth aspect, a terminal device is provided. An image sensor and an ambient light sensor are provided on the terminal device, and the terminal device includes the control device according to any one of the technical solutions of the above control device.

[0052] One or more of the above technical solutions of the present application have at least one or more of the following Beneficial effects:

[0053] In implementing the technical solution of the present application, the image sensor of the terminal device of the present application acquires the current image, obtains the current ambient light brightness according to the ambient light sensor, obtains the current ambient scene according to the ambient light brightness, and performs image correction on the image according to the ambient scene, so as to obtain the corrected image. Through the above configuration method, the present application can distinguish different ambient scenes according to different ambient light brightness, and thus perform differential image correction on the image according to different ambient scenes. Without applying rich ISP functions, it can effectively improve the image quality of the corrected image in different ambient scenes, and while reducing the power consumption of the terminal device, it can effectively ensure the recognition effect of the AON function.

[0054] Further, the present application respectively acquires the front ambient light brightness and the rear ambient light brightness of the terminal device, and obtains the ambient scene according to the front ambient light brightness and the rear ambient light brightness, which can accurately distinguish different ambient scenes, and thus perform image correction according to different ambient scenes, further improving the image quality.

[0055] Further, the present application sets a corresponding gamma coefficient for each ambient scene, and calls the corresponding gamma coefficient based on different ambient scenes to correct the image, which can make the image clearer, thereby further improving the image quality in different ambient scenes and effectively ensuring the recognition effect of the AON function.

[0056] Solution 1. An image processing method, characterized in that the method is applied to a terminal device, and an image sensor and an ambient light sensor are provided on the terminal device; the method includes:

[0057] Acquire the current image through the image sensor;

[0058] Acquire the ambient light brightness corresponding to the image through the ambient light sensor;

[0059] Obtain the ambient scene corresponding to the image according to the ambient light brightness;

[0060] Perform image correction on the image based on the ambient scene to obtain the corrected image.

[0061] Solution 2. The image processing method according to Solution 1, characterized in that

[0062] The performing image correction on the image based on the ambient scene to obtain the corrected image includes:

[0063] Obtain the gamma coefficient corresponding to the ambient scene according to the ambient scene;

[0064] Perform image correction on the image according to the gamma coefficient to obtain the corrected image.

[0065] Solution 3. The image processing method according to Solution 2, characterized in that

[0066] The method further includes obtaining the gamma coefficient corresponding to the environmental scene according to the following steps:

[0067] Collect an image set for obtaining the gamma coefficient, and obtain the ambient light luminance corresponding to each image in the image set;

[0068] Obtain the environmental scene corresponding to the image according to the ambient light luminance;

[0069] For the images of each environmental scene, obtain multiple adjusted gamma coefficients according to the preset initial gamma coefficient value and the preset step;

[0070] Perform image correction on the image according to each adjusted gamma coefficient to obtain the corrected image corresponding to each adjusted gamma coefficient;

[0071] Perform image recognition on the corrected image to obtain an image recognition result;

[0072] Obtain the gamma coefficient corresponding to the environmental scene according to the image recognition result.

[0073] Solution 4. The image processing method according to Solution 3, characterized in that

[0074] The obtaining the gamma coefficient corresponding to the environmental scene according to the image recognition result includes:

[0075] For each adjusted gamma coefficient, obtain the confidence mean value of the image recognition results of the images under the same environmental scene;

[0076] Select the adjusted gamma coefficient with the largest confidence mean value as the gamma coefficient corresponding to the environmental scene.

[0077] Solution 5. The image processing method according to any one of Solutions 1 to 4, characterized in that the ambient light luminance includes the front ambient light luminance and the rear ambient light luminance;

[0078] The obtaining the environmental scene corresponding to the image according to the ambient light luminance includes:

[0079] Obtain the environmental scene according to the front ambient light luminance and the rear ambient light luminance.

[0080] Solution 6. The image processing method according to Solution 5, characterized in that the image sensor is a front image sensor, and the environmental scene includes a dark scene, a front-light scene, a backlight scene, and a normal light scene;

[0081] Obtaining the environmental scene according to the front environmental light brightness and the rear environmental light brightness includes:

[0082] When both the front environmental light brightness and the rear environmental light brightness are less than a first preset brightness, determining that the environmental scene is the dark scene;

[0083] When both the front environmental light brightness and the rear environmental light brightness are greater than a second preset brightness, and the difference obtained by subtracting the rear environmental light brightness from the front environmental light brightness is greater than a third preset brightness, determining that the environmental scene is the front-light scene;

[0084] When both the front environmental light brightness and the rear environmental light brightness are greater than a second preset brightness, and the difference obtained by subtracting the front environmental light brightness from the rear environmental light brightness is greater than a third preset brightness, determining that the environmental scene is the backlight scene;

[0085] When both the front environmental light brightness and the rear environmental light brightness do not meet the above conditions, determining that the environmental scene is the normal light scene;

[0086] Wherein, the first preset brightness is less than the third preset brightness which is less than the second preset brightness, and the first preset brightness is greater than zero.

[0087] Solution 7. The image processing method according to Solution 1, characterized in that the environmental light sensor includes a front environmental light sensor and a rear environmental light sensor; the environmental light brightness includes the front environmental light brightness and the rear environmental light brightness;

[0088] Obtaining the environmental light brightness corresponding to the image through the environmental light sensor includes:

[0089] Obtaining the front environmental light brightness through the front environmental light sensor;

[0090] Obtaining the rear environmental light brightness through the rear environmental light sensor.

[0091] Solution 8. The image processing method according to any one of Solutions 1 to 7, characterized in that

[0092] Obtaining the current image through the image sensor includes:

[0093] Collecting a low-resolution image through the image sensor as the current image;

[0094] Among them, the low-resolution image is an image with a resolution less than the VGA standard resolution.

[0095] Solution 9. The image processing method according to Solution 8, characterized in that

[0096] The current image is an image with a QVGA standard resolution or a QQVGA standard resolution.

[0097] Solution 10. An image recognition method, characterized in that the method includes:

[0098] Obtain the image to be recognized, where the image is the corrected image obtained by the image processing method according to any one of Solutions 1 to 9;

[0099] Perform image recognition according to the image to be recognized to obtain an image recognition result.

[0100] Solution 11. The image recognition method according to Solution 10, characterized in that

[0101] The performing image recognition according to the image to be recognized to obtain an image recognition result includes:

[0102] Perform face recognition according to the image to be recognized to obtain a face recognition result.

[0103] Solution 12. The image recognition method according to Solution 10, characterized in that

[0104] The performing image recognition according to the image to be recognized to obtain an image recognition result includes:

[0105] Perform gaze recognition according to the image to be recognized to obtain a gaze recognition result.

[0106] Solution 13. A control device includes at least one processor and at least one storage device, the storage device is adapted to store multiple program codes, characterized in that the program codes are adapted to be loaded and run by the processor to execute the image processing method according to any one of Solutions 1 to 9 or the image recognition method according to any one of Solutions 10 to 12.

[0107] Solution 14. The image recognition method according to Solution 13, characterized in that the processor is a low-power processor.

[0108] Solution 15. A computer-readable storage medium stores multiple program codes, characterized in that the program codes are adapted to be loaded and run by a processor to execute the image processing method according to any one of Solutions 1 to 9 or the image recognition method according to any one of Solutions 10 to 12.

[0109] Solution 16. A terminal device, characterized in that an image sensor and an ambient light sensor are provided on the terminal device, and the terminal device includes the control device described in Solution 13 or 14. Description of the Drawings

[0110] With reference to the drawings, the disclosure of the present application will become more understandable. It is easily understood by those skilled in the art that these drawings are only for illustrative purposes and are not intended to limit the protection scope of the present application. Among them:

[0111] Figure 1 is a schematic main step flow diagram of an image processing method according to an embodiment of the present application;

[0112] Figure 2 is a schematic main step flow diagram of an image processing method according to an implementation manner of an embodiment of the present application;

[0113] Figure 3 is a schematic main step flow diagram of an image recognition method according to an embodiment of the present application;

[0114] Figure 4 is a schematic main structure block diagram of an image processing system according to an embodiment of the present application;

[0115] Figure 5 is a schematic main structure block diagram of an image recognition system according to an embodiment of the present application. Detailed Embodiments

[0116] Some embodiments of the present application will be described below with reference to the drawings. Those skilled in the art should understand that these embodiments are only for explaining the technical principle of the present application and are not intended to limit the protection scope of the present application.

[0117] In the description of this application, "module" and "processor" may include hardware, software, or a combination of both. A module may include a hardware circuit, various appropriate sensors, communication ports, memory, and may also include a software part, such as program code, or may be a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, in hardware, or in a combination of both. A non-transitory computer-readable storage medium includes any suitable medium that can store program code, such as magnetic disks, hard disks, optical discs, flash memories, read-only memories, random access memories, and so on. The term "A and / or B" represents all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and / or B" and may include only A, only B, or A and B. The singular terms "a" and "the" may also include the plural form.

[0118] Refer to the attached Figure 1 , Figure 1 is a schematic diagram of the main steps of an image processing method according to an embodiment of this application. As Figure 1 shown, the image processing method in the embodiment of this application is applied to a terminal device. An image sensor and an ambient light sensor may be provided on the terminal device. The image processing method mainly includes the following steps S101 - step S104.

[0119] Step S101: Obtain the current image through the image sensor.

[0120] In this embodiment, the current image may be collected through the image sensor.

[0121] In one implementation, the terminal device may be a smart phone, a tablet computer, a smart controller, or other devices.

[0122] In one implementation, the image sensor may be a camera provided on the terminal device. For an image used to implement the AON function, the image sensor may be a front camera for collecting face images.

[0123] In one implementation, a low-resolution image may be collected through the image sensor as the current image. Among them, the low-resolution image is image data with a resolution lower than the VGA specification resolution. That is, the low-resolution image that the image sensor can support is collected, which can effectively reduce power consumption. Among them, the VGA (Video Graphics Array) specification resolution is 640 pixels × 480 pixels. That is, the low-resolution image is an image with a resolution lower than 640 pixels × 480 pixels.

[0124] In one embodiment, the current image can be an image with a QVGA (Quarter Video Graphics Array) specification resolution or a QQVGA (Quarter Quarter Video Graphics Array) specification resolution, where the QVGA specification resolution is 320 pixels × 240 pixels and the QQVGA resolution is 120 pixels × 160 pixels. It should be noted that the above resolutions are only exemplary, and the present application is not limited to the above resolutions.

[0125] Step S102: Obtain the ambient light luminance corresponding to the image through an ambient light sensor.

[0126] In this embodiment, the current ambient light luminance can be obtained through an ambient light sensor.

[0127] In one embodiment, the ambient light sensor can include a front ambient light sensor and a rear ambient light sensor, that is, ambient light sensors are provided on both the side of the terminal device facing the user and the side not facing the user.

[0128] Step S103: Obtain the ambient scene corresponding to the image according to the ambient light luminance.

[0129] In this embodiment, the current ambient scene can be determined according to the collected ambient light luminance.

[0130] In one embodiment, the ambient scene can include a face light scene, a backlight scene, a dark scene, and a normal light scene. The current ambient scene can be distinguished according to different values of the ambient light luminance.

[0131] Step S104: Based on the ambient scene, perform image correction on the image to obtain a corrected image.

[0132] In this embodiment, image correction can be performed on the image based on the ambient scene, so as to obtain a corrected image.

[0133] In one embodiment, bright field uniformity correction can be performed on the image according to the ambient scene to obtain a corrected image.

[0134] In one embodiment, LUT (Look-Up Table) correction can be performed on the image according to the ambient scene to obtain a corrected image.

[0135] Based on the above steps S101 - S104, the image sensor of the terminal device in the embodiment of the present application acquires the current image, obtains the current ambient light brightness according to the ambient light sensor, obtains the current ambient scene according to the ambient light brightness, and performs image correction on the image according to the ambient scene, so as to obtain the corrected image. Through the above configuration method, the embodiment of the present application can distinguish different ambient scenes according to different ambient light brightnesses, and thus perform differential image correction on the image according to different ambient scenes. Without applying rich ISP functions, it can effectively improve the image quality of the corrected image in different ambient scenes, and while reducing the power consumption of the terminal device, it can effectively ensure the recognition effect of the AON function.

[0136] The following further describes steps S102, S103, and S104 respectively.

[0137] In an implementation manner of the embodiment of the present application, the ambient light brightness includes the front ambient light brightness and the rear ambient light brightness, and step S102 may further include the following steps S1021 and S1022:

[0138] Step S1021: Obtain the front ambient light brightness through the front ambient light sensor.

[0139] Step S1022: Obtain the rear ambient light brightness through the rear ambient light sensor.

[0140] In this implementation manner, the front ambient light brightness can be obtained through the front ambient light sensor respectively, and the rear ambient light brightness can be obtained through the rear ambient sensor respectively. Obtaining the front ambient light brightness and the rear ambient light brightness respectively can further obtain a more accurate ambient scene.

[0141] In an implementation manner of the embodiment of the present application, step S103 is further configured as:

[0142] Obtain the ambient scene according to the front ambient light brightness and the rear ambient light brightness.

[0143] In this implementation manner, the ambient scene may include a dark scene, a face light scene, a backlight scene, and a normal light scene. Step S103 may further include the following steps S1031 to S1034:

[0144] Step S1031: When both the front ambient light brightness and the rear ambient light brightness are less than the first preset brightness, determine that the ambient scene is a dark scene;

[0145] Step S1032: When both the front ambient light brightness and the rear ambient light brightness are greater than the second preset brightness, and the difference obtained by subtracting the rear ambient light brightness from the front ambient light brightness is greater than the third preset brightness, determine that the ambient scene is a face light scene.

[0146] Step S1033: When both the front ambient light brightness and the rear ambient light brightness are greater than the second preset brightness, and the difference obtained by subtracting the front ambient light brightness from the rear ambient light brightness is greater than the third preset brightness, determine that the environmental scene is a backlight scene.

[0147] Step S1034: When both the front ambient light brightness and the rear ambient light brightness do not meet the above conditions, determine that the environmental scene is a normal light scene; where the first preset brightness is less than the third preset brightness which is less than the second preset brightness, and the first preset brightness is greater than zero.

[0148] In this embodiment, assume that the front ambient light brightness is Lfront and the rear ambient light brightness is Lrear.

[0149] When Lfront < the first preset brightness and Lrear < the first preset brightness, it is a dark scene;

[0150] When Lfront > the second preset brightness, Lrear > the second preset brightness, and Lfront - Lrear > the third preset brightness, it is a front light scene;

[0151] When Lfront > the second preset brightness, Lrear > the second preset brightness, and Lrear - Lfront > the third preset brightness, it is a backlight scene

[0152] In other cases, it is a normal light scene.

[0153] Among them, the front light scene refers to the situation where the light projection direction is the same as the shooting direction of the image sensor. The backlight scene refers to the scene where the light is projected from the back or the oblique back of the subject to be photographed.

[0154] In one embodiment, the first preset brightness can be 5 lux, the second preset brightness can be 500 lux, and the third preset brightness can be 200 lux.

[0155] It should be noted that the above values of the first preset brightness, the second preset brightness, and the third preset brightness are only exemplary, and those skilled in the art can set the values of the first preset brightness, the second preset brightness, and the third preset brightness according to the actual application needs.

[0156] In one embodiment of the embodiments of the present application, step S104 may further include the following steps S1041 and S1042:

[0157] Step S1041: According to the environmental scene, obtain the gamma coefficient corresponding to the environmental scene.

[0158] Among them, the gamma coefficient is a coefficient used for gamma correction of an image. Gamma correction is a non-linear transformation used to correct the brightness of an image. The gamma coefficient determines the gray-scale mapping relationship between the input image and the output image during the gamma correction process.

[0159] In this embodiment, the gamma coefficient corresponding to the environmental scene can be obtained according to the following steps S201 to S206.

[0160] Step S201: Collect an image set for obtaining the gamma coefficient, and obtain the environmental light brightness corresponding to each image in the image set.

[0161] In this embodiment, an image set for obtaining the gamma coefficient can be collected, and the environmental light brightness corresponding to each image can be obtained.

[0162] Step S202: Obtain the environmental scene corresponding to the image according to the environmental light brightness.

[0163] In this embodiment, the method described in steps 1031 to S1034 can be used to obtain the environmental scene corresponding to the image according to the environmental light brightness.

[0164] Step S203: For the images of each environmental scene, obtain a plurality of adjusted gamma coefficients according to a preset initial value of the gamma coefficient and a preset step.

[0165] In this embodiment, for the images of each environmental scene, the initial value of the gamma coefficient and the preset step can be set to obtain a plurality of adjusted gamma coefficients.

[0166] In one embodiment, the initial value of the gamma coefficient can be 0.1, and the preset step can be 0.1. That is, taking 0.1 as the initial value and 0.1 as the step, a plurality of adjusted gamma coefficients can be obtained, and the maximum value of the gamma coefficient is 1.

[0167] Step S204: Correct the image according to each adjusted gamma coefficient to obtain a corrected image corresponding to each adjusted gamma coefficient.

[0168] In this embodiment, the image can be gamma-corrected according to each adjusted gamma coefficient to obtain a corrected image corresponding to each adjusted gamma coefficient.

[0169] Step S205: Perform image recognition on the corrected image to obtain an image recognition result.

[0170] In this embodiment, the corrected image is subjected to image recognition to obtain an image recognition result.

[0171] In one embodiment, the image recognition can be gaze recognition, that is, applying a gaze recognition algorithm to perform image recognition on the corrected image.

[0172] In one embodiment, the image recognition can be face recognition, that is, applying a face recognition algorithm to perform image recognition on the corrected image.

[0173] Step S206: According to the image recognition result, obtain the gamma coefficient corresponding to the environmental scene.

[0174] In this embodiment, step S206 can further include the following steps S2061 and S2062:

[0175] Step S2061: For each adjusted gamma coefficient, obtain the mean confidence of the image recognition results of the images under the same environmental scene.

[0176] Step S2062: Select the adjusted gamma coefficient with the largest mean confidence as the gamma coefficient corresponding to the environmental scene.

[0177] In this embodiment, for each environmental scene, the corrected images after gamma correction with different adjusted gamma coefficients can be used for image recognition to obtain the confidence of the image recognition results. Take the mean of the confidence of the image recognition results corresponding to the adjusted gamma coefficient to obtain the mean confidence. Select the adjusted gamma coefficient with the largest mean confidence as the gamma coefficient corresponding to the environmental scene.

[0178] Step S1042: According to the gamma coefficient, perform image correction on the image to obtain the corrected image.

[0179] In this embodiment, the image can be gamma-corrected according to the gamma coefficient to obtain the corrected image.

[0180] In one embodiment, reference can be made to the appendix Figure 2 , Figure 2 is a schematic diagram of the main step flow of an image processing method according to an embodiment of the present application. As Figure 2 shown, an image can be output based on the low-power mode through the front camera, and image M can be obtained through the low-power camera (AON). The front and rear ambient light brightnesses (front ambient light brightness and rear ambient light brightness) F_LUX and R_LUX can be obtained based on the front ambient light sensor F_ALS and the rear ambient light sensor R_AL. The dark, backlight, face light, and normal light scenes can be distinguished according to the front and rear ambient light brightnesses, and the corresponding gamma curves can be selected for image processing according to different scenes. The processed image can be recognized by an algorithm, and the face and gaze recognition results can be output and transmitted to the main processor.

[0181] Furthermore, the present application also provides an image recognition method.

[0182] Refer to the appendix Figure 3 , Figure 3 which is a schematic diagram of the main steps of an image recognition method according to an embodiment of the present application. As Figure 3 shown, the image recognition method in the embodiment of the present application mainly includes the following steps S301 - step S302.

[0183] Step S301: Obtain the image to be recognized, where the image is the corrected image obtained according to the image processing method described in the above-mentioned embodiment of the image processing method.

[0184] Step S302: Perform image recognition according to the image to be recognized to obtain an image recognition result.

[0185] In this embodiment, since the image obtained in the embodiment of the image processing method is corrected based on the environmental scene, it can ensure the image quality without a rich ISP function, and then perform image recognition based on the obtained image, which can ensure the accuracy of the image recognition result.

[0186] In one implementation manner, step S302 may include the following step S3021:

[0187] Step S3021: Perform face recognition according to the image to be recognized to obtain a face recognition result.

[0188] In this implementation manner, face recognition can be performed according to the image to be recognized to obtain a face recognition result.

[0189] In one implementation manner, step S302 may include the following step S3022:

[0190] Step S3022: Perform gaze recognition according to the image to be recognized to obtain a gaze recognition result.

[0191] In this implementation manner, gaze recognition can be performed according to the image to be recognized to obtain a gaze recognition result.

[0192] It should be noted that although the above steps are described in a specific order in the above embodiments, those skilled in the art can understand that in order to achieve the effects of the present application, different steps do not necessarily have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders, and these changes are all within the protection scope of the present application.

[0193] Furthermore, the present application also provides an image processing system.

[0194] Refer to the appendixFigure 4 , Figure 4 is the main structural block diagram of an image processing system according to an embodiment of the present application. As Figure 4 shown, the image processing system in the embodiment of the present application is applied to a terminal device, and an image sensor and an ambient light sensor are provided on the terminal device. The image processing system mainly includes an image acquisition module, an ambient light brightness acquisition module, an ambient scene acquisition module, and an image correction module. In this embodiment, the image acquisition module can be configured to acquire a current image through the image sensor. The ambient light brightness acquisition module can be configured to acquire the ambient light brightness corresponding to the image through the ambient light sensor. The ambient scene acquisition module can be configured to acquire the ambient scene corresponding to the image according to the ambient light brightness. The image correction module can be configured to perform image correction on the image based on the ambient scene to obtain a corrected image.

[0195] The above image processing system is used to execute Figure 1 the image processing method embodiment shown. The technical principles, the technical problems solved, and the technical effects produced by the two are similar. Those skilled in the art of the present technology can clearly understand that for the convenience and conciseness of description, the specific working process and related descriptions of the image processing system can refer to the content described in the embodiment of the image processing method, and will not be elaborated here.

[0196] Furthermore, the present application also provides an image recognition system.

[0197] Referring to the attached Figure 5 , Figure 5 is the main structural block diagram of an image recognition system according to an embodiment of the present application. As Figure 5 shown, the image recognition system in the embodiment of the present application mainly includes an image to be recognized acquisition module and an image recognition module. In this embodiment, the image acquisition module can be configured to acquire an image to be recognized, where the image is the corrected image obtained by the image processing method described in the above image processing method embodiment. The image recognition module can be configured to perform image recognition on the image to be recognized to obtain an image recognition result.

[0198] The above image recognition system is used to execute Figure 3 the image recognition method embodiment shown. The technical principles, the technical problems solved, and the technical effects produced by the two are similar. Those skilled in the art of the present technology can clearly understand that for the convenience and conciseness of description, the specific working process and related descriptions of the image recognition system can refer to the content described in the embodiment of the image recognition method, and will not be elaborated here.

[0199] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiment of the present application can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium that can carry the computer program code. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0200] Furthermore, the present application also provides a control device. In an embodiment of the control device according to the present application, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the image processing method of the above-mentioned method embodiment. The processor can be configured to execute the program in the storage device, and the program includes, but is not limited to, a program for executing the image processing method of the above-mentioned method embodiment. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The control device can be a control device formed by various electronic devices.

[0201] In the embodiment of the present application, the control device can be a control device formed by various electronic devices. In some possible implementation manners, the control device can include multiple storage devices and multiple processors. The program for executing the image processing method of the above-mentioned method embodiment can be divided into multiple sub-programs, and each sub-program can be loaded and run by a processor respectively to execute different steps of the image processing method of the above-mentioned method embodiment. Specifically, each sub-program can be stored in a different storage device respectively, and each processor can be configured to execute the program in one or more storage devices to jointly implement the image processing method of the above-mentioned method embodiment, that is, each processor respectively executes different steps of the image processing method of the above-mentioned method embodiment to jointly implement the image processing method of the above-mentioned method embodiment.

[0202] The above-mentioned multiple processors may be processors deployed on the same device. For example, the above-mentioned control device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors may be processors configured on the high-performance device. In addition, the above-mentioned multiple processors may also be processors deployed on different devices. For example, the above-mentioned control device may be a server cluster, and the above-mentioned multiple processors may be processors on different servers in the server cluster.

[0203] In one embodiment, the processor of the control device is a low-power processor.

[0204] Furthermore, the present application also provides a computer-readable storage medium. In an embodiment of the computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the image processing method in the above method embodiment. This program can be loaded and run by a processor to implement the above image processing method. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiments of the present application is a non-transitory computer-readable storage medium.

[0205] Furthermore, the present application also provides a control device. In an embodiment of the control device according to the present application, the control device includes a processor and a storage device. The storage device may be configured to store a program for executing the image recognition method in the above method embodiment. The processor may be configured to execute the program in the storage device, and this program includes but is not limited to the program for executing the image recognition method in the above method embodiment. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The control device may be a control device formed by various electronic devices.

[0206] In the embodiments of the present application, the control device may be a control device formed by various electronic devices. In some possible embodiments, the control device may include multiple storage devices and multiple processors. The program for executing the image recognition method in the above method embodiment may be divided into multiple sub-programs, and each sub-program can be loaded and run by a processor to execute different steps of the image recognition method in the above method embodiment. Specifically, each sub-program may be stored in a different storage device respectively, and each processor may be configured to execute the program in one or more storage devices to jointly implement the image recognition method in the above method embodiment, that is, each processor respectively executes different steps of the image recognition method in the above method embodiment to jointly implement the image recognition method in the above method embodiment.

[0207] The above-mentioned multiple processors may be processors deployed on the same device. For example, the above-mentioned control device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors may be processors configured on the high-performance device. In addition, the above-mentioned multiple processors may also be processors deployed on different devices. For example, the above-mentioned control device may be a server cluster, and the above-mentioned multiple processors may be processors on different servers in the server cluster.

[0208] Furthermore, the present application also provides a computer-readable storage medium. In an embodiment of the computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the image recognition method in the above method embodiment. This program can be loaded and run by a processor to implement the above image recognition method. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiments of the present application is a non-transitory computer-readable storage medium.

[0209] Furthermore, the present application also provides a terminal device. In an embodiment of the terminal device according to the present application, an image sensor and an ambient light sensor are provided on the terminal device, and the terminal device includes the control device in the control device embodiment.

[0210] Furthermore, it should be understood that since the setting of each module is only to illustrate the functional units of the device of the present application, the corresponding physical devices of these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part combined by software and hardware. Therefore, the number of each module in the figure is only illustrative.

[0211] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principle of the present application. Therefore, the technical solutions after splitting or combining will all fall within the protection scope of the present application.

[0212] Regarding the relevant personal information of users that may be involved in the embodiments of the present application, all are strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, for reasonable purposes based on business scenarios, and processing the personal information actively provided by users during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the authorization of users.

[0213] The user's personal information processed in this application may vary depending on the specific product / service scenario. It shall be subject to the specific scenario of the user using the product / service and may involve the user's account information, device information, driving information, vehicle information or other relevant information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0214] This application attaches great importance to the security of the user's personal information and has taken security protection measures that meet industry standards and are reasonable and feasible to protect the user's information and prevent the personal information from being accessed, publicly disclosed, used, modified, damaged or lost without authorization.

[0215] So far, the technical solutions of this application have been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of this application is obviously not limited to these specific embodiments. Without departing from the principle of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method is applied to a terminal device, and an image sensor and an ambient light sensor are arranged on the terminal device; the method includes: Obtain the current image through the image sensor; Obtain the ambient light brightness corresponding to the image through the ambient light sensor; Obtain the ambient scene corresponding to the image according to the ambient light brightness; Based on the ambient scene, perform image correction on the image to obtain a corrected image.

2. The image processing method according to claim 1, characterized in that, The performing image correction on the image based on the ambient scene to obtain a corrected image includes: Obtain the gamma coefficient corresponding to the ambient scene according to the ambient scene; Perform image correction on the image according to the gamma coefficient to obtain the corrected image.

3. The image processing method according to claim 2, characterized in that, The method further includes obtaining the gamma coefficient corresponding to the ambient scene according to the following steps: Collect an image set for obtaining the gamma coefficient, and obtain the ambient light brightness corresponding to each image in the image set; Obtain the ambient scene corresponding to the image according to the ambient light brightness; For the images of each ambient scene, obtain a plurality of adjusted gamma coefficients according to a preset initial value of the gamma coefficient and a preset step; Perform correction on the image according to each adjusted gamma coefficient to obtain a corrected image corresponding to each adjusted gamma coefficient; Perform image recognition on the corrected image to obtain an image recognition result; Obtain the gamma coefficient corresponding to the ambient scene according to the image recognition result.

4. The image processing method according to claim 3, characterized in that, The obtaining the gamma coefficient corresponding to the ambient scene according to the image recognition result includes: For each adjusted gamma coefficient, obtain the confidence mean of the image recognition results of the image under the same ambient scene; Select the adjusted gamma coefficient with the largest confidence mean as the gamma coefficient corresponding to the ambient scene.

5. The image processing method according to any one of claims 1 to 4, characterized in that, The ambient light brightness includes front ambient light brightness and rear ambient light brightness; The obtaining the ambient scene corresponding to the image according to the ambient light brightness includes: Obtain the ambient scene according to the front ambient light brightness and the rear ambient light brightness.

6. The image processing method according to claim 5, characterized in that, The image sensor is a front image sensor, and the ambient scenes include a dark scene, a face light scene, a backlight scene, and a normal light scene; The obtaining the ambient scene according to the front ambient light brightness and the rear ambient light brightness includes: When both the front ambient light brightness and the rear ambient light brightness are less than a first preset brightness, determine that the ambient scene is the dark scene; When both the front ambient light brightness and the rear ambient light brightness are greater than a second preset brightness, and the difference obtained by subtracting the rear ambient light brightness from the front ambient light brightness is greater than a third preset brightness, determine that the ambient scene is the face light scene; When both the front ambient light brightness and the rear ambient light brightness are greater than a second preset brightness, and the difference obtained by subtracting the front ambient light brightness from the rear ambient light brightness is greater than a third preset brightness, determine that the ambient scene is the backlight scene; When both the front ambient light brightness and the rear ambient light brightness do not meet the above conditions, determine that the environmental scene is the normal light scene; Wherein, the first preset brightness is less than the third preset brightness which is less than the second preset brightness, and the first preset brightness is greater than zero.

7. The image processing method according to claim 1, characterized in that, The ambient light sensor includes a front ambient light sensor and a rear ambient light sensor; the ambient light brightness includes the front ambient light brightness and the rear ambient light brightness; The obtaining of the ambient light brightness corresponding to the image through the ambient light sensor includes: Obtaining the front ambient light brightness through the front ambient light sensor; Obtaining the rear ambient light brightness through the rear ambient light sensor.

8. The image processing method according to any one of claims 1 to 7, characterized in that, The obtaining of the current image through the image sensor includes: Collecting a low-resolution image through the image sensor as the current image; Wherein, the low-resolution image is an image with a resolution less than the VGA standard resolution.

9. The image processing method according to claim 8, characterized in that, The current image is an image with a QVGA standard resolution or a QQVGA standard resolution.

10. An image recognition method, characterized in that, The method includes: Obtaining an image to be recognized, wherein the image is a corrected image obtained by the image processing method according to any one of claims 1 to 9; Performing image recognition on the image to be recognized to obtain an image recognition result.

Images