ISP adaptive adjustment control method and device based on face image

Abstract

The present invention discloses an ISP adaptive adjustment control method and device based on facial images. The method includes obtaining a facial image captured by an image acquisition device, and inputting the facial image into a facial image quality assessment model for analysis; obtaining the analysis result of the facial image quality assessment model as an image analysis parameter of the facial image, and judging whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameter; if so, adjusting the ISP according to the image analysis parameter until the parameter of the ISP meets the preset conditions, and re-triggering the execution of the above-mentioned operation of obtaining the facial image captured by the image acquisition device. It can be seen that the present invention can realize the dynamic adjustment of the ISP, so that the ISP is always in a state that is consistent with the facial application, and realizes the real-time and accurate acquisition of facial images, which helps to ensure that the facial application obtains relatively stable and high-quality facial input data, thereby improving the effectiveness and security of facial verification (for example: access control face recognition, etc.).

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an ISP adaptive adjustment control method and device based on face images. Background Art

[0002] With the rapid development of science and technology, facial recognition technology has become increasingly mature. Due to its advantages of fast response time and contactless identity authentication, facial recognition technology is increasingly being used in a wide range of applications, such as face unlocking, corporate attendance, community access control, and mobile payment.

[0003] In practical applications, the quality of the captured images directly affects the accuracy of face recognition, and the quality of the captured images depends on the adjustment of the camera's ISP (Image Signal Processor). Currently, ISP adjustment is mainly performed through global information obtained by the image sensor to adapt to environmental changes. However, in practice, it has been found that this ISP adjustment method completely isolates the ISP from the application that calls the camera. The application that calls the camera can only passively use the raw data provided by the camera and cannot guide the ISP to make targeted adjustments. In other words, the current ISP adjustment cannot be dynamically adjusted according to actual conditions, making it impossible to obtain real-time, accurate images. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an ISP adaptive adjustment control method and device based on facial images, which can dynamically adjust the ISP according to actual conditions to achieve real-time and accurate facial image acquisition.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses an ISP adaptive adjustment control method based on a face image, the method comprising:

[0006] Acquire a facial image captured by an image acquisition device, and input the facial image into a predetermined facial image quality assessment model for analysis;

[0007] Obtaining analysis results of the facial image quality assessment model as image analysis parameters of the facial image, and determining whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters;

[0008] When it is determined that the ISP needs to be adjusted, the ISP is adjusted according to the image analysis parameters until the parameters of the ISP meet the preset conditions, and the operation of obtaining the facial image captured by the image acquisition device is retriggered.

[0009] As an optional implementation, in the first aspect of the present invention, the image analysis parameter includes at least one of an exposure parameter of the facial image, a white balance parameter of the facial image, and a focus parameter of the facial image;

[0010] The determining, based on the image analysis parameters, whether the ISP corresponding to the image acquisition device needs to be adjusted includes:

[0011] When the image analysis parameter is an exposure parameter of the facial image, intercepting the facial image based on the calibration results corresponding to the determined facial key points to obtain a target facial region, wherein an area of ​​the target facial region is smaller than an area of ​​the facial image;

[0012] Obtaining a first brightness value corresponding to the target face area, and determining whether the first brightness value is greater than or equal to a preset brightness value threshold, and when it is determined that the first brightness value is greater than or equal to the preset brightness value threshold, determining that the ISP corresponding to the image acquisition device needs to be adjusted; or,

[0013] When the image analysis parameter is a white balance parameter of the facial image, determining the mean value of each of the three RGB channels of the target facial area, and judging whether the proportion distribution of all the channels matches a predetermined proportion distribution based on all the mean values, and when the judgment result is no, determining that the ISP corresponding to the image acquisition device needs to be adjusted; or

[0014] When the image analysis parameter is the focusing parameter of the facial image, the target facial area appears at the target position of the image acquisition device, and it is determined whether the target position is within a predetermined position range. When the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0015] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0016] When it is determined that the first brightness value is not greater than or equal to the preset brightness value threshold, expanding the effective area of ​​the target face region according to a preset expansion method to obtain an expanded target face region, wherein the preset expansion method is a method set based on the center position of the target face region;

[0017] Determining a second brightness value of a background area corresponding to the enlarged target facial area, and calculating backlighting of the facial image based on the first brightness value and the second brightness value;

[0018] It is determined whether the backlight level is less than or equal to a preset backlight level threshold, and when it is determined that the backlight level is less than or equal to the preset backlight level threshold, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0019] As an optional implementation manner, in the first aspect of the present invention, before acquiring the facial image captured by the image acquisition device, the method further includes:

[0020] determining whether the ISP needs to be adjusted according to a current state of the ISP, and triggering execution of the operation of acquiring the facial image captured by the image acquisition device when it is determined that the ISP needs to be adjusted;

[0021] The determining whether the ISP needs to be adjusted according to the current status of the ISP includes:

[0022] Read the target time currently displayed by the image processing system corresponding to the ISP, and obtain the completion time corresponding to the last adjustment of the ISP;

[0023] Calculate the interval between the target time and the completion time, and determine whether the interval is less than or equal to a predetermined interval threshold. When it is determined that the interval is less than or equal to the interval threshold, determine that the ISP needs to be adjusted.

[0024] As an optional implementation manner, in the first aspect of the present invention, after determining that the interval duration is less than or equal to the interval duration threshold and before determining that the ISP needs to be adjusted, the method further includes:

[0025] Obtain a total number of adjustments to the ISP within a first preset time period, and determine whether the total number of adjustments is less than a preset adjustment number threshold; when it is determined that the total number of adjustments is less than the preset adjustment number threshold, trigger the execution of the operation of determining that the ISP needs to be adjusted, wherein the end time of the first preset time period is the time when the ISP was last adjusted.

[0026] As an optional embodiment, in the first aspect of the present invention, after acquiring the facial image captured by the image acquisition device and before inputting the facial image into a predetermined facial image quality assessment model for analysis, the method further includes:

[0027] Determining image parameters corresponding to the facial image, the image parameters including at least one of an area of ​​the facial image, a position where the facial image appears on the image acquisition device, and facial information of the facial image, the facial information including a position of a left virtual pupil, a position of a right virtual pupil, a position of a left corner of a virtual mouth, a position of a right corner of a virtual mouth, and a position of a virtual nose tip;

[0028] Determine whether the facial image meets the predetermined adjustment conditions of the ISP based on the image parameters. When it is determined that the facial image meets the adjustment conditions of the ISP, trigger the operation of inputting the facial image into the predetermined facial image quality assessment model for analysis.

[0029] As an optional embodiment, in the first aspect of the present invention, after adjusting the ISP according to the image analysis parameters until the current parameters of the ISP meet preset conditions, and before re-triggering the operation of acquiring the facial image captured by the image acquisition device, the method further includes:

[0030] Determine whether the real-time display moment of the image processing system corresponding to the ISP reaches the end time of a second preset time length; when it is determined that the real-time display moment reaches the end time of the second preset time length, re-trigger the operation of acquiring the facial image captured by the image acquisition device, and the starting time of the second preset time length is the time when it is determined that the ISP needs to be adjusted.

[0031] A second aspect of the present invention discloses an ISP adaptive adjustment control device based on a face image, the device comprising an acquisition module, an analysis module, a first judgment module, and an adjustment module, wherein:

[0032] The acquisition module is used to acquire the face image acquired by the image acquisition device;

[0033] The analysis module is used to input the facial image into a predetermined facial image quality assessment model for analysis;

[0034] The acquisition module is further configured to acquire the analysis result of the facial image quality assessment model as an image analysis parameter of the facial image;

[0035] The first judgment module is used to judge whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters;

[0036] The adjustment module is used to adjust the ISP according to the image analysis parameters when the first judgment module determines that the ISP needs to be adjusted, until the parameters of the ISP meet the preset conditions, and trigger the acquisition module to re-execute the operation of acquiring the facial image captured by the image acquisition device.

[0037] As an optional implementation, in the second aspect of the present invention, the image analysis parameter includes at least one of an exposure parameter of the facial image, a white balance parameter of the facial image, and a focus parameter of the facial image;

[0038] The first judgment module includes a first judgment submodule, a second judgment submodule, or a third judgment submodule, wherein the first judgment submodule includes an interception unit, an acquisition unit, a judgment unit, and a determination unit, wherein:

[0039] The interception unit is configured to intercept the facial image based on the calibration results corresponding to the determined facial key points to obtain a target facial region when the image analysis parameter is an exposure parameter of the facial image, wherein the area of ​​the target facial region is smaller than the area of ​​the facial image;

[0040] The acquiring unit is configured to acquire a first brightness value corresponding to the target face area;

[0041] The judging unit is configured to judge whether the first brightness value is greater than or equal to a preset brightness value threshold;

[0042] The determining unit is configured to determine that the ISP corresponding to the image acquisition device needs to be adjusted when the judging unit determines that the first brightness value is greater than or equal to the preset brightness value threshold; or

[0043] The second judgment submodule is configured to, when the image analysis parameter is a white balance parameter of the facial image, determine the mean value of each of the three RGB channels of the target facial area, and determine whether the proportion distribution of all the channels matches a predetermined proportion distribution based on all the mean values; if the determination result is no, determine that the ISP corresponding to the image acquisition device needs to be adjusted; or

[0044] The third judgment submodule is used to obtain the target position of the target face area appearing in the image acquisition device when the image analysis parameter is the focus parameter of the face image, and to determine whether the target position is within a predetermined position range. When the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0045] As an optional embodiment, in the second aspect of the present invention, the acquiring unit is further configured to, when the judging unit determines that the first brightness value is not greater than or equal to the preset brightness value threshold, expand the effective area of ​​the target facial region according to a preset expansion method to obtain an expanded target facial region, wherein the preset expansion method is a method set based on the center position of the target facial region;

[0046] The determining unit is further configured to determine a second brightness value of a background area corresponding to the enlarged target face area;

[0047] The acquiring unit is further configured to calculate the backlight intensity of the facial image based on the first brightness value and the second brightness value;

[0048] The judging unit is configured to judge whether the backlight level is less than or equal to a preset backlight level threshold, and when it is judged that the backlight level is less than or equal to the preset backlight level threshold, trigger the determining unit to execute the operation of determining whether the ISP corresponding to the image acquisition device needs to be adjusted.

[0049] As an optional implementation, in the second aspect of the present invention, the apparatus further includes a second judgment module, wherein:

[0050] The second judgment module is configured to judge whether the ISP needs to be adjusted based on the current state of the ISP before the acquisition module acquires the facial image acquired by the image acquisition device; when it is determined that the ISP needs to be adjusted, trigger the acquisition module to execute the operation of acquiring the facial image acquired by the image acquisition device;

[0051] The second judgment module includes a reading submodule, an acquisition submodule, a calculation submodule and a fourth judgment submodule, wherein:

[0052] The reading submodule is used to read the target time currently displayed by the image processing system corresponding to the ISP;

[0053] The acquisition submodule is used to obtain the completion time corresponding to the last adjustment of the ISP;

[0054] The calculation submodule is used to calculate the interval between the target time and the completion time;

[0055] The fourth judgment submodule is used to judge whether the interval duration is less than or equal to a predetermined interval duration threshold;

[0056] The acquisition submodule is further configured to determine that the ISP needs to be adjusted when the fourth determination submodule determines that the interval duration is less than or equal to the interval duration threshold.

[0057] As an optional implementation manner, in the second aspect of the present invention, the obtaining submodule is further configured to, after the fourth determining submodule determines that the interval duration is less than or equal to the interval duration threshold and before determining that the ISP needs to be adjusted, obtain a total number of times the ISP is adjusted within a first preset duration, wherein the end time of the first preset duration is the time of the last adjustment of the ISP;

[0058] The fourth judgment submodule is further configured to judge whether the total number of adjustments is less than a preset adjustment number threshold, and when it is judged that the total number of adjustments is less than the preset adjustment number threshold, trigger the acquisition submodule to execute the operation of determining whether the ISP needs adjustment.

[0059] As an optional implementation, in the second aspect of the present invention, the apparatus further includes a determination module and a third judgment module, wherein:

[0060] The determination module is configured to determine image parameters corresponding to the facial image after the acquisition module acquires the facial image acquired by the image acquisition device and before the analysis module inputs the facial image into a predetermined facial image quality assessment model for analysis, the image parameters including at least one of an area of ​​the facial image, a position of the facial image at which the facial image appears on the image acquisition device, and facial information of the facial image, the facial information including a position of a left virtual pupil, a position of a right virtual pupil, a position of a left corner of a virtual mouth, a position of a right corner of a virtual mouth, and a position of a virtual nose tip of the facial image;

[0061] The third judgment module is used to judge whether the facial image meets the predetermined adjustment conditions of the ISP based on the image parameters. When it is judged that the facial image meets the adjustment conditions of the ISP, the analysis module is triggered to execute the operation of inputting the facial image into the predetermined facial image quality assessment model for analysis.

[0062] As an optional implementation manner, in the second aspect of the present invention, the apparatus further includes a fourth judgment module, wherein:

[0063] The fourth judgment module is used to adjust the ISP according to the image analysis parameters in the adjustment module until the current parameters of the ISP meet the preset conditions, and before triggering the acquisition module to re-trigger the execution of the operation of acquiring the facial image captured by the image acquisition device, to determine whether the real-time display moment of the image processing system corresponding to the ISP has reached the end time of a second preset duration; when it is determined that the real-time display moment has reached the end time of the second preset duration, trigger the acquisition module to re-trigger the execution of the operation of acquiring the facial image captured by the image acquisition device, and the starting time of the second preset duration is the time when it is determined that the ISP needs to be adjusted.

[0064] A third aspect of the present invention discloses another ISP adaptive adjustment control device based on a face image, the device comprising:

[0065] a memory storing executable program code;

[0066] a processor coupled to the memory;

[0067] The processor calls the executable program code stored in the memory to execute the ISP adaptive adjustment control method based on facial images disclosed in the first aspect of the present invention.

[0068] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the ISP adaptive adjustment control method based on facial images disclosed in the first aspect of the present invention.

[0069] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0070] In an embodiment of the present invention, a method and device for adaptive adjustment control of an ISP based on a facial image are disclosed. The method includes acquiring a facial image captured by an image acquisition device, and inputting the facial image into a predetermined facial image quality assessment model for analysis; acquiring an analysis result of the facial image quality assessment model as an image analysis parameter of the facial image, and judging whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameter; when it is judged that the ISP needs to be adjusted, adjusting the ISP based on the image analysis parameter until the parameter of the ISP meets a preset condition, and re-triggering the execution of the above-mentioned operation of acquiring the facial image captured by the image acquisition device. It can be seen that the embodiment of the present invention combines the ISP of the image acquisition device with the face-related application that carries the face image quality assessment model, inputs the collected face image into the face image quality assessment model for analysis, and can evaluate the parameter settings of the ISP based on the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the face image will be re-acquired to perform the same ISP adjustment operation. The dynamic adjustment of the ISP can be achieved, so that the ISP is always in a state that is consistent with the face application, and real-time and accurate face image acquisition is achieved, which helps to ensure that the face application obtains relatively stable high-quality face input data, thereby improving the effectiveness and security of face verification (for example: access control face recognition, etc.); and adaptively adjusts the ISP according to the face imaging results, fully utilizing the self-adjustment function of the existing image acquisition device ISP without increasing the cost, and significantly improving the ISP imaging effect of the existing image device. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 This is a flow chart of a method for adaptively adjusting an ISP based on a face image disclosed in an embodiment of the present invention;

[0073] Figure 2 This is a flow chart of another ISP adaptive adjustment control method based on a face image disclosed in an embodiment of the present invention;

[0074] Figure 3 1 is a structural diagram of an ISP adaptive adjustment control device based on a face image disclosed in an embodiment of the present invention;

[0075] Figure 42 is a schematic structural diagram of another ISP adaptive adjustment control device based on a face image disclosed in an embodiment of the present invention;

[0076] Figure 5 This is a structural diagram of another ISP adaptive adjustment control device based on facial images disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0078] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0079] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0080] The present invention discloses a method and device for adaptively adjusting and controlling an image service provider (ISP) based on facial images. The method combines the ISP of an image acquisition device with a facial application that carries a facial image quality assessment model, inputs the acquired facial image into the facial image quality assessment model for analysis, and evaluates the ISP parameter settings based on image analysis parameters, i.e., the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP is adjusted based on the image analysis parameters, and the same ISP adjustment operation is performed upon reacquiring a facial image. This method enables dynamic adjustment of the ISP, ensuring that the ISP is always in a state consistent with the facial application, enabling real-time and accurate facial image acquisition, and helping to ensure that the facial application obtains relatively stable, high-quality facial input data, thereby improving the effectiveness and security of facial verification (e.g., facial recognition for access control). Furthermore, the method adaptively adjusts the ISP based on the facial imaging results, fully utilizing the self-adjustment function of the existing ISP of the image acquisition device without increasing costs, significantly improving the ISP imaging effect of the existing imaging device. These are described in detail below.

[0081] Example 1

[0082] See also Figure 1 , Figure 1 This is a flow chart of a method for adaptively adjusting an ISP based on a face image disclosed in an embodiment of the present invention. Figure 1 The described ISP adaptive adjustment control method based on face images is applicable to an image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built into a terminal device that can capture faces, wherein the terminal device includes a user terminal and / or an access control device. Figure 1 As shown, the ISP adaptive adjustment control method based on face image may include the following operations:

[0083] 101. Obtain a facial image captured by an image acquisition device.

[0084] In the embodiment of the present invention, the image acquisition device is provided with an ISP, and the facial image can be a real-time image acquired by the image acquisition device, or an image acquired at intervals of a preset time (eg, 30 seconds).

[0085] In the embodiment of the present invention, after acquiring a facial image, a preprocessing operation is further performed on the facial image. In this case, the facial image in step 102 is a preprocessed facial image, wherein the preprocessing operation includes filtering operations, image segmentation operations, etc. Thus, performing the preprocessing operation on the facial image after acquiring the facial image can obtain a high-quality facial image, thereby improving the analysis results of the facial image and further improving the accuracy of the judgment of adjusting the ISP.

[0086] In an optional embodiment, before executing step 101, the ISP adaptive adjustment control method based on facial images may further include the following operations:

[0087] Initialize the image processing system, specifically, initialize the ISP adjustment information structure of the image processing system, wherein the ISP adjustment information structure is used to store variables of the ISP during the adjustment process, such as: facial information of the most recent frame detected (i.e., the specific position of the face appearing on the sensor included in the image acquisition device), the specific time of the last ISP adjustment, the number of ISP adjustments of the image processing system during the current operation, etc.;

[0088] After the image processing system is initialized, step 101 may be triggered to be executed.

[0089] It can be seen that this optional embodiment initializes the image processing system before acquiring the facial image, which can improve the information processing capability of the image processing system.

[0090] 102. Input the facial image into a predetermined facial image quality assessment model for analysis, and obtain an analysis result of the facial image quality assessment model as an image analysis parameter of the facial image.

[0091] In an embodiment of the present invention, the image analysis parameters (i.e., a representation of the imaging quality of the image acquisition device's sensor) include at least one of a facial image exposure parameter, a facial image white balance parameter, and a facial image focus parameter, which is not limited in this embodiment of the present invention. Furthermore, the image analysis parameters may also include, but are not limited to, a color correction matrix, sharpness, curve properties, and saturation. The more image analysis parameters included, the more accurate the determination of whether ISP adjustment is necessary.

[0092] 103. Determine whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters. If it is determined that the ISP needs to be adjusted, step 103 is triggered. If it is determined that the ISP does not need to be adjusted, the process ends.

[0093] 104 . Adjust the ISP according to the image analysis parameters until the ISP parameters meet the preset conditions, and re-trigger step 101 .

[0094] In an embodiment of the present invention, as an optional implementation manner, determining whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters includes:

[0095] When the image analysis parameters are exposure parameters of the facial image, intercepting the facial image based on the calibration results corresponding to the determined facial landmarks (e.g., 106 facial landmarks) to obtain a target facial region, wherein the area of ​​the target facial region is smaller than the area of ​​the facial image;

[0096] Obtain a first brightness value corresponding to the target face area, and determine whether the first brightness value is greater than or equal to a preset brightness value threshold. When it is determined that the first brightness value is greater than or equal to the preset brightness value threshold, determine that the ISP corresponding to the image acquisition device needs to be adjusted.

[0097] In this optional implementation, optionally, obtaining the first brightness value corresponding to the target face area may include:

[0098] The target face area is converted from RGB to YCbCr, and the mean of the Y component is calculated as the first brightness value corresponding to the target face area.

[0099] In this optional implementation, further, when it is determined that the first brightness value is not greater than or equal to the preset brightness value threshold, it is determined that the ISP corresponding to the image acquisition device does not need to be adjusted, that is, the exposure of the image acquisition device is normal, and the current process ends.

[0100] It can be seen that this optional implementation method can determine whether the ISP needs to be adjusted when the image acquisition device is overexposed by obtaining the brightness value of the facial image and comparing the brightness value with the preset brightness value; and before obtaining the brightness of the facial image, further intercepting the facial image through the calibration results corresponding to the facial key points can improve the efficiency and accuracy of determining the brightness value of the facial image, thereby helping to improve the accuracy and reliability of judging whether the ISP needs to be adjusted.

[0101] In an embodiment of the present invention, as another optional implementation, determining whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters includes:

[0102] When the above-mentioned image analysis parameters are the white balance parameters of the above-mentioned facial image, the mean of each of the three RGB channels of the target facial area is determined, and based on all the mean values, it is judged whether the proportion distribution of all channels matches the predetermined proportion distribution. When the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0103] It can be seen that this optional implementation method can determine whether the ISP needs to be adjusted by judging whether the distribution of the three RGB channels of the face image is reasonable.

[0104] In an embodiment of the present invention, as another optional implementation, determining whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters includes:

[0105] When the above-mentioned image analysis parameters are the focusing parameters of the above-mentioned facial image, the target facial area appears at the target position of the image acquisition device, and it is determined whether the target position is within a predetermined position range. When the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0106] It can be seen that this optional implementation method can determine whether the ISP needs to be adjusted by comparing the position where the acquired facial image appears on the image acquisition device with the predetermined position range.

[0107] In an embodiment of the present invention, when it is determined that the first brightness value is greater than or equal to the preset brightness value threshold and / or when it is determined that the proportional distribution of all channels does not match the predetermined proportional distribution and / or when it is determined that the target position is not within the predetermined position range, it is determined that the ISP needs to be adjusted. This can not only improve the accuracy of the judgment that the ISP needs to be adjusted, but also provide multiple judgment methods for whether the ISP needs to be adjusted, thereby improving the flexibility of the judgment that the ISP needs to be adjusted.

[0108] In the embodiment of the present invention, adjusting the ISP according to the image analysis parameters until the parameters of the ISP meet the preset conditions may include:

[0109] When it is determined that the first brightness value is greater than or equal to a preset brightness value threshold, obtaining an exposure parameter currently set by the ISP, and adjusting the ISP according to the exposure parameter currently set by the ISP and the exposure parameter of the face image until the exposure parameter of the ISP is less than or equal to the preset exposure parameter; or,

[0110] When it is determined that the ratio distribution of all channels does not match the predetermined ratio distribution, the mean value of the G channel of the target face image is fixed, and the gain of the R channel and the gain of the B channel are adjusted based on the R / G and G / B values ​​respectively, so that the ratio distribution of all channels matches the predetermined ratio distribution. Or,

[0111] When it is determined that the target position is not within the predetermined position range, the ISP is adjusted according to the target position until the target position is within the predetermined position range.

[0112] It can be seen that this optional implementation method adjusts the ISP according to different parameters so that various parameters of the ISP meet the image acquisition requirements, thereby improving the accuracy and efficiency of image acquisition.

[0113] In an optional embodiment, the ISP adaptive adjustment control method based on a face image may further include the following operations:

[0114] When it is determined that the first brightness value is not greater than or equal to the preset brightness value threshold, the effective area of ​​the target face region is enlarged according to a preset enlargement method to obtain an enlarged target face region, where the preset enlargement method is set based on the center position of the target face region;

[0115] Determining a second brightness value of a background area corresponding to the enlarged target facial area, and calculating backlighting of the facial image based on the first brightness value and the second brightness value;

[0116] It is determined whether the backlight level is less than or equal to a preset backlight level threshold. When it is determined that the backlight level is less than or equal to the preset backlight level threshold, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0117] In this optional embodiment, the preset enlargement method is a method set based on the center position of the target facial area. Specifically, the preset enlargement method is to use the center position of the target facial area as the center, the short side of the target facial area of ​​a first multiple (for example, 1 times) as the first short side, and the long side of the target facial area of ​​a second multiple (for example, 1 times) as the first long side to form a rectangular image area as the enlarged target facial area. For example, when the target facial area has a short side of 10 cm and a long side of 15 cm, the enlarged target facial area is still a rectangular image area formed by taking the center point of the target facial area as the center point, the short side becomes 15 cm, and the long side becomes 20 cm.

[0118] In this optional embodiment, when the expanded target facial region (foreground image) is obtained, the target facial region is removed from the facial image by an image separation method to obtain a background region, wherein the image separation method may include one or more combinations of separation methods such as background difference method, frame difference method, optical flow field method, and RPCA method. The method for removing the foreground image by the image separation method is an existing method and will not be repeated here. The method for calculating the second brightness value of the background region is the same as the method for calculating the target facial region and will not be repeated here.

[0119] In this optional embodiment, the specific calculation formula for the backlight intensity of the face image is as follows:

[0120]

[0121] Where, R is the backlight intensity of the face image; Y f is the first brightness value corresponding to the target face area; Y b is the second brightness value of the background area.

[0122] In this optional embodiment, further, when it is determined that the backlight level is not less than or equal to the preset backlight level threshold, it is determined that the ISP corresponding to the image acquisition device does not need to be adjusted, that is, the exposure of the image acquisition device is normal, and the current process ends.

[0123] It can be seen that this optional embodiment calculates the brightness value of the background area corresponding to the expansion of the face area when it is judged that the brightness value of the face area is small, and obtains the backlight of the face image based on these two brightness values, and determines that the ISP needs to be adjusted when it is determined that the backlight of the face image is small, thereby improving the judgment that the ISP needs to be adjusted when the image acquisition device is underexposed, and can improve the accuracy of determining that the ISP needs to be adjusted, which is beneficial to the all-round dynamic adjustment of the ISP, and further beneficial to the accuracy of subsequent acquisition of dynamic images.

[0124] In another optional embodiment, the ISP adaptive adjustment control method based on facial images may further include the following operations:

[0125] When it is determined that the backlight level is not less than or equal to the preset backlight level threshold, determining whether the first brightness value is greater than or equal to a certain preset brightness value threshold, and the certain preset brightness value threshold is less than the preset brightness value threshold;

[0126] When it is determined that the first brightness value is not greater than or equal to a preset brightness value threshold, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

[0127] In this optional embodiment, when it is determined that the first brightness value is greater than or equal to a preset brightness value threshold, it is determined that the ISP corresponding to the image acquisition device does not need to be adjusted, that is, the exposure of the image acquisition device is normal.

[0128] It can be seen that this optional embodiment continues to determine whether the brightness value of the face area is large when it is determined that the backlight degree of the face image is large. When it is determined that it is small, it is determined that the ISP needs to be adjusted, thereby improving the accuracy of determining whether the image acquisition device is underexposed, that is, improving the accuracy of determining that the ISP needs to be adjusted, and further improving the accuracy of the full range of ISP dynamic adjustment.

[0129] In yet another optional embodiment, before executing step 101, the ISP adaptive adjustment control method based on a facial image may further include the following operations:

[0130] Determine whether the ISP needs to be adjusted based on its current status. If it is determined that the ISP needs to be adjusted, step 101 is triggered.

[0131] In this optional embodiment, optionally, determining whether the ISP needs to be adjusted according to the current status of the ISP includes:

[0132] Read the target time currently displayed by the image processing system corresponding to the ISP, and obtain the completion time corresponding to the last adjustment of the ISP;

[0133] The interval between the target time and the completion time is calculated, and it is determined whether the interval is less than or equal to a predetermined interval threshold. When it is determined that the interval is less than or equal to the interval threshold, it is determined that the ISP needs to be adjusted.

[0134] In this optional embodiment, when it is determined that the interval duration is not less than or equal to the interval duration threshold, it is determined that the ISP does not need to be adjusted, and the total number of ISP adjustments of the image processing system during this operation up to the target time is set to zero. This makes it easier to reduce the calculation of invalid adjustments when it is determined that the ISP needs to be adjusted during the current system operation, thereby reducing the erroneous judgment that the ISP needs to be adjusted.

[0135] In this optional embodiment, the completion time corresponding to the last adjustment of the ISP is obtained from the adjustment information structure of the ISP.

[0136] It should be noted that this optional embodiment may also occur simultaneously with step 101 , or occur after step 101 , but is preferably performed before step 101 .

[0137] It can be seen that before acquiring a facial image, this optional embodiment first determines whether the ISP needs to be adjusted based on the current state of the ISP. If adjustment is required, the operation of acquiring the facial image can be performed. This can reduce the occurrence of adjustment operations on the ISP when there is no need to adjust the ISP, that is, when the various indicators of the image acquisition device (for example, white balance, etc.) match the current environment. This is beneficial to reducing the occurrence of incorrect ISP adjustment causing the image acquisition device to capture facial images of poor quality, reducing system power consumption, and improving the service life of the system.

[0138] In another optional embodiment, after determining that the interval duration is less than or equal to the interval duration threshold and before determining that the ISP needs to be adjusted, the face image-based ISP adaptive adjustment control method may further include the following operations:

[0139] Obtain the total number of ISP adjustments within a first preset time period, and determine whether the total number of adjustments is less than a preset adjustment number threshold. When it is determined that the total number of adjustments is less than the preset adjustment number threshold, trigger the execution of the above-mentioned operation of determining that the ISP needs to be adjusted, wherein the end time of the first preset time period is the time of the last ISP adjustment.

[0140] In this optional embodiment, when it is determined that the total number of adjustments is not less than a preset adjustment number threshold, it is determined that the ISP does not need to be adjusted, and this process is basically terminated.

[0141] It can be seen that this optional embodiment determines that the ISP needs to be adjusted only when it is determined that the time since the last ISP adjustment is short, and further when it is determined that the total number of ISP adjustments made by the image processing system during this operation is small. This can improve the accuracy of the determination that the ISP needs to be adjusted; and by limiting the total number of ISP adjustments during this system operation, it can meet the performance requirements of the ISP and adapt to complex environments, thereby improving the user experience.

[0142] In another optional embodiment, after adjusting the ISP according to the above-mentioned image analysis parameters until the current parameters of the ISP meet the preset conditions, and before re-triggering step 101, the ISP adaptive adjustment control method based on the face image may further include the following operations:

[0143] Determine whether the real-time display moment of the image processing system corresponding to the ISP has reached the end time of the second preset time. When it is determined that the real-time display moment has reached the end time of the second preset time, re-trigger the execution of step 101, and the start time of the second preset time is the time when it is determined that the ISP needs to be adjusted.

[0144] In this optional embodiment, when it is determined that the real-time display time has not reached the end time of the second preset duration, the above-mentioned operation of determining whether the real-time display time of the image processing system corresponding to the ISP has reached the end time of the second preset duration is triggered. In this way, by monitoring the real-time display time of the image processing system, the accuracy of triggering the re-execution of the facial image acquisition operation can be improved.

[0145] It can be seen that after the ISP adjustment is completed, this optional embodiment first determines whether the current moment has reached the preset moment. If so, the operation of obtaining the facial image is re-executed, which can meet the requirement that it takes a certain amount of time for the ISP parameters to take effect from being written to the register, thereby reducing the situation where subsequent direct reading of the image may destroy the stability of the image processing system, thereby ensuring the stability of the image processing system operation.

[0146] It can be seen that implementation Figure 1The described ISP adaptive adjustment control method based on face images combines the ISP of the image acquisition device with a face-related application that carries a face image quality assessment model, inputs the acquired face image into the face image quality assessment model for analysis, and can evaluate the parameter settings of the ISP based on the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted based on the image analysis parameters, and the same ISP adjustment operation will be performed on the face image again. This can realize dynamic adjustment of the ISP, so that the ISP is always in a state that is consistent with the face application, and achieve real-time and accurate face image acquisition, which helps to ensure that the face application obtains relatively stable high-quality face input data, thereby improving the effectiveness and security of face verification (for example: access control face recognition, etc.); and adaptively adjusts the ISP based on the face imaging results, fully utilizing the self-adjustment function of the existing image acquisition device ISP without increasing the cost, and significantly improving the ISP imaging effect of the existing image device.

[0147] Example 2

[0148] See also Figure 2 , Figure 2 This is a flow chart of another ISP adaptive adjustment control method based on face images disclosed in an embodiment of the present invention. Figure 2 The described ISP adaptive adjustment control method based on face images is applicable to an image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built into a terminal device that can capture faces, wherein the terminal device includes a user terminal and / or an access control device. Figure 2 As shown, the ISP adaptive adjustment control method based on face image may include the following operations:

[0149] 201. Acquire a facial image captured by an image acquisition device.

[0150] 202. Determine image parameters corresponding to the facial image.

[0151] In an embodiment of the present invention, the image parameters include at least one of the area of ​​the facial image, the position where the facial image appears on the image acquisition device, and the facial information of the facial image. The facial information of the facial image includes the position of the left virtual pupil of the facial image, the position of the right virtual pupil, the position of the left corner of the virtual mouth, the position of the right corner of the virtual mouth, and the position of the virtual nose tip.

[0152] 203. Determine whether the facial image satisfies the predetermined ISP adjustment conditions based on the image parameters. When it is determined that the facial image satisfies the ISP adjustment conditions, step 204 is triggered. When it is determined that the facial image does not satisfy the ISP adjustment conditions, step 201 may be triggered or the process may be terminated.

[0153] It can be seen that after acquiring a facial image, the embodiment of the present invention further determines whether the facial image meets the adjustment conditions of the ISP, and only performs subsequent operations when it is determined that the adjustment conditions of the ISP are met. This can reduce the occurrence of ISP adjustment errors caused by using the facial image to adjust the ISP even though the facial image does not meet the adjustment conditions of the ISP, which is beneficial to improving the accuracy of the dynamic adjustment of the ISP and reducing the power consumption of the image processing system. When it is determined that the adjustment conditions of the ISP are not met, the operation of acquiring the facial image is re-triggered, which is beneficial to obtaining an image that meets the adjustment conditions of the ISP, and further beneficial to the dynamic adjustment of the ISP.

[0154] 204. Input the facial image into a predetermined facial image quality assessment model for analysis, and obtain analysis results of the facial image quality assessment model as image analysis parameters of the facial image;

[0155] 205. Determine whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters. If it is determined that the ISP needs to be adjusted, step 206 is triggered. If it is determined that the ISP does not need to be adjusted, the process ends.

[0156] 206 . Adjust the ISP according to the image analysis parameters until the ISP parameters meet the preset conditions, and re-trigger step 201 .

[0157] In an embodiment of the present invention, as an optional implementation manner, determining whether a facial image satisfies a predetermined ISP adjustment condition based on image parameters includes:

[0158] When the above-mentioned image parameter is the area of ​​the above-mentioned facial image, it is determined whether the area of ​​the facial image is greater than or equal to a predetermined area range. When the judgment result is yes, it is determined that the facial image meets the predetermined ISP adjustment conditions.

[0159] It can be seen that this optional implementation method can determine whether the facial image meets the adjustment conditions of the ISP by calculating the area of ​​the acquired facial image and comparing the area with the preset area range.

[0160] In an embodiment of the present invention, as another optional implementation, determining whether a facial image satisfies a predetermined ISP adjustment condition based on image parameters includes:

[0161] When the above-mentioned image parameter is the first position of the above-mentioned facial image, the second position of the facial image collected last time is obtained, and the position offset between the first position and the second position is calculated, and it is determined whether the position offset is less than or equal to the preset position offset threshold. When the judgment result is yes, it is determined that the facial image meets the predetermined ISP adjustment conditions.

[0162] In this optional implementation, optionally, calculating the position offset between the first position and the second position may include:

[0163] An intersection-over-union ratio of the first position and the second position is calculated as a position offset between the first position and the second position.

[0164] It can be seen that this optional implementation method can realize the judgment that the facial image meets the adjustment conditions of ISP by calculating the position offset between the position of the facial image collected this time and the position of the facial image collected last time on the image acquisition device; and by calculating the intersection and union ratio of the positions of the two facial images, it can improve the calculation accuracy and efficiency of the position offset between the positions of the two facial images.

[0165] In an embodiment of the present invention, as another optional implementation, determining whether a facial image satisfies a predetermined ISP adjustment condition based on image parameters includes:

[0166] When the image parameter is facial information of the face image, calculating the mean of the pupil horizontal coordinates between the position of the left virtual pupil and the position of the right virtual pupil, and calculating the mean of the mouth corner horizontal coordinates between the position of the left corner of the mouth and the position of the right corner of the mouth;

[0167] Calculating a first abscissa difference between the pupil abscissa mean and the abscissa of the virtual nose tip position, and calculating a second abscissa difference between the mouth corner abscissa mean and the abscissa of the virtual nose tip position;

[0168] Calculate the target horizontal coordinate difference between the first horizontal coordinate difference and the second horizontal coordinate difference, and determine whether the target horizontal coordinate difference is less than or equal to the preset horizontal coordinate difference range. When the judgment result is yes, determine that the above-mentioned facial image meets the predetermined adjustment conditions of the above-mentioned ISP.

[0169] In this optional embodiment, a target calibration result that matches the facial image is determined from a predetermined set of facial key point calibration results to obtain facial information of the facial image. The set of facial key point calibration results includes multiple calibration results, for example, a calibration result including 5 key points, a calibration result including 68 key points, and a calibration result including 106 key points. The present invention selects the calibration result including 5 key points as the target calibration result.

[0170] It can be seen that this optional implementation method can realize the symmetry judgment of the virtual facial features of the captured facial image by calculating the positions of the left and right pupils, left and right corners of the mouth and the tip of the nose in the facial image, thereby realizing the judgment that the facial image meets the adjustment conditions of the ISP and improving the accuracy of the judgment that the facial image meets the adjustment conditions of the ISP.

[0171] In the embodiment of the present invention, it should be noted that when the judgment results of two or three of the above three judgment methods for whether the facial image meets the adjustment conditions of the ISP are yes, it is determined that the facial image meets the adjustment conditions of the ISP, which can improve the accuracy of the judgment that the facial image meets the adjustment conditions of the ISP, and is conducive to improving the subsequent adjustment accuracy of the ISP; by providing multiple judgment methods for whether the facial image meets the adjustment conditions of the ISP, the flexibility and possibility of judging whether the facial image meets the adjustment conditions of the ISP can be improved.

[0172] In another optional embodiment, the ISP adaptive adjustment control method based on facial images may further include the following operations:

[0173] When it is determined that the facial image does not meet the ISP adjustment conditions, the system analyzes the factors that led to the facial image not meeting the ISP adjustment conditions based on the facial image parameters and sends these factors to the image acquisition device, triggering the image acquisition device to recapture the facial image based on these factors. These factors may include incomplete facial features in the facial image, low ambient brightness, etc.

[0174] It can be seen that this optional embodiment further generates an image acquisition prompt after determining that the facial image does not meet the adjustment conditions of the ISP, and will help improve the accuracy of acquiring facial images that meet the adjustment conditions of the ISP, thereby improving the accuracy of the dynamic adjustment of the ISP.

[0175] In the embodiment of the present invention, for the description of step 201 and step 204 to step 206, please refer to the detailed description of step 101 to step 104 in embodiment 1, which will not be repeated in this embodiment of the present invention.

[0176] It can be seen that implementation Figure 2The described ISP adaptive adjustment control method based on face images combines the ISP of the image acquisition device with a face-related application that carries a face image quality assessment model, inputs the acquired face image into the face image quality assessment model for analysis, and can evaluate the parameter settings of the ISP based on the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted based on the image analysis parameters, and the same ISP adjustment operation will be performed on the face image again. This can realize dynamic adjustment of the ISP, so that the ISP is always in a state that is consistent with the face application, and achieve real-time and accurate face image acquisition, which helps to ensure that the face application obtains relatively stable high-quality face input data, thereby improving the effectiveness and security of face verification (for example: access control face recognition, etc.); and adaptively adjusts the ISP based on the face imaging results, fully utilizing the self-adjustment function of the existing image acquisition device ISP without increasing the cost, and significantly improving the ISP imaging effect of the existing image device. In addition, it can also reduce the occurrence of ISP adjustment errors caused by using the face image to adjust the ISP when it does not meet the ISP adjustment conditions, which is beneficial to improving the accuracy of the dynamic adjustment of the ISP and reducing the power consumption of the image processing system; it is also beneficial to obtain images that meet the ISP adjustment conditions, which is beneficial to the dynamic adjustment of the ISP; it can also provide a variety of judgment methods for whether the face image meets the ISP adjustment conditions, which improves the accuracy and flexibility of judging whether the face image meets the ISP adjustment conditions, and is beneficial to improving the subsequent ISP adjustment accuracy.

[0177] Example 3

[0178] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an ISP adaptive adjustment control device based on a face image disclosed in an embodiment of the present invention. Figure 3 The described ISP adaptive adjustment control device based on face image is applicable to an image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built into a terminal device that can capture faces, wherein the terminal device includes a user terminal and / or an access control device, etc. Figure 3 As shown, the ISP adaptive adjustment control device based on the face image may include an acquisition module 301, an analysis module 302, a first judgment module 303 and an adjustment module 304, wherein:

[0179] The acquisition module 301 is used to acquire a face image captured by an image acquisition device.

[0180] The analysis module 302 is configured to input the facial image into a predetermined facial image quality assessment model for analysis.

[0181] The acquisition module 301 is further configured to acquire the analysis result of the above-mentioned face image quality assessment model as the image analysis parameter of the above-mentioned face image;

[0182] The first judgment module 303 is configured to judge whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters.

[0183] The adjustment module 304 is used to adjust the ISP according to the above-mentioned image analysis parameters when the first judgment module 303 determines that the above-mentioned ISP needs to be adjusted, until the parameters of the ISP meet the preset conditions, and trigger the acquisition module 301 to re-execute the above-mentioned operation of acquiring the facial image captured by the image acquisition device.

[0184] It can be seen that implementation Figure 3 The described ISP adaptive adjustment control device based on face image combines the ISP of the image acquisition device with the face-related application that carries the face image quality assessment model, inputs the acquired face image into the face image quality assessment model for analysis, and can evaluate the parameter settings of the ISP based on the image analysis parameters, that is, the imaging quality of the ISP. When the ISP needs to be adjusted, the ISP will be adjusted according to the image analysis parameters, and the same ISP adjustment operation will be performed on the face image again. The dynamic adjustment of the ISP can be realized, so that the ISP is always in a state that is consistent with the face application, and real-time and accurate face image acquisition is achieved, which helps to ensure that the face application obtains relatively stable high-quality face input data, thereby improving the effectiveness and security of face verification (for example: access control face recognition, etc.); and adaptively adjusts the ISP according to the face imaging results, fully utilizing the self-adjustment function of the existing image acquisition device ISP without increasing the cost, and significantly improving the ISP imaging effect of the existing image device.

[0185] In an optional embodiment, the image analysis parameter includes at least one of an exposure parameter of the face image, a white balance parameter of the face image, and a focus parameter of the face image;

[0186] The first judgment module 303 may include a first judgment submodule, a second judgment submodule, or a third judgment submodule. The first judgment submodule includes an interception unit, an acquisition unit, a judgment unit, and a determination unit. It should be noted that these submodules and units are not shown in the corresponding figures.

[0187] A capture unit is used to capture the facial image based on the calibration results corresponding to the determined facial key points when the image analysis parameters are exposure parameters of the facial image, so as to obtain a target facial area, wherein the area of ​​the target facial area is smaller than the area of ​​the facial image.

[0188] The acquisition unit is used to obtain a first brightness value corresponding to the target face area.

[0189] The judging unit is configured to judge whether the first brightness value is greater than or equal to a preset brightness value threshold.

[0190] a determining unit, configured to determine that the ISP corresponding to the image acquisition device needs to be adjusted when the determining unit determines that the first brightness value is greater than or equal to the preset brightness value threshold; or

[0191] a second judgment submodule, configured to, when the image analysis parameter is a white balance parameter of the facial image, determine the mean of each of the three RGB channels of the target facial region, and determine, based on all the mean values, whether the proportion distribution of all the channels matches a predetermined proportion distribution; and if the determination result is no, determine that the ISP corresponding to the image acquisition device needs to be adjusted; or

[0192] The third judgment submodule is used to obtain the target position of the target face area appearing in the above-mentioned image acquisition device when the above-mentioned image analysis parameter is the focusing parameter of the above-mentioned face image, and to determine whether the target position is within a predetermined position range. When the judgment result is no, it is determined that the ISP corresponding to the above-mentioned image acquisition device needs to be adjusted.

[0193] It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image can also obtain the brightness value of the face image and compare the brightness value with the preset brightness value, and / or determine whether the distribution of the three RGB channels of the face image is reasonable, and / or compare the position where the acquired face image appears in the image acquisition device with the predetermined position range, so as to realize the judgment that the ISP needs to be adjusted when the image acquisition device is overexposed; and by providing multiple ways to determine whether the ISP needs to be adjusted, not only the accuracy and flexibility of the judgment that the ISP needs to be adjusted can be improved; and before obtaining the brightness of the face image, the face image is further intercepted through the calibration results corresponding to the face key points, which can improve the efficiency and accuracy of determining the brightness value of the face image, thereby helping to improve the accuracy and reliability of the judgment of whether the ISP needs to be adjusted.

[0194] In another optional embodiment, the acquisition unit is also used to expand the effective area of ​​the target face area according to a preset expansion method to obtain the expanded target face area when the judgment unit determines that the first brightness value is not greater than or equal to the preset brightness value threshold. The preset expansion method is a method set based on the center position of the target face area.

[0195] The determining unit is further configured to determine a second brightness value of the background area corresponding to the enlarged target face area.

[0196] The acquiring unit is further configured to calculate the backlight intensity of the facial image based on the first brightness value and the second brightness value.

[0197] The judging unit is used to judge whether the backlight level is less than or equal to a preset backlight level threshold. When it is judged that the backlight level is less than or equal to the preset backlight level threshold, the determining unit is triggered to execute the above-mentioned operation of determining whether the ISP corresponding to the image acquisition device needs to be adjusted.

[0198] It can be seen that the implementation of the ISP adaptive adjustment control device based on the face image can also, when it is determined that the brightness value of the face area is small, calculate the brightness value of the background area corresponding to the expansion of the face area, and obtain the backlight of the face image based on these two brightness values, and when it is determined that the backlight of the face image is small, determine that the ISP needs to be adjusted, thereby improving the judgment that the ISP needs to be adjusted when the image acquisition device is underexposed, and can improve the accuracy of determining that the ISP needs to be adjusted, which is beneficial to the all-round dynamic adjustment of the ISP, and further beneficial to the accuracy of subsequent acquisition of dynamic images.

[0199] In another optional embodiment, the above-mentioned ISP adaptive adjustment control device based on face image may further include a second judgment module 305. At this time, the ISP adaptive adjustment control device based on face image may be as follows: Figure 4 As shown, Figure 4 FIG. 1 is a structural diagram of another ISP adaptive adjustment control device based on face images, wherein:

[0200] The second judgment module 305 is used to determine whether the ISP needs to be adjusted based on the current status of the ISP before the acquisition module 301 acquires the facial image captured by the image acquisition device; when it is determined that the ISP needs to be adjusted, the acquisition module 301 is triggered to perform the above-mentioned operation of acquiring the facial image captured by the image acquisition device.

[0201] In this optional embodiment, optionally, as Figure 4 As shown, the second judgment module 305 may include a reading submodule 3051, an acquisition submodule 3052, a calculation submodule 3053 and a fourth judgment submodule 3054, wherein:

[0202] The reading submodule 3051 is used to read the target time currently displayed by the image processing system corresponding to the ISP.

[0203] The acquisition submodule 3052 is used to obtain the completion time corresponding to the last adjustment of the ISP.

[0204] The calculation submodule 3053 is used to calculate the interval between the target time and the completion time.

[0205] The fourth determination submodule 3054 is configured to determine whether the interval duration is less than or equal to a predetermined interval duration threshold.

[0206] The obtaining submodule 3052 is further configured to determine that the ISP needs to be adjusted when the fourth determining submodule 3054 determines that the interval duration is less than or equal to the interval duration threshold.

[0207] It can be seen that implementation Figure 4 The described ISP adaptive adjustment control device based on facial images can also determine whether the ISP needs to be adjusted according to the current state of the ISP before acquiring the facial image. If adjustment is required, the operation of acquiring the facial image can be performed. This can reduce the occurrence of ISP adjustment operations when there is no need to adjust the ISP, that is, when the various indicators of the image acquisition device (for example, white balance, etc.) match the current environment. This is conducive to reducing the occurrence of ISP error adjustment causing the image acquisition device to capture facial images of poor quality, reducing system power consumption, and improving the service life of the system.

[0208] In another optional embodiment, Figure 4 As shown, the acquisition submodule 3052 is further used to obtain the total number of adjustments of the above-mentioned ISP within a first preset time period after the fourth judgment submodule 3054 determines that the above-mentioned interval time is less than or equal to the above-mentioned interval time threshold, and before determining that the above-mentioned ISP needs to be adjusted, wherein the end time of the first preset time period is the time of the last ISP adjustment.

[0209] The fourth judgment submodule 3054 is further used to judge whether the total number of adjustments is less than a preset adjustment threshold. When it is judged that the total number of adjustments is less than the preset adjustment threshold, the acquisition submodule 3052 is triggered to perform the above-mentioned operation of determining whether the ISP needs adjustment.

[0210] It can be seen that implementation Figure 4 The described ISP adaptive adjustment control device based on facial images can also determine that the ISP needs to be adjusted only when it is determined that the time since the last ISP adjustment is short and further when it is determined that the total number of ISP adjustments during the current operation of the image processing system is small, thereby improving the accuracy of determining whether the ISP needs to be adjusted; and by limiting the total number of ISP adjustments during the current operation of the system, it can meet the performance requirements of the ISP and adapt to complex environments, thereby improving the user experience.

[0211] In another optional embodiment, Figure 4 As shown, the above-mentioned ISP adaptive adjustment control device based on facial images may further include a determination module 306 and a third judgment module 307, wherein:

[0212] The determination module 306 is used to determine the image parameters corresponding to the facial image after the acquisition module 301 acquires the facial image captured by the image acquisition device and before the analysis module 302 inputs the facial image into a predetermined facial image quality assessment model for analysis. The image parameters include the area of ​​the facial image, the position where the facial image appears on the above-mentioned image acquisition device, and at least one of the facial information of the facial image. The facial information of the facial image includes the position of the left virtual pupil of the facial image, the position of the right virtual pupil, the position of the left corner of the virtual mouth, the position of the right corner of the virtual mouth, and the position of the virtual nose tip.

[0213] The third judgment module 307 is used to judge whether the above-mentioned facial image meets the predetermined adjustment conditions of the above-mentioned ISP based on the above-mentioned image parameters. When it is judged that the facial image meets the adjustment conditions of the ISP, the analysis module 302 is triggered to execute the operation of analyzing the above-mentioned facial image input in the predetermined facial image quality assessment model.

[0214] It can be seen that implementation Figure 4 The described ISP adaptive adjustment control device based on facial images can also further determine whether the facial image meets the ISP adjustment conditions after acquiring the facial image, and only perform subsequent operations when it is determined that the ISP adjustment conditions are met. This can reduce the occurrence of ISP adjustment errors caused by using the facial image to adjust the ISP even though the facial image does not meet the ISP adjustment conditions, which is beneficial to improving the accuracy of the dynamic adjustment of the ISP and reducing the power consumption of the image processing system; when it is determined that the ISP adjustment conditions are not met, the operation of acquiring the facial image is re-triggered, which is beneficial to obtaining the image that meets the ISP adjustment conditions, and thus is beneficial to the dynamic adjustment of the ISP.

[0215] In another optional embodiment, Figure 4 As shown, the above-mentioned ISP adaptive adjustment control device based on facial image may further include a fourth judgment module 308, wherein:

[0216] The fourth judgment module 308 is used to judge whether the real-time display moment of the image processing system corresponding to the ISP reaches the end time of the second preset duration after the adjustment module 304 adjusts the above-mentioned ISP according to the above-mentioned image analysis parameters until the current parameters of the ISP meet the preset conditions, and before triggering the acquisition module 301 to re-trigger the execution of the above-mentioned operation of acquiring the facial image acquired by the image acquisition device. When it is judged that the real-time display moment reaches the end time of the second preset duration, the acquisition module 301 is triggered to re-trigger the execution of the above-mentioned operation of acquiring the facial image acquired by the image acquisition device. The starting time of the second preset time is the moment when it is determined that the ISP needs to be adjusted.

[0217] It can be seen that implementation Figure 4 The described ISP adaptive adjustment control device based on facial images can also determine whether the current moment has reached the preset moment after the ISP adjustment is completed. If so, the operation of obtaining the facial image is re-executed. This can meet the requirement that it takes a certain amount of time for the ISP parameters to take effect from being written to the register, thereby reducing the situation where subsequent direct reading of the image may destroy the stability of the image processing system, thereby ensuring the stability of the image processing system operation.

[0218] Example 4

[0219] See also Figure 5 , Figure 5 This is another ISP adaptive adjustment control device based on face images disclosed in an embodiment of the present invention. Figure 5 The described device can be applied to an image processing system, wherein the image processing system can be used to adjust the ISP, and the image processing system is built into a terminal device capable of capturing a face, wherein the terminal device includes a user terminal and / or an access control device. Figure 5 As shown, the ISP adaptive adjustment control device based on the face image may include:

[0220] A memory 501 storing executable program code;

[0221] a processor 502 coupled to the memory 501;

[0222] Furthermore, it may also include an input interface 503 and an output interface 504 coupled to the processor 502;

[0223] The processor 502 calls the executable program code stored in the memory 501 to execute the steps of the ISP adaptive adjustment control method based on the face image described in the first or second embodiment.

[0224] Example 5

[0225] An embodiment of the present invention discloses a computer-readable storage medium that stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the ISP adaptive adjustment control method based on a facial image described in Embodiment 1 or Embodiment 2.

[0226] Example 6

[0227] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps of the ISP adaptive adjustment control method based on facial images described in Example 1 or Example 2.

[0228] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0229] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0230] Finally, it should be noted that the ISP adaptive adjustment control method and device based on facial images disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for adaptively adjusting and controlling ISP based on face images, characterized in that: The method comprises: Acquire a facial image captured by an image acquisition device, and input the facial image into a predetermined facial image quality assessment model for analysis; Obtaining analysis results of the facial image quality assessment model as image analysis parameters of the facial image, and determining whether the ISP corresponding to the image acquisition device needs to be adjusted based on the image analysis parameters; wherein the image analysis parameters include at least one of an exposure parameter of the facial image, a white balance parameter of the facial image, and a focus parameter of the facial image; When it is determined that the ISP needs to be adjusted, adjusting the ISP according to the image analysis parameters until the parameters of the ISP meet the preset conditions, and re-triggering the operation of acquiring the facial image captured by the image acquisition device; After acquiring the facial image captured by the image acquisition device, the method further includes: Determining image parameters corresponding to the facial image, the image parameters including at least one of an area of ​​the facial image, a position where the facial image appears on the image acquisition device, and facial information of the facial image; determining, based on the image parameters, whether the facial image satisfies a predetermined adjustment condition of the ISP, and triggering, when it is determined that the facial image satisfies the adjustment condition of the ISP, executing the operation of inputting the facial image into a predetermined facial image quality assessment model for analysis; The image parameters include the position where the face image appears on the image acquisition device, and the position where the face image appears on the image acquisition device includes the first position of the face image; When the image parameter includes the first position of the facial image, determining whether the facial image satisfies a predetermined adjustment condition of the ISP according to the image parameter includes: Obtain the second position of the facial image captured last time, calculate the position offset between the first position and the second position, and determine whether the position offset is less than or equal to a preset position offset threshold. When the judgment result is yes, determine that the facial image meets the predetermined ISP adjustment conditions.

2. The ISP adaptive adjustment control method based on face image according to claim 1, characterized in that: The determining, based on the image analysis parameters, whether the ISP corresponding to the image acquisition device needs to be adjusted includes: When the image analysis parameter is an exposure parameter of the facial image, intercepting the facial image based on the calibration results corresponding to the determined facial key points to obtain a target facial region, wherein an area of ​​the target facial region is smaller than an area of ​​the facial image; Obtaining a first brightness value corresponding to the target face area, and determining whether the first brightness value is greater than or equal to a preset brightness value threshold, and when it is determined that the first brightness value is greater than or equal to the preset brightness value threshold, determining that the ISP corresponding to the image acquisition device needs to be adjusted; or, When the image analysis parameter is a white balance parameter of the facial image, determining the mean value of each of the three RGB channels of the target facial area, and judging whether the proportion distribution of all the channels matches a predetermined proportion distribution based on all the mean values, and when the judgment result is no, determining that the ISP corresponding to the image acquisition device needs to be adjusted; or When the image analysis parameter is the focusing parameter of the facial image, the target facial area appears at the target position of the image acquisition device, and it is determined whether the target position is within a predetermined position range. When the judgment result is no, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

3. The ISP adaptive adjustment control method based on face image according to claim 2, characterized in that: When the image analysis parameter is an exposure parameter of the facial image, the method further includes: When it is determined that the first brightness value is not greater than or equal to the preset brightness value threshold, expanding the effective area of ​​the target face region according to a preset expansion method to obtain an expanded target face region, wherein the preset expansion method is a method set based on the center position of the target face region; Determining a second brightness value of a background area corresponding to the enlarged target facial area, and calculating backlighting of the facial image based on the first brightness value and the second brightness value; It is determined whether the backlight level is less than or equal to a preset backlight level threshold, and when it is determined that the backlight level is less than or equal to the preset backlight level threshold, it is determined that the ISP corresponding to the image acquisition device needs to be adjusted.

4. The ISP adaptive adjustment control method based on facial images according to any one of claims 1 to 3, characterized in that: Before acquiring the facial image captured by the image acquisition device, the method further includes: determining whether the ISP needs to be adjusted according to a current state of the ISP, and triggering execution of the operation of acquiring the facial image captured by the image acquisition device when it is determined that the ISP needs to be adjusted; The determining whether the ISP needs to be adjusted according to the current status of the ISP includes: Read the target time currently displayed by the image processing system corresponding to the ISP, and obtain the completion time corresponding to the last adjustment of the ISP; Calculate the interval between the target time and the completion time, and determine whether the interval is less than or equal to a predetermined interval threshold. When it is determined that the interval is less than or equal to the interval threshold, determine that the ISP needs to be adjusted.

5. The ISP adaptive adjustment control method based on face image according to claim 4, characterized in that: After determining that the interval duration is less than or equal to the interval duration threshold and before determining that the ISP needs to be adjusted, the method further includes: Obtain a total number of adjustments to the ISP within a first preset time period, and determine whether the total number of adjustments is less than a preset adjustment number threshold; when it is determined that the total number of adjustments is less than the preset adjustment number threshold, trigger the execution of the operation of determining that the ISP needs to be adjusted, wherein the end time of the first preset time period is the time when the ISP was last adjusted.

6. The ISP adaptive adjustment control method based on facial images according to claim 1, 2, 3 or 5, characterized in that: The facial information of the face image includes the position of the left virtual pupil, the position of the right virtual pupil, the position of the left corner of the virtual mouth, the position of the right corner of the virtual mouth, and the position of the virtual nose tip of the face image.

7. The ISP adaptive adjustment control method based on face image according to claim 4, characterized in that: After adjusting the ISP according to the image analysis parameters until the current parameters of the ISP meet preset conditions, and before re-triggering the operation of acquiring the facial image captured by the image acquisition device, the method further includes: Determine whether the real-time display moment of the image processing system corresponding to the ISP reaches the end time of a second preset time length; when it is determined that the real-time display moment reaches the end time of the second preset time length, re-trigger the operation of acquiring the facial image captured by the image acquisition device, and the starting time of the second preset time length is the time when it is determined that the ISP needs to be adjusted.

8. An ISP adaptive adjustment control device based on face image, characterized in that: The device is used to execute the ISP adaptive adjustment control method based on a facial image according to any one of claims 1 to 7, and the device includes an acquisition module, an analysis module, a first judgment module, and an adjustment module, wherein: The acquisition module is used to acquire the face image acquired by the image acquisition device; The analysis module is used to input the facial image into a predetermined facial image quality assessment model for analysis; The acquisition module is further configured to acquire an analysis result of the facial image quality assessment model as an image analysis parameter of the facial image; wherein the image analysis parameter includes at least one of an exposure parameter of the facial image, a white balance parameter of the facial image, and a focus parameter of the facial image; The first judgment module is used to judge whether the ISP corresponding to the image acquisition device needs to be adjusted according to the image analysis parameters; The adjustment module is used to adjust the ISP according to the image analysis parameters when the first judgment module determines that the ISP needs to be adjusted, until the parameters of the ISP meet the preset conditions, and trigger the acquisition module to re-execute the operation of acquiring the facial image captured by the image acquisition device.

9. An ISP adaptive adjustment control device based on face image, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the ISP adaptive adjustment control method based on facial images as described in any one of claims 1-7.

10. A computer storage medium, characterized in that The computer storage medium stores computer instructions, which, when called, are used to execute the ISP adaptive adjustment control method based on facial images as described in any one of claims 1-7.

Images