Multi-region brightness coordinated adjustment method based on user facial position and touch point position

Abstract

The present invention relates to a multi-region brightness coordinated adjustment method based on a user facial position and a touch point position, and belongs to the technical field of display. The method comprises: capturing user facial image data by means of a camera, and obtaining user facial position information by means of a Dlib library face detection algorithm; on the basis of the user facial position information and the dimensions of a display screen, obtaining first adjustment region information by means of a first region delimitation method; capturing a display screen touch operation of a user by means of a touch sensor, obtaining initial touch point position information by means of a touch point detection algorithm, and on the basis of the initial touch point position information, obtaining touch point position information by means of a touch point position constraint condition; on the basis of the touch point position information and the first adjustment region information, obtaining second adjustment region information by means of a dynamic adjustment strategy; and on the basis of user preferences, adjusting the brightness of each region by means of a brightness adjustment algorithm, so as to obtain new brightness setting parameters, and dynamically adjusting the brightness of adjustment regions of the display screen on the basis of the new brightness setting parameters.

Description

Multi-region brightness cooperative adjustment method based on user face position and touch point position TECHNICAL FIELD

[0001] The present application belongs to the technical field of display, and particularly relates to a multi-region brightness cooperative adjustment method based on user face position and touch point position. BACKGROUND

[0002] Brightness adjustment is an intelligent display brightness optimization technology, which adjusts the brightness of a display device according to specific environmental conditions or user behavior, accurately determines the screen area where the user gazes by recognizing the user face position, and determines the user interaction area by monitoring the touch point position, so as to realize regional brightness adjustment. However, the display devices on the market mainly adjust the brightness of the full screen in terms of brightness adjustment, and cannot optimize the local brightness according to the specific use position and operation habits of the user, resulting in poor user experience. SUMMARY

[0003] To solve the above problems in the prior art, the present application provides a multi-region brightness cooperative adjustment method based on user face position and touch point position, which intelligently adjusts the local brightness of the display screen to improve the visual experience and operation comfort of the user.

[0004] The object of the present application can be achieved by the following technical solutions:

[0005] The multi-region brightness cooperative adjustment method based on user face position and touch point position comprises:

[0006] The camera captures the user face image data in real time, the user face position information is obtained from the user face image data by using the Dlib library face detection algorithm, and the user face position information includes a user face position coordinate set and a user face bounding box;

[0007] The first adjustment region information is obtained from the user face position information and the size of the display screen by using the first region limiting method, the first adjustment region information includes a first adjustment region bounding box and a first adjustment region coordinate set, the first adjustment region bounding box is a region obtained by introducing an offset and an expansion to the user face bounding box, and the expansion is the width and height added to the original region;

[0008] The touch sensor captures the user display screen touch operation to obtain touch sensor data, the initial touch point position information is obtained from the touch sensor data by using a touch point detection algorithm, and the touch point position information is obtained from the initial touch point position information by using a touch point position judgment condition, the touch point position information is the coordinates of the touch point on the display screen;

[0009] According to the touch point position information and the first adjustment region information, second adjustment region information is obtained through a dynamic adjustment strategy, the second adjustment region information including a second adjustment region boundary box and a second adjustment region coordinate set;

[0010] According to the first adjustment region information and the second adjustment region information, new brightness setting parameters are obtained through a brightness adjustment algorithm, and the brightness of the display screen adjustment region is adjusted according to the new brightness setting parameters, the new brightness setting parameters including first adjustment region new brightness setting parameters, second adjustment region new brightness setting parameters, and overlapping region new brightness setting parameters.

[0011] Preferably, the user face position information is obtained from the user face image data through a Dlib library face detection algorithm, including:

[0012] A face detector is initialized from the user face image data through a pre-trained classifier;

[0013] A suspected face region is obtained from the user face image data through a sliding window technique;

[0014] Face edge features and face shape features are extracted from the suspected face region through a HOG algorithm;

[0015] It is determined whether the suspected face region is a face region through the face detector based on the face edge features and the face shape features, and if not, the user face image data is recaptured through a camera and face detection is performed again; if yes, the user face position information is output.

[0016] Preferably, the first adjustment region information is obtained from the user face position information and the display screen size through a first region limiting method, including:

[0017] Effective user face position information is obtained from the user face position information through a face position determination condition, the face position determination condition including a face region coordinate set and a face region boundary box within the display screen boundary;

[0018] Initial first adjustment region information is obtained from the effective user face position information through offset calculation, the initial first adjustment region information including an initial first adjustment region boundary box and an initial first adjustment region coordinate set, and the expression of the offset calculation being: (x adjust ,y adjust )=(x-offset,y-offset) (xadjust_end,yadjust_end)=(x+w+extension,y+h+extension)

[0019] wherein, offset is an offset of the first adjustment region relative to the user face bounding box, extension is an extension of the first adjustment region relative to the user face bounding box, (x adjust ,y adjust ) is a top-left coordinate of the first adjustment region, (xadjust_end, yadjust_end) is a bottom-right coordinate of the first adjustment region;

[0020] the first adjustment region information is obtained according to the initial first adjustment region information and a first adjustment region determination condition, the first adjustment region determination condition comprising a first adjustment region coordinate set and a first adjustment region bounding box within a display screen boundary;

[0021] when the initial first adjustment region information does not satisfy the first adjustment region determination condition, the first adjustment region information is obtained according to the initial first adjustment region information and an adjustment strategy;

[0022] when the initial first adjustment region information satisfies the first adjustment region determination condition, the initial first adjustment region information is the first adjustment region information.

[0023] Preferably, the mathematical expression of the face position determination condition is: x+w≤display_width y+h≤display_height x≥0,y≥0 w>0, h>0

[0024] wherein, x is a horizontal pixel coordinate of the top-left corner of the user face position information, y is a vertical pixel coordinate of the top-left corner of the user face position information, w is a face width in the face bounding box, h is a face height in the face bounding box, display_width is the display screen width, and display_height is the display screen height.

[0025] Preferably, the mathematical expression of the first adjustment region determination condition is: xadjust_end≤display_width yadjust_end≤display_height x adjust ≥0,y adjust ≥0

[0026] wherein, x adjust is a horizontal coordinate of the top-left corner of the first adjustment region, y adjust is a vertical coordinate of the top-left corner of the first adjustment region, xadjust_end is a horizontal coordinate of the bottom-right corner of the first adjustment region, xadjust_end is a vertical coordinate of the bottom-right corner of the first adjustment region, display_width is the display screen width, and display_height is the display screen height.

[0027] Preferably, the adjustment strategy comprises:

[0028] When the horizontal coordinate of the upper left corner of the initial first adjustment region is less than 0, the horizontal coordinate of the upper left corner of the initial first adjustment region is set to 0;

[0029] When the vertical coordinate of the upper left corner of the initial first adjustment region is less than 0, the vertical coordinate of the upper left corner of the initial first adjustment region is set to 0;

[0030] When the horizontal coordinate of the lower right corner of the initial first adjustment region is greater than the width of the display screen, the horizontal coordinate of the lower right corner of the initial first adjustment region is set to be equal to the width of the display screen;

[0031] When the vertical coordinate of the lower right corner of the initial first adjustment region is greater than the height of the display screen, the vertical coordinate of the lower right corner of the initial first adjustment region is set to be equal to the height of the display screen.

[0032] Preferably, the obtaining of the touch point position information according to the initial touch point position information through a touch point position determination condition comprises:

[0033] When the initial touch point position information does not satisfy the touch point position determination condition, the system provides an error handling mechanism, and the error handling mechanism comprises system calibration and maintenance;

[0034] When the initial touch point position information satisfies the touch point position determination condition, the initial touch point position information is the touch point position information.

[0035] Preferably, the obtaining of the second adjustment region information according to the touch point position information and the first adjustment region information through a dynamic adjustment strategy comprises:

[0036] The initial second adjustment region information is obtained according to the touch point position information through a touch point determination condition;

[0037] The second adjustment region information is obtained according to the initial second adjustment region information through a second adjustment region determination condition.

[0038] Preferably, the mathematical expression of the touch point determination condition is: x c ≥x adjust_1 -offset1 y c ≥y adjust_1 -offset1 x c +w1≤x adjust_1 +w adjust_1 +offset1 y c +h1≤y adjust_1 +h adjust_1 +offset1

[0039] wherein, x adjust_1 is the horizontal coordinate of the lower left corner of the first adjustment region, y adjust_1 is the vertical coordinate of the lower left corner of the first adjustment region, x c is the horizontal coordinate of the touch point, y c is the vertical coordinate of the touch point, w1 is the width of the first adjustment region, h1 is the height of the first adjustment region, and offset1 is the offset of the touch point position relative to the first adjustment region.

[0040] Preferably, the initial second adjustment region information obtained by judging the touch point position information according to a touch point judgment condition comprises:

[0041] when the touch point position information satisfies the touch point judgment condition, the upper left corner coordinate of the initial second adjustment region is the touch point coordinate, and the lower right corner coordinate of the initial second adjustment region is the lower right corner coordinate of the first adjustment region;

[0042] when the touch point position information does not satisfy the touch point judgment condition, the upper left corner coordinate of the initial second adjustment region is the touch point coordinate, and the lower right corner coordinate of the initial second adjustment region is the touch point coordinate plus an expansion amount.

[0043] Preferably, the second adjustment region judgment condition comprises:

[0044] when the horizontal coordinate of the upper left corner of the initial second adjustment region is less than 0, the horizontal coordinate of the upper left corner of the second adjustment region is set to 0;

[0045] when the vertical coordinate of the upper left corner of the initial second adjustment region is less than 0, the vertical coordinate of the upper left corner of the second adjustment region is set to 0;

[0046] when the horizontal coordinate of the lower right corner of the initial second adjustment region is greater than the width of the display screen, the horizontal coordinate of the lower right corner of the second adjustment region is set to the width of the display screen;

[0047] when the vertical coordinate of the lower right corner of the initial second adjustment region is greater than the height of the display screen, the vertical coordinate of the lower right corner of the second adjustment region is set to the height of the display screen;

[0048] when the horizontal coordinate of the upper left corner of the initial second adjustment region and the vertical coordinate of the upper left corner of the initial second adjustment region are both greater than or equal to 0, the horizontal coordinate of the lower right corner of the initial second adjustment region is less than or equal to the width of the display screen, and the vertical coordinate of the lower right corner of the initial second adjustment region is less than or equal to the height of the display screen, the initial second adjustment region information is the second adjustment region information.

[0049] Preferably, the new brightness setting parameter obtained by the brightness adjustment algorithm according to the first adjustment region information and the second adjustment region information comprises:

[0050] The brightness adjustment parameter is obtained by customizing initialization according to the user preference through the brightness adjustment algorithm, and the brightness adjustment parameter comprises a brightness increase / decrease ratio and a brightness adjustment speed;

[0051] The first adjustment region brightness adjustment amount and the second adjustment region brightness adjustment amount are respectively obtained by adjustment amount calculation according to the brightness adjustment parameter, the first adjustment region information and the second adjustment region information, and the expression of the adjustment amount calculation is: A = B * Δ, wherein A is the brightness adjustment amount of the region, B is the current brightness of the region, and Δ is the brightness increase / decrease ratio;

[0052] The overlap region information is obtained by taking intersection according to the first adjustment region information and the second adjustment region information, the overlap region brightness adjustment amount is obtained by brightness balance method according to the overlap region information, and specifically, the overlap region brightness adjustment amount is obtained by taking mean value according to the first adjustment region brightness adjustment amount and the second adjustment region brightness adjustment amount;

[0053] The first adjustment region new brightness setting parameter, the second adjustment region new brightness setting parameter and the overlap region new brightness setting parameter are respectively obtained by brightness calculation according to the first adjustment region brightness adjustment amount, the second adjustment region brightness adjustment amount and the overlap region brightness adjustment amount, and the expression of the brightness calculation is: B new = B + A

[0054] Wherein, A is the brightness adjustment amount of the region, B is the current brightness of the region, and B new is the new brightness setting parameter of the region.

[0055] A multi-region brightness collaborative adjustment system based on user face position and touch point position based on the above-mentioned multi-region brightness collaborative adjustment method based on user face position and touch point position, comprising a system initialization module, a real-time detection and response module, a brightness adjustment feedback module, a user prompt and adjustment module;

[0056] The system initialization module is used for prompting the user to start the camera and the touch control permission for initial setting when the user uses the device for the first time, and the initial setting comprises camera calibration and touch sensitivity adjustment;

[0057] The real-time detection and response module is used for real-time monitoring of the user's face position and the user's display screen touch operation when the system is running in the background, and when the user approaches the device, the camera automatically captures the user's face image data and detects the face region to obtain the user's face position information; when the user performs a touch operation on the display screen, the touch sensor automatically collects and detects the touch data to obtain the touch point position information;

[0058] The brightness adjustment feedback module is used for automatically adjusting the display screen brightness according to the touch point position information and the user's face position information when the user operates the device, and the brightness change is displayed on the screen in real time, so that the user can directly see the brightness adjustment result.

[0059] The user prompt and adjustment module is used for displaying prompt information on the screen to guide the user to adjust or repair when the system detects an abnormal condition, and the abnormal condition includes that the camera is blocked and the touch sensor is faulty.

[0060] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the above-mentioned multi-region brightness collaborative adjustment method based on the user's face position and the touch point position.

[0061] A storage medium containing computer executable instructions for executing the above-mentioned multi-region brightness collaborative adjustment method based on the user's face position and the touch point position when executed by a computer processor.

[0062] The beneficial effects of the present application are:

[0063] (1) The user's face position and the user's display screen touch operation are monitored in real time by the camera and the touch sensor, the brightness adjustment region is automatically determined by the first region limiting method, the touch point position determination condition and the dynamic adjustment strategy, the user can realize different region brightness adjustment without manual operation, the user operation process is simplified, and the use convenience is improved.

[0064] (2) According to the touch point position information and the user's face position information, the brightness of the specific region of the display screen is dynamically adjusted by the brightness adjustment algorithm, which can quickly adjust to the most comfortable brightness environment for the user, improve the user experience, and at the same time can reduce unnecessary energy consumption and improve the energy utilization efficiency of the display device.

[0065] (3) The brightness of each region is adjusted according to the user's preference for parameter initialization setting and the brightness adjustment algorithm, which improves the user's satisfaction and comfort, and meets the individual needs of different users. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to facilitate the understanding of those skilled in the art, the present application will be further described below with reference to the accompanying drawings.

[0067] Fig. 1 is a flowchart of the method for adjusting the brightness of multiple regions based on the position of the user's face and the position of the touch point according to the present application. DETAILED DESCRIPTION

[0068] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined inventive purpose, the specific embodiments, structures, features and effects thereof according to the present application are described in detail below in combination with the drawings and preferred embodiments.

[0069] Referring to Fig. 1, the method for adjusting the brightness of multiple regions based on the position of the user's face and the position of the touch point comprises the following steps:

[0070] S1: capturing the image data of the user's face in real time through a camera, obtaining the position information of the user's face based on the image data of the user's face through the face detection algorithm of the Dlib library, wherein the position information of the user's face comprises a set of coordinates of the position of the user's face and a bounding box of the user's face;

[0071] S2: obtaining the first adjustment region information based on the position information of the user's face and the size of the display screen through the first region limiting method, wherein the first adjustment region information comprises a first adjustment region bounding box and a set of coordinates of the first adjustment region, and the first adjustment region bounding box is the region obtained by introducing an offset and an extension to the bounding box of the user's face, wherein the extension is the width and height added to the original region;

[0072] S3: capturing the touch operation of the user on the display screen through a touch sensor to obtain touch sensor data, obtaining the initial position information of the touch point based on the touch sensor data through a touch point detection algorithm, and obtaining the position information of the touch point based on the initial position information of the touch point through a touch point position determination condition, wherein the position information of the touch point is the coordinates of the touch point on the display screen;

[0073] S4: obtaining the second adjustment region information based on the position information of the touch point and the first adjustment region information through a dynamic adjustment strategy, wherein the second adjustment region information comprises a second adjustment region bounding box and a set of coordinates of the second adjustment region;

[0074] S5: obtaining the new brightness setting parameters based on the first adjustment region information and the second adjustment region information through a brightness adjustment algorithm, and dynamically adjusting the brightness of the adjustment region of the display screen based on the new brightness setting parameters, wherein the new brightness setting parameters comprise the new brightness setting parameters of the first adjustment region, the new brightness setting parameters of the second adjustment region, and the new brightness setting parameters of the overlapping region;

[0075] In step S1, obtaining the position information of the user's face based on the image data of the user's face through the face detection algorithm of the Dlib library comprises the following steps:

[0076] S101: initializing a face detector according to the user face image data through a pre-trained classifier;

[0077] S102: scanning the user face image data to obtain a suspected face region according to the user face image data through a sliding window technique;

[0078] S103: extracting a face edge feature and a face shape feature according to the suspected face region through a HOG algorithm;

[0079] S104: judging whether the suspected face region is a face region according to the face edge feature and the face shape feature through the face detector, if not, re-capturing the user face image data through a camera and repeating the steps S101 to S104, and if yes, outputting the user face position information.

[0080] In the step S2, obtaining first adjustment region information according to the user face position information and a display screen size through a first region limiting method, specifically including the following steps:

[0081] S201: obtaining effective user face position information according to the user face position information through a face position judgment condition, the face position judgment condition including a face region coordinate set and a face region boundary box being within a display screen boundary, and a mathematical expression being x+w≤display_width, y+h≤display_height, x≥0, y≥0, w>0, and h>0.

[0082] wherein x is a horizontal pixel coordinate of an upper left corner of the user face position information, y is a vertical pixel coordinate of the upper left corner of the user face position information, w is a face width in the face boundary box, h is a face height in the face boundary box, display_width is the display screen width, and display_height is the display screen height.

[0083] when the user face position information does not satisfy the face position judgment condition, adjusting the user face boundary box to obtain the effective user face position information through a boundary box scaling method; and when the user face position information satisfies the face position judgment condition, the user face position information is the effective user face position information.

[0084] S202: obtaining initial first adjustment region information according to the effective user face position information through offset calculation, the initial first adjustment region information including an initial first adjustment region boundary box and an initial first adjustment region coordinate set, and an expression of the offset calculation being (x adjust , y adjust(xadjust_end, yadjust_end) = (x + w + extension, y + h + extension)

[0085] wherein, offset is an offset of the first adjustment region relative to the user face bounding box, extension is an extension of the first adjustment region relative to the user face bounding box, (x adjust ,y adjust ) is a left upper corner coordinate of the first adjustment region, (xadjust_end, yadjust_end) is a right lower corner coordinate of the first adjustment region;

[0086] S203: obtaining the first adjustment region information according to the initial first adjustment region information through a first adjustment region determination condition, the first adjustment region determination condition including that a first adjustment region coordinate set is within a display screen boundary, and a mathematical expression is as follows: x adjust ≥0,y adjust ≥0

[0087] wherein, x adjust is a left upper corner horizontal coordinate of the first adjustment region, y adjust is a left upper corner vertical coordinate of the first adjustment region, xadjust_end is a right lower corner horizontal coordinate of the first adjustment region, xadjust_end is a right lower corner vertical coordinate of the first adjustment region, display_width is a display screen width, and display_height is a display screen height.

[0088] When the initial first adjustment region information does not satisfy the first adjustment region determination condition, obtaining the first adjustment region information according to the initial first adjustment region information through an adjustment strategy, the adjustment strategy including:

[0089] When the initial left upper corner horizontal coordinate of the first adjustment region is less than 0, setting the initial left upper corner horizontal coordinate of the first adjustment region as 0;

[0090] When the initial left upper corner vertical coordinate of the first adjustment region is less than 0, setting the initial left upper corner vertical coordinate of the first adjustment region as 0;

[0091] When the initial right lower corner horizontal coordinate of the first adjustment region is greater than the display screen width, setting the initial right lower corner horizontal coordinate of the first adjustment region as equal to the display screen width;

[0092] When the initial first adjustment region lower right corner longitudinal coordinate is greater than the display screen height, the initial first adjustment region lower right corner longitudinal coordinate is set to be equal to the display screen height;

[0093] When the initial first adjustment region information satisfies the first adjustment region determination condition, the initial first adjustment region information is the first adjustment region information.

[0094] It should be noted that the touch point detection algorithm in step S3 is a detection algorithm provided by the touch sensor itself.

[0095] In step S3, the touch point position information is obtained according to the initial touch point position information and a touch point position determination condition, and specifically includes: when the initial touch point position information does not satisfy the touch point position determination condition, an error processing mechanism is provided by the system, the error processing mechanism includes system calibration and maintenance; when the initial touch point position information satisfies the touch point position determination condition, the initial touch point position information is the touch point position information.

[0096] In step S4, the second adjustment region information is obtained according to the touch point position information and the first adjustment region information through a dynamic adjustment strategy, and specifically through the following steps:

[0097] S401: obtaining initial second adjustment region information according to the touch point position information through a touch point determination condition, and the mathematical expression of the touch point determination condition is: x c ≥x adjust_1 -offset1 y c ≥y adjust_1 -offset1 x c +w1≤x adjust_1 +w adjust_1 +offset1 y c +h1≤y adjust_1 +h adjust_1 +offset1

[0098] Wherein, x adjust_1 is the left lower corner horizontal coordinate of the first adjustment region, y adjust_1 is the left lower corner longitudinal coordinate of the first adjustment region, x c is the touch point horizontal coordinate, y c is the touch point longitudinal coordinate, w1 is the width of the first adjustment region, h1 is the height of the first adjustment region, offset1 is the offset of the touch point position relative to the first adjustment region.

[0099] When the touch point position information satisfies the touch point judgment condition, the left upper corner coordinate of the initial second adjustment region is the touch point coordinate, and the right lower corner coordinate of the initial second adjustment region is the first adjustment region right lower corner coordinate.

[0100] When the touch point position information does not satisfy the touch point judgment condition, the left upper corner coordinate of the initial second adjustment region is the touch point coordinate, and the right lower corner coordinate of the initial second adjustment region is the touch point coordinate plus an expansion amount.

[0101] S402: obtaining the second adjustment region information according to the initial second adjustment region information and a second adjustment region judgment condition, the second adjustment region judgment condition specifically including that when the left upper corner horizontal coordinate of the initial second adjustment region is less than 0, the left upper corner horizontal coordinate of the second adjustment region is set to 0.

[0102] When the left upper corner vertical coordinate of the initial second adjustment region is less than 0, the left upper corner vertical coordinate of the second adjustment region is set to 0.

[0103] When the right lower corner horizontal coordinate of the initial second adjustment region is greater than the display screen width, the right lower corner horizontal coordinate of the second adjustment region is set to the display screen width.

[0104] When the right lower corner vertical coordinate of the initial second adjustment region is greater than the display screen width, the right lower corner vertical coordinate of the second adjustment region is set to the display screen height.

[0105] When the left upper corner horizontal coordinate of the initial second adjustment region and the left upper corner vertical coordinate of the initial second adjustment region are both greater than or equal to 0, the right lower corner horizontal coordinate of the initial second adjustment region is less than or equal to the display screen width, and the right lower corner vertical coordinate of the initial second adjustment region is less than or equal to the display screen height, the initial second adjustment region information is the second adjustment region information.

[0106] In step S5, a new brightness setting parameter is obtained according to the first adjustment region information and the second adjustment region information through a brightness adjustment algorithm, which is specifically implemented by the following steps:

[0107] S501: obtaining a brightness adjustment parameter according to user preference through a brightness adjustment algorithm, the brightness adjustment parameter including a brightness increase / decrease ratio and a brightness adjustment speed.

[0108] S502: obtaining a first adjustment region brightness adjustment amount and a second adjustment region brightness adjustment amount through adjustment amount calculation according to the brightness adjustment parameter, the first adjustment region information and the second adjustment region information, the expression of the adjustment amount calculation being: A=B*Δ

[0109] Wherein, A is the brightness adjustment amount of the region, B is the current brightness of the region, and Δ is the brightness increase / decrease ratio.

[0110] S503: Obtain the overlapping region information by taking intersection according to the first adjustment region information and the second adjustment region information, and obtain the overlapping region brightness adjustment amount by brightness balancing method according to the overlapping region information, specifically including obtaining the overlapping region brightness adjustment amount by taking mean value according to the first adjustment region brightness adjustment amount and the second adjustment region brightness adjustment amount.

[0111] S504: Obtain the first adjustment region new brightness setting parameter, the second adjustment region new brightness setting parameter, and the overlapping region new brightness setting parameter by brightness calculation respectively according to the first adjustment region brightness adjustment amount, the second adjustment region brightness adjustment amount, and the overlapping region brightness adjustment amount, and the expression of the brightness calculation is: B new =B+A

[0112] Wherein, A is the brightness adjustment amount of the region, B is the current brightness of the region, and B new is the new brightness setting parameter of the region.

[0113] The multi-region brightness collaborative adjustment system based on user face position and touch point position includes a system initialization module, a real-time detection and response module, a brightness adjustment feedback module, and a user prompt and adjustment module.

[0114] The system initialization module is used for prompting the user to start the camera and touch permission for initial setting when the user uses the device for the first time, and the initial setting includes camera calibration and touch sensitivity adjustment.

[0115] The real-time detection and response module is used for real-time monitoring of user face position and user display screen touch operation when the system runs in the background, and when the user approaches the device, the camera automatically captures user face image data and detects the face region to obtain user face position information; when the user performs touch operation on the display screen, the touch sensor automatically collects and detects touch data to obtain touch point position information.

[0116] The brightness adjustment feedback module is used for automatically adjusting the display screen brightness according to the touch point position information and the user face position information when the user operates the device, and the brightness change is displayed on the screen in real time, so that the user can directly see the brightness adjustment result.

[0117] The user prompt and adjustment module is used for displaying prompt information on the screen to guide the user to adjust or repair when the system detects abnormal conditions, and the abnormal conditions include camera obstruction and touch sensor failure.

[0118] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0119] The computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave, in which computer-readable program code is embodied. Such propagated data signals can take a wide variety of forms, including but not limited to electro-magnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium that is not a storage medium, that is, that is not a tangible medium, and that can communicate, propagate or transport programming for use by or in connection with an instruction execution system, apparatus, or device.

[0120] The program code embodied on the computer-readable media can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above. Computer program code for carrying out operations of the present application can be written in one or more programming languages, or combinations of languages, including object oriented, such as Java, Smalltalk, C++, or conventional procedural, such as the "C" programming language, or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). The present application is directed to a computer program product, comprising a computer readable medium having stored therein program code means comprising program code means for implementing the steps of the present application.

[0121] The above merely describes the preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content to obtain equivalent embodiments with equivalent changes, as long as the changes or modifications do not deviate from the technical solution of the present application. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application still belong to the scope of the technical solution of the present application.

Claims

1. A method for multi-zone brightness coordination based on user face position and touch point position, characterized in that, The method comprises the following steps: Real-time capture user face image data through camera, get user face position information through Dlib library face detection algorithm according to the user face image data, the user face position information includes user face position coordinate set and user face boundary box; Get first adjustment area information through first area limiting method according to the user face position information and display screen size, the first adjustment area information includes first adjustment area boundary box and first adjustment area coordinate set, the first adjustment area boundary box is the area after introducing offset and expansion to the user face boundary box, the expansion is the width and height increased on the basis of original area; Capture user display screen touch operation through touch sensor to get touch sensor data, get initial touch point position information through touch point detection algorithm according to the touch sensor data, get touch point position information through touch point position judgment condition according to the initial touch point position information, the touch point position information is the coordinate of touch point on the display screen; Get second adjustment area information through dynamic adjustment strategy according to the touch point position information and the first adjustment area information, the second adjustment area information includes second adjustment area boundary box and second adjustment area coordinate set; Get new brightness setting parameter through brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information, adjust the brightness of the display screen adjustment area according to the new brightness setting parameter, the new brightness setting parameter includes first adjustment area new brightness setting parameter, second adjustment area new brightness setting parameter and overlapping area new brightness setting parameter. 2.The method of claim 1, wherein, The method comprises the following steps: Initialize face detector through pre-trained classifier according to the user face image data; Scan the user face image data through sliding window technology to get suspected face area according to the user face image data; Extract face edge features and face shape features through HOG algorithm according to the suspected face area; Judge whether the suspected face area is a face area through the face detector according to the face edge features and the face shape features, if not, recapture the user face image data through camera and detect face again, if yes, output the user face position information. 3.The method of claim 1, wherein, The method comprises the following steps: Get effective user face position information through face position judgment condition according to the user face position information, the face position judgment condition includes face area coordinate set and face area boundary box within the display screen boundary; Get initial first adjustment area information through offset calculation according to the effective user face position information, the initial first adjustment area information includes initial first adjustment area boundary box and initial first adjustment area coordinate set, the expression of the offset calculation is: (x adjust , y adjust ) = (x-offset, y-offset) (xadjust_end, yadjust_end) = (x + w + extension, y + h + extension) wherein offset is an offset of the first adjustment region relative to the user face bounding box, extension is an extension of the first adjustment region relative to the user face bounding box, (x adjust ,y adjust ) is a top-left corner coordinate of the first adjustment region, and (xadjust_end, yadjust_end) is a bottom-right corner coordinate of the first adjustment region. obtaining the first adjustment region information according to the initial first adjustment region information by a first adjustment region judging condition, the first adjustment region judging condition comprising a first adjustment region coordinate set and a first adjustment region bounding box within a display screen boundary; when the initial first adjustment region information does not satisfy the first adjustment region judging condition, obtaining the first adjustment region information according to the initial first adjustment region information by an adjustment strategy; when the initial first adjustment region information satisfies the first adjustment region judging condition, the initial first adjustment region information is the first adjustment region information.

4. The method of claim 3, wherein the method further comprises: The mathematical expression of the face position judging condition is: x + w ≤ displaywidth, y + h ≤ display_height, x ≥ 0, y ≥ 0, w > 0, h > 0, wherein x is the horizontal pixel coordinate of the upper left corner of the user face position information, y is the vertical pixel coordinate of the upper left corner of the user face position information, w is the face width in the face bounding box, h is the face height in the face bounding box, display_width is the display screen width, and display_height is the display screen height. The adjustment strategy comprises:

5. The method of claim 3, wherein the method further comprises: The mathematical expression of the first adjustment region determination condition is: xadjust_end ≤ display_width yadjust_end ≤ display_height x adjust ≥ 0, y adjust ≥ 0 wherein x adjust is the horizontal coordinate of the upper left corner of the first adjustment region, y adjust is the vertical coordinate of the upper left corner of the first adjustment region, xadjust_end is the horizontal coordinate of the lower right corner of the first adjustment region, yadjust_end is the vertical coordinate of the lower right corner of the first adjustment region, display_width is the width of the display screen, and display_height is the height of the display screen.

6. The method of claim 3, wherein the method further comprises: when the horizontal coordinate of the upper left corner of the initial first adjustment region is less than 0, setting the horizontal coordinate of the upper left corner of the initial first adjustment region as 0; when the vertical coordinate of the upper left corner of the initial first adjustment region is less than 0, setting the vertical coordinate of the upper left corner of the initial first adjustment region as 0; when the horizontal coordinate of the lower right corner of the initial first adjustment region is greater than the display screen width, setting the horizontal coordinate of the lower right corner of the initial first adjustment region as equal to the display screen width; when the vertical coordinate of the lower right corner of the initial first adjustment region is greater than the display screen height, setting the vertical coordinate of the lower right corner of the initial first adjustment region as equal to the display screen height. The obtaining of the touch point position information according to the initial touch point position information by a touch point position judging condition comprises:

7. The method of claim 1, wherein the method further comprises: when the initial touch point position information does not satisfy the touch point position judging condition, providing an error processing mechanism by the system, the error processing mechanism comprising system calibration and maintenance; when the initial touch point position information satisfies the touch point position judging condition, the initial touch point position information is the touch point position information. The obtaining of the second adjustment region information according to the touch point position information and the first adjustment region information by a dynamic adjustment strategy comprises: 8.The method of claim 1, wherein, obtaining initial second adjustment region information according to the touch point position information by a touch point judging condition; obtaining the second adjustment region information according to the initial second adjustment region information by a second adjustment region judging condition. The obtaining of the initial second adjustment region information according to the touch point position information by the touch point judging condition comprises: 9.The method of claim 8, wherein, The mathematical expression of the touch point judgment condition is: x c ≥ x adjust_1 -offset1 y c ≥ y adjust_1 -offset1 x c +w1≤ x adjust_1 +w adjust_1 +offset1 y c +h1≤ y adjust_1 +h adjust_1 +offset1 wherein x adjust_1 is a horizontal coordinate of a lower left corner of the first adjustment region, y adjust_1 is a vertical coordinate of the lower left corner of the first adjustment region, x c is a horizontal coordinate of the touch point, y c is a vertical coordinate of the touch point, w1 is a width of the first adjustment region, h1 is a height of the first adjustment region, and offset1 is an offset of the touch point position relative to the first adjustment region.

10. The method of claim 8, wherein the method further comprises: ​ When the touch point position information satisfies the touch point judgment condition, the left upper corner coordinate of the initial second adjustment region is the touch point coordinate, and the right lower corner coordinate of the initial second adjustment region is the right lower corner coordinate of the first adjustment region; When the touch point position information does not satisfy the touch point judgment condition, the left upper corner coordinate of the initial second adjustment region is the touch point coordinate, and the right lower corner coordinate of the initial second adjustment region is the touch point coordinate plus an expansion amount.

11. The method of claim 8, wherein the method further comprises: The second adjustment region judgment condition comprises: When the left upper corner horizontal coordinate of the initial second adjustment region is less than 0, the left upper corner horizontal coordinate of the second adjustment region is set to 0; When the left upper corner vertical coordinate of the initial second adjustment region is less than 0, the left upper corner vertical coordinate of the second adjustment region is set to 0; When the right lower corner horizontal coordinate of the initial second adjustment region is greater than the width of the display screen, the right lower corner horizontal coordinate of the second adjustment region is set to the width of the display screen; When the right lower corner vertical coordinate of the initial second adjustment region is greater than the width of the display screen, the right lower corner vertical coordinate of the second adjustment region is set to the height of the display screen; When the left upper corner horizontal coordinate of the initial second adjustment region and the left upper corner vertical coordinate of the initial second adjustment region are both greater than or equal to 0, the right lower corner horizontal coordinate of the initial second adjustment region is less than or equal to the width of the display screen, and the right lower corner vertical coordinate of the initial second adjustment region is less than or equal to the height of the display screen, the initial second adjustment region information is the second adjustment region information.

12. The method of claim 1, wherein, The new brightness setting parameter obtained through the brightness adjustment algorithm according to the first adjustment region information and the second adjustment region information comprises: A brightness adjustment parameter is obtained through a brightness adjustment algorithm according to a user preference, the brightness adjustment parameter comprising a brightness increase / decrease ratio and a brightness adjustment speed; A first adjustment region brightness adjustment amount and a second adjustment region brightness adjustment amount are respectively obtained through adjustment amount calculation according to the brightness adjustment parameter, the first adjustment region information and the second adjustment region information, an expression of the adjustment amount calculation being: A = B * Δ wherein A is a region brightness adjustment amount, B is a current brightness of the region, and Δ is the brightness increase / decrease ratio; An overlapping region information is obtained through intersection taking according to the first adjustment region information and the second adjustment region information, an overlapping region brightness adjustment amount is obtained through brightness balance according to the overlapping region information, specifically comprising an overlapping region brightness adjustment amount obtained through mean value taking according to the first adjustment region brightness adjustment amount and the second adjustment region brightness adjustment amount; The first adjustment region new brightness setting parameter, the second adjustment region new brightness setting parameter and the overlapping region new brightness setting parameter are respectively obtained through brightness calculation according to the first adjustment region brightness adjustment amount, the second adjustment region brightness adjustment amount and the overlapping region brightness adjustment amount, an expression of the brightness calculation being: B new = B + A Wherein, A is the region brightness adjustment amount, B is the region current brightness, B new is the region new brightness setting parameter.

13. A user face position and touch point position multi-region brightness coordination adjustment system based on the user face position and touch point position multi-region brightness coordination adjustment method of claim 1, comprising a system initialization module, a real-time detection and response module, a brightness adjustment feedback module, a user prompt and adjustment module, characterized in that: The system initialization module is used for prompting the user to turn on the camera and touch permission for initial setting when the user uses the device for the first time, and the initial setting includes camera calibration and touch sensitivity adjustment; The real-time detection and response module is used for monitoring the user face position and user display screen touch operation in real time when the system runs in the background, automatically capturing user face image data and detecting the face region to obtain user face position information when the user approaches the device; When the user performs touch operation on the display screen, the touch sensor automatically collects and detects touch data to obtain touch point position information; The brightness adjustment feedback module is used for adjusting the display screen brightness automatically according to the touch point position information and user face position information when the user operates the device, and the brightness change is displayed on the screen in real time, so that the user can directly see the brightness adjustment result; The user prompt and adjustment module is used for displaying prompt information on the screen to guide the user to adjust or repair when the system detects abnormal conditions, and the abnormal conditions include camera obstruction and touch sensor failure.

14. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the user face position and touch point position multi-region brightness coordination adjustment method of any one of claims 1-13.

15. A storage medium containing computer-executable instructions, wherein: The computer executable instructions are executed by the computer processor to perform the user face position and touch point position multi-region brightness coordination adjustment method of any one of claims 1-13.

Images