Multi-area brightness cooperative adjustment method based on user face position and touch point position

Abstract

The invention relates to a multi-area brightness cooperative adjustment method based on a user face position and a touch point position, and belongs to the technical field of display. The method comprises the following steps: capturing user face image data through a camera, obtaining user face position information through a Dlib library face detection algorithm, obtaining first adjustment area information through a first area limiting method according to the user face position information and the size of a display screen, the method comprises the following steps: capturing a user display screen touch operation through a touch sensor, obtaining initial touch point position information through a touch point detection algorithm, and obtaining touch point position information through a touch point position limiting condition according to the initial touch point position information; obtaining second adjustment area information through a dynamic adjustment strategy according to the touch point position information and the first adjustment area information, adjusting the brightness of each area according to user preference through a brightness adjustment algorithm to obtain a new brightness setting parameter, and dynamically adjusting the brightness of the display screen adjustment area according to the new brightness parameter.

Description

Technical Field

[0001] The present invention belongs to the field of display technology, and in particular relates to a method for coordinated adjustment of multi-zone brightness based on user face positions and touch point positions. Background Art

[0002] Brightness adjustment is an intelligent display brightness optimization technology that adjusts the brightness of the display device according to specific environmental conditions or user behavior. It accurately determines the screen area where the user is looking by identifying the user's face position, and determines the user interaction area by monitoring the touch point position, thereby achieving regional brightness adjustment. However, most display devices on the market mainly adjust the brightness of the entire screen, and are unable to optimize local brightness according to the user's specific usage location and operating habits, resulting in a poor user experience. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present invention proposes a multi-zone brightness coordinated adjustment method based on the user's face position and touch point position, which intelligently adjusts the local brightness of the display screen to enhance the user's visual experience and operating comfort.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] The multi-area brightness coordinated adjustment method based on the user's face position and the touch point position includes:

[0006] Capturing user face image data in real time through a camera, and obtaining user face position information through a Dlib library face detection algorithm according to the user face image data, wherein the user face position information includes a user face position coordinate set and a user face boundary box;

[0007] Obtaining first adjustment area information by a first area limitation method according to the user face position information and the display screen size, wherein the first adjustment area information includes a first adjustment area boundary box and a first adjustment area coordinate set, wherein the first adjustment area boundary box is an area after an offset and an expansion amount are introduced into the user face boundary box, and the expansion amount is a width and a height increased on the basis of the original area;

[0008] Capturing the user's touch operation on the display screen through the touch sensor to obtain touch sensor data, obtaining initial touch point position information through a touch point detection algorithm based on the touch sensor data, and obtaining touch point position information through a touch point position determination condition based on the initial touch point position information, wherein the touch point position information is the coordinates of the touch point on the display screen;

[0009] Obtaining second adjustment region information through a dynamic adjustment strategy according to the touch point position information and the first adjustment region information, wherein the second adjustment region information includes a second adjustment region boundary box and a second adjustment region coordinate set;

[0010] New brightness setting parameters are obtained through a brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information, and the brightness of the display adjustment area is dynamically adjusted according to the new brightness setting parameters. The new brightness setting parameters include new brightness setting parameters for the first adjustment area, new brightness setting parameters for the second adjustment area, and new brightness setting parameters for the overlapping area.

[0011] Preferably, obtaining the user face position information through the Dlib library face detection algorithm according to the user face image data includes:

[0012] Obtaining a face detector by initializing a pre-trained classifier according to the user face image data;

[0013] Scanning the user's face image data using a sliding window technique according to the user's face image data to obtain a suspected face area;

[0014] Extracting facial edge features and facial shape features from the suspected face region using the HOG algorithm;

[0015] The face detector is used to determine whether the suspected face area is a face area according to the face edge features and the face shape features. If not, the user face image data is recaptured through a camera and face detection is performed again; if yes, the user face position information is output.

[0016] Preferably, obtaining the first adjustment area information by a first area limitation method according to the user face position information and the display screen size includes:

[0017] Obtaining effective user face position information according to the user face position information through face position determination conditions, wherein the face position determination conditions include that the face region coordinate set is reasonable and the face region boundary frame is within the display screen boundary;

[0018] The initial first adjustment region information is obtained by offset calculation according to the effective user face position information, wherein the initial first adjustment region information includes an initial first adjustment region boundary box and an initial first adjustment region coordinate set, and the expression of the offset calculation is:

[0019] (x adjust ,y adjust ) = (x-offset, y-offset)

[0020] (x adjust_end ,y adjust_end)=(x+w+extension,y+h+extension)

[0021] Wherein, offset is the offset of the first adjustment area relative to the user face bounding box, extension is the extension of the first adjustment area relative to the user face bounding box, (x adjust ,y adjust ) is the coordinate of the upper left corner of the first adjustment area, (x adjust_end ,y adjust_end ) is the coordinate of the lower right corner of the first adjustment area;

[0022] The first adjustment area information is obtained according to the initial first adjustment area information through a first adjustment area determination condition, wherein the first adjustment area determination condition includes that the first adjustment area coordinate set is reasonable and the first adjustment area boundary frame is within the boundary of the display screen;

[0023] When the initial first adjustment region information does not meet the first adjustment region determination condition, obtaining the first adjustment region information through an adjustment strategy according to the initial first adjustment region information;

[0024] When the initial first adjustment region information satisfies the first adjustment region determination condition, the initial first adjustment region information is the first adjustment region information.

[0025] Preferably, the mathematical expression of the face position determination condition is:

[0026] x+w≤display_width

[0027] y+h≤display_hheight

[0028] x≥0,y≥0

[0029] w>0,h>0

[0030] Among them, x is the horizontal coordinate of the pixel in the upper left corner of the user's face position information, y is the vertical coordinate of the pixel in the upper left corner of the user's face position information, w is the face width in the face boundary box, h is the face height in the face boundary box, display_width is the width of the display screen, and display_height is the height of the display screen.

[0031] Preferably, the mathematical expression of the first adjustment area determination condition is:

[0032] x adjust_end ≤display_width

[0033] y adjust_end ≤display_height

[0034] x adjust ≥0,y adjust ≥0

[0035] Among them, x adjust is the horizontal coordinate of the upper left corner of the first adjustment area, y adjust is the ordinate of the upper left corner of the first adjustment area, x adjust_end is the horizontal coordinate of the lower right corner of the first adjustment area, y adjust_end is the vertical coordinate of the lower right corner of the first adjustment area, display_width is the width of the display screen, and display_hight is the height of the display screen.

[0036] Preferably, the adjustment strategy includes:

[0037] When the horizontal coordinate of the upper left corner of the initial first adjustment area is less than 0, the horizontal coordinate of the upper left corner of the initial first adjustment area is set to 0;

[0038] When the ordinate of the upper left corner of the initial first adjustment area is less than 0, setting the ordinate of the upper left corner of the initial first adjustment area to 0;

[0039] When the horizontal coordinate of the lower right corner of the initial first adjustment area is greater than the width of the display screen, setting the horizontal coordinate of the lower right corner of the initial first adjustment area to be equal to the width of the display screen;

[0040] When the ordinate of the lower right corner of the initial first adjustment area is greater than the height of the display screen, the ordinate of the lower right corner of the initial first adjustment area is set to be equal to the height of the display screen.

[0041] Preferably, obtaining the touch point position information according to the initial touch point position information through a touch point position determination condition comprises:

[0042] When the initial touch point position information does not meet the touch point position determination condition, the system provides an error handling mechanism, which includes system calibration and maintenance;

[0043] When the initial touch point position information satisfies the touch point position determination condition, the initial touch point position information is the touch point position information.

[0044] Preferably, obtaining the second adjustment area information through a dynamic adjustment strategy according to the touch point position information and the first adjustment area information includes:

[0045] Obtaining initial second adjustment area information by judging the touch point judgment condition according to the touch point position information;

[0046] The second adjustment region information is obtained according to the initial second adjustment region information through the second adjustment region determination condition.

[0047] Preferably, the mathematical expression of the touch point judgment condition is:

[0048] x c ≥x adjust_1 -offset1

[0049] y c ≥y adjust_1 -offset1

[0050] x c +w1≤x adjust_1 +w adjust_1 +offset1

[0051] y c +h1≤y adjust_1 +h adjust_1 +offset1

[0052] Among them, x adjust_1 is the lower left corner abscissa of the first adjustment area, y adjust_1 is the lower left corner ordinate of the first adjustment area, x c is the horizontal coordinate of the touch point, y c is the vertical coordinate of the touch point, w1 is the width of the first adjustment area, h1 is the height of the first adjustment area, and offset1 is the offset of the touch point position relative to the first adjustment area.

[0053] Preferably, the obtaining of the initial second adjustment area information by judging the touch point judgment condition according to the touch point position information includes:

[0054] When the touch point position information satisfies the touch point judgment condition, the coordinates of the upper left corner of the initial second adjustment area are the touch point coordinates, and the coordinates of the lower right corner of the initial second adjustment area are the coordinates of the lower right corner of the first adjustment area;

[0055] When the touch point position information does not meet the touch point judgment condition, the upper left corner coordinates of the initial second adjustment area are the touch point coordinates, and the lower right corner coordinates of the initial second adjustment area are the touch point coordinates plus the extension amount.

[0056] Preferably, the second adjustment region determination condition includes:

[0057] When the horizontal coordinate of the upper left corner of the initial second adjustment area is less than 0, the horizontal coordinate of the upper left corner of the second adjustment area is set to 0;

[0058] When the upper left corner ordinate of the initial second adjustment area is less than 0, the upper left corner ordinate of the second adjustment area is set to 0;

[0059] When the horizontal coordinate of the lower right corner of the initial second adjustment area is greater than the width of the display screen, the horizontal coordinate of the lower right corner of the second adjustment area is set to the width of the display screen;

[0060] When the lower right corner ordinate of the initial second adjustment area is greater than the width of the display screen, the lower right corner ordinate of the second adjustment area is set to the height of the display screen;

[0061] When the horizontal coordinate of the upper left corner of the initial second adjustment area and the vertical coordinate of the upper left corner of the initial second adjustment area are both greater than or equal to 0, the horizontal coordinate of the lower right corner of the initial second adjustment area is less than or equal to the width of the display screen, and the vertical coordinate of the lower right corner of the initial second adjustment area is less than or equal to the height of the display screen, the initial second adjustment area information is the second adjustment area information.

[0062] Preferably, obtaining a new brightness setting parameter through a brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information includes:

[0063] According to user preference, a brightness adjustment algorithm is used to customize and initialize the brightness adjustment parameters, wherein the brightness adjustment parameters include a brightness increase / decrease ratio and a brightness adjustment speed;

[0064] According to the brightness adjustment parameter, the first adjustment area information, and the second adjustment area information, the first adjustment area brightness adjustment amount and the second adjustment area brightness adjustment amount are respectively obtained by adjusting amount calculation, and the expression for calculating the adjustment amount is:

[0065] A=B*Δ

[0066] Wherein, A is the brightness adjustment amount of the area, B is the current brightness of the area, and Δ is the brightness increase or decrease ratio;

[0067] Obtaining overlapping area information by taking the intersection of the first adjustment area information and the second adjustment area information, and obtaining overlapping area brightness adjustment amount by using a brightness balance method according to the overlapping area information, specifically including obtaining overlapping area brightness adjustment amount by taking the average of the first adjustment area brightness adjustment amount and the second adjustment area brightness adjustment amount;

[0068] According to the brightness adjustment amount of the first adjustment area, the brightness adjustment amount of the second adjustment area, and the brightness adjustment amount of the overlapping area, the new brightness setting parameter of the first adjustment area, the new brightness setting parameter of the second adjustment area, and the new brightness setting parameter of the overlapping area are obtained through brightness calculation, and the expression of the brightness calculation is:

[0069] B new =B+A

[0070] Among them, A is the brightness adjustment amount of the area, B is the current brightness of the area, and B new Set parameters for the new brightness of the area.

[0071] A multi-region brightness collaborative adjustment system based on user face position and touch point position, which applies the multi-region brightness collaborative adjustment method based on user face position and touch point position, comprises a system initialization module, a real-time detection and response module, a brightness adjustment feedback module, and a user prompt and adjustment module;

[0072] The system initialization module is used when a user uses the device for the first time. The system prompts the user to enable the camera and touch permissions for initial settings, including camera calibration and touch sensitivity adjustment.

[0073] The real-time detection and response module is used to monitor the user's face position and the user's display screen touch operation in real time when the system is running in the background. When the user approaches the device, the camera automatically captures the user's face image data and detects the face area to obtain the user's face position information; when the user performs a touch operation on the display screen, the touch sensor automatically collects and detects the touch data to obtain the touch point position information;

[0074] The brightness adjustment feedback module is used when the user operates the device. The system automatically adjusts the display brightness according to the touch point location information and the user's face location information. The brightness change will be displayed on the screen in real time, and the user can directly see the brightness adjustment result;

[0075] The user prompt and adjustment module is used to display prompt information on the screen to guide the user to make adjustments or repairs when the system detects abnormal conditions, such as camera obstruction and touch sensor failure.

[0076] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the multi-zone coordinated brightness adjustment method based on the user's face position and the touch point position is implemented.

[0077] A storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute the above-mentioned multi-region brightness coordinated adjustment method based on user face position and touch point position.

[0078] The beneficial effects of the present invention are:

[0079] (1) The user's face position and the user's display screen touch operation are monitored in real time through the camera and touch sensor, and the brightness adjustment area is automatically determined through the first area limitation method, the touch point position determination condition and the dynamic adjustment strategy. The user can adjust the brightness of different areas without manual operation, which simplifies the user operation process and improves the convenience of use.

[0080] (2) The brightness of a specific area of ​​the display screen is dynamically adjusted based on the touch point location information and the user's face location information through a brightness adjustment algorithm. This can quickly adjust the brightness environment to the most comfortable level for the user, improve the user experience, and at the same time reduce unnecessary energy consumption and improve the energy efficiency of the display device.

[0081] (3) Parameters are initialized according to user preferences and the brightness of each area is adjusted through a brightness adjustment algorithm, which improves user satisfaction and comfort and meets the personalized needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0083] Figure 1 It is a flow chart of the method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position of the present invention. DETAILED DESCRIPTION

[0084] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0085] See also Figure 1 , a multi-area brightness coordinated adjustment method based on the user's face position and the touch point position, including:

[0086] S1: Capturing user face image data in real time through a camera, and obtaining user face position information through a Dlib library face detection algorithm according to the user face image data, wherein the user face position information includes a user face position coordinate set and a user face boundary box;

[0087] S2: obtaining first adjustment area information by a first area limitation method according to the user face position information and the display screen size, wherein the first adjustment area information includes a first adjustment area boundary box and a first adjustment area coordinate set, wherein the first adjustment area boundary box is an area after an offset and an expansion amount are introduced into the user face boundary box, and the expansion amount is a width and a height increased on the basis of the original area;

[0088] S3: Capturing the user's touch operation on the display screen through the touch sensor to obtain touch sensor data, obtaining initial touch point position information through a touch point detection algorithm according to the touch sensor data, and obtaining touch point position information through a touch point position determination condition according to the initial touch point position information, wherein the touch point position information is the coordinates of the touch point on the display screen;

[0089] S4: obtaining second adjustment region information through a dynamic adjustment strategy according to the touch point position information and the first adjustment region information, wherein the second adjustment region information includes a second adjustment region boundary box and a second adjustment region coordinate set;

[0090] S5: obtaining new brightness setting parameters through a brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information, and dynamically adjusting the brightness of the adjustment area of ​​the display screen according to the new brightness setting parameters, wherein the new brightness setting parameters include new brightness setting parameters for the first adjustment area, new brightness setting parameters for the second adjustment area, and new brightness setting parameters for the overlapping area;

[0091] In step S1, obtaining the user face position information through the Dlib library face detection algorithm according to the user face image data specifically includes the following steps:

[0092] S101: obtaining a face detector by initializing a pre-trained classifier according to the user face image data;

[0093] S102: Scanning the user face image data using a sliding window technique according to the user face image data to obtain a suspected face area;

[0094] S103: extracting face edge features and face shape features from the suspected face region using a HOG algorithm;

[0095] S104: Determine whether the suspected face area is a face area through the face detector according to the face edge features and the face shape features; if not, recapture the user face image data through the camera and repeat steps S101 to S104; if yes, output the user face position information.

[0096] In step S2, obtaining first adjustment area information by a first area limitation method according to the user face position information and the display screen size specifically includes the following steps:

[0097] S201: Obtain valid user face position information according to the user face position information through a face position determination condition, wherein the face position determination condition includes that the face region coordinate set is reasonable and the face region boundary frame is within the display screen boundary, and the mathematical expression is:

[0098] x+w≤displaay_width

[0099] y+h≤display_hheight

[0100] x≥0,y≥0

[0101] w>0,h>0

[0102] Among them, x is the horizontal coordinate of the pixel in the upper left corner of the user's face position information, y is the vertical coordinate of the pixel in the upper left corner of the user's face position information, w is the face width in the face boundary box, h is the face height in the face boundary box, display_width is the width of the display screen, and display_height is the height of the display screen.

[0103] When the user face position information does not meet the face position determination condition, the valid user face position information is obtained by adjusting the user face boundary frame through a frame scaling method; when the user face position information meets the face position determination condition, the user face position information is the valid user face position information;

[0104] S202: Obtaining initial first adjustment region information through offset calculation according to the effective user face position information, wherein the initial first adjustment region information includes an initial first adjustment region boundary box and an initial first adjustment region coordinate set, and the expression for the offset calculation is:

[0105] (x adjust ,y adjust ) = (x-offset, y-offset)

[0106] (x adjust_end ,y adjust_end )=(x+w+extension, y+h+extension)

[0107] Wherein, offset is the offset of the first adjustment area relative to the user face bounding box, extension is the extension of the first adjustment area relative to the user face bounding box, (x adjust ,y adjust ) is the coordinate of the upper left corner of the first adjustment area, (x adjust_end ,y adjust_end ) is the coordinate of the lower right corner of the first adjustment area;

[0108] S203: Obtain the first adjustment area information according to the initial first adjustment area information through the first adjustment area determination condition, wherein the first adjustment area determination condition includes that the first adjustment area coordinate set is reasonable and the first adjustment area boundary frame is within the display screen boundary, and the mathematical expression is:

[0109] x adjust_end ≤display_width

[0110] y adjust_end ≤display_height

[0111] x adjust≥0,y adjust ≥0

[0112] Among them, x adjust is the horizontal coordinate of the upper left corner of the first adjustment area, y adjust is the ordinate of the upper left corner of the first adjustment area, x adjust_end is the horizontal coordinate of the lower right corner of the first adjustment area, y adjust_end is the vertical coordinate of the lower right corner of the first adjustment area, display_width is the width of the display screen, and display_hight is the height of the display screen.

[0113] When the initial first adjustment region information does not meet the first adjustment region determination condition, the first adjustment region information is obtained by an adjustment strategy according to the initial first adjustment region information, and the adjustment strategy includes:

[0114] When the horizontal coordinate of the upper left corner of the initial first adjustment area is less than 0, the horizontal coordinate of the upper left corner of the initial first adjustment area is set to 0;

[0115] When the ordinate of the upper left corner of the initial first adjustment area is less than 0, setting the ordinate of the upper left corner of the initial first adjustment area to 0;

[0116] When the horizontal coordinate of the lower right corner of the initial first adjustment area is greater than the width of the display screen, setting the horizontal coordinate of the lower right corner of the initial first adjustment area to be equal to the width of the display screen;

[0117] When the ordinate of the lower right corner of the initial first adjustment area is greater than the height of the display screen, setting the ordinate of the lower right corner of the initial first adjustment area to be equal to the height of the display screen;

[0118] When the initial first adjustment region information satisfies the first adjustment region determination condition, the initial first adjustment region information is the first adjustment region information;

[0119] It should be noted that the touch point detection algorithm in step S3 is a detection algorithm provided by the touch sensor.

[0120] In step S3, the touch point position information is obtained according to the initial touch point position information through the touch point position determination condition, specifically including when the initial touch point position information does not meet the touch point position determination condition, the system provides an error handling mechanism, the error handling mechanism includes system calibration and maintenance; when the initial touch point position information meets the touch point position determination condition, the initial touch point position information is the touch point position information;

[0121] In step S4, obtaining the second adjustment region information through a dynamic adjustment strategy according to the touch point position information and the first adjustment region information is specifically implemented by the following steps:

[0122] S401: obtaining initial second adjustment area information according to the touch point position information through a touch point judgment condition, wherein the mathematical expression of the touch point judgment condition is:

[0123] x c ≥x adjust_1 -offset1

[0124] y c ≥y adjust_1 -offset1

[0125] x c +w1≤x adjust_1 +w adjust_1 +offset1

[0126] y c +h1≤y adjust_1 +h adjust_1 +offset1

[0127] Among them, x adjust_1 is the lower left corner abscissa of the first adjustment area, y adjust_1 is the lower left corner ordinate of the first adjustment area, x c is the horizontal coordinate of the touch point, y c is the vertical coordinate of the touch point, w1 is the width of the first adjustment area, h1 is the height of the first adjustment area, and offset1 is the offset of the touch point position relative to the first adjustment area.

[0128] When the touch point position information satisfies the touch point judgment condition, the coordinates of the upper left corner of the initial second adjustment area are the touch point coordinates, and the coordinates of the lower right corner of the initial second adjustment area are the coordinates of the lower right corner of the first adjustment area;

[0129] When the touch point position information does not meet the touch point judgment condition, the upper left corner coordinates of the initial second adjustment area are the touch point coordinates, and the lower right corner coordinates of the initial second adjustment area are the touch point coordinates plus the extension amount.

[0130] S402: Obtain the second adjustment area information according to the initial second adjustment area information through a second adjustment area determination condition, wherein the second adjustment area determination condition specifically includes that when the horizontal coordinate of the upper left corner of the initial second adjustment area is less than 0, the horizontal coordinate of the upper left corner of the second adjustment area is set to 0;

[0131] When the upper left corner ordinate of the initial second adjustment area is less than 0, the upper left corner ordinate of the second adjustment area is set to 0;

[0132] When the horizontal coordinate of the lower right corner of the initial second adjustment area is greater than the width of the display screen, the horizontal coordinate of the lower right corner of the second adjustment area is set to the width of the display screen;

[0133] When the lower right corner ordinate of the initial second adjustment area is greater than the width of the display screen, the lower right corner ordinate of the second adjustment area is set to the height of the display screen;

[0134] When the horizontal coordinate of the upper left corner of the initial second adjustment area and the vertical coordinate of the upper left corner of the initial second adjustment area are both greater than or equal to 0, the horizontal coordinate of the lower right corner of the initial second adjustment area is less than or equal to the width of the display screen, and the vertical coordinate of the lower right corner of the initial second adjustment area is less than or equal to the height of the display screen, the initial second adjustment area information is the second adjustment area information.

[0135] In step S5, obtaining a new brightness setting parameter through a brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information is specifically implemented by the following steps:

[0136] S501: Obtaining brightness adjustment parameters through a custom initialization of a brightness adjustment algorithm according to user preference, wherein the brightness adjustment parameters include a brightness increase / decrease ratio and a brightness adjustment speed.

[0137] S502: Obtain a first adjustment area brightness adjustment amount and a second adjustment area brightness adjustment amount respectively through adjustment amount calculation according to the brightness adjustment parameter, the first adjustment area information, and the second adjustment area information. The expression for calculating the adjustment amount is:

[0138] A=B*△

[0139] Wherein, A is the brightness adjustment amount of the area, B is the current brightness of the area, and Δ is the brightness increase or decrease ratio;

[0140] S503: Obtain overlapping area information by taking an intersection according to the first adjustment area information and the second adjustment area information, and obtain overlapping area brightness adjustment amount by a brightness balance method according to the overlapping area information, specifically including obtaining overlapping area brightness adjustment amount by taking an average according to the first adjustment area brightness adjustment amount and the second adjustment area brightness adjustment amount;

[0141] S504: According to the brightness adjustment amount of the first adjustment area, the brightness adjustment amount of the second adjustment area, and the brightness adjustment amount of the overlapping area, a new brightness setting parameter of the first adjustment area, a new brightness setting parameter of the second adjustment area, and a new brightness setting parameter of the overlapping area are obtained through brightness calculation, and the expression of the brightness calculation is:

[0142] B new =B+A

[0143] Among them, A is the brightness adjustment amount of the area, B is the current brightness of the area, and B new Set parameters for the new brightness of the area.

[0144] The multi-zone brightness collaborative adjustment system based on the user's face position and touch point position includes a system initialization module, a real-time detection and response module, a brightness adjustment feedback module, and a user prompt and adjustment module;

[0145] The system initialization module is used when a user uses the device for the first time. The system prompts the user to enable the camera and touch permissions for initial settings, including camera calibration and touch sensitivity adjustment.

[0146] The real-time detection and response module is used to monitor the user's face position and the user's display screen touch operation in real time when the system is running in the background. When the user approaches the device, the camera automatically captures the user's face image data and detects the face area to obtain the user's face position information; when the user performs a touch operation on the display screen, the touch sensor automatically collects and detects the touch data to obtain the touch point position information;

[0147] The brightness adjustment feedback module is used when the user operates the device. The system automatically adjusts the display brightness according to the touch point location information and the user's face location information. The brightness change will be displayed on the screen in real time, and the user can directly see the brightness adjustment result;

[0148] The user prompt and adjustment module is used to display prompt information on the screen to guide the user to make adjustments or repairs when the system detects abnormal conditions, such as camera obstruction and touch sensor failure.

[0149] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0150] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0151] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0152] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A multi-region brightness collaborative adjustment method based on the user's face position and touch point position, characterized in that: include: Capturing user face image data in real time through a camera, and obtaining user face position information through a Dlib library face detection algorithm according to the user face image data, wherein the user face position information includes a user face position coordinate set and a user face boundary box; Obtaining first adjustment area information by a first area limitation method according to the user face position information and the display screen size, wherein the first adjustment area information includes a first adjustment area boundary box and a first adjustment area coordinate set, wherein the first adjustment area boundary box is an area after an offset and an expansion amount are introduced into the user face boundary box, and the expansion amount is a width and a height increased on the basis of the original area; Capturing the user's touch operation on the display screen through the touch sensor to obtain touch sensor data, obtaining initial touch point position information through a touch point detection algorithm based on the touch sensor data, and obtaining touch point position information through a touch point position determination condition based on the initial touch point position information, wherein the touch point position information is the coordinates of the touch point on the display screen; Obtaining second adjustment region information through a dynamic adjustment strategy according to the touch point position information and the first adjustment region information, wherein the second adjustment region information includes a second adjustment region boundary box and a second adjustment region coordinate set; New brightness setting parameters are obtained through a brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information, and the brightness of the display adjustment area is dynamically adjusted according to the new brightness setting parameters. The new brightness setting parameters include new brightness setting parameters for the first adjustment area, new brightness setting parameters for the second adjustment area, and new brightness setting parameters for the overlapping area.

2. The method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position according to claim 1, characterized in that: The obtaining the user face position information by using the Dlib library face detection algorithm according to the user face image data includes: Obtaining a face detector by initializing a pre-trained classifier according to the user face image data; Scanning the user's face image data using a sliding window technique according to the user's face image data to obtain a suspected face area; Extracting facial edge features and facial shape features from the suspected face region using the HOG algorithm; The face detector is used to determine whether the suspected face area is a face area according to the face edge features and the face shape features. If not, the user face image data is recaptured through a camera and face detection is performed again; if yes, the user face position information is output.

3. The method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position according to claim 1, characterized in that: The obtaining of the first adjustment area information by a first area limiting method according to the user face position information and the display screen size includes: Obtaining effective user face position information according to the user face position information through face position determination conditions, wherein the face position determination conditions include that the face region coordinate set is reasonable and the face region boundary frame is within the display screen boundary; The initial first adjustment region information is obtained by offset calculation according to the effective user face position information, wherein the initial first adjustment region information includes an initial first adjustment region boundary box and an initial first adjustment region coordinate set, and the expression of the offset calculation is: (x adjust ,y adjust )=(x-offset,y-offset) (x adjust_end ,y adjust_end )=(x+w+extension,y+h+extension) Wherein, offset is the offset of the first adjustment area relative to the user face bounding box, extension is the extension of the first adjustment area relative to the user face bounding box, (x adjust ,y adjust ) is the coordinate of the upper left corner of the first adjustment area, (x adjust_end ,y adjust_end ) is the coordinate of the lower right corner of the first adjustment area; The first adjustment area information is obtained according to the initial first adjustment area information through a first adjustment area determination condition, wherein the first adjustment area determination condition includes that the first adjustment area coordinate set is reasonable and the first adjustment area boundary frame is within the boundary of the display screen; When the initial first adjustment region information does not meet the first adjustment region determination condition, obtaining the first adjustment region information through an adjustment strategy according to the initial first adjustment region information; When the initial first adjustment region information meets the first adjustment region determination condition, the initial first adjustment region information is the first adjustment region information.

4. The method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position according to claim 3, characterized in that: The mathematical expression of the face position determination condition is: X+w≤display_width y+h≤display_height x≥0,y≥0 w>0,h≥0 Among them, x is the horizontal coordinate of the pixel in the upper left corner of the user's face position information, y is the vertical coordinate of the pixel in the upper left corner of the user's face position information, w is the face width in the face boundary box, h is the face height in the face boundary box, display_width is the width of the display screen, and display_height is the height of the display screen.

5. The method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position according to claim 3, characterized in that: The mathematical expression of the first adjustment area determination condition is: x adjust_end ≤display_width y adjust_end ≤display_height x adjust ≥0,y adjust ≥0 Among them, x adjust is the horizontal coordinate of the upper left corner of the first adjustment area, y adjust is the ordinate of the upper left corner of the first adjustment area, x adjust_end is the horizontal coordinate of the lower right corner of the first adjustment area, y adjust_end is the vertical coordinate of the lower right corner of the first adjustment area, display_width is the width of the display screen, and display_height is the height of the display screen.

6. The method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position according to claim 3, characterized in that: The adjustment strategies include: When the horizontal coordinate of the upper left corner of the initial first adjustment area is less than 0, the horizontal coordinate of the upper left corner of the initial first adjustment area is set to 0; When the ordinate of the upper left corner of the initial first adjustment area is less than 0, setting the ordinate of the upper left corner of the initial first adjustment area to 0; When the horizontal coordinate of the lower right corner of the initial first adjustment area is greater than the width of the display screen, setting the horizontal coordinate of the lower right corner of the initial first adjustment area to be equal to the width of the display screen; When the ordinate of the lower right corner of the initial first adjustment area is greater than the height of the display screen, the ordinate of the lower right corner of the initial first adjustment area is set to be equal to the height of the display screen.

7. The method for coordinated brightness adjustment of multiple regions based on user face position and touch point position according to claim 1, characterized in that: The obtaining of the touch point position information according to the initial touch point position information by using a touch point position determination condition comprises: When the initial touch point position information does not meet the touch point position determination condition, the system provides an error handling mechanism, which includes system calibration and maintenance; When the initial touch point position information satisfies the touch point position determination condition, the initial touch point position information is the touch point position information.

8. The method for coordinated brightness adjustment of multiple regions based on user face position and touch point position according to claim 1, characterized in that: The obtaining the second adjustment area information by a dynamic adjustment strategy according to the touch point position information and the first adjustment area information comprises: Obtaining initial second adjustment area information by judging the touch point judgment condition according to the touch point position information; The second adjustment region information is obtained according to the initial second adjustment region information through the second adjustment region determination condition.

9. The method for coordinated brightness adjustment of multiple regions based on the user's face position and the touch point position according to claim 8, characterized in that: The mathematical expression of the touch point judgment condition is: x c ≥x adjust_1 -offset1 y c ≥y adjust_1 -offset1 x c +w1≤x adjust_1 +in adjust_1 +offset1 y c +h1≤y adjust_1 +h adjust_1 +offset1 Among them, x adjust_1 is the lower left corner abscissa of the first adjustment area, y adjust_1 is the lower left corner ordinate of the first adjustment area, x c is the horizontal coordinate of the touch point, y c is the vertical coordinate of the touch point, w1 is the width of the first adjustment area, h1 is the height of the first adjustment area, and offset1 is the offset of the touch point position relative to the first adjustment area.

10. The method for coordinated brightness adjustment of multiple regions based on user face position and touch point position according to claim 8, characterized in that: The obtaining of the initial second adjustment area information by judging the touch point judgment condition according to the touch point position information includes: When the touch point position information satisfies the touch point judgment condition, the coordinates of the upper left corner of the initial second adjustment area are the touch point coordinates, and the coordinates of the lower right corner of the initial second adjustment area are the coordinates of the lower right corner of the first adjustment area; When the touch point position information does not meet the touch point judgment condition, the upper left corner coordinates of the initial second adjustment area are the touch point coordinates, and the lower right corner coordinates of the initial second adjustment area are the touch point coordinates plus the extension amount.

11. The method for coordinated brightness adjustment of multiple regions based on user face position and touch point position according to claim 8, characterized in that: The second adjustment area determination condition includes: When the horizontal coordinate of the upper left corner of the initial second adjustment area is less than 0, the horizontal coordinate of the upper left corner of the second adjustment area is set to 0; When the upper left corner ordinate of the initial second adjustment area is less than 0, the upper left corner ordinate of the second adjustment area is set to 0; When the horizontal coordinate of the lower right corner of the initial second adjustment area is greater than the width of the display screen, the horizontal coordinate of the lower right corner of the second adjustment area is set to the width of the display screen; When the lower right corner ordinate of the initial second adjustment area is greater than the width of the display screen, the lower right corner ordinate of the second adjustment area is set to the height of the display screen; When the horizontal coordinate of the upper left corner of the initial second adjustment area and the vertical coordinate of the upper left corner of the initial second adjustment area are both greater than or equal to 0, the horizontal coordinate of the lower right corner of the initial second adjustment area is less than or equal to the width of the display screen, and the vertical coordinate of the lower right corner of the initial second adjustment area is less than or equal to the height of the display screen, the initial second adjustment area information is the second adjustment area information.

12. The method for coordinated brightness adjustment of multiple regions based on user face position and touch point position according to claim 1, characterized in that: The obtaining of a new brightness setting parameter by a brightness adjustment algorithm according to the first adjustment area information and the second adjustment area information comprises: According to user preference, a brightness adjustment algorithm is used to customize and initialize the brightness adjustment parameters, wherein the brightness adjustment parameters include a brightness increase / decrease ratio and a brightness adjustment speed; According to the brightness adjustment parameter, the first adjustment area information, and the second adjustment area information, the first adjustment area brightness adjustment amount and the second adjustment area brightness adjustment amount are respectively obtained by adjusting amount calculation, and the expression for calculating the adjustment amount is: A=B*△ Wherein, A is the brightness adjustment amount of the area, B is the current brightness of the area, and Δ is the brightness increase or decrease ratio; Obtaining overlapping area information by taking the intersection of the first adjustment area information and the second adjustment area information, and obtaining overlapping area brightness adjustment amount by using a brightness balance method according to the overlapping area information, specifically including obtaining overlapping area brightness adjustment amount by taking the average of the first adjustment area brightness adjustment amount and the second adjustment area brightness adjustment amount; According to the brightness adjustment amount of the first adjustment area, the brightness adjustment amount of the second adjustment area, and the brightness adjustment amount of the overlapping area, the new brightness setting parameter of the first adjustment area, the new brightness setting parameter of the second adjustment area, and the new brightness setting parameter of the overlapping area are obtained through brightness calculation, and the expression of the brightness calculation is: B neW =B+A Among them, A is the brightness adjustment amount of the area, B is the current brightness of the area, and B new Set parameters for the new brightness of the area.

13. A system for coordinating brightness of multiple regions based on user face position and touch point position using the method for coordinating brightness of multiple regions based on user face position and touch point position according to claim 1, comprising a system initialization module, a real-time detection and response module, a brightness adjustment feedback module, and a user prompt and adjustment module, characterized in that: The system initialization module is used when a user uses the device for the first time. The system prompts the user to enable the camera and touch permissions for initial settings, including camera calibration and touch sensitivity adjustment. The real-time detection and response module is used to monitor the user's face position and the user's display touch operation in real time when the system is running in the background. When the user approaches the device, the camera automatically captures the user's face image data and detects the face area to obtain the user's face position information; When the user performs a touch operation on the display screen, the touch sensor automatically collects and detects the touch data to obtain the touch point location information; The brightness adjustment feedback module is used when the user operates the device. The system automatically adjusts the display brightness according to the touch point location information and the user's face location information. The brightness change will be displayed on the screen in real time, and the user can directly see the brightness adjustment result; The user prompt and adjustment module is used to display prompt information on the screen to guide the user to make adjustments or repairs when the system detects abnormal conditions, such as camera obstruction and touch sensor failure.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the multi-zone brightness collaborative adjustment method based on the user face position and the touch point position as described in any one of claims 1-13 is implemented.

15. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the multi-zone brightness collaborative adjustment method based on the user face position and the touch point position as described in any one of claims 1-13.

Images