Method, apparatus and electronic device for distance measurement between human eye and display device

Abstract

This application provides a method, apparatus, electronic device, and storage medium for measuring the distance between the human eye and a display device. It obtains usage data from the display screen, determines whether the usage data meets the conditions for binocular ranging, and if so, uses a binocular ranging method to measure the distance between the human eye and the display device; otherwise, it uses a monocular ranging method. By analyzing the usage data of the display screen, it determines whether the usage data meets the conditions for binocular ranging. If the conditions are met, a more accurate binocular ranging method is used to measure the distance between the human eye and the display device; if the conditions are not met, a shorter monocular ranging method is used. This targeted processing of the distance measurement between the human eye and the display device enables real-time and accurate acquisition of distance measurement results, improving the accuracy and efficiency of distance measurement between the human eye and the display device.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for measuring distance between the human eye and a display device. Background Technology

[0002] Currently, with the widespread use of electronic devices such as computers, smartphones, and tablets, people's work, entertainment, and daily life heavily rely on electronic screens, i.e., display screens. When the human eye is too close to the display device, it can cause vision damage. In particular, when teenagers do not pay attention to maintaining the proper distance between their eyes and the display screen, it can easily lead to myopia or worsen existing myopia. Summary of the Invention

[0003] Based on this, this application provides a method, apparatus, electronic device, and storage medium for measuring distance between the human eye and a display device, which can improve the accuracy and efficiency of distance measurement between the human eye and the display device.

[0004] As a first aspect of the present application, a method for measuring the distance between a human eye and a display device is provided, comprising the following steps:

[0005] Monitor the usage data of the display device and determine whether the usage data meets the binocular ranging conditions;

[0006] Determine whether the data used meets the binocular ranging conditions. If it does, obtain the first image and the second image of the user to be tested captured by the two cameras on the display device. Perform human eye feature point detection on the first image and the second image respectively to obtain the first feature point coordinates and the second feature point coordinates corresponding to the same human eye feature point. Obtain the ranging result based on the first feature point coordinates and the second feature point coordinates.

[0007] If the conditions are not met, a third image of the user to be tested is obtained from a camera on the display device. Human eye feature detection is performed on the third image to obtain the coordinates of the left eye feature point and the right eye feature point. The distance measurement result is obtained based on the coordinates of the left eye feature point and the right eye feature point.

[0008] As a second aspect of this application, a ranging device between a human eye and a display device is provided, comprising:

[0009] The monitoring module is used to monitor the usage data of the display device and determine whether the usage data meets the binocular ranging conditions.

[0010] The first ranging module is used to determine whether the data used meets the binocular ranging conditions. If it does, it obtains the first image and the second image of the user to be measured captured by the two cameras on the display device. It performs human eye feature point detection on the first image and the second image respectively, obtains the first feature point coordinates and the second feature point coordinates corresponding to the same human eye feature point, and obtains the ranging result based on the first feature point coordinates and the second feature point coordinates.

[0011] The second ranging module is used to obtain a third image of the user to be measured taken by a camera on the display device if the conditions are not met, perform human eye feature detection on the third image to obtain the coordinates of the left eye feature point and the right eye feature point, and obtain the ranging result based on the coordinates of the left eye feature point and the right eye feature point.

[0012] As a third aspect of the present application, an electronic device is provided, including a processor, a memory, and a computer program stored in the memory and executable on the processor; when executed by the processor, the computer program implements the steps of the distance measurement method between the human eye and a display device as described in the first aspect.

[0013] As a fourth aspect of the present application, a storage medium is provided, the storage medium storing a computer program, which, when executed by a processor, implements the steps of the distance measurement method between the human eye and a display device as described in the first aspect.

[0014] This application embodiment analyzes the usage data of the display device to determine whether the usage data meets the conditions for binocular ranging. If the conditions are met, a highly accurate binocular ranging method is used to measure the distance between the human eye and the display device. If the conditions are not met, a shorter monocular ranging method is used to measure the distance between the human eye and the display device. This targeted approach to distance measurement between the human eye and the display device enables real-time and accurate acquisition of distance measurement results, improving the accuracy and efficiency of distance measurement between the human eye and the display device.

[0015] To better understand and implement this application, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0016] Figure 1 This provides the application environment for the distance measurement method between the human eye and the display device provided in the first embodiment of this application;

[0017] Figure 2 A schematic flowchart illustrating the distance measurement method between the human eye and a display device provided in the first embodiment of this application;

[0018] Figure 3This is a flowchart illustrating step S1 of the distance measurement method between the human eye and a display device provided in the first embodiment of this application.

[0019] Figure 4 This is a flowchart illustrating step S1 of the distance measurement method between the human eye and a display device provided in the second embodiment of this application.

[0020] Figure 5 This is a flowchart illustrating step S1 of the distance measurement method between the human eye and a display device provided in the third embodiment of this application.

[0021] Figure 6 This is a flowchart illustrating step S2 of the distance measurement method between the human eye and a display device provided in the first embodiment of this application.

[0022] Figure 7 This is a flowchart illustrating step S21 of the distance measurement method between the human eye and a display device provided in the first embodiment of this application.

[0023] Figure 8 This is a flowchart illustrating step S3 of the distance measurement method between the human eye and a display device provided in the first embodiment of this application.

[0024] Figure 9 This is a flowchart illustrating step S33 of the distance measurement method between the human eye and a display device provided in the fourth embodiment of this application.

[0025] Figure 10 A schematic flowchart illustrating the distance measurement method between the human eye and a display device provided in the fifth embodiment of this application;

[0026] Figure 11 This is a schematic diagram of the structure of the distance measuring device between the human eye and the display device provided in the sixth embodiment of this application;

[0027] Figure 12 This is a schematic diagram of the structure of an electronic device provided in the seventh embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Wherein, when the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0029] It should be understood that the embodiments described below do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0030] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, in the description of this application, unless otherwise stated, “a plurality” means two or more. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the listed items, for example, A and / or B, which can represent: A alone, A and B together, and B alone; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0031] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms, and these terms are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Depending on the context, the word "if" as used in this application can be interpreted as "when," "when," or "in response to determination."

[0032] Please see Figure 1 , Figure 1 The application environment for the distance measurement method between the human eye and a display device provided in the first embodiment of this application includes a display device 10 and a distance measuring device (not shown). The display device 10 is equipped with at least two cameras 20; the two cameras 20 are used to capture images of the user to be measured. The distance measuring device is connected to the display device 10.

[0033] In an optional embodiment, the display device 10 is connected to an external ranging device. The ranging device is used to execute the ranging method between the human eye and the display device of this application. In this case, the hardware pointed to by the ranging device essentially refers to computer equipment. For example, the ranging method device between the human eye and the display device can be a computer, mobile phone, tablet or smart interactive flat panel and other smart devices.

[0034] In another optional embodiment, the display device 10 is provided with a ranging device, which is used to perform the ranging method between the human eye and the display device of this application. In this case, the hardware pointed to by the ranging device can be a microcontroller (MCU) or the like.

[0035] During the user's use of the display device, the user faces the display device 10, and the ranging device controls the camera 11 on the display device 10 to take a picture of the user to be measured, thereby obtaining an image of the user to be measured. The ranging device then performs ranging processing on the image to obtain the ranging result between the user's eye and the electronic screen.

[0036] Please see Figure 2 , Figure 2 The flowchart illustrates the distance measurement method between the human eye and a display device provided in the first embodiment of this application. The method includes the following steps:

[0037] S1: Monitor the usage data of the display device and determine whether the usage data meets the conditions for binocular ranging.

[0038] Compared to monocular ranging, binocular ranging has higher accuracy and applicability, but it is time-consuming and resource-intensive. Therefore, in this embodiment, the ranging device acquires the usage data of the display device at preset time intervals, determines whether the usage data meets the binocular ranging conditions, and adopts a more suitable ranging scheme based on the current usage situation.

[0039] Usage data is used to indicate the current usage status of the display device. Usage data may include power consumption data, camera data, remaining battery power data, and resource usage data, etc.

[0040] In an optional embodiment, the usage data may include power consumption data, which reflects the power consumption of the display device. See also... Figure 3 , Figure 3 The flowchart of step S1 in the distance measurement method between the human eye and the display device provided in the first embodiment of this application is shown below, including step S101:

[0041] S101: If the power consumption data is less than the preset power consumption threshold, it is determined that the data used meets the binocular ranging conditions; if the power consumption data is greater than or equal to the preset power consumption threshold, it is determined that the data used does not meet the binocular ranging conditions.

[0042] Power consumption data can reflect the current resource usage of the display device. It is understandable that in actual use, especially when the ranging device is a microcontroller built into the display device and is part of the display device, the ranging device may also perform image display processing and data processing of other applications in addition to executing the ranging method of this application. This may lead to insufficient resources allocated to the ranging method, resulting in the inability to perform binocular ranging. Therefore, in order to perform effective ranging, it is necessary to judge the power consumption data.

[0043] Specifically, if the power consumption data is less than the preset power consumption threshold, it means that the display device can provide sufficient power resources for binocular ranging. Therefore, the ranging process between the human eye and the display device can be performed in real time, improving the accuracy of the ranging. If the power consumption data is greater than or equal to the preset power consumption threshold, it means that the ranging device cannot provide sufficient power for binocular ranging. If binocular ranging is forcibly used, the ranging process between the human eye and the display device cannot be performed in a timely manner, and there is even a possibility that the ranging device may freeze or be damaged.

[0044] This application embodiment achieves a balance between power consumption control of the display device and distance measurement processing between the human eye and the display device by setting a power consumption threshold.

[0045] In another alternative embodiment, the usage data may include camera data, which reflects the usage of the camera on the display device, such as whether the user is making a video call or live streaming, etc. See also Figure 4 , Figure 4 The flowchart of step S1 in the distance measurement method between the human eye and the display device provided in the second embodiment of this application is shown below, including steps S111 to S112:

[0046] S111: Based on camera data, obtain the number of cameras that can be used for distance measurement.

[0047] In this embodiment, the ranging device obtains the number of cameras on the display device that can be accessed based on camera data, thus avoiding interference with the user's current camera usage.

[0048] S112: If the number of cameras allowed to be used for ranging is two or more, determine whether the data used meets the binocular ranging conditions. If the number of cameras allowed to be used for ranging is less than two, determine that the monitoring data used does not meet the binocular ranging conditions.

[0049] When power consumption control of the display device is not implemented, the ranging device needs to perform binocular ranging through two cameras to obtain more accurate ranging results while ensuring normal user operation. In this embodiment, if the number of cameras allowed to be used for ranging is two or more, it indicates that normal binocular ranging processing can be performed, and the ranging device determines whether the data used meets the binocular ranging conditions. If the number of cameras allowed to be used for ranging is less than two, it indicates that normal binocular ranging processing cannot be performed, and the ranging device determines that the monitoring data used does not meet the binocular ranging conditions.

[0050] In yet another embodiment, please refer to Figure 5 , Figure 5The flowchart of step S1 in the distance measurement method between the human eye and the display device provided in the third embodiment of this application includes steps S121 to S122, as follows:

[0051] S121: If the power consumption data is less than the preset power consumption threshold, obtain the number of cameras that can be called for ranging based on the camera data.

[0052] S122: If the number of cameras allowed to be used for ranging is two or more, determine whether the data used meets the binocular ranging conditions. If the number of cameras allowed to be used for ranging is less than two, determine that the monitoring data used does not meet the binocular ranging conditions.

[0053] In this embodiment, if the power consumption data is less than a preset power consumption threshold, the ranging device obtains the number of cameras allowed to be called for ranging based on the camera data. If the number of cameras allowed to be called for ranging is two or more, it indicates that the ranging device can provide sufficient power consumption for binocular ranging, and then determines whether the data used meets the binocular ranging conditions. If the number of cameras allowed to be called for ranging is less than two, it is determined that the monitoring data used does not meet the binocular ranging conditions.

[0054] By combining power consumption data and camera data, and when both data meet the requirements for binocular ranging, the ranging device uses a highly accurate binocular ranging method to measure the distance between the human eye and the display device. If the conditions are not met, a shorter monocular ranging method is used. This targeted approach to distance measurement between the human eye and the display device enables real-time and accurate measurement results, improving the accuracy and efficiency of distance measurement between the human eye and the display device.

[0055] S2: If satisfied, obtain the first image and the second image of the user to be tested captured by the two cameras on the display device, perform human eye feature point detection on the first image and the second image respectively, obtain the coordinates of the first feature point and the second feature point corresponding to the same human eye feature point, and obtain the distance measurement result based on the coordinates of the first feature point and the second feature point.

[0056] The coordinates of the first feature point and the coordinates of the second feature point are coordinates in the camera coordinate system. In an optional embodiment, the ranging device can perform human eye feature point detection on the first image and the second image respectively through a target detection algorithm to obtain the coordinates of the first feature point and the second feature point corresponding to the same human eye feature point.

[0057] In another optional embodiment, the ranging device performs human eye feature point detection on the first and second images respectively using a target detection model to obtain the coordinates of the first and second feature points corresponding to the same human eye feature point. Specifically, the target detection model can be a MediaPipe deep learning model or a YOLO model.

[0058] To obtain the coordinates of the first and second feature points corresponding to the same human eye feature point, the left eye feature point can be selected as the target feature point, and the coordinates of the target feature point in the first image and the second image can be used as the first and second feature point coordinates, respectively. Similarly, the right eye feature point can be selected as the target feature point, and the coordinates of the target feature point in the first image and the second image can be used as the first and second feature point coordinates, respectively.

[0059] Please see Figure 6 , Figure 6 The flowchart of step S2 in the distance measurement method between the human eye and the display device provided in the first embodiment of this application includes steps S21 to S22, as follows:

[0060] S21: Subtract the x-coordinate of the first feature point from the x-coordinate of the second feature point to obtain the disparity parameter between the first image and the second image.

[0061] The parallax parameter is used to indicate the directional difference produced when the same target is captured by two cameras on a display device.

[0062] Typically, the two cameras used to capture the first and second images are set at the same height. In order to reduce the amount of computation and improve the efficiency of ranging, in this embodiment, the ranging device subtracts the abscissa of the first feature point from the abscissa of the second feature point to obtain the disparity parameter between the first and second images.

[0063] S22: Obtain the ratio between the preset focal length parameter and the disparity parameter between the first image and the second image, multiply the ratio by the preset baseline parameter, and obtain the ranging result.

[0064] The baseline parameter is used to indicate the distance between the two cameras; specifically, it refers to the baseline on the ideal imaging plane.

[0065] In this embodiment, the ranging device obtains the ratio between a preset focal length parameter and the parallax parameter between the first image and the second image. This ratio is then multiplied by a preset baseline parameter to obtain a first depth parameter, which serves as the ranging result, as follows:

[0066] Z c，1 =B×f / Disp

[0067] In the formula, Z c，1 is the first depth parameter, B is the baseline parameter, f is the preset focal length parameter, and Disp is the parallax parameter.

[0068] This application embodiment first detects the coordinates of feature points corresponding to the same human eye feature point in the first image and the second image, calculates the disparity parameter, and uses the calculated disparity parameter to measure the distance between the human eye and the display device, thereby achieving timely and accurate distance measurement between the human eye and the display device.

[0069] Please see Figure 7 , Figure 7 The flowchart of step S21 in the distance measurement method between the human eye and the display device provided in the first embodiment of this application includes steps S211 to S212, as follows:

[0070] S211: According to the preset correction parameters, the coordinates of the first feature point and the second feature point are corrected respectively to obtain the corrected coordinates of the first feature point and the second feature point.

[0071] The correction parameters include rotation matrix, imaging matrix, binocular baseline, camera intrinsic matrix, and distortion coefficients.

[0072] To calculate the disparity parameter between the first and second images, it is necessary to match the corresponding feature points of the first and second images. For this purpose, a binocular calibration method is usually used to match the first and second images. However, image distortion is easily generated during the binocular calibration process, which leads to inaccurate image matching and reduces the accuracy of binocular ranging.

[0073] In this embodiment, the ranging device can employ a binocular ranging method to obtain correction parameters for the cameras corresponding to the first and second images. Based on these correction parameters, the coordinates of the first and second feature points are corrected. Specifically, the ranging device performs distortion correction and line alignment on the coordinates of the first and second feature points according to the correction parameters, obtaining the corrected coordinates of the first and second feature points to improve the accuracy of binocular ranging.

[0074] S212: Subtract the x-coordinate of the first feature point from the x-coordinate of the second feature point after correction to obtain the disparity parameter between the first image and the second image.

[0075] In this embodiment, the ranging device subtracts the abscissa of the first feature point coordinates from the abscissa of the second feature point coordinates after correction to obtain the disparity parameter between the first image and the second image, thereby improving the accuracy of binocular ranging.

[0076] This application embodiment first detects the coordinates of feature points corresponding to the same human eye feature point in the first image and the second image, calculates the disparity parameter, and uses the calculated disparity parameter to measure the distance between the human eye and the display device. This avoids the need for image correction and stereo matching of all pixels in the image in the depth estimation of classic binocular ranging, greatly improving the efficiency of binocular ranging and realizing real-time binocular ranging.

[0077] S3: If not satisfied, obtain a third image of the user to be tested taken by a camera on the display device, perform human eye feature detection on the third image, obtain the coordinates of the left eye feature point and the right eye feature point, and obtain the distance measurement result based on the coordinates of the left eye feature point and the right eye feature point.

[0078] The coordinates of the left and right eye feature points are in the camera coordinate system. In this embodiment, if the conditions are not met, the ranging device obtains a third image of the user being tested captured by a camera on the display device, performs human eye feature detection on the third image, and obtains the coordinates of the left and right eye feature points.

[0079] Please see Figure 8 , Figure 8 The flowchart of step S3 in the distance measurement method between the human eye and the display device provided in the first embodiment of this application includes steps S31 to S32, as follows:

[0080] S31: Obtain the first horizontal axis spatial transformation parameter based on the horizontal coordinates of the left eye feature point, the horizontal coordinates of the right eye feature point, and the preset horizontal axis focal length parameter; obtain the first vertical axis spatial transformation parameter based on the vertical coordinates of the left eye feature point, the vertical coordinates of the right eye feature point, and the preset vertical axis focal length parameter.

[0081] The horizontal axis focal length parameter and the vertical axis focal length parameter are used to represent the focal length of the camera lens on the horizontal axis and the focal length on the vertical axis, respectively.

[0082] In this embodiment, the ranging device obtains the first horizontal axis spatial transformation parameters based on the horizontal coordinates of the left eye feature point, the horizontal coordinates of the right eye feature point, and the preset horizontal axis focal length parameters, as follows:

[0083]

[0084] In the formula, H1 is the first horizontal axis spatial transformation parameter, u1 is the horizontal coordinate of the left eye feature point corresponding to the first image, u2 is the horizontal coordinate of the right eye feature point corresponding to the first image, and f x This is the preset horizontal axis focal length parameter.

[0085] The ranging device obtains the first vertical axis spatial transformation parameters based on the ordinates of the left and right eye feature points and the preset vertical axis focal length parameters, as follows:

[0086]

[0087] In the formula, Y1 is the first vertical axis spatial transformation parameter, v1 is the vertical coordinate of the left eye feature point corresponding to the first image, v2 is the vertical coordinate of the right eye feature point corresponding to the first image, and f y This is the preset vertical axis focal length parameter.

[0088] S32: Obtain the distance measurement result based on the first horizontal axis spatial transformation parameters, the first vertical axis spatial transformation parameters, and the preset prior distance parameters.

[0089] The prior distance parameter is used to indicate the true distance between the left-eye feature point and the right-eye feature point.

[0090] In this embodiment, the ranging device obtains a second depth parameter as the ranging result based on the first horizontal axis spatial transformation parameter, the first vertical axis spatial transformation parameter, and a preset prior distance parameter, as follows:

[0091]

[0092] In the formula, Z c，2 d is the first depth parameter, and d is the prior distance parameter.

[0093] Based on pre-set prior distance parameters, the ranging device can determine the coordinates of the left and right eye feature points using only a third image acquired by a single camera. In the embodiments of this application, the ranging device, by pre-setting prior distance parameters, adopts a prior knowledge-based approach to measure distances based on the actual size of the target object and its size in the image. That is, the ranging device in this application can achieve distance measurement between the human eye and the display device using only a third image acquired by a single camera, thus improving the efficiency of distance measurement.

[0094] When different users use the display device, their prior distance parameters differ. Using the prior distance parameters of a user using a different device than the one currently using the display device for distance measurement between the human eye and the display device will lead to significant measurement errors and inaccurate results. To address this issue, in an optional embodiment, step S33 is included: updating the prior distance parameters. Please refer to [link to relevant documentation]. Figure 9 , Figure 9 The flowchart of step S33 in the distance measurement method between the human eye and the display device provided in the fourth embodiment of this application is shown below. It also includes steps S331 to S334, as follows:

[0095] S331: In response to the prior distance parameter update command, obtain the current usage data of the display device.

[0096] The prior distance parameter update command can be issued by the user or generated by the ranging device according to a preset time interval.

[0097] In this embodiment, the ranging device responds to the prior distance parameter update command and obtains the current usage data of the display device.

[0098] S332: If the data used at the current moment meets the binocular ranging conditions, the coordinates of the first feature point, the coordinates of the second feature point, and the corresponding ranging results at the current moment are respectively used as the first sampling coordinates, the second sampling coordinates, and the sampling ranging results; if the data used at the current moment does not meet the binocular ranging conditions, the coordinates of the first feature point, the coordinates of the second feature point, and the corresponding ranging results at the time closest to the current moment when the binocular ranging conditions are met are respectively used as the first sampling coordinates, the second sampling coordinates, and the sampling ranging results.

[0099] In this embodiment, if the usage data at the current moment meets the binocular ranging conditions, the ranging device uses the coordinates of the first feature point, the coordinates of the second feature point, and the corresponding ranging result at the current moment as the first sampling coordinates, the second sampling coordinates, and the sampling ranging result, respectively. If the usage data at the current moment does not meet the binocular ranging conditions, the coordinates of the first feature point, the coordinates of the second feature point, and the corresponding ranging result at the moment closest to the current moment when the binocular ranging conditions are met are used as the first sampling coordinates, the second sampling coordinates, and the sampling ranging result, respectively, to confirm that the user currently using the display device is the same user.

[0100] S333: Obtain the second horizontal axis spatial transformation parameter based on the horizontal coordinate in the first sampling coordinate, the horizontal coordinate in the second sampling coordinate, and the preset horizontal axis focal length parameter; obtain the second vertical axis spatial transformation parameter based on the vertical coordinate in the first sampling coordinate, the vertical coordinate in the second sampling coordinate, and the preset vertical axis focal length parameter.

[0101] In this embodiment, the ranging device obtains the second horizontal axis spatial transformation parameters based on the horizontal coordinates in the first sampling coordinates, the horizontal coordinates in the second sampling coordinates, and the preset horizontal axis focal length parameters, as follows:

[0102]

[0103] In the formula, H2 is the first horizontal axis spatial transformation parameter, U1 is the horizontal coordinate in the first sampling coordinate, and U2 is the horizontal coordinate in the second sampling coordinate.

[0104] The ranging device obtains the second vertical axis spatial transformation parameters based on the vertical coordinates of the first and second sampling coordinates, as well as the preset vertical axis focal length parameters, as follows:

[0105]

[0106] In the formula, Y2 is the first vertical axis spatial transformation parameter, V1 is the vertical coordinate in the first sampling coordinate, and V2 is the vertical coordinate in the second sampling coordinate.

[0107] S334: Based on the second horizontal axis spatial transformation parameters, the second vertical axis spatial transformation parameters, and the distance measurement results at the current moment, obtain the updated prior distance parameters and update the prior distance parameters.

[0108] In this embodiment, the ranging device obtains the updated prior distance parameters based on the second horizontal axis spatial transformation parameters, the second vertical axis spatial transformation parameters, and the ranging result at the current moment, and updates the prior distance parameters as follows:

[0109]

[0110] In the formula, d' is the updated prior distance parameter, and Z' c This represents the distance measurement result at the current moment.

[0111] By continuously updating the prior distance parameters, it is possible to not only perform more accurate distance measurement between the human eye and the display device for multiple users, but also to achieve more accurate distance measurement in real time when the data does not meet the binocular distance measurement conditions, i.e., when the display device switches from binocular distance measurement to monocular distance measurement in high power consumption scenarios, thus obtaining more accurate distance measurement results.

[0112] Please see Figure 10 , Figure 10 The flowchart of the distance measurement method between the human eye and the display device provided in the fifth embodiment of this application also includes step S4, as follows:

[0113] S4: When the ranging result is less than the preset ranging threshold, control the display device to issue a reminder.

[0114] In this embodiment, the ranging device controls the display device to perform a reminder operation if the ranging result is less than or equal to the ranging threshold, based on the ranging result and a preset ranging threshold. If the ranging result is greater than the ranging threshold, no reminder operation is performed. This can promptly remind the user that the current distance to the display screen is too close, reducing the risk of the user developing myopia or worsening myopia.

[0115] Please refer to Figure 11 , Figure 11 This is a schematic diagram of the structure of a distance measuring device between the human eye and a display device provided in the sixth embodiment of this application. This device can realize all or part of the distance measuring method between the human eye and the display device through software, hardware, or a combination of both. The distance measuring device 11 between the human eye and the display device includes:

[0116] Data monitoring module 111 is used to monitor the usage data of the display device and determine whether the usage data meets the binocular ranging conditions;

[0117] The first ranging module 112 is used to obtain a first image and a second image of the user to be measured captured by two cameras on the display device if the conditions are met, perform human eye feature point detection on the first image and the second image respectively, obtain the coordinates of the first feature point and the second feature point corresponding to the same human eye feature point, and obtain the ranging result based on the coordinates of the first feature point and the second feature point.

[0118] The second ranging module 113 is used to obtain a third image of the user to be measured taken by a camera on the display device if the conditions are not met, perform human eye feature detection on the third image, obtain the coordinates of the left eye feature point and the right eye feature point, and obtain the ranging result based on the coordinates of the left eye feature point and the right eye feature point.

[0119] The data monitoring module 111 includes a first sub-data monitoring module, a second sub-data monitoring module, and a third sub-data monitoring module.

[0120] The first sub-data monitoring module is used to determine if the power consumption data is less than the preset power consumption threshold, and to determine if the power consumption data does not meet the binocular ranging conditions if the power consumption data is greater than or equal to the preset power consumption threshold.

[0121] The second sub-data monitoring module is used to obtain the number of cameras that can be called for distance measurement based on camera data; if the number of cameras that can be called for distance measurement is two or more, it determines whether the data used meets the binocular distance measurement conditions; if the number of cameras that can be called for distance measurement is less than two, it determines that the data used does not meet the binocular distance measurement conditions.

[0122] The third sub-data monitoring module is used to obtain the number of cameras allowed to be called for ranging based on the camera data if the power consumption data is less than the preset power consumption threshold; if the number of cameras allowed to be called for ranging is two or more, it determines whether the data used meets the binocular ranging conditions; if the number of cameras allowed to be called for ranging is less than two, it determines that the monitoring data used does not meet the binocular ranging conditions.

[0123] The first ranging module 112 includes a disparity calculation module and a first sub-ranging module.

[0124] The disparity calculation module is used to subtract the abscissa of the first feature point from the abscissa of the second feature point to obtain the disparity parameter between the first image and the second image.

[0125] The first sub-ranging module is used to obtain the ratio between the preset focal length parameter and the parallax parameter between the first image and the second image, and multiply the ratio by the preset baseline parameter to obtain the ranging result; wherein, the baseline parameter is used to indicate the distance between the two cameras.

[0126] The disparity calculation module includes a correction module and a sub-disparity calculation module.

[0127] The correction module is used to correct the coordinates of the first feature point and the second feature point according to preset correction parameters, so as to obtain the corrected coordinates of the first feature point and the corrected coordinates of the second feature point; wherein, the correction parameters include rotation matrix, imaging matrix, binocular baseline, camera intrinsic parameter matrix and distortion coefficient;

[0128] The sub-disparity calculation module is used to subtract the x-coordinate of the corrected first feature point coordinates from the x-coordinate of the corrected second feature point coordinates to obtain the disparity parameter between the first image and the second image.

[0129] The second ranging module 113 includes a spatial transformation module and a second sub-ranging module.

[0130] The spatial transformation module is used to obtain the first horizontal axis spatial transformation parameters based on the horizontal coordinates of the left eye feature point coordinates, the horizontal coordinates of the right eye feature point coordinates, and the preset horizontal axis focal length parameters; and to obtain the first vertical axis spatial transformation parameters based on the vertical coordinates of the left eye feature point coordinates, the vertical coordinates of the right eye feature point coordinates, and the preset vertical axis focal length parameters.

[0131] The second sub-range measurement module is used to obtain the range measurement result based on the first horizontal axis spatial transformation parameter, the first vertical axis spatial transformation parameter, and the preset prior distance parameter; wherein, the prior distance parameter is used to indicate the true distance between the left eye feature point and the right eye feature point.

[0132] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of an electronic device provided in the seventh embodiment of this application. This application also provides an electronic device, including: a processor 121, a memory 122, and a computer program 123 stored in the memory 122 and executable on the processor; the electronic device can store multiple instructions, which are applicable to being loaded by the processor and executed by the method steps of embodiments one to five above. For the specific execution process, please refer to the detailed descriptions of embodiments one to five, which will not be repeated here.

[0133] The processor may include one or more processing cores. The processor 121 connects to various parts within the electronic device using various interfaces and lines. It executes instructions, programs, code sets, or instruction sets stored in the memory 122, and calls data from the memory 122 to perform various functions and process data of the ranging device 11 between the human eye and the display device. Optionally, the processor 121 may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 121 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 121 and may be implemented as a separate chip.

[0134] The memory 122 may include random access memory (RAM) or read-only memory. Optionally, the memory 122 may include a non-transitory computer-readable storage medium. The memory 122 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 122 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 122 may also be at least one storage device located remotely from the aforementioned processor 121.

[0135] This application also provides a storage medium that can store multiple instructions. The instructions are applicable to being loaded and executed by a processor using the method steps of Embodiments 1 to 5 described above. For the specific execution process, please refer to the detailed descriptions of Embodiments 1 to 5, which will not be repeated here.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0137] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0138] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0139] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0142] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0143] This application is not limited to the above-described embodiments. If any modifications or variations to this application do not depart from the spirit and scope of this application, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this application, then this application also intends to include such modifications and variations.

Claims

1. A method of ranging between a human eye and a display device, the method comprising: The method comprises the following steps: monitoring use data of the display device to determine whether the use data meets binocular distance measurement conditions; wherein the use data comprises power consumption data and camera data; the power consumption data is used to reflect the resource use of the current display device; the resource use is used to reflect whether the allocated resources can achieve binocular distance measurement; the camera data is used to reflect the use of the camera on the display device; the use of the camera is used to reflect whether the allowed camera call can achieve binocular distance measurement; if the conditions are met, obtaining a first image and a second image of a user to be measured photographed by two cameras on the display device; performing human eye feature point detection on the first image and the second image respectively to obtain first feature point coordinates and second feature point coordinates corresponding to the same human eye feature points; and obtaining a distance measurement result according to the first feature point coordinates and the second feature point coordinates; if the conditions are not met, obtaining a third image of a user to be measured photographed by one camera on the display device; performing human eye feature detection on the third image to obtain left eye feature point coordinates and right eye feature point coordinates; and obtaining a distance measurement result according to the left eye feature point coordinates and the right eye feature point coordinates.

2. The distance measurement method between the human eye and the display device according to claim 1, characterized in that: the step of determining whether the use data meets binocular distance measurement conditions comprises the steps of: if the power consumption data is less than a preset power consumption threshold, it is determined that the use data meets binocular distance measurement conditions; and if the power consumption data is greater than or equal to the preset power consumption threshold, it is determined that the use data does not meet binocular distance measurement conditions.

3. The distance measurement method between the human eye and the display device according to claim 1, characterized in that: the step of determining whether the use data meets binocular distance measurement conditions comprises the steps of: obtaining the number of cameras allowed to be called for distance measurement according to the camera data; if the number of cameras allowed to be called for distance measurement is two or more, it is determined whether the use data meets binocular distance measurement conditions; and if the number of cameras allowed to be called for distance measurement is less than two, it is determined that the use data does not meet binocular distance measurement conditions.

4. The distance measurement method between the human eye and the display device according to any one of claims 1 to 3, characterized in that: the step of obtaining a distance measurement result according to the first feature point coordinates and the second feature point coordinates comprises the steps of: subtracting the abscissa in the first feature point coordinates from the abscissa in the second feature point coordinates to obtain a parallax parameter between the first image and the second image; obtaining the ratio between a preset focal length parameter and the parallax parameter between the first image and the second image, multiplying the ratio by a preset baseline parameter to obtain a distance measurement result; wherein the baseline parameter is used to indicate the distance between the two cameras.

5. The distance measurement method between the human eye and the display device according to claim 4, characterized in that: The step of subtracting the horizontal coordinate in the first feature point coordinate from the horizontal coordinate in the second feature point coordinate to obtain the disparity parameter between the first image and the second image comprises the steps of: According to the preset correction parameter, the first feature point coordinate and the second feature point coordinate are respectively corrected to obtain the corrected first feature point coordinate and the corrected second feature point coordinate; wherein the correction parameter comprises a rotation matrix, a camera matrix, a binocular baseline, a camera intrinsic parameter matrix and a distortion coefficient; Subtract the horizontal coordinate in the corrected first feature point coordinate from the horizontal coordinate in the corrected second feature point coordinate to obtain the disparity parameter between the first image and the second image.

6. The distance measurement method between the human eye and the display device according to any one of claims 1 to 3, characterized in that: The step of obtaining the distance measurement result according to the left eye feature point coordinate and the right eye feature point coordinate comprises the steps of: According to the horizontal coordinate in the left eye feature point coordinate, the horizontal coordinate in the right eye feature point coordinate and the preset horizontal axis focal length parameter, a first horizontal axis space transformation parameter is obtained; According to the vertical coordinate in the left eye feature point coordinate, the vertical coordinate in the right eye feature point coordinate and the preset vertical axis focal length parameter, a first vertical axis space transformation parameter is obtained; According to the first horizontal axis space transformation parameter, the first vertical axis space transformation parameter and the preset prior distance parameter, a distance measurement result is obtained; wherein the prior distance parameter is used to indicate the real distance between the left eye feature point and the right eye feature point.

7. The method of ranging between a human eye and a display device of claim 6, wherein, Further comprising the steps of: In response to the prior distance parameter update instruction, the use data of the display device at the current time is obtained; If the use data at the current time meets the binocular distance measurement condition, the first feature point coordinate, the second feature point coordinate and the corresponding distance measurement result at the current time are taken as the first sampling coordinate, the second sampling coordinate and the sampling distance measurement result respectively; if the use data at the current time does not meet the binocular distance measurement condition, the first feature point coordinate, the second feature point coordinate and the corresponding distance measurement result when the binocular distance measurement condition is met at the time closest to the current time are taken as the first sampling coordinate, the second sampling coordinate and the sampling distance measurement result respectively; According to the horizontal coordinate in the first sampling coordinate, the horizontal coordinate in the second sampling coordinate and the preset horizontal axis focal length parameter, a second horizontal axis space transformation parameter is obtained; according to the vertical coordinate in the first sampling coordinate, the vertical coordinate in the second sampling coordinate and the preset vertical axis focal length parameter, a second vertical axis space transformation parameter is obtained; According to the second horizontal axis space transformation parameter, the second vertical axis space transformation parameter and the sampling distance measurement result, the prior distance parameter is updated.

8. A ranging device between a human eye and a display device, characterized in that, ​ A data monitoring module is configured to monitor usage data of the display device and determine whether the usage data satisfies binocular distance measurement conditions; wherein the usage data includes power consumption data and camera data; the power consumption data is used to reflect a resource usage condition of the current display device; the resource usage condition is used to reflect whether the allocated resources can achieve binocular distance measurement; the camera data is used to reflect a usage condition of a camera on the display device; and the usage condition of the camera is used to reflect whether the allowed camera call can achieve binocular distance measurement. A first distance measurement module is configured to, if the conditions are satisfied, obtain a first image and a second image of a to-be-measured user captured by two cameras on the display device, perform human eye feature point detection on the first image and the second image respectively, obtain first feature point coordinates and second feature point coordinates corresponding to the same human eye feature points, and obtain a distance measurement result according to the first feature point coordinates and the second feature point coordinates. A second distance measurement module is configured to, if the conditions are not satisfied, obtain a third image of a to-be-measured user captured by one camera on the display device, perform human eye feature detection on the third image, obtain left eye feature point coordinates and right eye feature point coordinates, and obtain a distance measurement result according to the left eye feature point coordinates and the right eye feature point coordinates.

9. An electronic device, comprising: Comprising: a processor, a memory, and a computer program stored on the memory and executable on the processor; when the computer program is executed by the processor, the steps of the distance measurement method between the human eye and the display device according to any one of claims 1 to 7 are implemented.

10. A storage medium characterized by: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the distance measurement method between the human eye and the display device according to any one of claims 1 to 7 are implemented.

Images