Display device and distance detection method based on portrait

Abstract

In some embodiments of the present application, a display device and a distance detection method based on a portrait are provided. The method can obtain a user image in real time during the operation of the display device, and input the user image into a first model and a second model respectively, so that the first model and the second model respectively calculate a first predicted distance and a second predicted distance using empirical modeling and deep learning. The first predicted distance and the second predicted distance are then weighted and summed to obtain a third predicted distance, so that display control is performed using the third predicted distance. The method can predict the distance between the user and the display device without obtaining lens parameters, and solve the third predicted distance based on empirical modeling and deep learning, which can improve the accuracy of distance detection.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a distance detection method based on a portrait. Background Art

[0002] Face detection is a target detection technology based on image recognition, which can be used as an auxiliary interaction method for display devices. Face detection technology can be used to identify the user's usage status to respond to corresponding control actions. For example, the display device can use face detection technology to identify whether there is a human target in front of the screen. After identifying the human target, the distance between the human target and the display device is calculated. The screen brightness is then adjusted according to the distance control to alleviate the damage to the user's eyes caused by excessive screen brightness when using the display device at close range.

[0003] When the display device performs face detection, it needs to first obtain image data containing the face through a built-in or external camera. After obtaining the image data, it uses image recognition to extract the face target from the image data. Then, it calculates the distance between the user and the camera based on the camera lens parameters and the number of pixels occupied by the face target. Finally, it uses the distance to match the appropriate control method.

[0004] However, the above face detection process needs to call the camera lens parameters when calculating the distance between the user and the display device. If the camera lens parameters cannot be calibrated, the distance cannot be calculated, resulting in the face detection method not being universal. In addition, in the above face detection process, the calculation of the distance also depends on the corresponding pixel points occupied by the face target. When the user does not face the display device in a frontal posture, the corresponding pixel points occupied by the face target will change, resulting in the calculation result of the distance being affected by the deflection angle, reducing the accuracy of face detection. Summary of the invention

[0005] Some embodiments of the present application provide a display device and a distance detection method based on portraits to solve the problem of low distance calculation accuracy when performing face detection on traditional display devices.

[0006] On the one hand, some embodiments of the present application provide a display device, including: a display, an image acquisition interface, and a controller. The display is configured to display a user interface; the image acquisition interface is configured to acquire a user image; and the controller is configured to execute the following program steps:

[0007] Acquire the user image;

[0008] Based on the first model and the user image, a first predicted distance is obtained, and based on the second model and the user image, a second predicted distance is obtained; the first model is a distance prediction model based on empirical modeling; the second model is a distance prediction model based on deep learning;

[0009] Calculating a third predicted distance, the third predicted distance being a weighted sum of the first predicted distance and the second predicted distance; the weight in the calculation process of the weighted sum is set according to the first predicted distance;

[0010] Display parameters are adjusted according to the third predicted distance.

[0011] On the other hand, in some embodiments of the present application, a distance detection method based on a portrait of a display device is also provided, which is applied to the above-mentioned display device, and the distance detection method comprises the following steps:

[0012] Acquire the user image;

[0013] Based on the first model and the user image, a first predicted distance is obtained, and based on the second model and the user image, a second predicted distance is obtained; the first model is a distance prediction model based on empirical modeling; the second model is a distance prediction model based on deep learning;

[0014] Calculating a third predicted distance, the third predicted distance being a weighted sum of the first predicted distance and the second predicted distance; the weight in the calculation process of the weighted sum is set according to the first predicted distance;

[0015] Display parameters are adjusted according to the third predicted distance.

[0016] It can be seen from the above technical solutions that the display device and the distance detection method based on the portrait provided in some embodiments of the present application can obtain the user image in real time during the operation of the display device, and input the user image into the first model and the second model respectively, so that the first model and the second model respectively calculate the first predicted distance and the second predicted distance using empirical modeling and deep learning methods. Then, the first predicted distance and the second predicted distance are weighted and summed to obtain the third predicted distance, so as to perform display control using the third predicted distance. The method can predict the distance between the user and the display device without obtaining the lens parameters, and solve the third predicted distance based on empirical modeling and deep learning, which can improve the accuracy of distance detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram showing an application scenario of a display device in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of the hardware structure of a display device in an embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of the structure of a display device with a built-in image collector in an embodiment of the present application;

[0021] Figure 4 This is a schematic diagram of the structure of a display device connected to an external image collector in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a distance detection process of a display device in an embodiment of the present application;

[0023] Figure 6 This is a schematic diagram of the model prediction process in the embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the second model in the embodiment of the present application;

[0025] Figure 8 A schematic diagram of the display brightness adjustment process in an embodiment of the present application;

[0026] Fig. 9 This is a schematic diagram of the distance interval in the embodiment of the present application;

[0027] Fig.10 This is a schematic diagram of key points in the embodiments of this application;

[0028] Fig.11 This is a schematic diagram of a process for correcting the first pixel distance in an embodiment of the present application;

[0029] Fig.12 This is a schematic diagram of the process of monitoring focal length parameters in an embodiment of the present application;

[0030] Fig.13 This is a schematic diagram of the image recording process in the embodiment of the present application;

[0031] Fig.14 A schematic diagram of updating the first model process in an embodiment of the present application;

[0032] Fig.15 A schematic diagram of updating the second model flow chart in an embodiment of the present application;

[0033] Fig.16 This is a timing diagram of the distance detection method in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0035] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0036] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0037] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0038] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc.

[0039] Figure 1 FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the control device 300 or the control apparatus 100 .

[0040] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.

[0041] In some embodiments, a control device 300 (such as a mobile phone, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the control device 300.

[0042] In some embodiments, the display device 200 may not use the above-mentioned control device 300 or control apparatus 100 to receive instructions, but may receive user control through touch or gestures.

[0043] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the control device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0044] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0045] like Figure 2 As shown, the display device 200 may include at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0046] In some embodiments, the controller 250 may include a processor, a video processor, an audio processor, a graphic processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0047] The display 260 may include the following components, namely: a display screen component for presenting images; a driving component for driving image display; a component for receiving image signals output from the controller 250 to display video content, image content, and a menu control interface, as well as a component for user control UI interface, etc.

[0048] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0049] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0050] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0051] The detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures.

[0052] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0053] The tuner-demodulator 210 receives broadcast television signals by wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. In some embodiments, the controller 250 and the tuner-demodulator 210 can be located in different split devices, that is, the tuner-demodulator 210 can also be in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0054] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0055] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0056] In some embodiments, the display device 200 can detect the user's usage status during operation, and automatically adjust the operating parameters according to the usage status to adapt to the user's usage status. For example, the display device 200 can detect the distance between the user and the display device 200 in real time during the playback of media resources, and automatically reduce the display brightness of the picture when the distance is less than a set distance threshold.

[0057] The usage status may include various status data, such as interval distance, environmental parameters, usage time, cumulative viewing time, etc. The adjusted operating parameters may also include various parameter forms, such as display brightness, color mode, playback volume, etc.

[0058] In order to detect the user's usage status, the display device 200 can have a built-in or external detection device. The detection device can be a sensor that can detect certain specific parameters, or it can be a device that runs independently and supports a specific user interaction method. For example, the display device 200 can have a built-in laser radar based on the time of flight (TOF) principle, and the laser radar can detect the position of the user in front of the display device 200, that is, obtain the distance between the user and the display device 200.

[0059] Since some display devices 200 usually do not have built-in or external distance sensors, in order to perform distance detection, in some embodiments, the display device 200 can acquire user images and have built-in image processing algorithms to detect the distance between the user and the display device 200 by real-time detection of the user images in combination with the image processing algorithms.

[0060] The display device 200 may have a built-in or external image collector 231. The image collector 231 built into the display device 200 may be used as a collector 230 of the display device 200. Figure 3 2 shows a schematic diagram of a display device structure with a built-in image collector in some embodiments. For example, some display devices 200 may have built-in cameras, which may be set at the top or bottom frame of the display device 200, such as Figure 3The camera of the display device 200 can directly use the data bus inside the display device 200 to connect with other components to achieve data transmission.

[0061] In some embodiments, the camera built into the display device 200 can also support lifting. That is, the camera can be set on a lifting mechanism. When image acquisition is required, the lifting mechanism is controlled to move through a specific lifting instruction, thereby driving the camera to rise for image acquisition. When image acquisition is not required, the lifting mechanism can also be controlled to move through a specific lifting instruction, thereby driving the camera to lower to hide the camera.

[0062] The image acquisition device 231 externally connected to the display device 200 may be an independent peripheral device and connected to the display device 200 via a specific image acquisition interface. Figure 4 FIG. 4 shows a schematic diagram of a display device structure of an external image collector in some embodiments. Figure 4 As shown, the image collector 231 may be an independent camera device, and the display device 200 may be provided with a Universal Serial Bus (USB) interface or a High Definition Multimedia Interface (HDMI) interface, and the image collector 231 is connected to the display device 200 via the USB interface or the HDMI interface. In order to facilitate the detection of the user's gesture interaction actions, in some embodiments, the image collector 231 externally connected to the display device 200 may be set at a position close to the display device 200, such as the image collector 231 is clamped on the top of the display device 200 by a clamping device, or the image collector 231 is placed on a desktop near the display device 200.

[0063] Obviously, the image collector 231 externally connected to the display device 200 can also support other connection modes according to the specific hardware configuration of the display device 200. In some embodiments, the image collector 231 can also establish a connection relationship with the display device 200 through the communicator 220 of the display device 200, and send the collected image data to the display device 200 according to the data transmission protocol corresponding to the communicator 220. For example, the display device 200 can be connected to the image collector 231 through a local area network or the Internet. After the network connection is established, the image collector 231 can send the collected data to the display device 200 through the network transmission protocol.

[0064] In some embodiments, the image collector 231 can also be connected to the display device 200 via a wireless network. For example, for a display device 200 that supports a WiFi wireless network, a WiFi module is provided in its communicator 220. Therefore, the image collector 231 and the display device 200 can be connected to the same wireless network to establish a wireless connection between the display device 200 and the image collector 231. After the image collector 231 collects the image data, the image data can be sent to a router device of the wireless network, and then forwarded to the display device 200 by the router device.

[0065] Obviously, the image collector 231 can also directly establish a wireless connection relationship with the display device 200 through other wireless connection methods. For example, the image collector 231 is a magnetic camera and is provided with a Bluetooth connection module. The camera can be adsorbed on the housing of the display device 200 through magnetic attraction, and a Bluetooth connection can be established with the display device 200 through the Bluetooth connection module to transmit image data. Among them, the wireless connection method includes but is not limited to WiFi direct connection, cellular network, analog microwave, Bluetooth, infrared, etc.

[0066] In some embodiments, when the display device 200 collects user images for distance detection, it first obtains the lens parameters of the image collector 231 such as the camera, including the focal length, model, photosensitive film resolution, photosensitive film area, and distance between the photosensitive film and the lens, and then uses these lens parameters in combination with the lens imaging principle to calculate the distance between the user image collector 231. For example, the human eye positioning distance detection algorithm based on the camera lens parameters can calculate the human eye distance according to the following formula (1) to infer the distance between the face and the camera.

[0067] D=αR / r Formula (1)

[0068] Where D is the distance between the target person and the camera; R is the actual distance between the pupils of the human eye; r is the pixel distance between the pupils of the human eye in the image; and α is the lens parameter of the camera.

[0069] Since some display devices 200 obtain user images by means of an external image collector 231, the display device 200 may not be able to obtain the lens parameters of the image collector 231. At this time, the display device 200 can predict the distance between the user and the display device 200 based on a combination of empirical modeling and deep learning algorithms, that is, a distance detection method based on a portrait is provided in some embodiments, which can be applied to the display device 200, and in order to meet the implementation of the method, the display device 200 should at least include a display 260, an image acquisition interface, and a controller 250. Among them, the display 260 is used to display the user interface, and the image acquisition interface is used to connect to the built-in or external image collector 231 to collect user images. Figure 5 FIG. 2 shows a schematic diagram of a process flow of display device distance detection in some embodiments. Figure 5 As shown, the controller 250 is configured to execute program steps corresponding to the above distance detection method, including the following contents:

[0070] Acquire user images. The display device 200 can acquire user images in real time through the image acquisition interface after starting to run applications and system functions related to distance detection. In this process, the display device 200 can generate a control instruction for starting shooting after the user starts to run related programs or turns on system functions, and send the control instruction to the image collector 231 through the image acquisition interface. After receiving the control instruction, the image collector 231 can start running in response to the control instruction and capture images of a specific area. The captured images are sent to the controller 250 of the display device 200 through the image acquisition interface. When the user is located in the shooting area of ​​the image collector 231, the display device 200 can acquire an image containing the user's image, that is, acquire the user's image.

[0071] The user image refers to an image obtained by shooting that contains a user image. Since the image collector 231 can be located on the top of the display device 200, such as Figure 3 As shown, or located on one side of the display device 200, such as Figure 4 As shown, therefore, the shooting range of the image collector 231 is usually a specific range in front of the screen of the display device 200, that is, in some embodiments, the user can be located in front of the screen of the display device 200, and the image containing the user's image captured by the image collector 231 is called a user image.

[0072] The user image may be a multi-frame image data stream acquired at a set shooting frequency. The shooting frequency of the user image may be the same as the shooting frame rate of the image collector 231. For example, when the image collector 231 is a 30fps ordinary camera, the display device 200 may obtain a data stream containing 30 frames of images per second. When the image collector 231 is a 60fps high frame rate camera, the display device 200 may obtain a data stream containing 60 frames of images per second.

[0073] Since the purpose of distance detection is to obtain the user's usage status, and the user's usage status does not change frequently during the process of using the display device 200, in some embodiments, the display device 200 does not need to obtain a high-frame rate image data stream, and only needs to capture images of the corresponding number of frames through the image collector 231 according to the set shooting cycle. For example, when the user turns on the function of adjusting the display brightness according to the distance of the display device 200, the display device 200 can send a shooting instruction to the image collector 231 every 1 minute. After receiving the shooting instruction, the image collector 231 captures an image of the user and sends it to the display device 200.

[0074] For some image collectors 231 that can support interval photography control instructions, the display device 200 can also send an instruction to start interval photography to the image collector 231 after the user starts an application or function that requires distance detection, so that the image collector 231 can take a photo once at a predetermined interval according to the instruction and send the captured user image to the display device 200. The image collector 231 can continue to send the user image to the display device 200 according to the interval photography instruction until the display device 200 sends a stop photography instruction to the image collector 231.

[0075] After acquiring the user image, the display device 200 may also detect whether the user image contains the target content that is used in the subsequent distance detection algorithm. For example, if the subsequent image processing algorithm uses portraits or facial content, the display device 200 may determine whether the user image contains portraits or facial content. If the user image contains portraits or facial content, the display device 200 may detect the user distance according to the subsequent distance detection algorithm. If the user image does not contain portraits or facial content, the subsequent distance detection algorithm will not be able to accurately predict the user distance, so the user may be prompted to enter the camera's shooting area by displaying a prompt message.

[0076] Figure 6 A schematic diagram of the model prediction process in some embodiments is shown. Figure 6As shown, after acquiring the user image, the display device 200 can input the user image into the first model to obtain the first predicted distance based on the first model and the user image. That is, the distance detection algorithm corresponding to the first model is used to perform distance prediction to obtain the first predicted distance. Among them, the first model is a distance prediction model based on empirical modeling. The first model is a set of algorithm programs for empirical modeling, which may include algorithm programs and comparison tables for calculating various intermediate parameters. Therefore, when the display device 200 uses the first model for distance detection, it can first call the model algorithm, then calculate the intermediate value for matching the predicted distance according to the model algorithm, and then match the first predicted distance corresponding to the current user image in the distance comparison table according to the intermediate value.

[0077] Among them, empirical modeling is a correlation between the pixel distance corresponding to a specific key point in the user image and the interval distance obtained by data processing methods such as regression analysis based on empirical data. For example, the display device 200 can start the empirical modeling process after connecting the image collector 231. The user is prompted to be at a position of 0.2m, 0.4m, 0.6m, 0.8m, 1m, 1.2m...5.8m, 6m, etc. from the display device 200, and a photo is taken at each position, and the pixel points between the user's eyes in the image obtained by taking the photo are recorded. Through regression analysis, the functional relationship between the pixel points in the image and the interval distance can be determined to construct a first model.

[0078] like Figure 6 As shown, while inputting the user image into the first model, the display device 200 can also input the user image into the second model to obtain a second predicted distance based on the second model and the user image. The second model is a distance prediction model based on deep learning. That is, the second model is a pre-trained machine learning model, which can take the user image as input and the classification probability of the predicted distance as output, and calculate the predicted distance corresponding to the user image through a deep learning algorithm.

[0079] Before inputting the user image into the second model, the display device 200 may also pre-process the user image to uniformly adjust the user image into a feature map of a specific form and remove image content in the user image that is likely to interfere with the face detection process. In some embodiments, after acquiring the user image, the display device 200 may identify a face area from the user image, and then define a face selection frame based on the face area. The face selection frame is a rectangular frame covering the face area. The user image is then cropped according to the face selection frame, and the cropped user image is scaled to a preset image size.

[0080] For example, the display device 200 can use the YOLO-Fastest deep learning network algorithm to obtain the position information of the face, including the position coordinate information (X, Y) and the width and height of the face selection box after receiving the user image data captured by the camera. Then, the original image captured by the camera is cropped based on the position information and the selection box information, and scaled to a feature image of 224×224 size for input into the second model for subsequent distance prediction.

[0081] In some embodiments, the second model may include a sampling module and a convolution kernel module, wherein the sampling module is used to perform sampling on the user image to compress the user image into a feature map; and the convolution kernel module is used to perform a convolution operation on the feature map to obtain a classification probability. The display device 200 may first call the second model, and then input the user image into the second model, so as to perform an operation on the user image through the sampling module and the convolution kernel module in the second model, so as to output the distance classification probability corresponding to the face area in the user image.

[0082] For example, Figure 7 The second model schematic diagram in some embodiments is shown. The neural network backbone network framework built into the display device 200 is as follows Figure 7 The input-output network (InpoNet) model shown in the figure differs from the general backbone network framework in that the standard modules are different and the number of layers of the network at different downsampling stages is different. Compared with the 0.8GFlops computing power of the common backbone network framework MobileNet in the industry, the computing power of the InpoNet network architecture built into the display device 200 can be reduced to 0.5GFlops, thereby adapting to the hardware configuration of the display device 200.

[0083] According to different prediction results of the network end, the second model used in the display device 200 can be divided into two branches: InpoNet I and InpoNet II. Figure 7As shown, the standard module of InpoNet can sequentially run the convolution operation of 1×1 convolution kernel, batch normalization, leaky linear rectification (leakyRelu), depthwise separable convolution of 3×3 convolution kernel, convolution operation of 1×1 convolution kernel, batch normalization, leakyRelu, and finally add the module input and the final leakyRelu result to obtain the output. In the neural network model applied in the display device 200, the downsampling module makes the convolution step size 2 during the depthwise separable convolution of 3×3 convolution kernel, producing the effect of downsampling. The InpoNet backbone network framework accepts RGB images with a resolution of 224×224×3, and after four consecutive downsampling modules, the feature map is compressed to 14×14, and then 22 consecutive convolution modules are performed. After another downsampling to compress the feature map to 7×7, 13 consecutive convolutions are performed to complete the operation of the backbone network framework.

[0084] Depending on the prediction results, InpoNet connects two different tails, that is, different prediction tasks have two different InpoNets connected to different tails. For InpoNetⅠ, which can be responsible for predicting at least 7 facial key points, including left eye, right eye, nose, left corner of mouth, right corner of mouth, upper lip, and lower lip, two fully connected layers are added after InpoNet to output a vector of 14 elements. For InpoNetⅡ, which is responsible for predicting the distance, three upsampling layers are added after InpoNet to output a 56×56 distance map.

[0085] After calculation by the second model, the display device 200 can obtain the classification probability of the current user image pair interval distance. The display device 200 then traverses each classification probability to find the highest distance value of the distance classification probability, that is, the algorithm distance, as the second predicted distance.

[0086] According to the first predicted distance calculated by the first model and the second predicted distance output by the second model, the display device 200 can calculate a third predicted distance. The third predicted distance is a weighted sum of the first predicted distance and the second predicted distance. For example, the weight of the first predicted distance can be set to 0.5, and the weight of the second predicted distance can be set to 0.5, then the third predicted distance = 0.5 × first predicted distance + 0.5 × second predicted distance.

[0087] Figure 8 Schematic diagram of the display brightness adjustment process in some embodiments is shown. In order to obtain the third predicted distance, as Figure 8As shown, the display device 200 can set a weight according to the first predicted distance output by the first model, and then perform a weighted sum calculation according to the set weight. Among them, the weight of the weighted sum calculation can be set according to the first predicted distance detected by the first model. For example, the display device 200 can estimate the interval between the user and the display device 200 according to the first predicted distance, preset a weight value for each interval, and then set the corresponding weight according to the interval to which the first predicted distance belongs.

[0088] When setting the weight, the display device 200 may also set the logarithmic function of the first prediction distance as the weight distribution. For example, the weight distribution function shown in the following formula (2) may be obtained:

[0089]

[0090] Where D 1 is the first prediction distance, the value unit is meter, and w is the weight.

[0091] The weight obtained in the above manner can be used as the weight of the first prediction distance in the process of calculating the third prediction distance. Correspondingly, a weight can be assigned to the second prediction distance, that is, the weight of the second prediction distance is equal to (1-w).

[0092] Combined with the set weights, the third prediction distance can be calculated according to the following formula (3):

[0093] D=w×D 1 +(1-w)×D 2 Formula (3)

[0094] Where D 1 is the first prediction distance; w is the weight; D 2 is the second prediction distance.

[0095] Through the above formulas (2) and (3), the weight distribution of the first predicted distance and the second predicted distance can be determined by the distance of the user. It can be seen that when the actual user distance is close, the final output distance tends to be a balanced distribution of the first predicted distance and the second predicted distance. However, considering that the facial image feature information actually captured when the user distance is far is limited, the error of the second predicted distance output by deep learning is large. Therefore, the third predicted distance output in the end tends to be the first predicted distance output by empirical modeling, that is, the first predicted distance has a higher weight.

[0096] The weight set for the predicted distance output by the model can be based on the number of updates of the first model and the second model of the display device 200. For example, during the initial application of the display device 200, since the second model does not have enough training data to improve the model parameters, the accuracy of the second model output result is lower than the accuracy of the first model output result. Therefore, the weight of the first model output result, that is, the first predicted distance, can be increased, and the weight of the second predicted distance can be reduced. As the amount of training data increases in the later period, the model parameters of the second model gradually tend to be improved, so that the accuracy of the second model output result is improved. At this time, the weight of the second predicted distance can be increased, and the weight of the first predicted distance can be reduced.

[0097] After the third predicted distance is calculated and obtained in the above manner, the display device 200 can execute the display adjustment instruction according to the third predicted distance. The display adjustment instruction can be expressed in an appropriate form according to the application started by the user or the system function enabled. For example, when the display device 200 enables automatic adjustment of brightness according to the user's viewing distance, after the display device 200 calculates and obtains the third predicted distance in the above manner, the third predicted distance can be judged, that is, the third predicted distance is compared with the distance threshold for brightness adjustment. If the third predicted distance is less than or equal to the distance threshold, the display device 200 can generate a display adjustment instruction for reducing the brightness of the display 260, and in response to the display adjustment instruction, control the reduction of the display brightness of the display 260.

[0098] Obviously, under different third prediction distances, the display adjustment instructions executed by the display device 200 are also different, that is, different third prediction distances may correspond to different adjustment parameters, so that the adjustment instructions are generated and responded to according to the corresponding adjustment parameters. Fig. 9 FIG. 2 shows a schematic diagram of distance intervals in an embodiment. Fig. 9 As shown, the display device 200 can preset multiple viewing distance intervals according to the screen size, including an ultra-close distance interval of 0 to 1m, a close distance interval of 1 to 3m, a medium distance interval of 3 to 5m, and a long distance interval of more than 5m. When the third predicted distance is calculated to be 0.5m, it is determined that the viewing distance interval to which the third predicted distance belongs is an ultra-close distance interval of 0 to 1m. Such a viewing distance is not suitable for high-brightness viewing. At this time, a display adjustment instruction for adjusting the brightness of the display 260 to the lowest can be generated, and in response to the display adjustment instruction, the display brightness of the display 260 is controlled to be adjusted to the lowest value. If the third predicted distance is 2.5m, it is determined that the user is in the close distance interval. At this time, the display device 200 can generate a display adjustment instruction for reducing the brightness by 50%, and in response to the display adjustment instruction, the display brightness is controlled to be reduced by 50%.

[0099] It should be noted that the operating parameters of the display device 200 specifically adjusted by the display adjustment instruction can be based on absolute values. For example, when it is determined that the third predicted distance belongs to the ultra-short distance interval, the display brightness is adjusted to 1% brightness; when it is determined that the third predicted distance belongs to the close distance interval, the display brightness is adjusted to 30% brightness. The operating parameter adjustment value can also be based on a relative value, that is, the adjusted operating parameter is comprehensively calculated based on the current operating parameter and the adjustment amount. For example, when it is determined that the third predicted distance belongs to the ultra-short distance interval, the display brightness is adjusted to 10% of the current brightness; when it is determined that the third predicted distance belongs to the close distance interval, the display brightness is adjusted to 50% of the current brightness.

[0100] It can be seen from the above technical solutions that the distance detection method provided in some of the above embodiments can obtain the user image in real time during the operation of the display device, and input the user image into the first model and the second model respectively, so that the first model and the second model respectively calculate the first predicted distance and the second predicted distance using empirical modeling and deep learning methods. Then, the first predicted distance and the second predicted distance are weighted and summed to obtain the third predicted distance, so as to perform display control using the third predicted distance. The method can predict the distance between the user and the display device 200 without obtaining the lens parameters, and solve the third predicted distance based on empirical modeling and deep learning, which can improve the accuracy of distance detection.

[0101] Since the user does not completely face the screen of the display device 200 in the process of taking pictures, that is, during the use of part of the display device 200, the user has a certain degree of deflection angle relative to the screen of the display device 200, and these deflection angles will cause the pixel distance corresponding to the specific key point in the image captured by the image collector 231 to change. For example, when the user faces the image collector 231 in the front, the interval between the two eyes in the captured user image is 560 pixels, and when the user deflects 60° to take pictures at the same position, the interval between the two eyes in the obtained user image is 280 pixels. At this time, if the predicted distance is matched according to the middle value of the interval of 280 pixels, the predicted distance detected by the display device 200 is farther than the actual distance, which reduces the accuracy of the distance prediction.

[0102] In order to reduce the influence of the user's deflection angle on distance judgment, the first model may also include a deflection angle detection algorithm and a deflection angle comparison table. After acquiring the user image, the display device 200 can predict the deflection angle according to the relative position relationship between specific key points in the user image, thereby correcting the distance based on the predicted deflection angle to obtain a more accurate distance value.

[0103] Therefore, in some embodiments, the display device 200 may first call the first model algorithm in the step of inputting the user image into the first model to obtain the first predicted distance, and then search for the first key point in the user image according to the first model algorithm. The first key point is the imaging position of two facial organs, and the two organs are symmetrically distributed on the face. For example, two organs such as eyes, ears, and eyebrows on the user's face can be used as the identification subject of the first key point. That is, after acquiring the user image, the display device 200 can identify the image content area formed by any one of the eyes, ears, and eyebrows in the user image by matching factors such as shape and color, and find the first key point.

[0104] While searching for the first key point, the display device 200 can also search for a second key point in the user image. The second key point is an organ imaging position of one facial organ located on the symmetry axis of the two first key points. For example, one organ on the user's face, such as the nose tip, mouth, chin, etc., can be used as the identification subject of the second key point.

[0105] The first key point and the second key point can be represented in the form of coordinates. The coordinate unit can be the number of pixels. For example, taking any vertex of each frame of the user image as the coordinate origin, determine the number of pixels between the first key point and the coordinate origin to obtain the position coordinates of the first key point. Since the image content of facial organs usually occupies multiple pixels in the user image, in the process of searching for the first key point and the second key point, the display device 200 can combine the pixels belonging to the same organ to form a set, calculate the center point of the set, and use the coordinates of the center point as the position coordinates of the first key point or the second key point.

[0106] Obviously, since the relative positions of the facial organs are fixed, the user's deflection state can be determined through two first key points and one second key point, that is, after finding the first key point and the second key point, the display device 200 can calculate the pixel deflection degree according to the relative position relationship between the first key point and the second key point. The pixel deflection degree is the horizontal coordinate of the foot point of the perpendicular line from the second key point to the two first key points.

[0107] For example, Fig.10 Schematic diagrams of key points in some embodiments are shown. Fig.10As shown, the first key point can be the center of the left and right eyes, and the second key point is the nose tip. Then, the deflection angle can be solved by empirical modeling based on the triangular position coordinates. That is, the display device 200 can first use a camera to collect data to obtain a user image, and then use the center of both eyes as the origin, and the line connecting the center of both eyes as the x-axis coordinate, that is, the coordinates of both eyes are (-1, 0) and (1, 0) respectively. When a perpendicular line is drawn from the nose tip to the line connecting the two eyes, the x-axis coordinate of the point where the foot of the perpendicular point P (-0.9, 0) is located is marked as the deflection degree of the face pixel.

[0108] After calculating and obtaining the pixel deflection degree, the display device 200 may also use the pixel deflection degree to query the first predicted distance. In some embodiments, the display device 200 may call a deflection angle comparison table and a distance comparison table in the first model, wherein the deflection angle comparison table includes pixel deflection degrees and deflection angles having a mapping relationship. For example, the mapping relationship between pixel deflection degrees and deflection angles may be stored in the deflection angle comparison table, as shown in the following table:

[0109]

[0110]

[0111] After calculating and obtaining the pixel deflection degree, the display device 200 can query the user's deflection angle at this time based on the above deflection angle comparison table, that is, use the pixel deflection degree to match the deflection angle prediction value in the deflection angle comparison table.

[0112] Similarly, the distance comparison table includes pixel distances and predicted distances having a mapping relationship. After the display device 200 calculates the number of interval pixels between two first key points, ie, the pixel distance, the current predicted distance can be queried based on the distance comparison table.

[0113] When a deflection of the user's posture is detected in the user image, the display device 200 may also use the deflection angle to correct the pixel distance before querying the predicted distance, that is, Fig.11 FIG. 4 shows a schematic diagram of a process for correcting the distance of the first pixel in some embodiments. Fig.11 As shown, after obtaining the pixel distance between the two first key points in the user image before cropping, the pixel distance is corrected using the deflection angle prediction value to obtain a corrected distance, and then the first predicted distance is matched in the distance comparison table according to the corrected distance.

[0114] For example, when the display device 200 detects that the perpendicular coordinate corresponding to the pixel deflection degree is P(0.87,0), the corresponding deflection angle is determined to be α=30° by querying the deflection angle comparison table. At this time, the number of pixels between the two first key points can be obtained, that is, r=320px. Then, the pixel distance is corrected according to the deflection angle of 30° to obtain the corrected distance, that is, the corrected distance r'=r / cosα=370px. Finally, the corresponding distance is queried in the distance comparison table according to the corrected distance r', and the first predicted distance is obtained.

[0115] It can be seen from the above technical solutions that the distance detection method provided in some of the above embodiments can detect the user's deflection angle while performing distance prediction, so as to use the detected deflection angle to correct the pixel point distance measured in the user image, thereby reducing the influence of the user's deflection posture on the distance prediction result and improving the accuracy of the distance detection result.

[0116] In the above embodiment, since the mapping relationship between the pixel point distance and the interval distance is affected by the lens parameters of the image collector 231, different image collectors 231 have different lens parameters, which may correspond to different distance comparison tables. Therefore, in order to allow users to use different models of cameras according to their needs, the display device 200 can also read the camera model connected to the image acquisition interface. The first model can store deflection angle comparison tables and distance comparison tables corresponding to multiple models of cameras. After reading the camera model connected to the image acquisition interface, the display device 200 can call the corresponding deflection angle comparison table and distance comparison table in the first model according to the read camera model.

[0117] For some cameras, since their lenses have an optical zoom structure, their focal lengths are not fixed, that is, the lens parameters are not fixed. Therefore, in order to allow users to use the camera's variable focal length and meet distance detection requirements. Fig.12 FIG. 4 shows a schematic diagram of a process for monitoring focal length parameters in some embodiments. Fig.12 As shown, the display device 200 can also monitor in real time the change value of the pixel distance within a preset unit time, for example, the change value of the pixel distance within 1 second.

[0118] Then compare the change value with the preset zoom threshold. If the change value is less than the preset zoom threshold, it means that the focal length of the current camera has not changed, so the distance prediction can continue to be performed using the current distance comparison table. If the change value is greater than or equal to the preset zoom threshold, that is, the focal length of the current camera has changed, the currently used distance comparison table can no longer meet the current camera focal length conditions, and the deflection angle comparison table and distance comparison table need to be re-called.

[0119] Therefore, the display device 200 can obtain the camera callback information and extract the current focal length parameter from the camera callback information. For example, the display device 200 can obtain the camera callback information through the standard camera manager interface in the operating system, and use the focal length reading interface (CaptureResult.LENS_FOCAL_LENGTH) to obtain the specific camera focal length parameter. Then, the deflection angle comparison table and the distance comparison table that meet the current focal length parameter are called from the first model to obtain the deflection angle and the first predicted distance according to the corresponding comparison table.

[0120] It can be seen from the above technical solution that the distance detection method provided in some of the above embodiments can obtain the lens state corresponding to the current shooting process by monitoring the camera model and camera focal length parameters connected to the image acquisition interface, thereby calling different comparison tables from the model according to different lens states, reducing the impact of camera model changes and focal length changes on the calculation process, and improving the accuracy of the distance detection results.

[0121] In some embodiments, the display device 200 may pre-build the first model and the second model. Fig.13 FIG. 4 shows a schematic diagram of the image recording process in some embodiments, such as Fig.13 As shown, the display device 200 can obtain the control instruction input by the user for entering the user information. For example, in the setting interface of the display device 200, an information entry menu can be provided, and the information entry menu can include a position detection information entry option. When the user controls the focus cursor to select the position detection information entry option, the display device 200 can obtain the control instruction for entering the user information and enter the information entry state.

[0122] The display device 200 that enters the information entry state can respond to the control instruction and obtain multiple frames of recorded images through the image acquisition interface. Among them, the multiple frames of recorded images include images obtained by shooting multiple users at multiple deflection angles at multiple shooting positions. For example, after entering the information entry state, the display device 200 can use multiple cameras, and the same camera at different focal lengths to collect data. And start the face detection algorithm, and render the recorded image data shot by the camera and the pixel distance information of the key points of the human eyes on the screen. At this time, with the screen of the display device 200 as the distance 0, determine the distance of 0-6 meters perpendicular to the screen of the display device 200 as the range of information entry. Then, by displaying the prompt information, 5 or more tested users display the facial information in the left, middle and right areas of the information entry interface respectively, and take pictures at the positions of 0.2m, 0.4m, 0.6m, 0.8m, 1m, 1.2m...5.8m, 6m from the screen respectively to obtain multiple frames of recorded images.

[0123] Then receive the classification label set by the user for each frame of recorded image, wherein the classification label includes a distance label and a deflection angle label. The distance label is the distance between the user's position and the display device 200 during the recording process, such as 0.2m, 0.4m, 0.6m, 0.8m, 1m, 1.2m...5.8m, 6m, etc. The deflection angle label is the deflection state of the user at each specific position according to the prompt information during the recording process, such as -90°, -60°, -30°, 0°, 30°, 60°, 90°, etc.

[0124] By setting a distance label and a deflection angle label for each frame of the recorded image, the recorded image can be converted into empirical modeling data and training data, so as to update the first model and the second model according to the recorded image and the classification label. Fig.14 A schematic diagram of updating the first model process in some embodiments is shown. Fig.14 As shown, for the first model, the display device 200 can first obtain multiple recorded images of the same deflection angle posture at each shooting position, and calculate the average value of the pixel distance in the recorded image, wherein the pixel distance is the number of interval pixels between two first key points. Then, a mapping relationship between the average pixel distance and the distance label of the current position is established to create a distance comparison table, and then the distance comparison table is added to the first model.

[0125] For example, the display device 200 can use the face detection algorithm of the first model after calling the first model. Take the center of both eyes as the origin, the line connecting the key points of both eyes as the x-axis coordinate, and obtain the coordinates of both eyes as (-1, 0) and (1, 0) respectively. Then, make a vertical line from the nose tip coordinate point to the line connecting the two eyes, and obtain the x-coordinate of the intersection point as the face pixel deflection degree. At the same time, the user image data and face pixel deflection degree data captured by the camera are rendered in the input interface. Then, take the screen of the display device 200 as the distance 0, and determine the distance 3m perpendicular to the screen of the display device 200 as the deflection angle input range. Through the prompt message, prompt the user to turn his head to -90°, -60°, -30°, 0°, 30°, 60°, 90°, and count the corresponding pixel deflection degrees, and then establish the mapping relationship between the pixel deflection degree and the user head steering angle during the input process through regression analysis, so as to generate a steering angle comparison table and store it in the first model.

[0126] Fig.15 A schematic diagram of updating the second model process in some embodiments is shown. Fig.15As shown, for the second model, the display device 200 can input the input image with the classification label into the second model, and obtain the distance classification probability output by the second model. Then, the distance classification probability and the distance label are compared to obtain the prediction value loss, so as to back-propagate the prediction value loss in the second model to adjust the model parameters in the second model.

[0127] For example, the neural network architecture corresponding to the second model can support multi-layer perceptron (MLP), using the 14-element vector output by InpoNetⅠ, i.e., the coordinates of the 7 key points of the face, as input data, to establish 3 updateable weight matrices connecting the 3 key points and the yaw angle. And the fitting loss function provided by the following formula (4) is:

[0128]

[0129] Among them, L is the prediction value loss or nonlinear fitting loss; B is the batch in one training; y is the predicted deflection angle, and y' is the label deflection angle. After calculating the prediction value loss using the above formula, back propagation is performed based on the calculated prediction value loss to adjust the model parameters of each layer in the neural network model to update the neural network model so that the updated neural network model can more accurately predict the deflection angle of the face coordinates.

[0130] It can be seen from the above technical solutions that the distance detection method provided in some of the above embodiments can support the user to enter information, and update the first model and the second model based on the data entered by the user. By entering information, the display device 200 can make adaptive adjustments for different users so that the first model and the second model can better meet the user's portrait characteristics and improve the accuracy of the detection results.

[0131] Fig.16 1 shows a time sequence diagram of the distance detection method in some embodiments. Based on the above distance detection method, a display device 200 is also provided in some embodiments. The display device 200 includes: a display 260, an image acquisition interface and a controller 250. The display 260 is configured to display a user interface; the image acquisition interface is configured to acquire a user image; Fig.16 As shown, the controller 250 is configured to perform the following program steps:

[0132] Acquire the user image;

[0133] Based on the first model and the user image, a first predicted distance is obtained, and based on the second model and the user image, a second predicted distance is obtained; the first model is a distance prediction model based on empirical modeling; the second model is a distance prediction model based on deep learning;

[0134] Calculating a third predicted distance, the third predicted distance being a weighted sum of the first predicted distance and the second predicted distance; the weight in the calculation process of the weighted sum is set according to the first predicted distance;

[0135] Display parameters are adjusted according to the third predicted distance.

[0136] It can be seen from the above technical solutions that the display device 200 provided in some of the above embodiments can obtain a user image in real time during operation, and input the user image into the first model and the second model respectively, so that the first model and the second model respectively calculate the first predicted distance and the second predicted distance using empirical modeling and deep learning methods. The first predicted distance and the second predicted distance are then weighted and summed to obtain a third predicted distance, so as to perform display control using the third predicted distance. The display device 200 can predict the distance between the user and the display device 200 without obtaining lens parameters, and solve the third predicted distance based on empirical modeling and deep learning, which can improve the accuracy of distance detection.

[0137] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; An image acquisition interface, configured to acquire user images; The controller is configured as: Acquire the user image; Based on the first model and the user image, a first predicted distance is obtained, and based on the second model and the user image, a second predicted distance is obtained; the first model is a distance prediction model based on empirical modeling; the second model is a distance prediction model based on deep learning; Calculating a third predicted distance, the third predicted distance being a weighted sum of the first predicted distance and the second predicted distance; the weight in the calculation process of the weighted sum is set according to the first predicted distance; Display parameters are adjusted according to the third predicted distance.

2. The display device according to claim 1, characterized in that The controller is configured to: After the step of acquiring the user image, identifying a face area from the user image; Defining a face selection frame according to the face area, wherein the face selection frame is a rectangular frame covering the face area; Crop the user image according to the face selection frame; The cropped user image is scaled to a preset image size.

3. The display device according to claim 1, characterized in that The controller is configured to: In the step of obtaining a first predicted distance based on the first model and the user image, calling a first model algorithm; Searching for a first key point in the user image according to the first model algorithm, where the first key point is an organ imaging position of two facial organs, where the two organs are symmetrically distributed on the face; Searching for a second key point in the user image, where the second key point is an organ imaging position in the facial organ, which is one in number and located on a symmetry axis between two first key points; Calculate a pixel deflection degree, where the pixel deflection degree is the horizontal coordinate of a foot point of a perpendicular line from the second key point to the two first key points; The first prediction distance is queried using the pixel deflection degree.

4. The display device according to claim 3, characterized in that The controller is configured to: In the step of using the pixel deflection degree to query the first predicted distance, a deflection angle comparison table and a distance comparison table are called in the first model, wherein the deflection angle comparison table includes the pixel deflection degrees and deflection angles having a mapping relationship; and the distance comparison table includes the pixel point distance and the predicted distance having a mapping relationship; Matching the deflection angle prediction value in the deflection angle comparison table using the pixel deflection degree; Obtaining a pixel distance between two first key points in the user image before cropping; Correcting the pixel point distance using the deflection angle prediction value to obtain a corrected distance; The first predicted distance is matched in the distance comparison table according to the corrected distance.

5. The display device according to claim 4, characterized in that The controller is configured to: Monitor the change value of the pixel point distance within a preset unit time; If the change value is greater than or equal to the preset zoom threshold, obtain camera callback information; Extract the current focal length parameter from the camera callback information; The deflection angle comparison table and the distance comparison table that conform to the current focal length parameters are called in the first model.

6. The display device according to claim 1, characterized in that The controller is configured to: In the step of obtaining a second prediction distance based on the second model and the user image, the second model is called, and the second model includes a sampling module and a convolution kernel module, the sampling module is used to perform sampling on the user image to compress the user image into a feature map; the convolution kernel module is used to perform a convolution operation on the feature map to obtain a classification probability; inputting the user image into the second model; Performing operations on the user image through the sampling module and the convolution kernel module to output a distance classification probability corresponding to a face area in the user image; Find the algorithm distance with the highest distance classification probability as the second predicted distance.

7. The display device according to claim 1, characterized in that The controller is configured to: Obtaining control instructions input by the user for entering user information; In response to the control instruction, a plurality of frames of recorded images are acquired through the image acquisition interface, wherein the plurality of frames of recorded images include images obtained by photographing a plurality of users at a plurality of deflection angles at a plurality of shooting positions; receiving a classification label set by a user for each frame of the input image, wherein the classification label includes a distance label and a deflection angle label; The first model and the second model are updated according to the input image and the classification label.

8. The display device according to claim 7, characterized in that The controller is configured to: In the step of updating the first model and the second model according to the recorded images and the classification labels, a plurality of the recorded images with the same deflection angle posture at each shooting position are obtained; Calculate the average value of the pixel distance in the input image, where the pixel distance is the number of interval pixels between two first key points; Establishing a mapping relationship between the average pixel distance value and the distance label of the current position to create a distance comparison table; The distance comparison table is added to the first model to update the first model.

9. The display device according to claim 7, characterized in that: The controller is configured to: In the step of updating the first model and the second model according to the input image and the classification label, the input image is input into the second model; Obtaining the distance classification probability output by the second model; Comparing the distance classification probability and the distance label to obtain a prediction value loss; The prediction value loss is back-propagated in the second model to adjust the model parameters of the second model and update the second model.

10. A distance detection method based on human portrait, characterized in that: Applied to a display device, the distance detection method includes: Get user image; Based on the first model and the user image, a first predicted distance is obtained, and based on the second model and the user image, a second predicted distance is obtained; the first model is a distance prediction model based on empirical modeling; the second model is a distance prediction model based on deep learning; Calculating a third predicted distance, the third predicted distance being a weighted sum of the first predicted distance and the second predicted distance; the weight in the calculation process of the weighted sum is set according to the first predicted distance; Display parameters are adjusted according to the third predicted distance.

Images