Display apparatus, display method, and storage medium

Abstract

The embodiment of the invention belongs to the display technology, and provides a display method, display equipment and a storage medium. The display equipment comprises a processor and an image acquisition device, and the processor is configured to control the image acquisition device to obtain the definition of object recognition frames of different objects acquired by the image acquisition device under different control currents in the first focusing process; determining a first control current of the image acquisition device according to the definition of the object recognition frame, and controlling the image acquisition device to focus according to the first control current; and during target object close-up, if a difference value between the definition of the object recognition frame of the target object and the upper limit value of the definition of the object recognition frame of the target object is greater than a preset threshold value, controlling the image acquisition device to focus the target object according to the upper limit value of the definition of the object recognition frame. According to the invention, the focusing frequency in the display process is reduced, so that the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to display technology, and more specifically, to a display device, a display method, and a storage medium. Background Art

[0002] When conducting conferences through display devices, speaker tracking has basically become a must-have product feature.

[0003] Currently, at the beginning of a meeting, the image acquisition device of the display device (for example, a camera) will focus once. When a participant speaks, it will refocus according to the position of the speaker. The picture of the display device will go through a process of clear-blurred-clear. If there are multiple speakers, the picture of the display device will repeatedly go through the process of clear-blurred-clear. The focusing frequency is high, and the user experience is poor. Summary of the invention

[0004] The exemplary embodiments of the present application provide a display device, a display method and a storage medium, which can reduce the focusing frequency and improve the user experience.

[0005] In a first aspect, the present application provides a display device, the display device comprising a processor and an image acquisition device, the processor being configured to:

[0006] Controlling the image acquisition device to obtain the clarity of object recognition frames of different objects acquired by the image acquisition device under different control currents during the first focusing process; the control current is used to drive the image acquisition device to focus;

[0007] Determining a first control current of the image acquisition device according to the clarity of the object recognition frame, and controlling the image acquisition device to focus according to the first control current;

[0008] When taking a close-up of a target object, if the difference between the clarity of an object recognition frame of the target object and an upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold, controlling the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame;

[0009] If the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, the clarity of the current object recognition frame of the target object is maintained for a close-up.

[0010] In some embodiments, the processor is configured to:

[0011] Acquire a second control current corresponding to an upper limit value of the clarity of the object recognition frame of the target object;

[0012] The image acquisition device is controlled to focus on the target object according to the difference between the second control current and the first control current.

[0013] In some embodiments, the processor is configured to:

[0014] Obtain the fusion clarity of the object recognition frame under different control currents;

[0015] The control current value corresponding to the upper limit value of the fusion clarity of the object recognition frame is used as the first control current value.

[0016] In some embodiments, the processor is configured to:

[0017] For any control current, obtaining the size of the object recognition frame of different objects acquired by the image acquisition device under the control current;

[0018] Obtaining a corresponding weight value according to the size of the object recognition frame;

[0019] The clarity of the object recognition frame is weighted according to the weight value to obtain the fusion clarity of the object recognition frame.

[0020] In some embodiments, the display device further includes a sound collection device, and the processor is configured to:

[0021] Controlling the sound collection device to obtain the sound source direction of the target object;

[0022] The target object is determined according to the picture captured by the image acquisition device and the direction of the sound source of the target object.

[0023] In some embodiments, the processor is configured to:

[0024] Determine whether there is an event that triggers refocusing; the event includes: at least one of a change in the number and / or position of the objects, and a change in the position of the image acquisition device;

[0025] If so, the image acquisition device is controlled to refocus.

[0026] In some embodiments, the processor is configured to:

[0027] Controlling the image acquisition device to perform object recognition and determine the number and position of the objects in the acquired picture;

[0028] When determining, based on the number and positions of the objects, that the number and / or positions of the objects change, obtaining the position coordinates of the changed objects;

[0029] According to the position coordinates, the image acquisition device is controlled to refocus.

[0030] In some embodiments, the processor is configured to:

[0031] If the ordinate value of the position coordinate is outside a preset range, the image acquisition device is controlled to refocus; the preset range is determined according to the ordinate value of the acquired object during the first focusing process;

[0032] If the vertical coordinate value of the position coordinate is within a preset range, the clarity of the current picture is maintained for display.

[0033] In a second aspect, the present application provides a display method, comprising:

[0034] Controlling the image acquisition device to obtain the clarity of the object recognition frame of objects at different positions acquired by the image acquisition device under different control currents during the first focusing process; the control current is used to drive the image acquisition device to focus;

[0035] Determining a first control current value of the image acquisition device according to the clarity of the object recognition frame, and controlling the image acquisition device to focus according to the first control current value;

[0036] When taking a close-up of a target object, if the difference between the clarity of an object recognition frame of the target object and an upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold, controlling the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame;

[0037] If the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, the clarity of the current object recognition frame of the target object is maintained for a close-up.

[0038] In a third aspect, the present application provides a display device, including:

[0039] An acquisition module, used to control the image acquisition device to obtain the clarity of the object recognition frame of objects at different positions acquired by the image acquisition device under different control currents during the first focusing process; the control current is used to drive the image acquisition device to focus;

[0040] A focusing module, configured to determine a first control current value of the image acquisition device according to the clarity of the object recognition frame, and to control the image acquisition device to focus according to the first control current value;

[0041] A close-up module, configured to control the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame when taking a close-up of the target object, if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold;

[0042] The processing module is used to maintain the clarity of the current object recognition frame of the target object for close-up if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0044] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0045] The display device, display method and storage medium provided by the embodiments of the present application include a processor and an image acquisition device, wherein the processor is configured to: control the image acquisition device to obtain the clarity of object recognition frames of different objects acquired by the image acquisition device under different control currents during the first focusing process; the control current is used to drive the image acquisition device to focus; determine a first control current of the image acquisition device according to the clarity of the object recognition frame, and control the image acquisition device to focus according to the first control current; when taking a close-up of a target object, if the difference between the clarity of the object recognition frame of the target object and the upper limit of the clarity of the object recognition frame of the target object is greater than a preset threshold, control the image acquisition device to focus on the target object according to the upper limit of the clarity of the object recognition frame; if the difference between the clarity of the object recognition frame of the target object and the upper limit of the clarity of the object recognition frame of the target object is less than or equal to the preset threshold, maintain the clarity of the current object recognition frame of the target object for close-up. The different sharpness of the object is recorded during the initial focusing process. When a close-up of the target object is taken later, when the difference between the clarity of the object recognition frame of the target object and the upper limit of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, there is no need to refocus on the target object, thereby reducing the number of focus times, reducing the number of times the captured image is blurred due to focusing, and thus improving the user's video experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0047] Figure 1 A schematic diagram of a scenario provided for this application;

[0048] Figure 2 A schematic diagram of the hardware configuration of a display device provided in this application;

[0049] Figure 3 A flow chart of a display method provided by the present application;

[0050] Figure 4 A schematic diagram of the clarity of an object recognition frame provided in this application;

[0051] Figure 5 A schematic diagram of a flow chart of a display method provided in this application;

[0052] Figure 6 A schematic diagram of the size of an object recognition frame provided in this application;

[0053] Figure 7 A schematic diagram of a flow chart of a display method provided in this application;

[0054] Figure 8 A schematic diagram for determining a preset range provided for this application;

[0055] Fig. 9 A schematic diagram of the first focusing process provided by this application;

[0056] Fig.10 A process diagram of a close-up of a target object provided for this application;

[0057] Fig.11 A schematic diagram of a personnel change process provided for this application;

[0058] Fig.12 A schematic diagram of the structure of a display device provided in this application. DETAILED DESCRIPTION

[0059] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0060] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0061] In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0062] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described in the present application as "exemplary" or "for example" should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0063] With the development of technology, the demand for video conferencing is increasing, and large conference screens have become a commonly used display device for users. Among the current large conference screens, the speaker tracking function has basically become a necessary product function.

[0064] Figure 1 A schematic diagram of a scenario of conducting a conference through a display device provided in an embodiment of the present application, such as Figure 1 As shown, at the beginning of a meeting, the processor of the display device can control the image acquisition device (eg, camera) to perform a focus once in a center focus manner to determine an initial focus position of the display image.

[0065] When a participant in the meeting speaks, the processor can control the image acquisition device to zoom in on the speaker. Since the relative relationship between the speaker and the initial focus position cannot be determined, it is impossible to ensure that the clarity of the image at the speaker is at the best focus position. The processor will control the image acquisition device to automatically focus on the speaker's position for the second time.

[0066] If other participants take turns to speak later, it is equivalent to the speaker's position constantly changing. Every time the speaker is changed, the focus needs to be refocused. The display screen of the display device will repeat the process of blur-clear-blur-clear. Due to too many focus times, the user's meeting experience is greatly reduced.

[0067] In view of the above-mentioned problems existing in the prior art, the present application provides a display device, a display method and a storage medium, which can reduce the number of focusing times during a meeting and improve the user experience.

[0068] Among them, the display device provided in the embodiment of the present application can have various implementation forms, for example, it can be a television, a smart TV, a laser projection device, a monitor, an electronic whiteboard (electronic bulletin board), an electronic table (electronic table), etc., and the embodiment of the present application does not limit this.

[0069] The hardware configuration of the above-mentioned display device is introduced below.

[0070] Figure 2 A possible hardware configuration diagram of a display device provided in this application. Figure 2 As shown, in some embodiments, the display device 200 may include at least one of a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0071] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0072] The display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from the controller, and display video content, image content, and menu control interface components and user control UI interface.

[0073] The display 260 can be a liquid crystal display, an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), and a projection display, etc. It can also be a projection device and a projection screen.

[0074] The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0075] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0076] The detector 230 is used to collect signals from the external environment or the external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0077] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0078] The tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0079] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0080] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0081] In some embodiments, the controller includes a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0082] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0083] "User interface" is the medium interface for interaction and information exchange between application programs or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The most common form of user interface is graphical user interface (GUI), which refers to the user interface related to computer operation displayed in a graphical way. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0084] The technical solution of the present application is described in detail below in conjunction with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0085] Figure 3 A flow chart of a display method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the steps include:

[0086] S301 , controlling an image acquisition device to obtain the clarity of object recognition frames of different objects acquired by the image acquisition device under different control currents during the first focusing process.

[0087] The application scenario of the embodiment of the present application can be a video conference scenario, or a scenario such as displaying items through video, and the embodiment of the present application does not limit the application scenario. For the sake of ease of description, the technical solution of the embodiment of the present application will be described by taking the conference scenario as an example. Correspondingly, the object in the embodiment of the present application is the corresponding conference participant.

[0088] In the embodiment of the present application, the control current is used to drive the image acquisition device to focus. During the focusing process of the image acquisition device, the lens is driven to move for focusing according to the magnitude of the input control current, and different magnitudes of the control current correspond to different moving distances of the lens. That is, different magnitudes of the control current are different focusing parameters of the image acquisition device.

[0089] In the embodiment of the present application, the processor starts to perform the first focus when receiving an external instruction, for example, a user triggers a control to start a meeting on a display device, or starts a meeting at a preset time point.

[0090] At the beginning of the focusing process, the image acquisition device will perform focusing multiple times in sequence according to different input control currents.

[0091] In a possible implementation, the control current may be increased from zero to a maximum current allowed by the image acquisition device in a preset interval. For example, the control current may be 0 mA, 10 mA, 20 mA, 30 mA, and finally a maximum current.

[0092] During each focusing process, the processor can control the image acquisition device to perform object recognition, such as Figure 4 As shown, the image acquisition device can perform face recognition on the participants during the focusing process. In the acquired image, the head of each participant has a corresponding face recognition frame 401 and a clarity 402 of the face recognition frame 401 .

[0093] It should be understood that when performing object recognition, the picture in the object recognition box can be a partial picture of the object or a full picture of the object, and the embodiment of the present application does not limit this.

[0094] Taking the case where three people attend a meeting and the upper limit of the control current is 60 mA as an example, during the first focusing process, the corresponding relationship between the control current and the clarity of the face recognition frame of different users (participants) can be obtained as shown in Table 1 below.

[0095] Table 1

[0096]

[0097] S302: Determine a first control current of the image acquisition device according to the clarity of the object recognition frame, and control the image acquisition device to focus according to the first control current.

[0098] In the embodiment of the present application, the first control current may be a focus parameter determined during the first focusing process, that is, the image acquisition device may use the first control current as a focus parameter to complete the first focusing.

[0099] In some embodiments, the processor may perform fusion processing on the clarity of the object recognition frame to determine the first control current.

[0100] Exemplarily, the fusion clarity of the object recognition frame under different control currents is obtained; and the control current value corresponding to the upper limit value of the fusion clarity of the object recognition frame is used as the first control current value.

[0101] The fusion clarity of the object recognition frame may be the clarity obtained by fusion processing the clarity of the object recognition frames of multiple objects under the same control current. That is, each control current corresponds to the fusion clarity of an object recognition frame.

[0102] In a possible implementation, the clarity of each object recognition frame may be summed to obtain the fusion clarity of the object recognition frame.

[0103] Taking the data in Table 1 as an example, the fusion clarity of the object recognition frame corresponding to 0 mA can be the sum of the clarity of the three face recognition frames, that is, 0.6+0.8+0.3=1.7.

[0104] In a possible implementation, the clarity of each object recognition frame may be averaged to obtain a fusion clarity of the object recognition frame.

[0105] Taking the data in Table 1 as an example, the fusion clarity of the object recognition frame corresponding to 0 mA can be the average clarity of the three face recognition frames, that is, (0.6+0.8+0.3) / 3=0.57.

[0106] In a possible implementation, the clarity of each object recognition frame may be weighted to obtain the fusion clarity of the object recognition frame.

[0107] Among them, the weight of the clarity of each object recognition frame can be determined based on prior knowledge, and the embodiments of the present application are not limited to this.

[0108] Taking the data in Table 1 as an example, the weights of the clarity of the object recognition frame corresponding to 0mA are 0.8, 0.7, and 0.6, respectively. The fusion clarity of the corresponding object recognition frame can be the weighted value of the clarity of the three face recognition frames, that is, (0.6*0.8+0.8*0.7+0.3*0.6)=1.22.

[0109] It should be understood that the fusion clarity of the object recognition frame may also be determined in other ways, and the embodiments of the present application are not limited to this.

[0110] When determining the fusion clarity of the object recognition frame corresponding to each control current, the first control current may be determined according to an upper limit value of the fusion clarity of the object recognition frame.

[0111] In a possible implementation, the upper limit value of the fusion clarity of the object recognition frame may be the maximum value of the fusion clarity of the object recognition frame obtained under each control current.

[0112] For example, under a control current of 20 mA, the maximum value of the fusion clarity of the object recognition frame is A, that is, the upper limit value of the fusion clarity of the object recognition frame is A. The control current corresponding to the maximum value of the fusion clarity of the object recognition frame can be used as the first control current, that is, the first control current is 20 mA.

[0113] In some embodiments, if there are multiple fusion sharpnesses of the same maximum object recognition frame, the smaller one of the corresponding control currents may be used as the first control current.

[0114] In a possible implementation, the upper limit value of the fusion clarity of the object recognition frame may be a preset value, and the control current corresponding to the fusion clarity of the object recognition frame that is less than the upper limit value and closest to the upper limit value (maximum) may be used as the first control current.

[0115] For example, the fusion clarity of the object recognition frame corresponding to 0mA, 10mA, 20mA, and 30mA are 1.22, 1.36, 1.19, and 1.41 respectively, and the upper limit of the fusion clarity is 1.40. Then, the control current of 10mA corresponding to the fusion clarity of the object recognition frame of 1.36 can be used as the first control current.

[0116] When determining the first control current, the processor may feed back the first control current to the image acquisition device, so that the image acquisition device controls the lens to move a corresponding distance according to the first control current to complete the initial focusing.

[0117] S303. When taking a close-up of a target object, if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold, control the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame.

[0118] S304: If the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, maintain the current clarity of the object recognition frame of the target object for close-up.

[0119] In the embodiment of the present application, a close-up of a target object can be used to track the target object, and the processor can control the image acquisition device to use the object recognition function to determine the target object.

[0120] The definition of the object recognition frame of the target object is the definition of the object recognition frame in the current display picture, that is, the definition of the object recognition frame of the target object under the first control current.

[0121] Taking the data in Table 1 as an example, if the first control current is 30 mA and the target object is user 1, the clarity of the object recognition frame of the target object is 0.6.

[0122] The upper limit value of the clarity of the object recognition frame of the target object may be a maximum value of the clarity of the object recognition frame of the target object obtained during the first focusing process, or may be a preset value, which is not limited in this embodiment of the present application.

[0123] Exemplarily, if the upper limit value of the clarity of the object recognition frame of the target object is the maximum value of the clarity of the object recognition frame of the target object obtained during the first focusing process, taking the data in Table 1 as an example, the target object is user 1, then the upper limit value of the clarity of the object recognition frame of the target object can be 0.8.

[0124] When the processor obtains the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object, it determines whether the difference is greater than a preset threshold. If it is greater than the preset threshold, it may indicate that the clarity of the current target object is poor and the target object needs to be refocused to improve the close-up effect of the target object, thereby improving the user experience.

[0125] If the difference is less than the preset threshold, it can indicate that the clarity of the current target object is better and no refocusing is required. The processing can control the image acquisition device to take a close-up of the target object with the current clarity. The preset threshold can be adjusted according to actual needs, and the embodiment of the present application does not limit this.

[0126] The display method provided by the embodiment of the present application obtains the clarity of the object recognition frame of different objects captured by the image acquisition device under different control currents during the first focusing process by controlling the image acquisition device; the control current is used to drive the image acquisition device to focus; according to the clarity of the object recognition frame, a first control current of the image acquisition device is determined, and the image acquisition device is controlled to focus according to the first control current; when taking a close-up of a target object, if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold, the image acquisition device is controlled to focus on the target object according to the upper limit value of the clarity of the object recognition frame; if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to the preset threshold, the clarity of the current object recognition frame of the target object is maintained for close-up. The different sharpness of the object is recorded during the initial focusing process. When a close-up of the target object is taken later, when the difference between the clarity of the object recognition frame of the target object and the upper limit of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, there is no need to refocus on the target object, thereby reducing the number of focus times, reducing the number of times the captured image is blurred due to focusing, and thus improving the user's video experience.

[0127] Based on the above embodiment, a method for determining a target object is introduced below.

[0128] Exemplarily, the sound acquisition device is controlled to acquire the direction of the sound source of the target object; and the target object is determined according to the image acquired by the image acquisition device and the direction of the sound source of the target object.

[0129] In the embodiment of the present application, the display device further includes a sound collection device (e.g., a microphone), and the processor can control the sound collection device to collect sound and obtain the sound source orientation of the target object, that is, the location information of the source of the target object. According to the picture collected by the image collection device, the target object is screened out from the picture in combination with the sound source orientation of the target object.

[0130] Based on the above embodiment, the process of focusing on the target object according to the upper limit value of the clarity of the object recognition frame is introduced below.

[0131] Exemplarily, a second control current corresponding to an upper limit value of the clarity of the object recognition frame of the target object is obtained; and according to a difference between the second control current and the first control current, the image acquisition device is controlled to focus on the target object.

[0132] In one possible implementation, if the upper limit value of the clarity of the object recognition frame of the target object is the maximum value of the clarity of the object recognition frame of the target object obtained during the first focusing process, the control current corresponding to the maximum value of the clarity of the object recognition frame of the target object can be used as the second control current.

[0133] Exemplarily, taking the data in Table 1 as an example, if the target object is user 1, the upper limit value of the clarity of the object recognition frame of the target object may be 0.8, and the second control current may be 20 mA.

[0134] In some embodiments, if there are multiple object recognition frame definitions of the same largest target object, a smaller one of the corresponding control currents may be used as the second control current.

[0135] In one possible implementation, if the upper limit value of the clarity of the object recognition frame of the target object is a preset value, the second control current may be a preset current corresponding to the preset value, or the second control current may be determined based on the clarity of the object recognition frame of the target object obtained during the first focusing process, which is less than the preset value and closest to the preset value.

[0136] Exemplarily, taking the data in Table 1 as an example, the target object is user 1, and if the preset value is 0.75, the clarity of the object recognition frame of the target object that is smaller than the preset value and closest to the preset value is 0.7, and the corresponding second control current is 10 mA.

[0137] In some embodiments, if there are multiple identical sharpnesses of the object recognition frames of the target object that are smaller than a preset value and closest to the preset value, the smaller one of the corresponding control currents may be used as the second control current.

[0138] When the first control current and the second control current are determined again, the first control current and the second control current can be input into the image acquisition device as new control parameters, so that the image acquisition device controls the lens to move according to the new control parameters to directly reach the position with the best clarity.

[0139] Exemplarily, if the first control current is 20mA and the second control current is 40mA, the difference of 20mA can be directly fed back to the image acquisition device. The image acquisition device can directly control the lens to move a corresponding distance according to the received control current to reach the optimal clarity, thereby optimizing the conventional focusing process and improving the speed of focusing on the target object.

[0140] Based on the above embodiment, a method for obtaining the fusion clarity of the object recognition frame is further introduced below.

[0141] Figure 5A flow chart of another display method provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the steps include:

[0142] S501. For any control current, obtain the sizes of object recognition frames of different objects acquired by the image acquisition device under the control current.

[0143] In the embodiment of the present application, during the first focusing process, when the processor controls the image acquisition device to focus according to the control current, the image acquisition device can simultaneously identify the size of the object recognition frame when performing object recognition.

[0144] For example, Figure 6 As shown, during the focusing process of the image acquisition device, there is a corresponding object recognition frame size 601 at the face recognition frame of each participant in the acquired image.

[0145] Taking the case where three people attend a meeting and the upper limit of the control current is 60 mA as an example, during the first focusing process, the corresponding relationship between the control current and the face recognition frame size of different users (participants) can be obtained as shown in Table 2 below.

[0146] Table 2

[0147]

[0148]

[0149] S502: Obtain a corresponding weight value according to the size of the object recognition box.

[0150] In the embodiment of the present application, object recognition frames of different sizes may have the same or different weight values. The weight values ​​may be determined based on prior knowledge. For example, they may be debugged and set during functional testing.

[0151] The weight value may be pre-stored in a storage unit of the display device, and when the processor determines the size of the object recognition box, the weight value corresponding to the size of the object recognition box may be obtained therefrom.

[0152] S503: Perform weighted processing on the clarity of the object recognition frame according to the weight value to obtain the fusion clarity of the object recognition frame.

[0153] In the embodiment of the present application, when the corresponding weight of the clarity of the object recognition frame is determined, the corresponding relationship between the control current and the clarity of the face recognition frame of different users (participants) and the weight is obtained.

[0154] Taking the case where three people attend a meeting and the upper limit of the control current is 60 mA as an example, during the first focusing process, the corresponding relationship between the control current and the clarity of the face recognition frame of different users (participants) and the weight can be obtained as shown in Table 3 below.

[0155] Table 3

[0156]

[0157] The clarity of the object recognition frame is weighted according to the weight to obtain the fusion clarity of the object recognition frame corresponding to the control current.

[0158] For example, taking 20mA in Table 3 as an example, the fusion clarity of the object recognition frame corresponding to 20mA can be: 0.8*0.8+0.6*0.7+0.5*0.6=1.36

[0159] The display method provided in the embodiment of the present application can quickly determine the optimal focus control current during the first focusing process by acquiring the fusion clarity of the object recognition frame, thereby obtaining a display picture with the best comprehensive clarity and a better video for the user.

[0160] In the embodiment of the present application, during the video display process of the display device, the objects therein may change. For example, in a video conference, the number of participants may increase or decrease, the positions of the participants may move, and the position of the display device may move. In this case, the processor needs to determine whether refocusing is required. Figure 7 The above situation is explained.

[0161] Figure 7 A flow chart of another display method provided in an embodiment of the present application is shown as follows: Figure 7 As shown, the steps include:

[0162] S701. Determine whether there is an event that triggers refocusing; the event includes: at least one of a change in the number and / or position of the objects, and a change in the position of the image acquisition device.

[0163] In the embodiment of the present application, the processor can determine whether there is an event that triggers refocusing based on the data collected by the image acquisition device.

[0164] Exemplarily, when focusing for the first time, the processor can record the number of objects and the position of each object in the current picture according to the captured picture of the image capture device, and determine the position of the image capture device according to the positioning data. The position of the object can be the coordinates of the object in the captured picture. The positioning data can be data determined based on longitude and latitude, or data determined by a global positioning system.

[0165] During the display process, the processor uses the image acquisition device to detect objects in the captured picture. If a change in the number and / or position of the objects is determined, the refocusing process is performed. Also, when a change in the position of the image acquisition device is monitored, the refocusing process is performed.

[0166] S702: If yes, control the image acquisition device to refocus.

[0167] In one possible implementation, if the refocusing event is a change in the position of the image acquisition device, after entering the refocusing process, in order to reduce the number of focus times and improve the user's video experience, it can be determined whether refocusing is needed based on the distance of the change in the position of the image acquisition device.

[0168] Exemplarily, if the distance of the change in the position of the image acquisition device is less than or equal to the preset distance, the clarity of the current picture is maintained for display; if the distance of the change in the position of the image acquisition device is greater than the preset distance, the focus is re-performed. The distance of the change in the position of the image acquisition device may be a distance of left-right change or a distance of front-back change, and the present application embodiment does not limit this. The re-focusing method may refer to the first focusing method. The present application embodiment does not elaborate on this.

[0169] In a possible implementation, if the refocusing event is a change in the number and / or position of objects, after entering the refocusing process, refocusing can be performed according to the changed position of the object.

[0170] Exemplarily, the image acquisition device is controlled to perform object recognition to determine the number and positions of the objects in the captured image; based on the number and positions of the objects, when it is determined that the number and / or positions of the objects change, the position coordinates of the changed objects are obtained; and based on the position coordinates, the image acquisition device is controlled to refocus.

[0171] In the embodiment of the present application, the position of the object can be represented by the coordinates of the acquisition object recognition frame, for example, the coordinates of the center point of the object recognition frame.

[0172] In the process of continuously monitoring the objects in the picture, if it is determined that the number of objects increases or decreases, and the position of the objects moves, it is determined that the number and / or position of the objects changes, and the position coordinates of the changed objects are obtained.

[0173] In a possible implementation, when controlling the image acquisition device to refocus according to the position coordinates, in order to reduce the number of focusing times and improve the user's video experience, it can be determined whether refocusing is needed based on the distance of the change amplitude of the position coordinates.

[0174] Exemplarily, if the ordinate value of the position coordinate is outside a preset range, the image acquisition device is controlled to refocus; the preset range is determined according to the ordinate of the acquired object during the first focusing process; if the ordinate value of the position coordinate is within the preset range, the clarity of the current picture is maintained for display.

[0175] In the embodiment of the present application, an important parameter affecting the clarity of the image during focusing is the depth of field, which, when mapped to an object in the image, may be the vertical coordinate in the object position coordinates.

[0176] For example, Figure 8 As shown, during the first focusing process, the processor can determine the object 801 with the largest ordinate value in the picture and the object 802 with the smallest ordinate value according to the picture captured by the image acquisition device. The ordinate values ​​of the objects 801 and 802 are used as the upper limit and lower limit of the preset range. For example, if the ordinate values ​​of the objects 801 and 802 are a and b respectively, the preset range is [a, b].

[0177] The processor determines that there is a new object or an object is reduced, or the position of the object is moved, and determines the relationship between the vertical coordinate of the new object, the vertical coordinate of the reduced object, or the vertical coordinate after the position is moved and the preset range. If it is within the preset range, it can be considered that the changed position is close to the center of the image, and there is no need to refocus. If it is outside the preset range, the image acquisition device is controlled to refocus. The refocusing method is similar to the first focusing method, which will not be repeated here.

[0178] Exemplarily, the ordinate of the newly added object is c. If a<c<b, it means that the newly added object is close to the center of the picture, the clarity is good, and no refocusing is required. If c>b, or c<a, it means that the newly added object is far from the center of the picture, the clarity is poor, and refocusing is required.

[0179] In some embodiments, if there are multiple changed objects, the vertical coordinates of the multiple objects can be obtained, and the object with the largest vertical coordinate or the object with the smallest vertical coordinate is selected as the vertical coordinate for determining whether refocusing is required.

[0180] In some embodiments, if an object is moving, when the object stops moving, it is determined whether refocusing is required based on the vertical coordinate of the position when the object stops moving.

[0181] The display method provided in the embodiment of the present application can determine whether refocusing is required based on the coordinates of the changed object when the object in the picture changes. If the changed object is near the center of the picture, the current clarity is maintained for display, thereby reducing the number of refocusing times and reducing the number of times the displayed picture becomes blurred, thereby improving the user's video experience.

[0182] In summary, the display method provided in the embodiment of the present application includes the display method of the three processes of initial focus, speaker close-up and personnel change. Figure 9-11 , the complete flow of the three processes is briefly explained, still taking video conferencing as an example.

[0183] like Fig. 9 A schematic diagram of the initial focusing process provided in the embodiment of the present application is shown in FIG. Fig. 9 As shown, including:

[0184] A1. Face recognition of participants.

[0185] A2. Use different control currents for focusing.

[0186] A3. Determine the size, coordinates and clarity of the face recognition frame.

[0187] A4. Assign weight values.

[0188] A5. Determine the corresponding table between the control current value and the clarity at each face recognition frame.

[0189] A6. Add weight to calculate the fusion clarity value.

[0190] A7. Take the control current value A when the fusion clarity value is maximum.

[0191] A8. Focus according to the control current A.

[0192] like Fig.10 A flow chart of a close-up of a spokesperson provided in the embodiment of the present application is as follows: Fig.10 As shown, including:

[0193] B1. Determine the target spokesperson.

[0194] B2. Query the clarity and control current value of the initial focusing process for the target speaker.

[0195] B3: Whether the difference between the current clarity of the target speaker and the optimal clarity is greater than a threshold.

[0196] B4. If yes, send the current value at the best definition to the image acquisition device. If no, keep the current definition for close-up.

[0197] B5. The image acquisition device reaches the specified focus position.

[0198] like Fig.11 A flow chart of personnel changes provided in the embodiment of the present application is as follows: Fig.11 As shown, including:

[0199] C1. Image acquisition device face recognition.

[0200] C2. Confirm the coordinates of all faces in initial focus.

[0201] C3: Whether the number of faces has changed. If so, execute the step shown in C4; if not, the process ends.

[0202] C4. Whether the vertical axis is within the preset range.

[0203] C5. If not, refocus. If yes, keep the current clarity for a close-up. The process ends.

[0204] It should be understood that Figure 9-11 The specific implementation methods of the display methods of the three processes provided, namely, initial focus, speaker close-up, and personnel change, are similar to those in the aforementioned embodiments and will not be described in detail here.

[0205] The embodiment of the present application also provides a display device.

[0206] Fig.12 A schematic diagram of the structure of the display device 120 provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, including:

[0207] The acquisition module 1201 is used to control the image acquisition device to obtain the clarity of the object recognition frame of different objects acquired by the image acquisition device under different control currents during the first focusing process; the control current is used to drive the image acquisition device to focus.

[0208] The focusing module 1202 is used to determine a first control current of the image acquisition device according to the clarity of the object recognition frame, and control the image acquisition device to focus according to the first control current.

[0209] The close-up module 1203 is used to control the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame when taking a close-up of the target object, if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold.

[0210] The processing module 1204 is used to maintain the clarity of the current object recognition frame of the target object for close-up if the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold.

[0211] In some embodiments, the close-up module 1203 is further used to obtain a second control current corresponding to an upper limit value of the clarity of the object recognition frame of the target object; and control the image acquisition device to focus on the target object according to the difference between the second control current and the first control current.

[0212] In some embodiments, the focusing module 1202 is further used to obtain the fusion clarity of the object recognition frame under different control currents; and use the control current value corresponding to the upper limit value of the fusion clarity of the object recognition frame as the first control current value.

[0213] In some embodiments, the focusing module 1202 is further used to obtain, for any control current, the size of the object recognition frame of different objects captured by the image acquisition device under the control current; obtain the corresponding weight value according to the size of the object recognition frame; and perform weighted processing on the clarity of the object recognition frame according to the weight value to obtain the fusion clarity of the object recognition frame.

[0214] In some embodiments, the close-up module 1203 is further used to control the sound acquisition device to obtain the sound source direction of the target object; determine the target object according to the picture acquired by the image acquisition device and the sound source direction of the target object.

[0215] In some embodiments, the focusing module 1202 is further used to determine whether there is an event that triggers refocusing; the event includes: at least one of a change in the number and / or position of the objects, and a change in the position of the image acquisition device; if present, the image acquisition device is controlled to refocus.

[0216] In some embodiments, the focusing module 1202 is also used to control the image acquisition device to perform object recognition and determine the number and position of the objects in the captured picture; determine, based on the number and position of the objects, when the number and / or position of the objects change, and obtain the position coordinates of the changed objects; and control the image acquisition device to refocus based on the position coordinates.

[0217] In some embodiments, the focusing module 1202 is also used to control the image acquisition device to refocus if the vertical coordinate value of the position coordinate is outside a preset range; the preset range is determined according to the vertical coordinate of the acquired object during the first focusing process; if the vertical coordinate value of the position coordinate is within the preset range, the clarity of the current picture is maintained for display.

[0218] The display device provided in the present application is used to execute the technical solution of the display method provided in any of the aforementioned embodiments. Its implementation principle and technical effect are similar and will not be described in detail.

[0219] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0220] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one control module of the display device can read the execution instruction from the readable storage medium, and at least one control module executes the execution instruction so that the display device implements the display method provided by the various embodiments described above.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0222] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: The display device includes a processor and an image acquisition device, and the processor is configured to: Controlling the image acquisition device to obtain the clarity of object recognition frames of different objects acquired by the image acquisition device under different control currents during the first focusing process; The control current is used to drive the image acquisition device to focus; Determining a first control current of the image acquisition device according to the clarity of the object recognition frame, and controlling the image acquisition device to focus according to the first control current; When taking a close-up of a target object, if the difference between the clarity of an object recognition frame of the target object and an upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold, controlling the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame; If the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, the clarity of the current object recognition frame of the target object is maintained for a close-up.

2. The display device according to claim 1, characterized in that The processor is configured to: Acquire a second control current corresponding to an upper limit value of the clarity of the object recognition frame of the target object; The image acquisition device is controlled to focus on the target object according to the difference between the second control current and the first control current.

3. The display device according to claim 2, characterized in that The processor is configured to: Obtain the fusion clarity of the object recognition frame under different control currents; The control current value corresponding to the upper limit value of the fusion clarity of the object recognition frame is used as the first control current value.

4. The display device according to claim 3, characterized in that The processor is configured to: For any control current, obtaining the size of the object recognition frame of different objects acquired by the image acquisition device under the control current; Obtaining a corresponding weight value according to the size of the object recognition frame; The clarity of the object recognition frame is weighted according to the weight value to obtain the fusion clarity of the object recognition frame.

5. The display device according to claim 2, characterized in that The display device further includes a sound collection device, and the processor is configured to: Controlling the sound collection device to obtain the sound source direction of the target object; The target object is determined according to the picture captured by the image acquisition device and the direction of the sound source of the target object.

6. The display device according to claim 1, characterized in that The processor is configured to: Determine whether there is an event that triggers refocusing; the event includes: at least one of a change in the number and / or position of the objects, and a change in the position of the image acquisition device; If so, the image acquisition device is controlled to refocus.

7. The display device according to claim 6, characterized in that The processor is configured to: Controlling the image acquisition device to perform object recognition and determine the number and position of the objects in the acquired picture; When determining, based on the number and positions of the objects, that the number and / or positions of the objects change, obtaining the position coordinates of the changed objects; According to the position coordinates, the image acquisition device is controlled to refocus.

8. The display device according to claim 7, characterized in that The processor is configured to: If the ordinate value of the position coordinate is outside a preset range, the image acquisition device is controlled to refocus; the preset range is determined according to the ordinate value of the acquired object during the first focusing process; If the vertical coordinate value of the position coordinate is within a preset range, the clarity of the current picture is maintained for display.

9. A display method, characterized in that: include: Controlling the image acquisition device to obtain the clarity of the object recognition frame of the object at different positions acquired by the image acquisition device under different control currents during the first focusing process; The control current is used to drive the image acquisition device to focus; Determining a first control current value of the image acquisition device according to the clarity of the object recognition frame, and controlling the image acquisition device to focus according to the first control current value; When taking a close-up of a target object, if the difference between the clarity of an object recognition frame of the target object and an upper limit value of the clarity of the object recognition frame of the target object is greater than a preset threshold, controlling the image acquisition device to focus on the target object according to the upper limit value of the clarity of the object recognition frame; If the difference between the clarity of the object recognition frame of the target object and the upper limit value of the clarity of the object recognition frame of the target object is less than or equal to a preset threshold, the clarity of the current object recognition frame of the target object is maintained for a close-up.

10. A storage medium, characterized in that: The storage medium stores computer-executable instructions, which are used to implement the method according to claim 9 when executed by a processor.

Images