Display device and processing method for display device

Abstract

The invention relates to a display device and a processing method for the display device, in particular to the technical field of display devices. The display device includes: a memory configured to store computer instructions and / or data associated with the display device; the at least one processor is connected with the memory and is configured to execute the computer instruction to enable the display equipment to monitor a camera state broadcast message sent by a camera service in an operating system, and the camera state broadcast message is used for indicating whether a camera is started by an application program or not; if the camera state broadcast message is a camera release message, calling a camera to collect a camera image through a global application of the display device, the camera release message being used for indicating that the camera is not started by the application program; human body feature data is obtained from the camera image, and the display device is controlled to execute the corresponding function according to the control instruction corresponding to the human body feature data, so that the problem that conflicts exist when the application program calls the camera is solved.

Description

Display device and processing method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on December 2, 2022, with application number 202211544807.2; filed on December 19, 2022, with application number 202211635507.5; filed on December 28, 2022, with application number 202211697408.X; filed on December 29, 2022, with application number 202211711091.0; and filed on January 9, 2023, with application number 202310027361.4, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a processing method for the display device. Background Art

[0004] With the improvement of chip performance and the use of accelerated processing units (APUs) and graphics processing units (GPUs) in display devices, the ability to perform image processing and deep learning operations on displays has been further enhanced. During image processing, the camera is centrally managed by the display device's operating system, and only one application can use the camera at a time. Currently, to ensure that artificial intelligence (AI) algorithms that require camera access in all scenarios can call the camera, related technologies will terminate the processes of other applications using the camera and allocate camera resources to the AI ​​algorithm. This affects other applications' use of the camera and can easily lead to confusion in camera resource management.

[0005] Summary of the Invention

[0006] According to an embodiment of the present application, a display device includes: a memory configured to store computer instructions and / or data associated with the display device; at least one processor connected to the memory and configured to execute the computer instructions to enable the display device to: monitor a camera status broadcast message sent by a camera service in an operating system, the camera status broadcast message being used to indicate whether the camera is enabled by an application; if the camera status broadcast message is a camera release message, the camera is called through a global application of the display device to capture a camera image, the camera release message being used to indicate that the camera is not enabled by the application; and obtain human feature data from the camera image, and control the display device to perform corresponding functions according to control instructions corresponding to the human feature data.

[0007] According to an embodiment of the present application, a processing method for a display device includes: monitoring a camera status broadcast message sent by a camera service in an operating system, the camera status broadcast message being used to indicate whether the camera is enabled by an application; if the camera status broadcast message is a camera release message, calling the camera through a global application of the display device to capture a camera image, the camera release message being used to indicate that the camera is not enabled by the application; obtaining human body feature data from the camera image, and controlling the display device to perform corresponding functions according to control instructions corresponding to the human body feature data. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to some embodiments of the present application;

[0009] FIG2 is a schematic diagram of an application scenario of a display device according to some embodiments of the present application;

[0010] FIG3 is a configuration block diagram of a control device 100 according to some embodiments of the present application;

[0011] FIG4 is a block diagram of a hardware configuration of a display device 200 according to some embodiments of the present application;

[0012] FIG5A is a schematic diagram of software configuration in a display device 200 according to some embodiments of the present application;

[0013] FIG5B is a schematic diagram showing an icon control interface of an application in a display device 200 according to one or more embodiments of the present application;

[0014] FIG6 is a flowchart diagram 1 of a processing method for a display device according to some embodiments of the present application;

[0015] FIG7 is a schematic diagram of a user interface according to some embodiments of the present application;

[0016] FIG8 is a schematic diagram of a camera image according to some embodiments of the present application;

[0017] FIG9 is a second flow chart of a processing method for a display device according to some embodiments of the present application;

[0018] FIG10 is a third flow chart of a processing method for a display device according to some embodiments of the present application;

[0019] FIG11 is a schematic diagram of a horizontal detection area and a vertical detection area according to some embodiments of the present application;

[0020] FIG12 is a flowchart diagram 1 of another processing method for a display device according to some embodiments of the present application;

[0021] FIG13 is a schematic diagram of a video page according to some embodiments of the present application;

[0022] FIG14 is a schematic diagram of a rectangular area of ​​a human body contour according to some embodiments of the present application;

[0023] FIG15 is a schematic diagram of vertex coordinates of a preset effective control area according to some embodiments of the present application;

[0024] FIG16 is a schematic diagram of a video page according to some embodiments of the present application;

[0025] FIG17 is a second schematic diagram of a video page according to some embodiments of the present application;

[0026] FIG18 is a schematic diagram of a process for calculating a first size of a human body part that is not captured according to some embodiments of the present application;

[0027] FIG19 is a second flow chart of another processing method for a display device according to some embodiments of the present application;

[0028] FIG20 is a schematic diagram of second prompt information according to some embodiments of the present application;

[0029] FIG21A is a schematic diagram of a scenario of switching channels using a gesture display device according to some embodiments of the present application;

[0030] FIG21B is a schematic diagram of another scenario of switching channels using a gesture display device according to some embodiments of the present application;

[0031] FIG22A is a framework diagram of a gesture setting system for a display device according to one or more embodiments of the present application;

[0032] FIG22B is a diagram illustrating an architecture for gesture settings for a display device according to one or more embodiments of the present application;

[0033] FIG23A is a flowchart diagram 1 of a third processing method for a display device according to some embodiments of the present application;

[0034] FIG23B is a schematic diagram of a scene of a gesture setting interface according to some embodiments of the present application;

[0035] FIG23C is a second flow chart of a third processing method for a display device according to some embodiments of the present application;

[0036] FIG23D is a third flow chart of a third processing method for a display device according to some embodiments of the present application;

[0037] FIG23E is a schematic diagram of another scene of a gesture setting interface according to some embodiments of the present application;

[0038] FIG23F is a schematic diagram of a scenario of a user reminder interface according to some embodiments of the present application;

[0039] 23G is a fourth flow chart of a third processing method for a display device according to some embodiments of the present application;

[0040] FIG23H is a schematic diagram of a scenario for starting a gesture setting function interface according to some embodiments of the present application;

[0041] FIG23I is a fifth flow chart of a third processing method for a display device according to some embodiments of the present application;

[0042] FIG24 is a flowchart of a fourth processing method for a display device according to some embodiments of the present application;

[0043] FIG25 is a schematic diagram of a preset gesture according to some embodiments of the present application;

[0044] FIG26 is a schematic diagram of a first gesture image according to some embodiments of the present application;

[0045] FIG27 is a schematic diagram of a second gesture image according to some embodiments of the present application;

[0046] FIG28 is a schematic diagram of determining a gesture movement direction and a gesture movement speed according to some embodiments of the present application;

[0047] FIG29 is a schematic diagram of a coordinate range to be identified according to some embodiments of the present application;

[0048] FIG30 is a schematic diagram of a third gesture image according to some embodiments of the present application;

[0049] FIG31 is a flowchart of a processing method for a display device in the related art;

[0050] FIG32 is a system framework diagram of a fifth processing method for a display device according to one or more embodiments of the present application;

[0051] FIG33A is a flowchart diagram 1 of a fifth processing method for a display device according to some embodiments of the present application;

[0052] FIG33B is a schematic diagram of an interface when a display shows prompt information according to some embodiments of the present application;

[0053] FIG34A is a second flow chart of a fifth processing method for a display device according to some embodiments of the present application;

[0054] FIG34B is a schematic diagram of an interface when a display shows target information of resource management according to some embodiments of the present application;

[0055] FIG35 is a schematic diagram of an interaction process between modules for implementing a fifth processing method for a display device according to some embodiments of the present application;

[0056] FIG36 is a schematic diagram of the interaction process between modules of another implementation of the fifth processing method for a display device according to some embodiments of the present application. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0059] At present, display devices use cameras to capture images, and use AI algorithms to process the images to determine the corresponding control instructions, and control the display devices to perform corresponding operations. For example, in a global control scenario, the display device meets the needs of user gesture control by detecting user gestures. The camera will automatically start after the display device is turned on, but the camera is an exclusive resource and can only be used by one application at a time. When other applications use the camera to take pictures or record videos, the AI ​​algorithm's call to the camera will be preempted. In related technologies, in order to meet the AI ​​algorithm's call to the camera, the use of the camera by other applications will be immediately terminated, thereby transferring the camera resources to the AI ​​algorithm for use, which affects the normal operation of other applications and causes each application to snatch camera resources, resulting in chaotic camera resource management, which in turn affects the user's experience.

[0060] In order to solve the above problems, an embodiment of the present application provides a display device and a processing method for a display device, wherein the display device includes a memory configured to store computer instructions and / or data associated with the display device; at least one processor, connected to the memory, configured to execute a computer program to enable the display device to: monitor a camera status broadcast message sent by a camera service in an operating system to determine whether the camera is enabled by an application based on the camera status broadcast message; if the camera status broadcast message is a camera release message, indicating that the camera is not enabled by the application, then the global application of the display device calls the camera to capture a camera image, and then obtains human feature data from the camera image, so as to control the display device to perform a corresponding function according to the control instruction corresponding to the human feature data. By monitoring the status of whether other applications have enabled the camera, when the camera is not enabled by the application, that is, when the camera is idle, the global application calls the camera to capture the camera image to identify the camera image to realize control of the display device, so that the management of the camera becomes orderly, and the global application calls the camera more conveniently, which is conducive to human-computer interaction between the user and the display device.

[0061] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application. As shown in Figure 1, the diagram includes a control device 100, a display device 200, a smart device 300, and a server 400. A user can operate the display device 200 through the smart device 300 or the control device 100 to turn on the display device 200.

[0062] In some embodiments, the control device 100 may be a remote controller, and communication between the remote controller and the display device may include infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the display device 200. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc. In some embodiments, a mobile terminal, tablet computer, computer, laptop computer, or other smart device may also be used to control the display device 200.

[0063] In some embodiments, the display device 200 can also be controlled in ways other than control devices and smart devices. For example, the display device 200 can directly receive the user's voice command control through a module for obtaining voice commands configured inside the display device 200, or it can receive the user's voice command control through a voice control device set outside the display device 200, or it can receive the user's control through touch or gestures.

[0064] In some embodiments, the smart device 300 can install software applications on the display device 200 and connect to the display device 200 through a network communication protocol to achieve one-to-one control operations and data communication. Audio and video content displayed on the smart device 300 can also be transmitted to the display device 200 for synchronized display.

[0065] In some embodiments, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200. The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. In addition to providing broadcast reception television functionality, the display device 200 may also provide an intelligent network television function that provides computer support functionality.

[0066] The display device provided in the embodiments of the present application can have various implementation forms, for example, it can be a television, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. The embodiments of the present application do not limit the specific type of the display device.

[0067] Figure 2 is a schematic diagram of an application scenario of a display device according to some embodiments of the present application, illustrating a user controlling a display device 200 through gestures. Display device 200 is equipped with a camera 201, which can be used to capture images and record videos. Camera 201 can be a monocular camera, a binocular camera, or an infrared camera, which is shown in the figure for illustrative purposes only and is not intended to be limiting.

[0068] In the scenario shown in FIG2 , a user makes a corresponding gesture, and at least one processor 250 of the display device 200 is configured to execute computer instructions stored in the display device's memory, causing the display device 200 to: monitor a camera status broadcast message sent by a camera service in the operating system, which indicates whether the camera is enabled by an application. If the camera status broadcast message is a camera release message, indicating that the camera is not enabled by an application, the global application of the display device 200 calls the camera to capture a camera image, thereby allowing the camera to capture the user's gesture, and then obtain human body feature data from the camera image, so as to control the display device to perform corresponding functions according to the control instructions corresponding to the human body feature data.

[0069] Figure 3 is a configuration block diagram of the control device 100 provided in an embodiment of the present application. As shown in Figure 3, the control device 100 includes a processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that can be recognized and responded to by the display device 200, acting as an interactive intermediary between the user and the display device 200. The communication interface 130 is used to communicate with the outside world and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button or an alternative module.

[0070] FIG4 is a block diagram of the hardware configuration of the display device 200 provided in an embodiment of the present application. As shown in FIG4 , the display device 200 includes: a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and at least one of a user interface (i.e., a user input interface). Among them, the at least one processor 250 includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output. The display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen. The tuner and demodulator 210 receives broadcast and television signals by wired or wireless reception, and demodulates audio and video signals, such as and EPG data signals, from multiple wireless or wired broadcast and television signals. The communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include a Wifi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chips or near-field communication protocol chips, and at least one of an infrared receiver. The display device 200 can establish the sending and receiving of control signals and data signals with the external control device 100 or the server 400 through the communicator 220. The detector 230 is used to collect signals from the external environment or interactions with the outside. At least one processor 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be in an external device of the main device where at least one processor 250 is located, such as an external set-top box. The user interface may be used to receive control signals from a control device (such as an infrared remote control, etc.).

[0071] In some embodiments, the above-mentioned display device is a terminal device with a display function, such as a television, a mobile phone, a computer, a learning machine, etc.

[0072] In some embodiments, at least one processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The at least one processor 250 controls the overall operation of the display device 200. A user can enter user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user can enter user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0073] The output interface (display 260 and / or audio output interface 270 ) is configured to output user interaction information; the communicator 220 is used to communicate with the server 400 or other devices.

[0074] According to an embodiment of the present application, a display device 200 includes:

[0075] a memory configured to store computer instructions and / or data associated with the display device;

[0076] At least one processor 250, connected to the memory, is configured to execute the computer instructions to cause the display device to:

[0077] Monitor the camera status broadcast message sent by the camera service in the operating system. The camera status broadcast message is used to indicate whether the camera is enabled by the application.

[0078] If the camera status broadcast message is a camera release message, the camera is called by the global application of the display device to capture the camera image. The camera release message is used to indicate that the camera is not enabled by the application;

[0079] Obtain human feature data from the camera image, and control the display device to perform corresponding functions according to the control instructions corresponding to the human feature data.

[0080] The display device 200 monitors the camera status broadcast message sent by the camera service in the operating system to determine whether the camera is enabled by the application based on the camera status broadcast message. If the camera status broadcast message is a camera release message, indicating that the camera is not enabled by the application, the global application of the display device calls the camera to capture the camera image, and then obtains human feature data from the camera image to control the display device to perform the corresponding function according to the control instructions corresponding to the human feature data. By monitoring the status of whether other applications have enabled the camera, when the camera is not enabled by the application, that is, when the camera is idle, the global application calls the camera to capture the camera image, and then recognizes the camera image to control the display device. This makes the camera management more orderly and the global application's call to the camera more convenient, which is conducive to human-computer interaction between the user and the display device.

[0081] In some embodiments, if the camera status broadcast message is a camera release message, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to:

[0082] If the camera status broadcast message is a camera activation message, the display is controlled to display a camera activation icon on the current user interface to prompt the user that the camera is being activated by the application;

[0083] At least one processor 250 is configured to, if the camera status broadcast message is a camera release message, cause a global application of the display device to call a camera to capture a camera image, and is specifically configured to execute the computer instructions to cause the display device to:

[0084] If the camera status broadcast message is a camera release message, the display is controlled to cancel the display of the camera enable icon on the current user interface, and the camera is called through the global application of the display device to capture the camera image.

[0085] In some embodiments, at least one processor 250 obtains human feature data from a camera image and is specifically configured to execute the computer instructions to enable the display device to:

[0086] Detecting whether there are valid human features in the camera image, where valid human features include: gesture features and / or human posture features;

[0087] If there are valid human features in the camera image, the human feature data is obtained from the camera image.

[0088] In some embodiments, after detecting whether there is a valid human feature in the camera image, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to:

[0089] If there are no valid human features in the camera image, determine whether the camera is blocked;

[0090] If the camera is blocked, adjusting the frequency at which the global application calls the camera from the initial frequency to the target frequency, and / or outputting a first prompt message;

[0091] The target frequency is lower than the initial frequency, and the first prompt information is used to prompt the user that the camera of the display device is blocked.

[0092] In some embodiments, if the camera is blocked, at least one processor 250 adjusts the frequency at which the global application calls the camera from the initial frequency to the target frequency, and / or outputs a first prompt message, and is specifically configured to execute the computer instructions to cause the display device to:

[0093] Collect the image to be detected through the camera;

[0094] If the difference between the brightness value of the image to be detected and the minimum brightness value is less than or equal to the first preset value, the frequency of the global application calling the camera is adjusted from the initial frequency to the target frequency, and / or a first prompt message is output.

[0095] In some embodiments, the brightness value is represented by an RGB value or a grayscale value;

[0096] and / or,

[0097] The brightness value of the image to be detected includes at least one of the following:

[0098] The brightness value of each pixel in the image to be detected, the brightness value of each pixel in a partial area of ​​the image to be detected, the average brightness value of all pixels in the image to be detected, and the average brightness value of pixels in a partial area of ​​the image to be detected.

[0099] In some embodiments, if the camera is blocked, at least one processor 250 adjusts the frequency at which the global application calls the camera from the initial frequency to the target frequency, and / or outputs a first prompt message, and is specifically configured to execute the computer instructions to cause the display device to:

[0100] Collect the image to be detected through the camera;

[0101] Determining at least one horizontal detection area and at least one vertical detection area from the image to be detected;

[0102] If the differences between the brightness values ​​of any pixel points in the target direction in the target detection area are less than or equal to a second preset value, adjusting the frequency of the global application calling the camera from the initial frequency to the target frequency, and / or outputting a first prompt message;

[0103] The target detection area is a horizontal detection area, and the target direction is a vertical direction; or the target detection area is a vertical detection area, and the target direction is a horizontal direction.

[0104] As shown in Figure 5A, Figure 5A is a schematic diagram of the software configuration in the display device 200 provided in an embodiment of the present application. As shown in Figure 5A, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as "application layer"), the software development layer (Software Development Kit, SDK), the application framework layer (referred to as "framework layer"), the Android runtime (Android runtime) and system runtime library layer (referred to as "system library layer"), and the kernel layer.

[0105] In some embodiments, at least one application program runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the applications in the application layer include but are not limited to the above examples.

[0106] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0107] In some embodiments, the kernel layer is a layer between hardware and software, and includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0108] Among them, the application layer is responsible for calling the camera driver and calling the SDK layer interface to pass the captured image to the SDK layer for AI calculation, so as to receive the AI ​​algorithm results such as gestures and human key point data recognized by the SDK layer. Global applications in the application layer (such as AI Sense) are used to manage the status of the camera, including but not limited to: whether the camera is idle, whether the camera is blocked, the camera usage status, etc. Hishow represents the application that needs to occupy the camera for shooting, and Hishow and AI applications preempt the camera. The global application registers for the power-on broadcast when the display device is turned on. After receiving the power-on broadcast, it starts the service to register for the camera broadcast and monitor the camera status. Then, according to the monitored camera status, it executes operations such as calling the AI ​​interface and adjusting the frequency of the AI ​​interface, and processes the returned AI results to convert the AI ​​results into control instructions to control the display device to perform corresponding operations to achieve corresponding functions.

[0109] The SDK layer encapsulates image processing and image recognition models and provides external service access interfaces. The kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, Wi-Fi driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0110] The framework layer is used to modify the logic of the system library layer. When the system library layer obtains the status information of the camera being used, it records the package name and usage status of the application using the camera, and sends a broadcast indicating that the camera is occupied; when the camera is released, the usage mark is cleared and a broadcast message indicating that the camera is idle is sent.

[0111] Figure 5B is a schematic diagram of the icon control interface display of an application in the display device 200 according to one or more embodiments of the present application. As shown in Figure 5B, the application layer includes at least one application that can display corresponding icon controls in the display, such as: live TV application icon controls, video on demand application icon controls, media center application icon controls, application center icon controls, game application icon controls, etc.

[0112] A processing method for a display device according to an embodiment of the present application can be implemented based on the above-mentioned display device 200 .

[0113] In order to illustrate this solution in more detail, the following will be explained in an exemplary manner in conjunction with Figure 6. It can be understood that the steps involved in Figure 6 may include more steps or fewer steps in actual implementation, and the order of these steps may also be different, so as to be able to implement a processing method for a display device in an embodiment of the present application.

[0114] As shown in FIG6 , FIG6 is a flowchart of a processing method for a display device according to some embodiments of the present application. The method includes the following steps S601 to S603:

[0115] S601: Monitor the camera status broadcast message sent by the camera service in the operating system.

[0116] The camera status broadcast message is used to indicate whether the camera is enabled by the application. The camera status broadcast message is divided into a camera enable message and a camera release message. The camera enable message indicates that the camera is enabled by the application, and the camera release message indicates that the camera is not enabled by the application.

[0117] In some embodiments, if the camera status information broadcast by the camera service in the operating system is a camera activation message, indicating that the camera is being activated by an application, the present embodiment provides an implementation method: if the camera status broadcast message is a camera activation message, the display is controlled to display a camera activation icon on the current user interface to prompt the user that the camera is being activated by the application. This allows the user to be aware that the camera is being activated, which helps protect user privacy.

[0118] As shown in Figure 7, Figure 7 is a schematic diagram of a user interface according to some embodiments of the present application, showing a user interface 700 and a camera activation icon 701. When the camera status information sent by the camera service in the operating system is monitored as a camera activation message, the user interface 700 displays the camera activation icon 701.

[0119] S602: If the camera status broadcast message is a camera release message, call the camera through the global application of the display device to capture a camera image.

[0120] The camera release message is used to indicate that the camera is not enabled by the application.

[0121] Global apps are AI applications that use the camera in all scenarios. For example, a general video app can capture the user's image while playing a movie and recognize user gestures to control the volume. Global apps are invisible background apps to the user. When a global app uses the camera, the user cannot tell whether the camera is on.

[0122] In some embodiments, when the camera status information sent by the camera service in the operating system is monitored to be a camera release message, it indicates that the camera has not been enabled by the application and is in an idle state. The embodiment of the present application provides an implementation method: if the camera status broadcast message is a camera release message, the display is controlled to cancel the display of the camera enable icon on the current user interface, and the camera is called through the global application of the display device to capture the camera image.

[0123] S603: Acquire human feature data from the camera image, and control the display device to execute corresponding functions according to control instructions corresponding to the human feature data.

[0124] The human body feature data includes but is not limited to: human body contour feature data, hand gesture feature data, human body posture feature data, human body key point feature data, and facial feature data.

[0125] In some embodiments, in the process of obtaining human body feature data from the camera image, it is first detected whether there are valid human body features in the camera image. The valid human body features include: gesture features, and / or human posture features. In the case of detecting the presence of valid human body features in the camera image, human body feature data is obtained from the camera image. It is understandable that the valid human body features are pre-set partial human body feature data corresponding to the existence of control instructions. For example, when the camera image contains a complete human body image of the user, and the valid human body features are gesture features, it is detected that the camera image includes gesture features. Therefore, the camera image can determine the corresponding control instructions after AI calculation to meet the user's gesture control display device needs.

[0126] As shown in FIG8 , FIG8 is a schematic diagram of a camera image according to some embodiments of the present application, showing human body feature data 801 and valid human body features 802 in camera image 800. After detecting camera image 800, it is determined that valid human body features 802 exist in camera image 800, and then human body feature data 801 included in camera image 800 is obtained to perform AI calculations on human body feature data 801, thereby controlling the display device to perform corresponding functions according to control instructions corresponding to human body feature data 801.

[0127] If no valid human features are detected in the camera image, the camera is determined to be obstructed. It is understandable that, to protect user privacy, the camera of the display device is currently equipped with a mechanical switch. When the camera is not in use, turning off the mechanical switch can fundamentally prevent the camera from shooting without the user's awareness, effectively protecting privacy. Therefore, if the camera application collected by the global application of the display device calling the camera does not contain human features, it can further determine whether the camera is obstructed.

[0128] If the camera is blocked, the frequency of the global application calling the camera is adjusted from the initial frequency to the target frequency, where the target frequency is lower than the initial frequency. This reduces the frequency of the camera shooting when the camera is blocked, which helps save power consumption of the display device.

[0129] In some embodiments, if the camera is blocked, a first prompt message is output to inform the user that the display device's camera is blocked, so that the user can promptly remove the camera blockage when the global application needs to control the display device. Alternatively, if the camera is blocked, the frequency at which the global application calls the camera is adjusted from the initial frequency to the target frequency, and the first prompt message is output.

[0130] In some embodiments, as shown in FIG9 , FIG9 is a second flow chart of a processing method for a display device according to some embodiments of the present application, including the following steps S901 to S903:

[0131] S901: Capture an image to be detected through a camera.

[0132] In some embodiments, after determining that the camera is blocked, the camera is used to capture an image to be detected, so as to determine whether the camera is actually blocked by re-calling the camera for shooting, thereby reducing misjudgment and lowering the error rate.

[0133] S902: Determine whether the difference between the brightness value of the image to be detected and the minimum brightness value is less than or equal to a first preset value.

[0134] The first preset value is a preset brightness threshold corresponding to the image captured when the camera is blocked. It is understood that when the camera is blocked, the image captured by the camera is a pure black image. Mechanical switches blocking the camera may also result in light leakage caused by human error or error. The specific value of the first preset value can be set according to actual circumstances and is not limited by this application.

[0135] In some embodiments, the brightness value is represented by an RGB value or a grayscale value, and / or the brightness value of the image to be detected includes at least one of the following situations: the brightness value of each pixel in the image to be detected, the brightness value of each pixel in a partial area of ​​the image to be detected, the average brightness of all pixels in the image to be detected, and the average brightness of pixels in a partial area of ​​the image to be detected.

[0136] In some feasible embodiments, the first preset value is 30, the difference between the R value and the minimum B value of each pixel in the image to be detected is calculated, and then it is determined whether the difference is less than or equal to the first preset value 30.

[0137] When the difference between the brightness value of the image to be detected and the minimum brightness value is less than or equal to the first preset value, step S903 is executed.

[0138] When the difference between the brightness value of the image to be detected and the minimum brightness value is greater than the first preset value, it indicates that the camera is not blocked. Optionally, the frequency of calling the camera by the global application is increased.

[0139] S903: Adjust the frequency of the global application calling the camera from the initial frequency to the target frequency, and / or output a first prompt message.

[0140] The above steps collect the image to be detected by re-calling the camera, thereby accurately judging whether the camera is blocked according to the image to be detected.

[0141] In some embodiments, as shown in FIG10 , FIG10 is a flowchart diagram of a processing method for a display device according to some embodiments of the present application, including the following steps S1001 to S1003:

[0142] S1001. Collect an image to be detected through a camera.

[0143] S1002: Determine at least one horizontal detection area and at least one vertical detection area from the image to be detected.

[0144] As shown in Figure 11, Figure 11 is a schematic diagram of the horizontal detection area and the vertical detection area provided in an embodiment of the present application. Figure 11 shows the image to be detected 11A. In the coordinate system where the image to be detected 11A is located, at least one horizontal detection area 11B is determined from the coordinate origin along the positive direction of the horizontal axis, as shown in a in Figure 11; and at least one vertical detection area 11C is determined from the coordinate origin along the vertical axis, as shown in b in Figure 11.

[0145] S1003: Determine whether the differences between the brightness values ​​of any pixel points in the target direction in the target detection area are all less than or equal to a second preset value.

[0146] The brightness value is represented by RGB or grayscale value.

[0147] The second preset value is a pre-set brightness threshold. The specific value can be set according to actual conditions and is not limited in this application.

[0148] The target detection area is a horizontal detection area, and the target direction is a vertical direction; or, the target detection area is a vertical detection area, and the target direction is a horizontal direction.

[0149] In some embodiments, the difference in brightness between any pixel points in the target direction within the target area is calculated to determine whether the difference is less than or equal to a second preset value. For example, the difference in brightness between any pixel points in the vertical direction within the horizontal detection area is calculated to determine whether the difference is less than or equal to the second preset value.

[0150] When the differences between the brightness values ​​of any pixel points in the target direction in the target detection area are all less than or equal to the second preset value, step S1004 is executed.

[0151] When the difference between the brightness values ​​of any pixel points in the target direction in the target detection area is greater than the second preset value, it means that the camera is not actually blocked. Optionally, the frequency of calling the camera by the global application is increased.

[0152] S1004: Adjust the frequency of the global application calling the camera from the initial frequency to the target frequency, and / or output a first prompt message.

[0153] If the difference between the brightness values ​​of any pixel points in the target direction in the target detection area is less than or equal to the second preset value, the frequency of the global application calling the camera is adjusted from the initial frequency to the target frequency, and / or the first prompt information is output.

[0154] The above steps divide the re-shot image to be detected into at least one horizontal detection area and at least one vertical detection area, and accurately and quickly determine whether the camera is blocked by comparing the difference between any pixel points in the horizontal direction, or comparing the difference between any pixel points in the vertical direction. In this way, when the camera is blocked, the frequency of the camera is reduced to reduce the energy consumption of the display device; and / or, a first prompt message is output to prompt the user that the camera is blocked.

[0155] In summary, the embodiment of the present application provides a first processing method for a display device, which first monitors the camera status broadcast message sent by the camera service in the operating system to determine whether the camera is enabled by the application based on the camera status broadcast message; if the camera status broadcast message is a camera release message, indicating that the camera is not enabled by the application, the global application of the display device calls the camera to capture the camera image, and then obtains human feature data from the camera image to control the display device to perform the corresponding function according to the control instructions corresponding to the human feature data. By monitoring the status of whether other applications have enabled the camera, when the camera is not enabled by the application, that is, when the camera is idle, the global application calls the camera to capture the camera image to identify the camera image to realize the control of the display device, so that the management of the camera becomes orderly, and the global application calls the camera more conveniently, which is conducive to the human-computer interaction between the user and the display device.

[0156] Under related technologies, display devices use built-in or external cameras to capture user images for image recognition, enabling functions such as gesture control, AI fitness, and somatosensory gaming. However, due to camera angle limitations, fixed positions, or the limited movable distance between the user and the camera, it is difficult to capture images that include complete gestures or the user's full body. The captured images contain incomplete data and information that the image recognition algorithm needs to process, which can lead to inaccurate results and affect human-computer interaction performance.

[0157] In order to solve some or all of the above-mentioned technical problems, some embodiments of the present application provide another processing method for a display device, which includes: controlling the camera to capture a target image and displaying a video page on the display, in which the target image is displayed; then performing human body image segmentation on the target image to determine the human body contour rectangular area in the target image, and obtaining the vertex coordinates of the human body contour rectangular area in the video page, and then determining the adjustment parameters of the camera according to the vertex coordinates of the human body contour rectangular area to adjust the camera position according to the adjustment parameters. The display device can therefore control the camera to capture a target image including a complete human body contour rectangular area, thereby achieving fast and accurate adjustment of the camera position and improving human-computer interaction performance.

[0158] Still in the scenario shown in FIG. 2 , the user makes a corresponding gesture, and the display device 200 controls the camera to capture the target image and displays the video page on the display, so that the target image is displayed in real time on the video page. The target image is then segmented to determine the human body outline rectangular area in the target image, and the vertex coordinates of the human body outline rectangular area in the video page are obtained. Adjustment parameters are then determined based on the vertex coordinates of the human body outline rectangular area, and the camera position is adjusted based on the adjustment parameters, so that the camera captures the complete gesture image made by the user. By obtaining the vertex coordinates of the human body outline rectangular area in the video page in real time, it is possible to quickly and accurately analyze whether the human body outline rectangular area is completely within the video page to determine whether the complete user gesture has been captured, thereby facilitating the user to control the display device through gestures.

[0159] Based on the above-mentioned another processing method for a display device, the display device 200 according to some embodiments of the present application includes at least one processor 250, which is further configured to execute computer instructions stored in the memory of the display device 200 so that the display device 200:

[0160] Control the camera to capture the target image, display the video page on the monitor, and display the target image on the video page;

[0161] Perform human body image segmentation on the target image to determine the human body contour rectangular area, and obtain the vertex coordinates of the human body contour rectangular area in the video page;

[0162] Determine adjustment parameters according to vertex coordinates of the rectangular area of ​​the human body contour;

[0163] Adjust the camera position based on the adjustment parameters.

[0164] The above-mentioned display device 200 controls the camera to capture the target image and displays the video page on the display. The target image will be displayed in the video interface, and then the target image is segmented to determine the human body contour rectangular area, and the vertex coordinates of the human body contour rectangular area in the video page are obtained; then the adjustment parameters are determined according to the vertex coordinates of the human body contour rectangular area, and the camera position is adjusted according to the adjustment parameters, so that the human body contour rectangular area is completely in the video interface. The display device then captures the target image including the complete human body, and then recognizes such a target image, determines the corresponding control instructions to perform the corresponding operation, which can meet the user's diverse needs such as gesture control, somatosensory games, AI fitness, etc., and improves the human-computer interaction performance.

[0165] In some embodiments, before determining the adjustment parameters based on the vertex coordinates of the rectangular area of ​​the human body outline, the at least one processor 250 is further configured to execute computer instructions to enable the display device to:

[0166] Get the vertex coordinates of the preset effective control area in the video page;

[0167] At least one processor 250 determines adjustment parameters based on vertex coordinates of the rectangular area of ​​the human body outline, and is specifically configured to execute computer instructions to enable the display device to:

[0168] The adjustment parameters are determined according to the vertex coordinates of the rectangular area of ​​the human body outline and the vertex coordinates of the effective control area preset in the video page.

[0169] In some embodiments, at least one processor 250 determines adjustment parameters based on vertex coordinates of the human body outline rectangular area and vertex coordinates of a preset valid control area in the video page, and is specifically configured to execute computer instructions to enable the display device to:

[0170] Determine, based on vertex coordinates of the human body outline rectangular area and vertex coordinates of a preset effective control area in the video page, whether the human body outline rectangular area overlaps with a margin area in the video page, where the margin area is an area in the video page excluding the effective control area;

[0171] When the human body outline rectangular area overlaps with the margin area in the video page, determining the orientation information of the overlapping area in the video page;

[0172] The adjustment parameters are determined according to the position information of the overlapping area in the video page.

[0173] In some embodiments, the adjustment parameter is an adjustment direction. The at least one processor 250 determines the adjustment parameter based on the direction information of the overlapping area in the video page, and is specifically configured to execute computer instructions to cause the display device to:

[0174] When the overlapping area is in the first direction in the video page, the adjustment direction of the camera is determined to be the first direction.

[0175] In some embodiments, the adjustment parameter is an adjustment direction. The at least one processor 250 determines the adjustment parameter based on the vertex coordinates of the rectangular area of ​​the human body contour, and is specifically configured to execute computer instructions to enable the display device to:

[0176] When it is determined that the human body outline rectangular area does not include the complete human body, determining a second direction of a position where the human body part is not captured relative to the video page;

[0177] Determine that the adjustment direction of the camera is in the second direction.

[0178] In some embodiments, the at least one processor 250 is further configured to execute computer instructions to cause the display device to:

[0179] When it is determined that the human body outline rectangular area does not include the complete human body, calculating a first size of the human body portion that is not captured according to the vertex coordinates of the human body outline rectangular area;

[0180] Obtaining a second size of a target background area, where the target background area is a background area in a direction opposite to a direction where a human body portion is not captured on the video page;

[0181] When the first size is less than or equal to the second size, determining a second direction of a position where the human body part is not captured relative to the video page; and determining that the adjustment direction of the camera is the second direction;

[0182] When the first size is larger than the second size, outputting a first prompt message, the first prompt message being used to prompt the user to stay away from the camera;

[0183] The first size is the first width, and the second size is the second width; or, the first size is the first length, and the second size is the second length.

[0184] In some embodiments, at least one processor 250 determines a third size of the captured human body part in the video page;

[0185] Calculate the first size of the uncaptured human body part on the video page based on the ratio of the third size to the target size;

[0186] The target size ratio is the size ratio of the uncaptured human body part to the complete human body.

[0187] In some embodiments, the at least one processor 250 determines the size of the complete human body in the video page when determining that the human body outline rectangular area includes the complete human body;

[0188] When the size of the complete human body in the video page is less than half of the size of the video page, a second prompt message is output, and the second prompt message is used to prompt the user to get closer to the camera.

[0189] Another processing method for a display device provided in some embodiments of the present application can be implemented based on the above-mentioned display device.

[0190] As shown in FIG. 12 , FIG. 12 is a flowchart diagram of another processing method for a display device according to some embodiments of the present application. The method includes the following steps S1201 to S1204 :

[0191] S1201: Control the camera to capture a target image and display a video page on the display.

[0192] Among them, the target image is displayed in the video page.

[0193] As shown in Figure 13, which is a schematic diagram of a video page according to some embodiments of the present application, the figure shows a video page 1300, in which a target image 1301 is displayed. The target image 1301 is displayed in the video page 1300 in a full-screen manner.

[0194] It should be noted that when the user uses the display device to collect and identify full-body limb images, a certain amount of margin space will be reserved for the camera to collect the target image. A part of the area in the video page can be pre-set as the effective control area, and the remaining area can be set as the margin area in the form of a gray mask. Referring to the preset effective control area 1302 and margin area 1303 in Figure 13, the display device will analyze and process the image information included in the preset effective control area.

[0195] S1202: Perform human body image segmentation on the target image to determine a human body contour rectangular area, and obtain vertex coordinates of the human body contour rectangular area in the video page.

[0196] In some embodiments, AI algorithms such as image processing models and image recognition models encapsulated by the SDK layer perform human image segmentation on the target image to separate the human image from the target image, and then mark the human outline rectangular area in the target image, as shown in Figure 14, which is a schematic diagram of the human outline rectangular area according to some embodiments of the present application. The figure shows a video page 1300, which displays a target image 1301. After performing human image segmentation on target image 1301, a human outline rectangular area 1401 is obtained.

[0197] In some embodiments, the coordinates of the human body outline rectangular area in the video page are calculated by an AI algorithm. It can be understood that the AI ​​algorithm outputs the coordinate values ​​of the four sides of the human body outline rectangular area: top, bottom, left, and right, where top is the upper vertical coordinate of the human body outline rectangular area, bottom is the lower vertical coordinate of the human body outline rectangular area, left is the left horizontal coordinate of the human body outline rectangular area, and right is the right horizontal coordinate of the human body outline rectangular area. Then, according to the four boundary values ​​of top, bottom, left, and right, the vertex coordinates of the human body outline boundary area are obtained, which can be the upper left vertex coordinates and the lower right vertex coordinates, or the upper right vertex coordinates and the lower left vertex coordinates. Referring to Figure 14, the upper left vertex bodyPointA (left, top) and the lower right vertex bodyPointB (right, bottom) are obtained.

[0198] S1203: Determine adjustment parameters according to the vertex coordinates of the rectangular area of ​​the human body contour.

[0199] The adjustment parameter may be a parameter for adjusting the direction of the camera or a parameter for adjusting the angle of the camera.

[0200] In some embodiments, before determining the adjustment parameters based on the vertex coordinates of the rectangular area of ​​the human body contour, the vertex coordinates of the preset effective control area in the video page are obtained, wherein the size of the preset effective control area can be fixed, so that the vertex coordinates of the preset effective control area are obtained in the video coordinate system corresponding to the video page, as shown in Figure 9. Figure 15 is a schematic diagram of the vertex coordinates of the effective control area according to some embodiments of the present application, the upper left corner vertex coordinates controlPointA (X1, Y1) and the lower right corner vertex coordinates controlPointB (X2, Y2) of the preset effective control area 1302; then, the adjustment parameters are determined based on the vertex coordinates of the rectangular area of ​​the human body contour and the vertex coordinates of the effective control area preset in the video page.

[0201] Optionally, compare the horizontal coordinate left of the upper left corner vertex bodyPointA of the human body contour rectangular area to see whether it is less than the horizontal coordinate X1 of the preset upper left corner vertex coordinate controlPointA of the effective control area; or, compare the horizontal coordinate right of the lower right corner vertex bodyPointB of the human body contour rectangular area to see whether it is greater than the horizontal coordinate X2 of the preset lower right corner vertex coordinate controlPointB of the effective control area; or, compare the vertical coordinate top of the upper left corner vertex bodyPointA of the human body contour rectangular area to see whether it is greater than the vertical coordinate Y1 of the preset upper left corner vertex coordinate controlPointA of the effective control area; or, compare the vertical coordinate bottom of the lower right corner vertex bodyPointB of the human body contour rectangular area to see whether it is less than the vertical coordinate Y2 of the preset lower right corner vertex coordinate controlPointB of the effective control area.

[0202] When left is less than X1, it means that the rectangular area of ​​the human body outline exceeds the left boundary of the preset effective control area, and the camera is determined to be adjusted to the left. The camera is adjusted to the left with the camera itself as the reference system. The adjustment angle is calculated based on the difference between left and X1. For details, please refer to the relevant technology, and this application will not elaborate on this. Optionally, when left is less than X1, a prompt message is generated, which is used to prompt the user to adjust the camera to the right or to move to the left. This is to adjust the position of the camera or move the user by himself with the user himself as the reference system, so that the rectangular area of ​​the human body outline is in the preset effective control area, which is beneficial for the display device to recognize the image information within the effective control area, so as to determine the user's interaction intention and improve the interaction performance.

[0203] When right is greater than X2, it means that the rectangular area of ​​the human body outline exceeds the right boundary of the preset effective control area, and the camera is determined to be adjusted to the right. The adjustment angle is calculated based on the difference between right and X2. For details, please refer to the relevant technology, and this application will not go into details. Optionally, when right is greater than X2, a prompt message is generated, which is used to prompt the user to adjust the camera to the right, or to prompt the user to move to the left, so that the rectangular area of ​​the human body outline is within the preset effective control area, which is beneficial for the display device to recognize the image information within the effective control area, so as to determine the user's interaction intention and improve the interaction performance.

[0204] If top is less than Y1, it indicates that the human body outline rectangular area exceeds the upper boundary of the preset effective control area, and the camera is determined to be adjusted upward. The adjustment angle is calculated based on the difference between top and Y1. For details, please refer to the relevant technology, and this application will not elaborate on this. Optionally, if top is greater than Y1, a prompt message is generated to prompt the user to adjust the camera upward or to prompt the user to move away from the camera, so that the human body outline rectangular area is within the preset effective control area.

[0205] If bottom is greater than Y2, it indicates that the human body contour area exceeds the lower boundary of the preset effective control area, and the camera is determined to be adjusted downward. The adjustment angle is calculated based on the difference between bottom and Y2. For details, please refer to the relevant technology, and this application will not elaborate on this. Optionally, if bottom is greater than Y2, a prompt message is generated to prompt the user to adjust the camera downward or to prompt the user to move away from the camera, so that the human body contour rectangular area is within the preset effective control area.

[0206] As shown in Figure 16, Figure 16 is a schematic diagram of a video page according to some embodiments of the present application, showing a video page 1300, a target image 1301, a preset effective control area 1302, and a human body outline rectangular area 1401. When left is greater than X1, right is less than X2, top is greater than Y1, and bottom is less than Y2, it means that the human body outline rectangular area 1401 is completely within the preset effective control area 1302. The display device can perform image recognition on the effective control area 1302 of the video screen 1301, which contains complete user limb information, and can accurately identify the user's interaction intention, thereby improving interaction performance. In some embodiments, in the process of determining the adjustment parameters based on the vertex coordinates of the human body outline rectangular area and the vertex coordinates of the effective control area preset in the video page, first, based on the vertex coordinates of the human body outline rectangular area and the vertex coordinates of the effective control area preset in the video page, it is determined whether the human body outline rectangular area overlaps with the margin area in the video page. The margin area is the area in the video page other than the effective control area. Optionally, determine whether the human body contour rectangular area overlaps with the margin area in the video page based on the vertex coordinates of the human body contour rectangular area: bodyPointA (left, top) and bodyPointB (right, bottom), and the vertex coordinates of the preset effective control area controlPointA (X1, Y1) and controlPointB (X2, Y2).

[0207] Optionally, determine whether the horizontal coordinate left of the upper left corner vertex bodyPointA of the human body contour rectangular area is less than the horizontal coordinate X1 of the preset upper left corner vertex coordinate controlPointA of the effective control area; or, determine whether the horizontal coordinate right of the lower right corner vertex bodyPointB of the human body contour rectangular area is greater than the horizontal coordinate X2 of the preset lower right corner vertex coordinate controlPointB of the effective control area; or, determine whether the vertical coordinate top of the upper left corner vertex bodyPointA of the human body contour rectangular area is greater than the vertical coordinate Y1 of the preset upper left corner vertex coordinate controlPointA of the effective control area; or, determine whether the vertical coordinate bottom of the lower right corner vertex bodyPointB of the human body contour rectangular area is less than the vertical coordinate Y2 of the preset lower right corner vertex coordinate controlPointB of the effective control area.

[0208] When the horizontal coordinate left of the upper left corner vertex bodyPointA of the human body contour rectangular area is less than the horizontal coordinate X1 of the upper left corner vertex coordinate controlPointA of the preset effective control area, it means that the human body contour rectangular area overlaps with the margin area in the video page, and then the orientation information of the overlapping area in the video page is determined, and the first direction of the overlapping area in the video interface is obtained as left, so it is determined to adjust the camera to the left, and the adjustment angle is calculated according to the difference between left and X1.

[0209] As shown in Figure 17, Figure 17 is a second schematic diagram of a video page according to some embodiments of the present application, showing a video page 1300, a target image 1301, a preset effective control area 1302, a margin area 1303, and a human body contour rectangular area 1401. The horizontal coordinate left of the upper left corner vertex bodyPointA of the human body contour rectangular area 1401 is smaller than the horizontal coordinate X1 of the upper left corner vertex coordinate controlPointA of the preset effective control area 1302, and overlaps with the margin area 1303. The overlapping area is the left side of the video page 1300. Then, it is determined that the camera will be adjusted to the left, and the human body contour rectangular area 1401 will move to the right relative to the video page 1300 until it is completely in the preset effective control area 1302, so as to facilitate subsequent AI calculations to identify the image information contained in the preset effective control area 1302.

[0210] When the horizontal coordinate right of the vertex bodyPointB in the lower right corner of the human body contour rectangular area is greater than the horizontal coordinate X2 of the vertex coordinate controlPointB in the lower right corner of the preset effective control area, it means that the human body contour rectangular area overlaps with the margin area in the video page, and then the orientation information of the overlapping area in the video page is determined, and the first direction of the overlapping area in the video interface is right, so it is determined to adjust the camera to the right, and the adjustment angle is calculated according to the difference between right and X2.

[0211] When the vertical coordinate top of the upper left corner vertex bodyPointA of the human body contour rectangular area is greater than the vertical coordinate Y1 of the upper left corner vertex coordinate controlPointA of the preset effective control area, it means that the human body contour rectangular area overlaps with the margin area in the video page, and then the orientation information of the overlapping area in the video page is determined, and the first direction of the overlapping area in the video interface is obtained as upward, so it is determined to adjust the camera upward, and the adjustment angle is calculated according to the difference between top and Y1.

[0212] When the vertical coordinate bottom of the vertex bodyPointB in the lower right corner of the human body contour rectangular area is less than the vertical coordinate Y2 of the vertex coordinate controlPointB in the lower right corner of the preset effective control area, it means that the human body contour rectangular area overlaps with the margin area in the video page, and then the orientation information of the overlapping area in the video page is determined, and the first direction of the overlapping area in the video interface is obtained as downward, so it is determined to adjust the camera downward, and the adjustment angle is calculated according to the difference between bottom and Y2.

[0213] In some embodiments, the adjustment parameter is an adjustment direction, and the adjustment parameter is determined based on the vertex coordinates of the human body outline rectangular area, including: when it is determined that the human body outline rectangular area does not include a complete human body, determining the second direction of the position of the human body part that is not captured relative to the video page. The embodiment of the present application provides an implementation method, when it is determined that the human body outline rectangular area does not include a complete human body, in the process of determining the second direction of the position of the human body part that is not captured relative to the video page, first calculate the first size of the human body part that is not captured based on the vertex coordinates of the human body outline rectangular area, the first size being a first width or a first length.

[0214] As shown in FIG18 , FIG18 is a schematic diagram of a process for calculating a first size of a human body portion not captured according to some embodiments of the present application. Calculating the first size of a human body portion not captured includes the following steps S1801 to S1802:

[0215] S1801: Determine the third size of the captured human body part in the video page.

[0216] In some embodiments, the third size of the captured human body part in the video page is determined based on the vertex coordinates of the rectangular area of ​​the human body contour: bodyPointA (left, top) and bodyPointB (right, bottom). The third size can be a third length or a third width.

[0217] As shown in the following formula (1): pictureBodyHight=bottom-top (1)

[0218] Formula (1) represents the third length of the human body part captured by the camera in the video page.

[0219] Alternatively, the following formula (2) pictureBodyWidth = right - left (2)

[0220] Formula (2) represents the third width of the human body part captured by the camera in the video page.

[0221] S1802: Calculate a first size of the uncaptured human body part in the video page based on the ratio of the second size to the target size.

[0222] The target size ratio is the ratio of the uncaptured body part to the entire body. For example, if the limb key point data returned by the AI ​​algorithm does not include the key point data of the lower limbs, it means that the camera did not capture the user's lower limbs. Therefore, the length of the uncaptured lower limbs must be calculated based on the ratio of the lower limb length to the height.

[0223] As shown in the following formula (3):

[0224] Formula (3) represents the actual height length of the human body part in the video page, where p1 is the length ratio of the unfilmed human body part to the complete human body.

[0225] Or, as in formula (4)

[0226] Formula (4) represents the width of the actual height of the human body part in the video page, where p2 is the width ratio of the unfilmed human body part to the complete human body.

[0227] Furthermore, the first length of the human body part not captured is calculated according to formula (5): nobodyHeight=BodyHeight*p1 (5)

[0228] Alternatively, the first width of the human body part not photographed is calculated according to formula (6): nobodyWidth=BodyWidth*p2 (6)

[0229] After calculating the first size of the uncaptured human body part: the first length or the first width, as shown in FIG19 , FIG19 is a second flow chart of another processing method for a display device according to some embodiments of the present application, including the following steps S1901 to S1904:

[0230] S1901: Obtain a second size of the target background area.

[0231] The target background area is the background area in the video page that is opposite to the direction where the human body part is not captured.

[0232] The following formula (7) calculates the second length of the target background area based on the vertex coordinates of the human body contour boundary area: bodyPointA (left, top) and bodyPointB (right, bottom), and the vertex coordinates of the preset effective control area: controlPointA (X1, Y1) and controlPointB (X2, Y2): diffHight1 = top - Y1 (7)

[0233] The direction where the human body part is not captured is the lower side of the video page, and the opposite direction is the upper side. The second length diffHight1 of the target background area on the upper side is calculated according to formula (7).

[0234] Or the following formula (8): diffHight2=Y2-bottom (8)

[0235] The direction where the human body part is not captured is the upper side of the video page, and the opposite direction is the lower side. The second length diffHight2 of the target background area on the lower side is calculated according to formula (8).

[0236] Alternatively, the second width of the target background area is calculated according to formula (9): diffWidth1 = X1-left (9)

[0237] The direction where the human body part is not captured is the right side of the video page, and the opposite direction is the left. The second width diffWidth1 of the target background area on the left side is calculated according to formula (9).

[0238] Or the following formula (10): diffWidth2=right-X2 (10)

[0239] The direction where the human body part is not captured is the left side of the video page, and the opposite direction is the right side. The second width diffWidth2 of the target background area on the right side is calculated according to formula (10).

[0240] S1902: Determine whether the first size is smaller than or equal to the second size.

[0241] When the first size is less than or equal to the second size, that is, the first length nobodyHeight is less than the second length diffHight1, or the first length nobodyHeight is less than the second length diffHight1; or, the first width nobodyWidth is less than the second width diffWidth1, and the first width nobodyWidth is less than the second width diffWidth2, step S1203 is executed; the specific first size is compared with which of the above-mentioned second sizes is determined by the limb key point data output by the AI ​​algorithm. For example, if the output limb key points do not include lower limb key point data, the first size nobodyHeight is compared with diffHight1. Other optional implementations are the same or similar, and this application will not elaborate on them here.

[0242] If the first size is larger than the second size, execute step S1904;

[0243] S1903: Determine a second direction of the position where the human body part is not captured relative to the video page; and determine that the adjustment direction of the camera is the second direction.

[0244] When the first size is less than or equal to the second size, taking the first length being less than the second length as an example, the first length is less than or equal to the second length, indicating that the first length where the human body part is not photographed is less than the second length of the target background area, and the second direction is the direction where the human body part is not photographed. After adjusting the camera according to the second direction, the complete human body part is displayed.

[0245] For example, the limb key point data output by the AI ​​algorithm indicates that the human body part that is not captured is the lower limbs, and the lower limbs are located at the bottom of the video page. When the length of the target background area, that is, the area above the user's head, is greater than the length of the lower limbs, the camera movement direction is determined to be downward to display the user's lower limbs on the video page, thereby fully displaying the user's human body parts. This is conducive to the AI ​​algorithm obtaining accurate calculation results and improving human-computer interaction performance.

[0246] S1904: Output a first prompt message, where the first prompt message is used to prompt the user to stay away from the camera.

[0247] When the first size is larger than the second size, it means that the height or width of the human body part that is not captured is larger, and it is difficult to capture the target image containing the complete human body part by simply adjusting the camera. Therefore, the first prompt information is output to prompt the user to move away from the camera, so as to control the camera to capture the complete human body part and display it in the effective control area preset in the video page.

[0248] In some other embodiments, when it is determined that the human body contour area includes a complete human body, the size of the complete human body in the video page is determined, such as the following formula (11):

[0249] Where q represents the size of the complete human body in the video page, expressed as a percentage. pictureBodyHight is calculated according to the aforementioned formula (1). When the complete human body is included in the human body contour area, it represents the length of the complete human body. controlHight = Y2-Y1 (12)

[0250] controlHight indicates the length of the preset effective control area.

[0251] When the size of the complete human body in the video page is less than half the size of the video page, that is, when q is less than 0.5, it means that the complete human body occupies a small proportion in the video screen, which may be not conducive to the subsequent AI algorithm to recognize it. Therefore, the second prompt information is output, and the second prompt information is used to prompt the user to move closer to the camera.

[0252] As shown in FIG20 , FIG20 is a schematic diagram of the second prompt information according to some embodiments of the present application, showing: a video page 1300, a target image 1301, a preset effective control area 1302, a human body outline rectangular area 1401, and second prompt information 2010. Therefore, if the user's complete body is displayed too small on the video page, the user is prompted to move closer to the camera so that the complete body occupies a larger area on the video page, which facilitates accurate recognition by the subsequent AI algorithm and improves human-computer interaction performance.

[0253] S1204: Adjust the camera position based on the adjustment parameters.

[0254] After executing the above step S1203, the adjustment direction of the camera is determined, or the adjustment direction and adjustment angle are determined, and then the camera position is adjusted according to the adjustment direction; or, the camera position is adjusted according to the adjustment direction and adjustment angle.

[0255] In summary, the embodiment of the present application provides another processing method for a display device, which first controls the camera to capture a target image and displays a video page on the display, in which the target image is displayed; then, the target image is segmented to determine the human body contour rectangular area in the target image, and the vertex coordinates of the human body contour rectangular area in the video page are obtained, and then the adjustment parameters of the camera are determined according to the vertex coordinates of the human body contour rectangular area, so as to adjust the camera position according to the adjustment parameters. The display device can therefore control the camera to capture a target image including a complete human body contour rectangular area, thereby achieving fast and accurate adjustment of the camera position and improving human-computer interaction performance.

[0256] In related art, when a display device leaves the factory, gestures corresponding to different functional operations of the display device are pre-set, allowing users to use gestures to control the display device to achieve different functional operations when using the display device. Referring to Figure 21A, when a user is watching Channel 1, Gesture 1 is pre-set for the display device 200 when it leaves the factory to achieve the channel switching function. When the display device 200 detects and recognizes Gesture 1, it executes the channel switching function, and as shown in Figure 21B, it switches from Channel 1 to Channel 2.

[0257] However, since the display device is pre-set with gestures corresponding to different functional operations of the display device when it leaves the factory, users cannot set corresponding gestures for different functional operations according to their preferences and habits. There is a possibility that the pre-set gestures are not suitable for users, or users are not clear about the functional operations corresponding to each gesture, resulting in a low utilization rate of users using gestures to control the display device.

[0258] In order to solve the above problems, the embodiment of the present application proposes a third processing method for a display device. In the above technical solution, the display device calls out a gesture setting interface based on a preset operation, wherein the gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function; in response to the target preset gesture input by the user in the gesture setting interface, and the preset function corresponding to the target preset gesture, a first correspondence between the target preset gesture and the preset function is established and stored, so that when the display device detects the target preset gesture, it executes the preset function based on the first correspondence between the target preset gesture and the preset function. In the above technical solution, the gesture setting function can be started according to the preset operation, and the gesture setting interface can be called out, so that the user can set the target preset gesture corresponding to the preset function, and the first correspondence corresponding to the preset function is established through the target preset gesture input by the user, so that when the display device detects the target preset gesture, it can use the first correspondence to execute the preset function, thereby meeting the user's habits and preferences, thereby improving the user's usage rate of using gestures to control the display device to achieve different functional operations and improving the user experience. In some embodiments, the above display device is a terminal device with a display function, such as a TV, a mobile phone, a computer, a learning machine, etc. In the display device:

[0259] an output interface (display 260 and / or audio output interface 270 ), configured to output user interaction information;

[0260] Communicator 220, for communicating with the server;

[0261] a memory configured to: store computer instructions and / or data associated with a display device;

[0262] At least one processor 250 is connected to the output interface, the communicator 220 and the memory, and is further configured to execute the computer instructions to enable the display device to:

[0263] Based on the preset operation, calling out a gesture setting interface, wherein the gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function;

[0264] In response to the target preset gesture input by the user in the gesture setting interface, and the preset function corresponding to the target preset gesture, a first correspondence between the target preset gesture and the preset function is established and stored, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence between the target preset gesture and the preset function.

[0265] In some embodiments, at least one processor 250 is specifically configured to execute the computer instructions to cause the display device to:

[0266] Based on the number of times the preset function is used and the number of times the display device is turned on, calling out the gesture setting interface;

[0267] Based on the detected first preset gesture, calling out the gesture setting interface; or

[0268] Based on the operation of starting the gesture setting function, the gesture setting interface is called out.

[0269] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to:

[0270] Counting the number of times the preset function is used and the number of times the display device is turned on within the first time period;

[0271] A first ratio of the number of times the preset function is used to the number of times the display device is turned on is calculated, and when the first ratio is greater than a first preset threshold, the gesture setting interface is called out.

[0272] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to:

[0273] Counting the number of times a plurality of initial preset gestures are used and the number of times the display device is turned on within the second time period;

[0274] Calculating a second ratio of the number of times each of the initial preset gestures is used to the number of times the display device is turned on;

[0275] The target preset gesture is determined from the plurality of initial preset gestures based on the second ratio corresponding to each of the initial preset gestures.

[0276] In some embodiments, at least one processor 250 is specifically configured to execute the computer instructions to cause the display device to:

[0277] The second ratio corresponding to each of the initial preset gestures is compared with a second preset threshold value. When any of the second ratios is smaller than the second preset threshold value, the initial preset gesture corresponding to the current second ratio is determined to be the target preset gesture.

[0278] In some embodiments, at least one processor 250 is specifically configured to execute the computer instructions to cause the display device to:

[0279] The second ratios corresponding to the plurality of initial preset gestures are compared, and the initial preset gesture corresponding to the smallest second ratio is determined as the target preset gesture.

[0280] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to:

[0281] Obtain parameter information corresponding to the preset function;

[0282] A second correspondence between the preset function and the parameter information is established and stored, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence and the second correspondence.

[0283] In summary, the present application executes the above-mentioned processing method for a display device on a display device, and calls out a gesture setting interface based on a preset operation through at least one processor of the display device, wherein the gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function; in response to a target preset gesture input by a user in the gesture setting interface, and a preset function corresponding to the target preset gesture, a first correspondence between the target preset gesture and the preset function is established and stored, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence between the target preset gesture and the preset function. In the above-mentioned technical solution, the gesture setting function can be started according to the preset operation, and the gesture setting interface can be called out, so that the user can set the target preset gesture corresponding to the preset function, and the first correspondence corresponding to the preset function is established through the target preset gesture input by the user, so that when the display device detects the target preset gesture, it can use the first correspondence to execute the preset function, thereby satisfying the user's habits and preferences, thereby improving the user's utilization rate of using gestures to control the display device to achieve different functional operations, and improving the user experience.

[0284] FIG22A is a framework diagram of a gesture setting system for a display device according to one or more embodiments of the present application. As shown in FIG22A , the system may include a channel gesture setting interface calling module 2201 and an establishing module 2202. The system uses the gesture setting interface calling module 2201 to call out a gesture setting interface based on a preset operation, wherein the gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function, and the establishing module 2202 is used to establish and store the first correspondence between the target preset gesture and the preset function in response to a target preset gesture input by a user in the gesture setting interface and the preset function corresponding to the target preset gesture, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence between the target preset gesture and the preset function. In the above technical solution, the gesture setting function can be started according to the preset operation, and the gesture setting interface can be called out, so that the user can set the corresponding target preset gesture for the preset function, and establish a first correspondence corresponding to the preset function through the target preset gesture input by the user. When the display device detects the target preset gesture, it can use the first correspondence to execute the preset function, thereby satisfying the user's habits and preferences, thereby increasing the user's usage rate of using gestures to control the display device to achieve different functional operations and improving the user experience.

[0285] Figure 22B is a diagram illustrating the gesture setting architecture for a display device according to one or more embodiments of the present application. Based on the aforementioned system framework, the present application is implemented as shown in Figure 22B. The Android system primarily includes an application layer, a framework layer, a system runtime layer, and a kernel layer. The implementation logic is primarily embodied in the application layer, which includes a channel first target area display module, a second target area display module, and a processing module. The functions of each module have been described in detail in the aforementioned embodiments and will not be further elaborated here to avoid repetition.

[0286] In the third processing method for a display device provided in an embodiment of the present application, the display device first calls out a gesture setting interface based on a preset operation, wherein the gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function, and then, in response to the target preset gesture input by the user in the gesture setting interface, and the preset function corresponding to the target preset gesture, establishes and stores a first correspondence between the target preset gesture and the preset function, so that when the display device detects the target preset gesture, it executes the preset function based on the first correspondence between the target preset gesture and the preset function. In the above technical solution, the gesture setting function can be started according to the preset operation, and the gesture setting interface can be called out, so that the user can set the target preset gesture corresponding to the preset function, and the first correspondence corresponding to the preset function is established through the target preset gesture input by the user, so that when the display device detects the target preset gesture, it can use the first correspondence to execute the preset function, thereby satisfying the user's habits and preferences, thereby improving the user's utilization rate of using gestures to control the display device to achieve different functional operations and enhancing the user experience.

[0287] FIG23A is a flow chart of a third processing method for a display device according to some embodiments of the present application. As shown in FIG23A , the method specifically includes the following steps:

[0288] S231, based on the preset operation, calling out the gesture setting interface.

[0289] Among them, the preset operation refers to an operation that can start the gesture setting function and call out the gesture setting interface. For example, the preset operation can be an operation in which the user starts the setting gesture through a control device such as a remote control, but it is not limited to this. This application does not specifically limit it. The gesture setting interface is used to set the first correspondence between the target preset gesture and the preset function. Referring to Figure 23B, for the display device 200, the target preset gesture and preset function can be set through the gesture setting interface 2301. However, it is not limited to this. This application does not specifically limit it. Those skilled in the art can make specific settings according to actual conditions.

[0290] The above-mentioned preset function refers to the function of controlling the display device. The preset function can be an application function, such as a video playback application function, an audio playback application function, a web application function, etc. The preset function can also be a control function of the display device, such as a channel switching function, a return to the home page function, a volume adjustment function, a display screen brightness adjustment function, etc., but is not limited to this. This application does not specifically limit it, and technical personnel in this field can make specific settings according to actual conditions.

[0291] Specifically, the display device starts the gesture setting function according to the preset operation, and calls out a gesture setting interface for setting a first correspondence between a target preset gesture and a preset function.

[0292] FIG23C is a second flow chart of a third processing method for a display device according to some embodiments of the present application. FIG23C further describes a possible implementation of S231 based on the embodiment shown in FIG23A , as shown in FIG23C :

[0293] S2311: Based on the number of times a preset function is used and the number of times the display device is turned on, a gesture setting interface is called out.

[0294] Among them, the number of times the preset function is used and the number of times the display device is turned on can be calculated by setting a counter, and the counter is used to count the number of times the preset function is used and the number of times the display device is turned on. For a preset function such as a video playback application function, when the video playback application function is turned on, the counter is increased by 1 to obtain the number of times the preset function is used, but this is not limited to this. This application does not specifically limit it, and technical personnel in this field can make specific settings according to actual conditions.

[0295] Specifically, the display device counts the number of times a preset function is used and the number of times the display device is turned on, and starts the gesture setting function according to the relationship between the number of times the preset function is used and the number of times the display device is turned on to call out the gesture setting interface.

[0296] FIG23D is a third flow diagram of a third processing method for a display device according to some embodiments of the present application. FIG23D further describes a possible implementation of S2311 based on the embodiment shown in FIG23C , as shown in FIG23D :

[0297] S23111, counting the number of times a preset function is used and the number of times the display device is turned on within the first time period.

[0298] Among them, the first duration refers to a parameter used to limit the number of times a preset function is used within a fixed duration, and the number of times the display device is turned on. The first duration can be, for example, 30 days, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0299] It should be noted that, for the first time period, assuming it is 30 days, the number of times the preset functions are used and the number of times the display device is turned on are counted starting from May 1, XXXX. When it comes to the 31st day, that is, May 31, the data of May 1 needs to be deleted to ensure that the counted number of times the preset functions are used and the number of times the display device is turned on are the latest data, and further save memory.

[0300] Specifically, the display device counts the number of times a preset function is used and the number of times the display device is turned on within the first period of time.

[0301] Continuing with the above embodiment, for the first time period of 30 days, the preset function is the video playback application function, then the number of times the video playback application function is used within 30 days is counted as N1, and the number of times the display device is turned on is N2, but not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0302] S23112, calculating a first ratio of the number of times a preset function is used to the number of times the display device is turned on.

[0303] Specifically, the display device calculates a ratio between the number of times the preset function is used and the number of times the display device is turned on, and the calculation result is a first ratio.

[0304] Continuing with the above embodiment, the number of times the video playback application function is used within 30 days is N1, and the number of times the display device is turned on is N2. The first ratio of the number of times the video playback application function is used N1 to the number of times the display device is turned on N2 is calculated to be N1 / N2=4, but not limited to this. This application is not specifically limited, and those skilled in the art can set it according to actual conditions.

[0305] S23113, determine whether the first ratio is greater than a first preset threshold.

[0306] Among them, the first preset threshold is used to determine whether to start the gesture setting function to call out the parameters of the gesture setting interface. For example, the first preset threshold can be 3, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0307] S23114: When the first ratio is greater than a first preset threshold, calling out a gesture setting interface.

[0308] Specifically, the display device determines whether the first ratio is greater than a first preset threshold, and when the first ratio is greater than the first preset threshold, starts the gesture setting function to call out the gesture setting interface.

[0309] Continuing with the above embodiment, referring to FIG. 23E , when the first ratio N1 of the number of times the video playback application function is used and the number of times the display device is powered on N2 is calculated to be N1 / N2=4, and the first preset threshold is 3, if the first ratio 4 is greater than the first preset threshold 3, the gesture setting function is activated to call out gesture setting interface 2301. However, this is not a limitation and is not specifically limited in this application. Persons skilled in the art may configure gesture settings according to actual circumstances.

[0310] Optionally, based on the above embodiments, in some embodiments of the present application, when it is determined that the first ratio is greater than a first preset threshold, a prompt message is sent to the user to determine whether to set a target preset gesture for the preset function.

[0311] Referring to Figure 23F, when it is determined that the first ratio is greater than the first preset threshold, an interface 2302 is actively popped up in the display device 200 to remind the user to determine whether to set the gesture. The interface includes "Do you agree to set the target preset gesture for Function 2?" The user can use the virtual buttons "Reject" and "Agree" in the lower right corner of the interface to determine whether to agree to set the target preset gesture for Function 2 according to user needs. When the user selects "Agree", the target preset gesture is set for Function 2 and the gesture setting interface is entered. When the user selects "Reject", the target preset gesture is not set for Function 2, but it is not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0312] In the technical solution provided by the embodiment of the present application, in the above process, by calculating the first ratio of the number of times the preset function is used and the number of times the display device is turned on within the first time period, and comparing the first ratio with the first preset threshold, when it is determined that the first ratio is greater than the first preset threshold, it can be determined that the current preset function is a function frequently used by the user. At this time, the gesture setting function is started, the gesture setting interface is called out, and the target preset gesture is set for the current preset function, so that the user can use the target preset gesture to directly control the preset function operation when using the display device in the future, thereby improving the user experience.

[0313] FIG23G is a fourth flow chart of a third processing method for a display device according to some embodiments of the present application. FIG23G further describes another possible implementation of S231 based on the embodiment shown in FIG23A , as shown in FIG23G :

[0314] S2312: Based on the detected first preset gesture, call out a gesture setting interface.

[0315] The first preset gesture is used to control the display device to start the gesture setting function. Referring to Figure 23H, gesture 2 is the first preset gesture, but is not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0316] Specifically, the display device starts a gesture setting function according to the detected first preset gesture and calls out a gesture setting interface.

[0317] In some feasible embodiments, when the display device 200 detects gesture 2, the display device 200 starts the gesture setting function and calls out the gesture setting interface, but is not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0318] It should be noted that when a display device's built-in camera captures multiple frames of continuous images and a first preset gesture is detected in each of the multiple frames, the gesture setting function is activated and the gesture setting interface is displayed to avoid erroneous activation of the gesture setting function. However, this is not a limitation of this application and those skilled in the art may configure the gesture setting function based on actual circumstances.

[0319] FIG23I is a fifth flow chart of a third processing method for a display device according to some embodiments of the present application. FIG23I further describes another possible implementation of S231 based on the embodiment shown in FIG23A , as shown in FIG23I :

[0320] S2313, based on the operation of starting the gesture setting function, calling out the gesture setting interface.

[0321] Among them, the gesture setting function operation refers to the operation of the user starting the gesture setting function through a control device such as a remote control display device, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0322] Specifically, when the display device receives an operation from the user to start the gesture setting function, it starts the gesture setting function and calls out the gesture setting interface.

[0323] In the technical solution provided by the embodiment of the present application, in the above process, the gesture setting function is started and the gesture setting interface is called out by calculating the relationship between the number of times the preset function is used and the number of times the display device is turned on within the first time period, detecting the first preset gesture, or starting the gesture setting function operation. In this process, since the target preset gestures corresponding to different preset functions are set according to the user's habits and preferences, the usage rate of the user using gestures to control the display device to achieve different function operations when using the display device in the subsequent use is improved, thereby improving the user experience.

[0324] S232, in response to the target preset gesture input by the user in the gesture setting interface, and the preset function corresponding to the target preset gesture, establish and store a first correspondence between the target preset gesture and the preset function, so that when the display device detects the target preset gesture, it executes the preset function based on the first correspondence between the target preset gesture and the preset function.

[0325] Among them, the corresponding relationship refers to the preset function. When the user inputs the target preset gesture in the gesture setting interface, the preset function is bound to the target preset gesture to determine that different target preset gestures can control the preset function corresponding to the display device actuator.

[0326] Specifically, when the display device receives the target preset gesture input by the user in the gesture setting interface, and the preset function corresponding to the target preset gesture, it responds to the target preset gesture and the preset function corresponding to the target preset gesture, establishes a first correspondence between the target preset gesture and the preset function corresponding to the target preset gesture, and stores the first correspondence between the target preset gesture and the preset function, so that when the display device detects the target preset gesture, it can execute the preset function according to the first correspondence between the target preset gesture and the preset function.

[0327] When the display device detects a target preset gesture, it can compare the target preset gesture with multiple stored preset gestures. When it is determined that the target preset gesture exists among the multiple stored preset gestures, the preset function corresponding to the target preset gesture is executed according to the first correspondence between the stored target preset gesture and the preset function, but is not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0328] In the technical solution provided by the embodiment of the present application, the display device calls out a gesture setting interface based on a preset operation, wherein the gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function; in response to a target preset gesture input by a user in the gesture setting interface, and a preset function corresponding to the target preset gesture, a first correspondence between the target preset gesture and the preset function is established and stored, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence between the target preset gesture and the preset function. In the above technical solution, the gesture setting function can be started according to the preset operation, and the gesture setting interface can be called out, so that the user can set the target preset gesture corresponding to the preset function, and the first correspondence corresponding to the preset function is established through the target preset gesture input by the user, so that when the display device detects the target preset gesture, it can use the first correspondence to execute the preset function, thereby satisfying the user's habits and preferences, thereby improving the user's utilization rate of using gestures to control the display device to achieve different functional operations and enhancing the user experience.

[0329] Optionally, based on the above embodiments, in some embodiments of the present application, further comprising:

[0330] S61, counting the number of times a plurality of initial preset gestures are used and the number of times the display device is turned on within a second time period.

[0331] Among them, the second time length refers to a parameter used to limit the number of times multiple initial preset gestures are used within a fixed time length, and the number of times the display device is turned on. The second time length can be, for example, 30 days, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0332] The above-mentioned multiple initial preset gestures refer to the preset gestures that have been pre-set when the display device leaves the factory, and can also be the preset gestures set by the user according to his or her own habits and preferences, but are not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0333] Specifically, the display device counts the usage counts corresponding to the plurality of initial preset gestures and the number of times the display device is turned on within the second time period.

[0334] Continuing with the above embodiment, for the second time period of 30 days, the multiple initial preset gestures are gesture 1, gesture 2, and gesture 3. The number of times gesture 1, gesture 2, and gesture 3 are used within 30 days is N3, N4, and N5 respectively, and the number of times the display device is turned on is N2, but not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0335] S62, calculating a second ratio of the number of times each initial preset gesture is used to the number of times the display device is turned on.

[0336] Specifically, the display device calculates a ratio between the number of times each initial preset gesture is used and the number of times the display device is turned on, and the calculation result is a second ratio.

[0337] Continuing with the above embodiment, the number of times gesture 1, gesture 2, and gesture 3 are used within 30 days is counted as N3, N4, and N5, and the number of times the display device is turned on is N2. The second ratio of the number of times gesture 1, gesture 2, and gesture 3 are used, N3, N4, and N5, to the number of times the display device is turned on N2, is calculated respectively. However, this is not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0338] S63: Determine a target preset gesture from the multiple initial preset gestures based on the second ratio corresponding to each initial preset gesture.

[0339] Specifically, the display device determines a target preset gesture from a plurality of initial preset gestures according to the calculated second ratio corresponding to each initial preset gesture.

[0340] Optionally, based on the above embodiment, in some embodiments of the present application, further, a possible implementation of S63 includes:

[0341] S631: Compare the second ratio corresponding to each initial preset gesture with the second preset threshold. When any second ratio is smaller than the second preset threshold, determine that the initial preset gesture corresponding to the current second ratio is the target preset gesture.

[0342] Among them, the second preset threshold is a parameter used to determine the target preset gesture among multiple initial preset gestures. For example, the second preset threshold can be 4, but is not limited to this. This application does not specifically limit it, and those skilled in the art can set it according to actual conditions.

[0343] Specifically, the display device determines in turn whether the second ratio corresponding to each initial preset gesture is greater than the second preset threshold. When there is a second ratio greater than the second preset threshold, the initial preset gesture corresponding to the second ratio currently greater than the second preset threshold is used as the target preset gesture.

[0344] Continuing with the above embodiment, when the second ratios of the number of times gestures 1, 2, and 3 are used (N3, N4, and N5) to the number of times the display device is powered on (N2) are calculated to be 5, 1, and 3, respectively, the second ratios corresponding to gestures 1, 2, and 3 are compared with the second preset threshold 2, respectively. When the second ratio 1 corresponding to gesture 2 is less than the second preset threshold 2, gesture 2 is determined to be the target preset gesture. However, this is not a limitation and is not specifically limited in this application. Persons skilled in the art may configure the gesture based on actual circumstances.

[0345] Optionally, based on the above embodiment, in some embodiments of the present application, further, another possible implementation of S63 includes:

[0346] S632: Compare the second ratios corresponding to the multiple initial preset gestures, and determine the initial preset gesture corresponding to the smallest second ratio as the target preset gesture.

[0347] Specifically, the display device compares the second ratios corresponding to the multiple initial preset gestures, determines the smallest second ratio among the multiple second ratios, and uses the initial preset gesture corresponding to the smallest second ratio as the target preset gesture.

[0348] Continuing with the above embodiment, when the second ratios of the number of times gestures 1, 2, and 3 are used (N3, N4, and N5) to the number of times the display device is powered on (N2) are calculated to be 5, 1, and 3, respectively, the second ratios corresponding to gestures 1, 2, and 3 are compared, and gesture 2 corresponding to the second ratio of 1 is determined to be the target preset gesture. However, this is not a limitation and is not specifically limited in this application. Persons skilled in the art may configure the gesture based on actual circumstances.

[0349] In the technical solution provided by the embodiment of the present application, in the above process, by calculating the second ratio of the number of times multiple initial preset gestures are used and the number of times the display device is turned on within the second time period, and based on the multiple second ratios, determining the target preset gesture among the multiple initial preset gestures, in this way, the preset functions of the infrequently used initial preset gestures among the pre-set initial preset gestures are updated, without the user having to input a new preset gesture, thus saving memory.

[0350] Optionally, based on the above embodiment, in some embodiments of the present application, further, when executing S232, the following steps are further included:

[0351] S2321, obtaining parameter information corresponding to the preset function.

[0352] Among them, parameter information refers to the parameter information corresponding to the current preset function when the current display device is executing a preset function. For example, when the preset function is a video playback application function, and the video playback application function is playing a movie at this time, the playback mode is a movie mode, and the movie mode includes the frequency information, audio information, etc. of the display screen. That is, when the current parameter information is the video playback application function playing a movie, the movie mode includes the frequency information, audio information, etc. of the display screen, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0353] Specifically, when determining to establish a target preset gesture for a preset function, the display device obtains parameter information corresponding to the preset function.

[0354] S2322: Establish and store a second correspondence between the preset function and the parameter information, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence and the second correspondence.

[0355] Specifically, the display device obtains parameter information corresponding to the preset function, establishes a second correspondence between the preset function and the parameter information corresponding to the preset function, and stores the second correspondence established between the preset function and the parameter information corresponding to the preset function, so that when the display device detects the target preset gesture, it can execute the preset function according to the first correspondence between the preset gesture and the preset function, and the second correspondence between the preset function and the parameter information corresponding to the preset function.

[0356] In the above-mentioned technical solution provided by the embodiment of the present application, in the above-mentioned process, not only is the corresponding target preset gesture suggested for the preset function, but a second correspondence between the preset function and its corresponding parameter information is also established, so that when the display device detects the target preset gesture, it can execute the preset function based on the first correspondence and the second correspondence, without the user having to further adjust the parameters, thereby facilitating user operation and improving user experience.

[0357] At present, gesture control is widely used in display devices (such as smart large-screen devices, etc.). In related technologies, gesture recognition is performed on the acquired image to determine the control instruction corresponding to the gesture, and then the display device is controlled to perform the corresponding operation. In actual applications, when the display device is performing gesture recognition, there are cases where other gestures suddenly break into the recognizable range of the display device. At this time, the image acquired by the display device contains multiple gestures, and it is difficult for the display device to accurately determine which gesture is the gesture for controlling the display device, which interferes with the control of the display device and affects the accuracy of gesture recognition; in addition, the user will inevitably shake when making a gesture, which may cause the display device to determine the wrong control instruction based on the shaking of the gesture. For example, when the user controls the playback volume of the display device through gestures, the small-scale shaking of the gesture causes the playback volume of the display device to fluctuate, affecting the user's experience.

[0358] In order to solve some or all of the above-mentioned technical problems, according to the display device of some embodiments of the present application, a fourth processing method for the display device is provided, in which the display device first obtains a continuous first gesture image and a second gesture image through a camera, and then determines the gesture movement direction and gesture movement speed of the change from the first gesture image to the second gesture image based on the first gesture image and the second gesture image, further controls the camera to obtain a third gesture image, and determines the coordinate range to be identified in the third gesture image according to the gesture movement direction and the gesture movement speed, and then performs gesture recognition on the coordinate range to be identified; when the gesture is identified and matches the preset gesture, the control instruction of the preset gesture is executed, so that the display device can perform gesture recognition on the coordinate range to be identified in a targeted manner, avoid the interference of sudden gestures on the display device, and improve the accuracy of gesture control of the display device.

[0359] Continuing with the scenario shown in FIG. 2 , the user performs a corresponding gesture, and the display device 200 controls the camera 201 to capture a first gesture image and a second gesture image, both of which contain the user's gesture. The display device then performs preliminary gesture recognition on the first and second gesture images, primarily to determine the direction and speed of the gesture during the transition from the first gesture image to the second gesture image. The display device then controls the camera 201 to capture images again, obtaining a third gesture image. Based on the aforementioned direction and speed of the gesture, the display device determines the coordinate range within which the gesture may appear in the third gesture image. The display device then performs gesture recognition within this coordinate range. If a gesture is recognized, the display device determines whether the recognized gesture matches a preset gesture. If so, the control instruction for the preset gesture is executed.

[0360] By analyzing the gesture movement direction and gesture movement speed in two consecutive gesture images: the first gesture image and the second gesture image, the coordinate range to be identified where the gesture may appear in the third gesture image at the next moment is predicted, and gesture recognition is performed within the coordinate range to be identified to avoid other gestures that suddenly appear in the third gesture image from interfering with the control of the display device, thereby improving the accuracy of gesture control of the display device.

[0361] According to some embodiments of the present application, a display device includes at least one processor 250, which is further configured to execute computer instructions stored in a memory of the display device so that the display device:

[0362] Controlling the camera to acquire continuous first gesture images and second gesture images;

[0363] determining, based on the first gesture image and the second gesture image, a gesture movement direction and a gesture movement speed of changing from the first gesture image to the second gesture image;

[0364] Controlling the camera to acquire a third gesture image, and determining a coordinate range to be identified in the third gesture image according to the gesture movement direction and gesture movement speed;

[0365] Perform gesture recognition on the coordinate range to be recognized, and execute the control instruction of the preset gesture when a preset gesture is matched.

[0366] The above-mentioned display device analyzes and predicts the coordinate range of the gesture that may appear by identifying and recording the gesture coordinates, so as to identify the gesture that may appear within the coordinate range to be identified, use the gesture as a control gesture, determine the control instruction of the preset gesture that matches the control gesture, so that the display device executes the operation corresponding to the control instruction, and realizes gesture control of the display device; gestures appearing in other areas outside this position are determined to be interfering gestures and are not processed, thereby effectively preventing other gestures that suddenly intrude during the gesture recognition process from interfering with the gesture recognition results, thereby improving the accuracy and stability of gesture control of the display device.

[0367] In some embodiments, at least one processor 250 determines, based on the first gesture image and the second gesture image, a gesture movement direction and a gesture movement speed for changing from the first gesture image to the second gesture image, and is configured to execute computer instructions to cause the display device to:

[0368] Performing gesture recognition on the first gesture image and the second gesture image to determine first gesture coordinates corresponding to the first gesture image and second gesture coordinates corresponding to the second gesture image;

[0369] determining a first time difference between the first gesture image and the second gesture image;

[0370] The gesture movement direction and gesture movement speed are determined according to the first time difference, the first gesture coordinates, and the second gesture coordinates.

[0371] In some embodiments, at least one processor 250 performs gesture recognition on the first gesture image and the second gesture image, determines first gesture coordinates corresponding to the first gesture image and second gesture coordinates corresponding to the second gesture image, and is configured to execute computer instructions to cause the display device to:

[0372] In a case where a plurality of first candidate gestures and a plurality of second candidate gestures are obtained by performing gesture recognition on the first gesture image and the second gesture image, determining a first gesture from the plurality of first candidate gestures and determining a second gesture from the plurality of second candidate gestures according to similarities between the respective first candidate gestures and the respective second candidate gestures;

[0373] The first gesture is a gesture whose gesture similarity is greater than or equal to a preset similarity among the plurality of first candidate gestures; the second gesture is a gesture whose gesture similarity is greater than or equal to a preset similarity among the plurality of second candidate gestures;

[0374] First gesture coordinates corresponding to the first gesture and second gesture coordinates corresponding to the second gesture are determined.

[0375] In some embodiments, at least one processor 250 determines a gesture movement direction and a gesture movement speed based on the first time difference, the first gesture coordinates, and the second gesture coordinates, and is configured to execute computer instructions to cause the display device to:

[0376] Calculate the distance difference between the first gesture coordinates and the second gesture coordinates;

[0377] When the distance difference is greater than a preset distance threshold, the gesture movement direction and gesture movement speed are determined according to the first time difference and the distance difference.

[0378] In some embodiments, after calculating the distance difference between the first gesture coordinates and the second gesture coordinates, the at least one processor 250 is further configured to execute computer instructions to cause the display device to:

[0379] When the distance difference is less than the preset distance threshold, a control instruction corresponding to the first gesture in the first gesture image is determined, so as to execute the control instruction corresponding to the first gesture to operate the display device.

[0380] In some embodiments, at least one processor 250 performs gesture recognition within a coordinate range to be recognized and, if a preset gesture is matched, executes a control instruction of the preset gesture, and is configured to execute computer instructions to cause the display device to:

[0381] Gesture recognition is performed on the coordinate range to be recognized. If a preset gesture is matched and the gesture movement speed is greater than a preset threshold, the control instruction of the preset gesture is executed.

[0382] In some embodiments, at least one processor 250 performs gesture recognition within a coordinate range to be recognized and is configured to execute computer instructions to enable the display device to:

[0383] cropping the third gesture image according to the coordinate range to be identified to obtain a fourth gesture image;

[0384] The fourth gesture image is input into the gesture recognition model to obtain a gesture recognition result output by the gesture recognition model, where the gesture recognition result includes no gesture, gesture category, and gesture similarity.

[0385] As shown in FIG. 24 , FIG. 24 is a flowchart of a fourth processing method for a display device according to some embodiments of the present application. The method includes the following steps S2401 to S2404:

[0386] S2401: Control a camera to acquire consecutive first gesture images and second gesture images.

[0387] In some embodiments, the display device obtains continuous first gesture images and second gesture images through a configured camera, wherein the camera can be a monocular camera, a binocular camera, an infrared camera, etc., and this application does not limit this.

[0388] It is understandable that the camera captures images at a fixed acquisition frequency, and the time difference between multiple images captured by the camera is fixed.

[0389] The first gesture image and the second gesture image are continuous, and a time difference between the first gesture image and the second gesture image is a first time difference.

[0390] S2402: Determine, based on the first gesture image and the second gesture image, a gesture movement direction and a gesture movement speed when the first gesture image changes to the second gesture image.

[0391] In some embodiments, after acquiring consecutive first and second gesture images, gesture recognition is performed on each of the first and second gesture images. Gesture recognition involves performing artificial intelligence calculations on images captured by a camera or millimeter wave to infer the gestures in the images.

[0392] In some embodiments, still referring to the software configuration diagram of the display device shown in FIG5A , the display device application layer calls the camera to capture images, obtains the first gesture image and the second gesture image, and then calls the Software Development Kit (SDK) interface to pass the first gesture image and the second gesture image to the SDK layer for artificial intelligence calculation. The SDK layer performs image processing, image recognition, and other operations on the first gesture image and the second gesture image to obtain a recognized gesture. The application layer receives the gesture returned by the SDK layer and determines the first gesture coordinates of the first gesture in the first gesture image and the second gesture coordinates of the second gesture in the second gesture image.

[0393] Optionally, the SDK layer may further obtain the category and similarity of the first gesture contained in the first gesture image, and the category and similarity of the second gesture contained in the second gesture image by recognizing the first gesture image and the second gesture image;

[0394] Among them, similarity refers to the similarity between the recognized gesture and the preset gesture. The preset gesture is shown in Figure 25. Figure 25 is a schematic diagram of the preset gesture according to some embodiments of the present application. The figure only exemplifies some preset gestures, and the present application does not limit them here.

[0395] In some embodiments, gesture recognition is performed on a first gesture image and a second gesture image to obtain multiple first candidate gestures and multiple second candidate gestures. It is understood that the gesture image captured by the camera includes multiple gestures of a single user or multiple gestures of multiple users. Then, based on the similarity of each first candidate gesture, a candidate gesture with a similarity greater than a preset similarity is determined from the multiple first candidate gestures as the first gesture corresponding to the first gesture image; and based on the similarity of each second candidate gesture, a candidate gesture with a similarity greater than a preset similarity is determined from the multiple second candidate gestures as the second gesture corresponding to the second gesture image. The preset similarity is a pre-set similarity threshold used to screen valid gestures, typically set to 0.8. Gestures with a similarity greater than or equal to 0.8 are determined to be valid gestures, while gestures with a similarity less than 0.8 are determined to be invalid gestures. After the first and second gestures are screened and obtained, the first gesture coordinates of the first gesture in the first gesture image are determined, and the second gesture coordinates of the second gesture in the second gesture image are determined.

[0396] The first gesture coordinates may be the coordinates of any one or more of the four vertices of the first gesture detection frame, or the coordinates of the center point of the first gesture detection frame, which is not limited in this application. The second gesture coordinates may be the coordinates of any one or more of the four vertices of the second gesture detection frame, or the coordinates of the center point of the second gesture detection frame.

[0397] In some feasible embodiments, the first gesture image is recognized to obtain the number of gestures (handsCount), the gesture category (handType), the similarity (Score), and the gesture coordinates (Rect) included in the first gesture image. The gesture category (handType) can include gestures 1, 2, 3, fist, OK, and so on. During gesture recognition of the first gesture image, a determination is first made as to whether the number of recognized gestures is zero. If not, indicating that the first gesture image contains gestures, the gesture category and similarity of the gestures included in the first gesture image are then identified.

[0398] As shown in Figure 26, Figure 26 is a schematic diagram of a first gesture image according to some embodiments of the present application. The first gesture coordinates are connected or annotated in the first gesture image to obtain a first gesture detection frame 2601. Optionally, the present application can control the display of the display device to display the image captured by the camera in real time, and display the first gesture image 2600 annotated with the first gesture detection frame 2601 during the gesture recognition process of the first gesture image. The gesture category and similarity of the first gesture can be annotated on the first gesture detection frame 2601. For example, in Figure 26, the gesture category of the first gesture is "Five" and the similarity is 0.998.

[0399] As shown in FIG27 , which is a schematic diagram of a second gesture image according to some embodiments of the present application, the display device now displays the acquired second gesture image 2700 , the first gesture detection frame 2601 , and the second gesture detection frame 2701 .

[0400] When the first gesture coordinates corresponding to the first gesture image and the second gesture coordinates corresponding to the second gesture image are obtained, the distance difference from the first gesture to the second gesture and the gesture movement direction from the first gesture to the second gesture are first calculated, that is, the magnitude and direction of the displacement change in the process of changing from the first gesture coordinates to the second gesture coordinates are calculated.

[0401] The embodiments of the present application provide an implementation method for calculating the distance difference and movement direction of the gesture in the horizontal coordinate direction based on the horizontal coordinate of the first gesture coordinate and the horizontal coordinate of the second gesture coordinate, and calculating the distance difference and movement direction of the gesture in the vertical coordinate direction based on the vertical coordinate of the first gesture coordinate and the vertical coordinate of the second gesture coordinate. As shown in Figure 10, Figure 28 is a schematic diagram of determining the gesture movement direction and gesture movement speed according to some embodiments of the present application. For ease of explanation, the first gesture detection frame 2601 and the second gesture detection frame 2701 are displayed in the same image in the figure. However, those skilled in the art will understand that the actual situation is not necessarily so. The first gesture detection frame shown in the image is only used to illustrate the position of the first gesture, and does not mean that the first gesture actually appears at that position in the gesture image.

[0402] In Figure 28, the first gesture coordinate A1 (x1, y1) and the second gesture coordinate A2 (x2, y2) are used to calculate the distance difference between the gestures in the horizontal direction according to the following formula (1): diff X = x2 - x1 (1)

[0403] Wherein, if diff X is greater than 0, it indicates that the gesture moves in the positive direction of the horizontal axis, and if it is less than 0, it indicates that the gesture moves in the negative direction of the horizontal axis.

[0404] The distance difference of the gesture in the vertical coordinate direction is calculated according to the following formula (2): diff Y = y2 - y1 (2)

[0405] Among them, diff Y greater than 0 indicates that the gesture moves in the positive direction of the vertical axis, and less than 0 indicates that the gesture moves in the negative direction of the vertical axis.

[0406] Optionally, a distance difference d between the first gesture coordinates and the second gesture coordinates is calculated according to a Euclidean distance formula, and the gesture movement direction is from the first gesture coordinates to the second gesture coordinates.

[0407] In some embodiments, after calculating the distance difference between the first gesture coordinates and the second gesture coordinates, it is determined whether the distance difference is greater than a preset distance threshold, which is a preset maximum distance for determining gesture jitter. When the distance difference is less than or equal to the preset distance threshold, it indicates that there is natural jitter in the gesture, and the first gesture is matched with the preset gesture to determine the control instruction corresponding to the first gesture, so as to execute the control instruction corresponding to the first gesture to operate the display device. The embodiment of the present application provides an implementation method for recording the distance difference between the first gesture coordinates and the second gesture coordinates in a tabular form, so as to subsequently obtain other gesture coordinates and calculate the distance difference between the second gesture coordinates and the other gesture coordinates. If the aforementioned distance difference is less than or equal to the preset threshold, and the positive and negative values ​​change alternately, it is determined that the gesture is jittery. The present application ignores the distance difference between the gesture coordinates, and any gesture can be matched with the preset gesture to determine the corresponding control instruction to control the display device to perform the corresponding operation.

[0408] The inventors found that in the actual gesture recognition process, there is inevitable shaking when the user makes a gesture, which causes a small displacement of the gestures recognized before and after. The current gesture recognition will recognize the displacement caused by gesture shaking as part of the user's gesture control. For example, when a user spreads out five fingers to select a certain media for playback, the five-finger spread gesture has up and down shaking, and the display device mistakenly recognizes it as controlling the up and down page turning, affecting the user's experience.

[0409] The above embodiment judges the distance difference of gesture movement and ignores gesture changes with a distance difference less than a preset distance threshold, thereby avoiding misrecognition caused by natural jitter of static gestures and accurately identifying the control instructions corresponding to the gestures, thereby improving the accuracy of gesture control of the display device.

[0410] If the distance difference between the first gesture coordinates and the second gesture coordinates is greater than a preset distance threshold, it indicates that the user's gesture has caused a significant displacement relative to the display device, rather than natural jitter. The gesture movement speed corresponding to the change from the first gesture coordinates to the second gesture coordinates is then calculated based on the first time difference between the first gesture image and the second gesture image, and the distance difference d between the second gesture coordinates and the second gesture coordinates.

[0411] In some feasible embodiments, the first time difference between the first gesture image and the second gesture image is t1, and the gesture movement speed in the horizontal axis direction is calculated according to the following formula (3):

[0412] The gesture movement speed in the vertical axis direction is calculated according to the following formula (4).

[0413] S2403: Control the camera to acquire a third gesture image, and determine a coordinate range to be recognized in the third gesture image according to the gesture movement direction and gesture movement speed.

[0414] In some embodiments, after analyzing the previously captured first gesture image and second gesture image, it is determined that the gesture changes from the position corresponding to the first gesture coordinates to the position corresponding to the second gesture coordinates, and the gesture movement direction and gesture movement speed are obtained, and then the camera is controlled to obtain a third gesture image. Based on the second time difference between the third gesture image and the first gesture image, or the second time difference between the third gesture image and the second gesture image, and the gesture movement speed, the candidate coordinate range where the third gesture may appear in the third gesture image is determined. Furthermore, based on the gesture movement direction from the first gesture coordinates to the second gesture coordinates, the coordinate range to be identified in the third gesture image is determined to accurately perform gesture recognition on the coordinate range to be identified.

[0415] Optionally, the second time difference between the third gesture image and the second gesture image is t2, the movement speed along the horizontal axis is Vx, and the candidate horizontal coordinate range is x2±Vx*t2; the movement speed along the vertical axis is Vy, and the candidate vertical coordinate range is y2±Vy*t2.

[0416] As shown in FIG. 29 , FIG. 29 is a schematic diagram of a coordinate range to be identified according to some embodiments of the present application, showing a third gesture image 29 , a second gesture detection frame 2701 , and a coordinate range 29A to be identified.

[0417] S2404: Perform gesture recognition on the coordinate range to be recognized, and if a preset gesture is matched, execute a control instruction of the preset gesture.

[0418] The control commands of the preset gestures include but are not limited to: mute, unmute, control volume, start / pause playback, switch channels, power on / off, etc.

[0419] During gesture recognition within the coordinate range to be recognized, the third gesture image can optionally be cropped based on the coordinate range to obtain a fourth gesture image. It is understood that the size of the fourth gesture image is the same as the size of the image corresponding to the coordinate range to be recognized. The fourth gesture image is then input into a gesture recognition model to obtain a gesture recognition result output by the gesture recognition model. The gesture recognition model is described in detail in related art and is not further described in this application. The gesture recognition results include: no gesture, gesture category, and gesture similarity.

[0420] When the third gesture image is identified as including the third gesture, and the gesture category and gesture similarity of the third gesture match a certain gesture in the preset gestures, the control instruction of the matched preset gesture can be determined and the control instruction of the preset gesture can be executed to achieve accurate gesture recognition and control of the display device through gestures when the user gesture moves.

[0421] As shown in Figure 30, Figure 30 is a schematic diagram of the third gesture image according to some embodiments of the present application. The figure shows the third gesture image 29. Gesture recognition is performed on the coordinate range 29A to be identified in the third gesture image 29, and the five-finger open gesture can be recognized. However, the OK gesture shown in the figure exceeds the coordinate range 29A to be identified. The OK gesture is not processed, and the control instruction corresponding to the five-finger open gesture is executed.

[0422] The above embodiment analyzes and predicts the possible location of the gesture by identifying and recording the gesture coordinates, identifies the gesture that may appear at that location, uses the gesture as a control gesture, determines the control instruction of the preset gesture that matches the control gesture, and enables the display device to execute the operation corresponding to the control instruction, thereby realizing gesture control of the display device; gestures that appear in other areas outside the location are determined to be interference gestures and are not processed, thereby effectively preventing other gestures that suddenly intrude during the gesture recognition process from interfering with the gesture recognition results, thereby improving the accuracy and stability of gesture control of the display device.

[0423] In summary, the fourth processing method for a display device in some embodiments of the present application, wherein the method first controls the camera to acquire continuous first gesture images and second gesture images, and then determines the gesture movement direction and gesture movement speed of changing from the first gesture image to the second gesture image based on the first gesture image and the second gesture image, further controls the camera to acquire a third gesture image and determines the coordinate range to be identified in the third gesture image according to the gesture movement direction and the gesture movement speed, and then performs gesture recognition on the coordinate range to be identified; when the gesture is identified to match a preset gesture, the control instruction of the preset gesture is executed, so that the display device can perform gesture recognition on the coordinate range to be identified in a targeted manner, avoid interference of sudden gestures on the display device, and improve the accuracy of gesture control of the display device.

[0424] With the rapid development of the internet and the widespread adoption of AI technology in recent years, display devices such as smart TVs have gradually become essential devices in people's daily lives and the preferred entertainment medium for the living room. Especially during family gatherings, the large screen size of smart TVs becomes increasingly prominent, making them a popular entertainment option.

[0425] With the continuous development of technology, the memory of display devices is getting larger and larger. At the same time, more and more applications are installed on display devices. Various resources and applications will occupy the available storage space of the display device. Therefore, the problem of shooting failure may occur due to insufficient storage space of the display device.

[0426] FIG31 is a flow chart of a processing method for a display device in the related art. As shown in FIG31 , the process specifically includes the following steps:

[0427] S3101: The display device determines that the available storage space of the display device is less than a preset threshold.

[0428] The preset value may be a pre-set value, such as 200M, or 10% of the total storage space, etc. It may also depend on specific circumstances, and this embodiment does not impose any specific limitation on this.

[0429] When the display device determines that the available storage space of the display device is less than a preset threshold, it indicates that the available storage space of the display device is insufficient.

[0430] S3102: Display device deletes resources or exports resources.

[0431] When the display device receives a resource deletion operation or resource export operation triggered by the user, the display device performs the deletion operation or the export operation on the resource selected by the user to increase the available storage space of the display device.

[0432] S3103: Determine whether the available storage space of the display device is less than a preset threshold.

[0433] If so, execute S3102; if not, execute S3104.

[0434] S3104, controlling the image acquisition module to continue acquiring images.

[0435] In the above method, when the available storage space of the display device is insufficient, the available storage space of the display device can only be increased by deleting resources or exporting resources. If the user does not want to delete or export resources, the problem of shooting failure caused by insufficient storage space of the display device cannot be solved. In addition, the above method provides users with few options and cannot meet the needs of users.

[0436] In response to the above problems, the present application proposes a fifth processing method for a display device. In this method, when the display device determines that the available storage space of the display device is less than a preset threshold, the display is controlled to display a prompt message, wherein the prompt message is used to prompt the user whether to turn on the memory saving mode. When an operation to turn on the memory saving mode is received, in response to the operation, the resolution of the camera is lowered from a first value to a second value, and the camera is controlled to capture images at a resolution corresponding to the second value. In the above technical solution, by lowering the resolution of the camera, the camera captures images at the lowered resolution, thereby reducing the size of the captured image, which can save the available storage space of the display device, enable the image capture device to capture more images, avoid the problem of shooting failure, and improve the user experience.

[0437] In some embodiments, the display device is a terminal device with display and image acquisition functions, such as a television, a mobile phone, a computer, etc. The at least one processor 250 in the display device is further configured to execute computer instructions stored in the memory of the display device so that the display device:

[0438] When determining that the available storage space of the display device is less than a preset threshold, controlling the display to display a prompt message, wherein the prompt message is used to prompt the user whether to enable a memory saving mode;

[0439] In response to the operation of enabling the memory saving mode, lowering the resolution of the camera from a first value to a second value;

[0440] The camera is controlled to capture images at a resolution corresponding to the second value.

[0441] In some embodiments, the at least one processor 250 is further configured to execute computer instructions to cause the display device to:

[0442] In response to an operation in which the memory saving mode is not enabled, controlling the display to display target information for resource management, wherein the target information includes at least one option of delete, export and delete, export, and reduce image quality;

[0443] In response to an operation of determining a target option from the target information and an operation of determining a first resource from local resources of the display device, performing an operation of the target option on the first resource to increase available storage space of the display device;

[0444] The camera is controlled to continue collecting images.

[0445] In some embodiments, the target options include reducing image quality;

[0446] At least one processor 250 is specifically configured to execute computer instructions to cause the display device to:

[0447] In response to determining the operation of reducing the image quality from the target information and determining the operation of the first resource from the local resources of the display device, lowering the quality value of the first resource to obtain a second resource;

[0448] The first resource and / or the second resource are stored in a target device, a replacement operation is performed on the first resource based on the second resource, and the first resource in the local resources is deleted to increase the available storage space of the display device.

[0449] In some embodiments, the target options include export and delete;

[0450] At least one processor 250 is specifically configured to execute computer instructions to cause the display device to:

[0451] In response to determining the export and deletion operation from the target information and determining the first resource from the local resources of the display device, the first resource is stored in the target device and a deletion operation is performed on the first resource to increase the available storage space of the display device.

[0452] In some embodiments, the at least one processor 250 is further configured to execute computer instructions to cause the display device to:

[0453] Before determining that the available storage space of the display device is less than the preset threshold, detecting that the camera is started, and determining whether the memory saving mode is turned on;

[0454] When the memory saving mode is enabled, the resolution of the camera is lowered from a first value to a second value.

[0455] In some embodiments, at least one processor 250 is specifically configured to execute computer instructions to cause the display device to:

[0456] Acquiring available storage space of the display device through a storage status manager of the display device;

[0457] According to a size relationship between the available storage space and the preset threshold, it is determined that the available storage space is smaller than the preset threshold.

[0458] In some embodiments, the aspect ratio of the first value is equal to the aspect ratio of the second value.

[0459] To summarize, the present application executes the fifth processing method for a display device on a display device. When the display device determines that the available storage space of the display device is less than a preset threshold, the display is controlled to display a prompt message, wherein the prompt message is used to prompt the user whether to turn on the memory saving mode. When an operation to turn on the memory saving mode is received, in response to the operation, the resolution of the camera is lowered from a first value to a second value, and the camera is controlled to capture images at a resolution corresponding to the second value. In the above technical solution, by lowering the resolution of the camera, the camera captures images at the lowered resolution, thereby reducing the size of the captured image, which can save the available storage space of the display device, enable the image capture device to capture more images, avoid the problem of shooting failure, and improve the user experience.

[0460] Figure 32 is a system framework diagram of a fifth processing method for a display device according to one or more embodiments of the present application. As shown in Figure 32, in some embodiments, the software implementation of this method mainly includes two layers: an application layer and a framework layer. The application layer includes a storage space acquisition module, a resource management module, and a camera; the framework layer includes a storage status manager and an information storage module. Specifically, the storage space acquisition module is mainly used to record the current available storage space of the display device, so that when the display device determines that the available storage space is less than the preset threshold based on the size relationship between the available storage space and the preset threshold, it controls the display of the display device to display a prompt message, which is used to remind the user; the resource management module is mainly used to store and operate local resources. For example, it can perform operations such as deleting, replacing or exporting some resources to the target device. The target device can be a USB flash drive, a server or a cloud, etc. This embodiment does not specifically limit this; the camera is mainly used for operations such as image capture, video recording and parameter adjustment; the storage status manager (Storage Stats Manager) is a storage space query and management function supported by the Android system. Through this part of the logic, the display device can determine the current storage space usage of the entire machine; the information storage module is the camera function of the Android system, mainly used to adjust the resolution and quality value, etc. At the same time, this part stores relevant information of the camera.

[0461] FIG33A is a flowchart diagram of a fifth processing method for a display device according to some embodiments of the present application. This embodiment can be used to illustrate the image acquisition process when the available storage space of the display device is insufficient. As shown in FIG33A , the method specifically includes the following steps:

[0462] S3310: When it is determined that the available storage space of the display device is less than a preset threshold, control the display to display a prompt message.

[0463] The prompt message prompts the user whether to enable memory-saving mode. Memory-saving mode can be understood as reducing the camera's resolution and the size of captured images to avoid capture failures due to insufficient available storage on the display device. A memory-saving mode switch can be configured on the display device to allow the user to enable memory-saving mode.

[0464] Specifically, the display device can obtain the usage of the storage space of the display device. When it is determined based on the usage that the available storage space of the display device is less than a preset threshold, it means that the available storage space of the display device is insufficient, which may cause shooting failure. At this time, the display of the display device is controlled to display a prompt message, and the user is prompted through the prompt message to allow the user to determine whether to turn on the memory saving mode.

[0465] S3320: In response to the operation of enabling the memory saving mode, lower the resolution of the camera from the first value to a second value.

[0466] The first value is the resolution value supported by the camera, such as 1920*1080, or other values, which are not specifically limited in this embodiment. The second value is the resolution value supported by the camera, such as 864*480, or other values, which are not specifically limited in this embodiment. A camera usually supports at least two resolution values. For example, a camera may support resolution values ​​of 800*600, 864*480, 1024*576, 1680*896, and 1920*1080, with 1920*1080 having the highest resolution and 800*600 having the lowest resolution.

[0467] When a user activates memory saving mode via a smart device, a control device, or voice, the display device, upon receiving the operation, lowers the camera resolution from a first value to a second value in response to the operation. Specifically, the second value may be determined by at least one processor in the display device or may be set by the user, and this embodiment is not limited thereto.

[0468] In some embodiments, the aspect ratio of the first value is equal to the aspect ratio of the second value.

[0469] In this embodiment, since the resolution is composed of width and height, when the aspect ratio of the first value is equal to the aspect ratio of the second value, it can be ensured that after the resolution is lowered, the captured image will not be stretched or deformed, and no black edges will appear on the upper and lower sides or the left and right sides, which is conducive to improving the user experience and satisfaction.

[0470] In some embodiments, the second value may be set by a user, and therefore, the aspect ratio of the first value may not be equal to the aspect ratio of the second value, which is not limited in this embodiment.

[0471] S3330: Control the camera to capture images at a resolution corresponding to the second value.

[0472] After the display device lowers the resolution of the camera from the first value to the second value, the camera is controlled to capture images at the resolution corresponding to the second value, which can reduce the size of the image captured at this time, thereby reducing the storage space occupied by the image captured at this time, thereby ensuring the normal use of the camera.

[0473] In some embodiments, when the first value is 1920*1080, when this resolution is used to capture an image of a certain scene, the resulting image size is 511KB. When the second value is 864*480, when this resolution is used to capture an image of the same scene, the resulting image size is 115KB. This indicates that lowering the camera resolution in the above manner can reduce the image size by a factor of four. This effect is particularly significant when the file size resulting from capturing an image at the resolution corresponding to the first value is larger.

[0474] The above method provided in the embodiment of the present application, when the display device determines that the available storage space of the display device is less than a preset threshold, controls the display to display a prompt message, wherein the prompt message is used to prompt the user whether to turn on the memory saving mode. When an operation to turn on the memory saving mode is received, in response to the operation, the resolution of the camera is lowered from a first value to a second value, and the camera is controlled to capture images at the resolution corresponding to the second value. In the above technical solution, by lowering the resolution of the camera, the camera captures images at the lowered resolution, thereby reducing the size of the captured image, which can save the available storage space of the display device, enable the image capture device to capture more images, avoid the problem of shooting failure, and improve the user experience. This solution is particularly suitable for scenarios where users do not have high requirements for resolution.

[0475] Figure 33B is a schematic diagram of an interface when a display displays a prompt message according to some embodiments of the present application. As shown in Figure 33B, when the display device determines that the available storage space of the display device is less than a preset threshold, the display is controlled to display a prompt message, which reads: Insufficient available storage space, you can choose to save memory mode or manage resources to increase available storage space, and two options are available for the user to select: save memory mode and resource management. Resource management means: Memory saving mode is not enabled, which can be understood as managing the local resources of the display device to increase the available storage space of the display device.

[0476] In some embodiments, optionally, the above method may further specifically include:

[0477] Before determining that the available storage space of the display device is less than a preset threshold, detecting that the camera is activated, and determining whether the memory saving mode is enabled;

[0478] When the memory saving mode is on, the camera resolution is lowered from a first value to a second value.

[0479] Specifically, before determining that the available storage space of the display device is less than a preset threshold, if the display device detects that the camera is activated, it determines whether the memory saving mode is enabled. If the memory saving mode is enabled, the display device lowers the resolution of the camera from the first value to the second value so that the camera can subsequently capture images at the resolution corresponding to the second value.

[0480] In this embodiment, when the available storage space of the display device is relatively sufficient, if the memory saving mode is turned on, the display device will lower the resolution of the camera from the first value to the second value, so that the camera can capture images at the resolution corresponding to the second value, thereby reducing the size of the captured image, saving storage space for the display device, and allowing more resources to be stored in the display device.

[0481] In some embodiments, when determining that the available storage space of the display device is less than a preset threshold, controlling the display to display a prompt message may specifically include:

[0482] Obtaining the available storage space of the display device through the storage status manager of the display device;

[0483] When it is determined based on the size relationship between the available storage space and the preset threshold that the available storage space is less than the preset threshold, the display is controlled to display a prompt message.

[0484] Specifically, the storage status manager of the display device has storage space query and management functions, so the display device can obtain the current available storage space of the display device through the storage status manager of the display device, compare the size of the available storage space with the preset threshold, and control the display to display a prompt message when it is determined that the available storage space is less than the preset threshold based on the size relationship.

[0485] In this embodiment, the above process can determine that the available storage space of the display device is less than the preset threshold, and provide prompt information in a timely manner to facilitate subsequent operations of the user.

[0486] FIG34A is a second flow chart of a fifth processing method for a display device according to some embodiments of the present application. This embodiment is optimized based on the above embodiment. Optionally, this embodiment can be applied to illustrate the image acquisition process when the memory saving mode is not enabled. As shown in FIG34A , the method specifically includes the following steps:

[0487] S3410: When it is determined that the available storage space of the display device is less than a preset threshold, control the display to display a prompt message.

[0488] S3420: In response to the operation of not turning on the memory saving mode, control the display to display target information of resource management.

[0489] The target information includes at least one option of deletion, export and deletion, export, and reduction of image quality.

[0490] Specifically, if the user has not enabled the memory saving mode, it indicates that the user has selected the resource management option in the prompt information. In this case, the display device controls the display to display target information for resource management in response to the operation of not enabling the memory saving mode. This target information is used for the user to select, so that the display device can subsequently perform the corresponding operation based on the user's selection.

[0491] S3430: In response to the operation of determining the target option from the target information and the operation of determining the first resource from the local resources of the display device, perform the operation of the target option on the first resource to increase the available storage space of the display device.

[0492] The target option can be understood as the option determined by the user from the options included in the target information. The local resource can be understood as a resource already stored in the display device, such as an image or video, which is not specifically limited in this embodiment. The first resource can be understood as the resource to be edited determined by the user from the local resources of the display device.

[0493] Specifically, when a user determines a target option from options included in target information for resource management displayed on a display, and determines a first resource from local resources of the display device, the display device can receive the user's operation of determining the target option from the target information and the user's operation of determining the first resource from local resources of the display device. In response to the user's operation of determining the target option from the target information and the user's operation of determining the first resource from local resources of the display device, the display device performs the operation of selecting the target option on the first resource, thereby increasing available storage space of the display device and facilitating subsequent camera operations to continue image acquisition.

[0494] S3440: Control the camera to continue collecting images.

[0495] Specifically, after the display device performs the target option operation on the first resource, the available storage space has increased. At this time, the camera is controlled to continue collecting images so that the camera can shoot normally.

[0496] The fifth processing method for a display device provided in an embodiment of the present application is that when the display device determines that the available storage space of the display device is less than a preset threshold, the display is controlled to display a prompt message. When an operation is received in which the memory saving mode is not turned on, that is, when an operation is received in which the user selects to manage resources, the display is controlled to display target information of resource management in response to the operation. When an operation is received in which the user determines a target option from the target information and a first resource from the local resources of the display device, the target option operation is performed on the first resource in response to the operation of determining the target option from the target information and the operation of determining the first resource from the local resources of the display device to increase the available storage space of the display device, and finally the camera is controlled to continue capturing images. In the above technical solution, by performing the target option operation on the first resource selected by the user, the available storage space of the display device can be increased, so that the image acquisition device can capture more images, avoid the problem of shooting failure, and improve the user experience. This solution is particularly suitable for scenarios where users have high resolution requirements, and can ensure that high-resolution images are captured and that the image acquisition device can shoot normally.

[0497] Figure 34B is a schematic diagram of an interface showing resource management target information on a display according to some embodiments of the present application. As shown in Figure 34B , the target information primarily includes four options: Delete, Export and Delete, Export, and Reduce Image Quality. Resource 1, Resource 2, Resource 3, and Resource 4 are the first resources identified. This display interface indicates that the target option identified from the target information is Reduce Image Quality.

[0498] It should be noted that the options included in the target information in Figure 34B, resource 1, resource 2, resource 3 and resource 4, are only used to illustrate this embodiment and are not used to limit it.

[0499] In some embodiments, the target options include reducing image quality;

[0500] In response to an operation of determining a target option from target information and an operation of determining a first resource from local resources of the display device, performing an operation of the target option on the first resource to increase available storage space of the display device may specifically include:

[0501] In response to determining an operation of reducing the image quality from the target information and determining an operation of the first resource from the local resources of the display device, lowering the quality value of the first resource to obtain a second resource;

[0502] The first resource and / or the second resource are stored in the target device, a replacement operation is performed on the first resource based on the second resource, and the first resource in the local resource is deleted to increase the available storage space of the display device.

[0503] The target device may be understood as a device with a storage function, such as a USB flash drive, a server, or a cloud, etc., and this embodiment does not impose any specific limitation on this.

[0504] Specifically, when the target option is to reduce the image quality, the display device responds to the user's operation of determining the image quality reduction from the target information and the operation of determining the first resource from the local resources of the display device. Since the display device provides an entry for compressed resources (non-packaged compression), the quality (i.e., clarity) of the first resource can be reduced through this entry to obtain a second resource, thereby reducing the storage space occupied by the resource and obtaining more storage space for the user. After obtaining the second resource, the display device can store the first resource to the target device, perform a replacement operation on the first resource based on the second resource, and delete the first resource from the local resources to increase the available storage space of the display device; the display device can also store the second resource to the target device, perform a replacement operation on the first resource based on the second resource, and delete the first resource from the local resources to increase the available storage space of the display device; the display device can also store both the first resource and the second resource to the target device, perform a replacement operation on the first resource based on the second resource, and delete the first resource from the local resources to increase the available storage space of the display device.

[0505] In some feasible embodiments, when reducing the image quality, the solution for reducing resource quality provided by the Android system can be used to obtain a bitmap corresponding to the first resource. The bitmap can support adjustment of the image quality. Specifically, any quality value from 0 to 100 can be selected, where 100 is the original image quality. The smaller the quality value, the worse the image clarity and the less storage space occupied.

[0506] In some feasible embodiments, the size of an original image (with a quality value of 100) is 1.02 MB. After the quality value of the image is lowered to 25, the size of the resulting image is 341 KB.

[0507] In this embodiment, by lowering the quality value of the first resource to obtain the second resource, retaining the second resource in the local resources of the display device and deleting the first resource, it can not only achieve the effect of retaining resources, but also save the available storage space of the display device, so that the image acquisition device can capture more images and avoid the problem of shooting failure.

[0508] In some embodiments, optionally, the target options include export and delete;

[0509] In response to an operation of determining a target option from target information and an operation of determining a first resource from local resources of the display device, performing an operation of the target option on the first resource to increase available storage space of the display device may specifically include:

[0510] In response to determining the export and deletion operation from the target information and the first resource from the local resources of the display device, the first resource is stored in the target device and the deletion operation is performed on the first resource to increase the available storage space of the display device.

[0511] Specifically, when the target option is to export and delete, the display device responds to the user's operation of determining the export and deletion from the target information and the operation of determining the first resource from the local resources of the display device, stores the first resource to the target device, and performs a deletion operation on the first resource, thereby increasing the available storage space of the display device.

[0512] In this embodiment, by storing the first resource to the target device and performing a deletion operation on the first resource, the first resource can be backed up and the available storage space of the display device can be saved, so that the image acquisition device can capture more images and avoid the problem of shooting failure.

[0513] In some embodiments, optionally, when the target option is delete, the display device performs a deletion operation on the first resource in response to the user's operation of determining the deletion from the target information and the operation of determining the first resource from the local resources of the display device. This can increase the available storage space of the display device, allowing the image acquisition device to capture more images and avoid the problem of shooting failure.

[0514] In some embodiments, optionally, when the target option is to export, the display device performs an export operation on the first resource in response to the user's operation of determining the export from the target information and the operation of determining the first resource from the local resources of the display device. Specifically, the export can be to any storage device other than the display device, thereby increasing the available storage space of the display device, enabling the image acquisition device to capture more images and avoiding the problem of shooting failure.

[0515] FIG35 is a schematic diagram of the interaction process between modules in implementing the fifth processing method for a display device according to some embodiments of the present application. Specifically, the process includes the following steps:

[0516] S7001: At least one processor of a display device detects that a camera is activated.

[0517] Specifically, at least one processor of the display device can detect in real time whether the camera is activated.

[0518] S7002: At least one processor of the display device obtains available storage space of the display device.

[0519] At least one processor of the display device can obtain available storage space of the display device through the storage status manager.

[0520] S7003: When at least one processor of the display device determines that the available storage space of the display device is less than a preset threshold, the at least one processor of the display device controls the display to display a prompt message.

[0521] Please refer to the detailed description in S3310. To avoid repetition, it will not be repeated here.

[0522] S7004: In response to the operation of enabling the memory saving mode, at least one processor of the display device adjusts the resolution of the camera from a first value to a second value.

[0523] Please refer to the specific description in S3320. To avoid repetition, it will not be repeated here.

[0524] S7005: At least one processor of the display device controls the camera to capture images at a resolution corresponding to the second value.

[0525] Please refer to the specific description in S3330. To avoid repetition, it will not be repeated here.

[0526] S7006: The camera stores the captured image in the resource management module.

[0527] After the camera captures an image at a resolution corresponding to the second value, the captured image is stored in the resource management module for subsequent viewing by users.

[0528] In this embodiment, the above S7001-S7006 mainly describe the situation after the user selects the memory saving mode option in the prompt information when the available storage space of the display device is insufficient. This process is mainly suitable for scenarios where the user does not have high requirements for resolution. By reducing the size of the captured image, the available storage space of the display device is saved, allowing the image capture device to capture more images and avoiding the problem of shooting failure.

[0529] FIG36 is a schematic diagram of the interaction process between modules of another embodiment of the fifth processing method for a display device according to some embodiments of the present application. Specifically, the process includes the following steps:

[0530] S8001: At least one processor of a display device detects that a camera is activated.

[0531] Please refer to the detailed description in S7001. To avoid repetition, it will not be repeated here.

[0532] S8002: At least one processor of the display device obtains available storage space of the display device.

[0533] Please refer to the detailed description in S7002. To avoid repetition, it will not be repeated here.

[0534] S8003: When at least one processor of the display device determines that the available storage space of the display device is less than a preset threshold, the at least one processor of the display device controls the display to display a prompt message.

[0535] Please refer to the detailed description in S3310. To avoid repetition, it will not be repeated here.

[0536] S8004: In response to the operation of not enabling the memory saving mode, at least one processor of the display device controls the display to display target information of resource management.

[0537] Please refer to the detailed description in S3420. To avoid repetition, it will not be repeated here.

[0538] S8005. In response to an operation of determining a target option from the target information and an operation of determining a first resource from local resources of the display device, at least one processor of the display device performs an operation of the target option on the first resource to increase available storage space of the display device.

[0539] Please refer to the detailed description in S3430. To avoid repetition, it will not be repeated here.

[0540] S8006: At least one processor of the display device controls the camera to continue capturing images.

[0541] Please refer to the detailed description in S3440. To avoid repetition, it will not be repeated here.

[0542] S8007: The camera stores the captured image in the resource management module.

[0543] After at least one processor of the display device performs a target option operation on the first resource to increase the available storage space of the display device, the camera stores the captured image in the resource management module for subsequent user viewing.

[0544] In this embodiment, the above S8001-S8007 mainly describe the situation after the user selects the resource management option in the prompt information when the available storage space of the display device is insufficient. This process is mainly suitable for scenarios where the user has high requirements for resolution. By executing the target option operation on the first resource selected by the user, the available storage space of the display device can be increased, which can not only ensure the capture of high-resolution images, but also ensure the normal shooting of the image acquisition device, so that the image acquisition device can capture more images and avoid the problem of shooting failure.

[0545] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various processes of the processing method for the display device in the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0546] The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0547] An embodiment of the present application provides a computer program product, which stores a computer program. When the computer program is executed by a processor, it implements the various processes of the processing method for the display device in the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0548] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0549] In this application, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0550] In this application, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0551] In this application, computer-readable media includes permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0552] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion of some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms of the above applications. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, comprising: a memory configured to store computer instructions and / or data associated with the display device; At least one processor, connected to the memory, is configured to execute the computer instructions to cause the display device to: Monitor the camera status broadcast message sent by the camera service in the operating system, where the camera status broadcast message is used to indicate whether the camera is enabled by the application; If the camera status broadcast message is a camera release message, the camera is called by a global application of the display device to capture a camera image, and the camera release message is used to indicate that the camera is not enabled by the application; Human body feature data is acquired from the camera image, and the display device is controlled to execute corresponding functions according to control instructions corresponding to the human body feature data.

2. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: If the camera status broadcast message is a camera activation message, controlling the display to display a camera activation icon on the current user interface to prompt the user that the camera is being activated by the application; The at least one processor, if the camera status broadcast message is a camera release message, causing the global application of the display device to call the camera to capture a camera image, is specifically configured to execute the computer instruction to enable the display device to: If the camera status broadcast message is a camera release message, the display is controlled to cancel the display of the camera activation icon on the current user interface, and the camera is called through the global application of the display device to capture the camera image.

3. The display device according to claim 1 , wherein the at least one processor, which obtains the human feature data from the camera image, is specifically configured to execute the computer instructions to enable the display device to: Detect whether there are valid human features in the camera image, the valid human features including: Hand gesture features and / or human posture features; If valid human features exist in the camera image, the human feature data is obtained from the camera image.

4. The display device according to claim 3, wherein the at least one processor, after detecting whether there is a valid human feature in the camera image, is further configured to execute the computer instruction to enable the display device to: If no valid human features exist in the camera image, determining whether the camera is blocked; If the camera is blocked, adjusting the frequency at which the global application calls the camera from the initial frequency to the target frequency, and / or outputting a first prompt message; in, The target frequency is less than the initial frequency, and the first prompt information is used to prompt the user that the camera of the display device is blocked.

5. The display device according to claim 4, wherein the at least one processor is configured to, if the camera is blocked, adjust the frequency at which the global application calls the camera from an initial frequency to a target frequency, and / or output a first prompt message, and is specifically configured to execute the computer instructions to cause the display device to: Collecting the image to be detected by the camera; If the difference between the brightness value of the image to be detected and the minimum brightness value is less than or equal to a first preset value, the frequency of the global application calling the camera is adjusted from the initial frequency to the target frequency, and / or a first prompt message is output.

6. The display device according to claim 5, The brightness value is represented by RGB value or grayscale value; and / or, The brightness value of the image to be detected includes at least one of the following situations: The brightness value of each pixel in the image to be detected, the brightness value of each pixel in a partial area of ​​the image to be detected, the average brightness value of all pixels in the image to be detected, and the average brightness value of pixels in a partial area of ​​the image to be detected.

7. The display device according to claim 4, wherein the at least one processor is configured to, if the camera is blocked, adjust the frequency at which the global application calls the camera from an initial frequency to a target frequency, and / or output a first prompt message, and is specifically configured to execute the computer instructions to cause the display device to: Collecting the image to be detected by the camera; Determining at least one horizontal detection area and at least one vertical detection area from the image to be detected; If the differences between the brightness values ​​of any pixel points in the target direction in the target detection area are less than or equal to a second preset value, adjusting the frequency at which the global application calls the camera from the initial frequency to the target frequency, and / or outputting a first prompt message; in, The target detection area is a horizontal detection area, and the target direction is a vertical direction; or the target detection area is a vertical detection area, and the target direction is a horizontal direction.

8. The display device according to claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Controlling the camera to capture a target image and displaying a video page on a display, wherein the target image is displayed on the video page; Performing human body image segmentation on the target image to determine a human body outline rectangular area, and obtaining vertex coordinates of the human body outline rectangular area in the video page; Determining adjustment parameters according to vertex coordinates of the rectangular area of ​​the human body contour; Based on the adjustment parameters, the camera position is adjusted.

9. The display device according to claim 8, wherein the at least one processor, before determining the adjustment parameters based on the vertex coordinates of the human body outline rectangular area, is further configured to execute the computer instructions to cause the display device to: Obtaining vertex coordinates of a preset effective control area in the video page; The at least one processor determines the adjustment parameter according to the vertex coordinates of the rectangular area of ​​the human body outline, and is specifically configured to execute the computer instruction to enable the display device to: The adjustment parameters are determined according to the vertex coordinates of the human body outline rectangular area and the vertex coordinates of the effective control area preset in the video page.

10. The display device according to claim 9, wherein the at least one processor determines the adjustment parameter based on vertex coordinates of the human body outline rectangular area and vertex coordinates of a preset valid control area in the video page, and is specifically configured to execute the computer instructions to cause the display device to: Determining, based on vertex coordinates of the human body outline rectangular area and vertex coordinates of a preset effective control area in the video page, whether the human body outline rectangular area overlaps with a margin area in the video page, where the margin area is an area in the video page excluding the effective control area; In a case where the human body outline rectangular area overlaps with the margin area in the video page, determining the orientation information of the overlapping area in the video page; The adjustment parameter is determined according to the position information of the overlapping area in the video page.

11. The display device according to claim 10, wherein the adjustment parameter is an adjustment direction, and the at least one processor determines the adjustment parameter based on direction information of the overlapping area in the video page, and is specifically configured to execute the computer instructions to cause the display device to: When the overlapping area is in the first direction on the video page, the adjustment direction of the camera is determined to be the first direction.

12. The display device according to claim 8, wherein the adjustment parameter is an adjustment direction, and the at least one processor determines the adjustment parameter according to vertex coordinates of the rectangular area of ​​the human body outline, and is specifically configured to execute the computer instructions to enable the display device to: In the case where it is determined that the human body outline rectangular area does not include a complete human body, it is determined that no human body part is captured. in a second direction relative to the video page; Determine that the adjustment direction of the camera is the second direction.

13. The display device according to claim 8, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: When it is determined that the human body outline rectangular area does not include a complete human body, calculating a first size of a human body portion that is not captured according to vertex coordinates of the human body outline rectangular area; Obtaining a second size of a target background area, wherein the target background area is a background area in the video page in a direction opposite to a direction where the human body portion is not captured; When the first size is less than or equal to the second size, determining a second direction of a position where the human body part is not captured relative to the video page; and determining that the adjustment direction of the camera is the second direction; When the first size is larger than the second size, outputting a first prompt message, wherein the first prompt message is used to prompt the user to stay away from the camera; in, The first dimension is a first width, and the second dimension is a second width; or, the first dimension is a first length, and the second dimension is a second length.

14. The display device according to claim 13, wherein the at least one processor is configured to calculate a first size of the uncaptured human body portion based on the coordinates of the human body outline rectangular area, and is specifically configured to execute the computer instructions to enable the display device to: Determine a third size of the captured human body portion in the video page; Calculating the first size of the uncaptured human body part on the video page according to the ratio of the third size to the target size; in, The target size ratio is the size ratio of the unphotographed human body part in the complete human body.

15. The display device according to claim 12, wherein the at least one processor is specifically configured to execute the computer instructions to enable the display device to: In the case of determining that the human body outline rectangular area includes a complete human body, determining the size of the complete human body in the video page; In the case that the size of the complete human body in the video page is smaller than half of the size of the video page, a second prompt information is output, where the second prompt information is used to prompt the user to get closer to the camera.

16. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Based on the preset operation, call out the gesture setting interface, where: The gesture setting interface is used to set a first correspondence between a target preset gesture and a preset function; In response to the target preset gesture input by the user in the gesture setting interface, and the preset function corresponding to the target preset gesture, a first correspondence between the target preset gesture and the preset function is established and stored, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence between the target preset gesture and the preset function.

17. The display device according to claim 16, wherein the at least one processor is specifically configured to execute the computer instructions to enable the display device to: Based on the number of times the preset function is used and the number of times the display device is turned on, calling out the gesture setting interface; Based on the detected first preset gesture, calling out the gesture setting interface; or Based on the operation of starting the gesture setting function, the gesture setting interface is called out.

18. The display device according to claim 17, wherein the at least one processor is further configured to execute the computer instructions to enable the display device to: Counting the number of times the preset function is used and the number of times the display device is turned on within the first time period; A first ratio of the number of times the preset function is used to the number of times the display device is turned on is calculated, and when the first ratio is greater than a first preset threshold, the gesture setting interface is called out.

19. The display device according to claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Counting the number of times a plurality of initial preset gestures are used and the number of times the display device is turned on within the second time period; Calculating a second ratio of the number of times each of the initial preset gestures is used to the number of times the display device is turned on; The target preset gesture is determined from the plurality of initial preset gestures based on the second ratio corresponding to each of the initial preset gestures.

20. The display device according to claim 19, wherein the at least one processor is specifically configured to execute the computer instructions to enable the display device to: The second ratio corresponding to each of the initial preset gestures is compared with a second preset threshold value. When any of the second ratios is smaller than the second preset threshold value, the initial preset gesture corresponding to the current second ratio is determined to be the target preset gesture.

21. The display device according to claim 19, wherein the at least one processor is specifically configured to execute the computer instructions to cause the display device to: The second ratios corresponding to the plurality of initial preset gestures are compared, and the initial preset gesture corresponding to the smallest second ratio is determined as the target preset gesture.

22. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Obtain parameter information corresponding to the preset function; A second correspondence between the preset function and the parameter information is established and stored, so that when the display device detects the target preset gesture, the preset function is executed based on the first correspondence and the second correspondence.

23. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: controlling the camera to acquire a continuous first gesture image and a second gesture image; and determining, based on the first gesture image and the second gesture image, a gesture movement direction and a gesture movement speed for changing from the first gesture image to the second gesture image; controlling the camera to acquire a third gesture image, and determining a coordinate range to be identified in the third gesture image according to the gesture movement direction and the gesture movement speed; Gesture recognition is performed on the coordinate range to be recognized, and when a preset gesture is matched, a control instruction of the preset gesture is executed.

24. The display device of claim 23, wherein the at least one processor determines, based on the first gesture image and the second gesture image, a gesture movement direction and a gesture movement speed for changing from the first gesture image to the second gesture image, and is specifically configured to execute the computer instructions to cause the display device to: Performing gesture recognition on the first gesture image and the second gesture image to determine first gesture coordinates corresponding to the first gesture image and second gesture coordinates corresponding to the second gesture image; determining a first time difference between the first gesture image and the second gesture image; The gesture movement direction and the gesture movement speed are determined according to the first time difference, the first gesture coordinates, and the second gesture coordinates.

25. The display device according to claim 24, wherein the at least one processor performs gesture recognition on the first gesture image and the second gesture image to determine first gesture coordinates corresponding to the first gesture image and second gesture coordinates corresponding to the second gesture image, and is specifically configured to execute the computer instructions to enable the display device to: In a case where a plurality of first candidate gestures and a plurality of second candidate gestures are obtained by performing gesture recognition on the first gesture image and the second gesture image, determining a first gesture from the plurality of first candidate gestures and determining a second gesture from the plurality of second candidate gestures according to similarities between the respective first candidate gestures and the respective second candidate gestures; in, The first gesture is a gesture whose gesture similarity among the multiple first candidate gestures is greater than or equal to a preset similarity; The second gesture is a gesture whose gesture similarity among the plurality of second candidate gestures is greater than or equal to the preset similarity; Determine first gesture coordinates corresponding to the first gesture and second gesture coordinates corresponding to the second gesture.

26. The display device according to claim 24, wherein the at least one processor determines a gesture movement direction and a gesture movement speed based on the first time difference, the first gesture coordinates, and the second gesture coordinates, and is specifically configured to execute the computer instructions to cause the display device to: Calculating a distance difference between the first gesture coordinates and the second gesture coordinates; When the distance difference is greater than a preset distance threshold, the gesture movement direction and the gesture movement speed are determined according to the first time difference and the distance difference.

27. The display device according to claim 26, wherein after calculating the distance difference between the first gesture coordinates and the second gesture coordinates, the at least one processor is further configured to execute the computer instructions to cause the display device to: When the distance difference is less than the preset distance threshold, a control instruction corresponding to the first gesture in the first gesture image is determined, so as to execute the control instruction corresponding to the first gesture to operate the display device.

28. The display device according to claim 23, wherein the at least one processor performs gesture recognition on the coordinate range to be recognized and, if a preset gesture is matched, executes a control instruction of the preset gesture, is specifically configured to execute the computer instruction to cause the display device to: Gesture recognition is performed on the coordinate range to be recognized, and when a preset gesture is matched and a movement speed of the gesture is greater than a preset threshold, a control instruction of the preset gesture is executed.

29. The display device according to claim 23, wherein the at least one processor performs gesture recognition on the coordinate range to be recognized, and is specifically configured to execute the computer instructions to enable the display device to: cropping the third gesture image according to the coordinate range to be identified to obtain a fourth gesture image; The fourth gesture image is input into a gesture recognition model to obtain a gesture recognition result output by the gesture recognition model, where the gesture recognition result includes no gesture, gesture category, and gesture similarity.

30. The display device according to claim 23, wherein the at least one processor determines a coordinate range to be recognized in the third gesture image based on the gesture movement direction and the gesture movement speed, and is specifically configured to execute the computer instructions to cause the display device to: Acquire a second time difference between the third gesture image and the first gesture image or the second gesture image; determining a candidate coordinate range in the third gesture image according to the second time difference and the gesture movement speed; The coordinate range to be identified is determined according to the candidate coordinate range and the gesture movement direction.

31. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: When it is determined that the available storage space of the display device is less than a preset threshold, the display is controlled to display a prompt message, wherein: The prompt information is used to prompt the user whether to turn on the memory saving mode; In response to the operation of enabling the memory saving mode, lowering the resolution of the camera from a first value to a second value; The camera is controlled to capture images at a resolution corresponding to the second value.

32. The display device of claim 31 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: In response to the operation of not turning on the memory saving mode, controlling the display to display target information of resource management, wherein, The target information includes at least one option of delete, export and delete, export, and reduce image quality; In response to an operation of determining a target option from the target information and an operation of determining a first resource from local resources of the display device, performing an operation of the target option on the first resource to increase available storage space of the display device; The camera is controlled to continue collecting images.

33. The display device according to claim 32, wherein the target option comprises reducing picture quality; The at least one processor is specifically configured to execute the computer instructions to enable the display device to: In response to determining the operation of reducing the image quality from the target information and determining the operation of the first resource from the local resources of the display device, lowering the quality value of the first resource to obtain a second resource; The first resource and / or the second resource are stored in a target device, a replacement operation is performed on the first resource based on the second resource, and the first resource in the local resources is deleted to increase the available storage space of the display device.

34. The display device according to claim 32, wherein the target options include export and delete; The at least one processor is specifically configured to execute the computer instructions to enable the display device to: In response to determining the export and deletion operation from the target information and determining the first resource from the local resources of the display device, the first resource is stored in the target device and a deletion operation is performed on the first resource to increase the available storage space of the display device.

35. The display device of claim 31 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Before determining that the available storage space of the display device is less than the preset threshold, detecting that the camera is started, and determining whether the memory saving mode is turned on; When the memory saving mode is enabled, the resolution of the camera is lowered from a first value to a second value.

36. The display device according to claim 31 , wherein the at least one processor is specifically configured to execute the computer instructions to cause the display device to: Acquiring available storage space of the display device through a storage status manager of the display device; According to a size relationship between the available storage space and the preset threshold, it is determined that the available storage space is smaller than the preset threshold.

37. The display device according to any one of claims 31 to 36, wherein the aspect ratio of the first value is equal to the aspect ratio of the second value.

38. A processing method for a display device, comprising: Monitor the camera status broadcast message sent by the camera service in the operating system, where the camera status broadcast message is used to indicate whether the camera is enabled by the application; If the camera status broadcast message is a camera release message, the camera is called by a global application of the display device to capture a camera image, and the camera release message is used to indicate that the camera is not enabled by the application; Human body feature data is acquired from the camera image, and the display device is controlled to execute corresponding functions according to control instructions corresponding to the human body feature data.

Images