Display device and display device control method

Abstract

The invention discloses a display device and a display device control method.The display device comprises a display module, a control module, a gesture sensing module and a light sensing module, and the light sensing module is used for detecting the brightness of ambient light and generating an ambient light brightness signal; the control module is used for adjusting working parameters of the gesture sensing module according to the ambient light brightness signal; the gesture sensing module is used for sensing an instruction gesture made by a user and generating a corresponding sensing signal; and the control module is also used for generating a corresponding control signal according to the sensing signal and controlling the display module to display through the control signal. Therefore, the display device can be controlled by identifying the gesture of the user, so that diversified requirements in different scenes are met, the operation convenience is improved, the working parameters of the gesture sensing module are dynamically adjusted according to the ambient light brightness condition, the accuracy and reliability of instruction gesture identification can be improved, and the energy loss is reduced.

Description

Technical Field

[0001] The present application relates to the field of image display technology, and in particular to a display device and a display device control method. Background Art

[0002] With the increasing prevalence of digital interaction, traditional display system interaction methods that rely on physical buttons or touch screens present numerous inconveniences and limitations in certain scenarios. For example, in medical surgical settings, doctors must adhere to sterile procedures and cannot directly touch display devices. In industrial manufacturing, workers' hands may be greasy, and contact with the display system can easily stain the screen, affecting the viewing experience. Furthermore, in many other application scenarios, such as conference presentations, medical imaging, and industrial monitoring, users need to frequently switch display viewing angles or manipulate displayed content. Traditional display system interaction methods are inconvenient and hinder the user's viewing experience. While voice control of display systems can also achieve contactless control for greater convenience, voice control has high noise requirements and low voice recognition accuracy in noisy environments. Furthermore, it is not suitable for environments where quietness is essential, such as libraries and conference rooms.

[0003] Therefore, it is necessary to provide an improved technical solution to overcome the above technical problems existing in the prior art. Summary of the Invention

[0004] The purpose of this application is to provide a display device and a display device control method, which can control the display device by recognizing user gestures, thereby meeting diverse needs in different scenarios and improving the convenience of operation.

[0005] To achieve the above objectives: In a first aspect, an embodiment of the present application provides a display device, comprising a display module, a control module, a gesture sensing module, and a light sensing module, wherein the display module, the gesture sensing module, and the light sensing module are all communicatively connected to the control module; The light sensing module is used to detect the brightness of the ambient light and generate an ambient light brightness signal; The control module is used to adjust the working parameters of the gesture sensing module according to the ambient light brightness signal; The gesture sensing module is used to sense the command gesture made by the user and generate a corresponding sensing signal; The control module is further configured to generate a corresponding control signal according to the sensing signal, and control the display module to perform display via the control signal.

[0006] In one embodiment, the gesture sensing module includes a first sensing unit for sensing the instruction gesture in a non-contact sensing manner and generating a corresponding sensing signal.

[0007] In one embodiment, the first sensing unit includes at least one of an infrared sensor, a visual sensor, a microwave sensor, a laser sensor, and an ultrasonic sensor.

[0008] In one embodiment, the gesture sensing module further includes a second sensing unit for sensing the pressure generated by the instruction gesture and generating a corresponding piezoelectric signal as the sensing signal based on the pressure.

[0009] In one embodiment, the display device further includes a user matching module, which is communicatively connected to the control module and is used to detect the user's operating authority for the display device.

[0010] In a second aspect, an embodiment of the present application provides a display device control method for the display device described in the above embodiment, the display device control method comprising the following steps: S1, detecting the brightness of ambient light through the light sensing module and generating an ambient light brightness signal; S2, using the control module to adjust the operating parameters of the gesture sensing module according to the ambient light brightness signal; S3, using the gesture sensing module to sense the instruction gesture made by the user and generate a corresponding sensing signal; S4, using the control module to generate a corresponding control signal according to the sensing signal; S5, controlling the display module to display via the control signal.

[0011] In one embodiment, the step S1 includes: sensing the instruction gesture in a non-contact sensing manner and generating a corresponding sensing signal.

[0012] In one embodiment, the step S1 further includes: sensing the pressure generated by the instruction gesture, and generating a corresponding piezoelectric signal based on the pressure as the sensing signal.

[0013] In one embodiment, in step S1, generating the corresponding sensing signal includes: Matching the command gesture with a preset gesture template, and determining whether the command gesture complies with an operating specification of the display device based on a matching result; If it is determined that the command gesture complies with the operating specification of the display device, generating a corresponding sensing signal according to the command gesture; If it is determined that the instruction gesture does not comply with the operation specification of the display device, a preset input auxiliary operation is performed.

[0014] In one embodiment, before step S1, the method further includes the following steps: S0, detecting the user's operating authority for the display device, and determining whether to activate the gesture sensing module according to the detection result.

[0015] The display device and display device control method provided in the embodiments of the present application detect the brightness of ambient light through a light sensing module and generate an ambient light brightness signal. The control module adjusts the working parameters of the gesture sensing module according to the ambient light brightness signal. The gesture sensing module senses the command gesture made by the user and generates a corresponding sensing signal. The control module generates a corresponding control signal according to the sensing signal to control the display module to display. In this way, the display device can be controlled by recognizing the user's gestures, thereby meeting the diverse needs in different scenarios and improving the convenience of operation. The working parameters of the gesture sensing module are dynamically adjusted according to the ambient light brightness conditions, which can also improve the accuracy and reliability of command gesture recognition and reduce energy loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 This is a structural block diagram of a display device provided in one embodiment of the present application.

[0018] Figure 2 This is a structural diagram of a display device provided in one embodiment of the present application.

[0019] Figure 3 This is a structural block diagram of a first sensing unit provided in one embodiment of the present application.

[0020] Figure 4 This is a circuit structure diagram of a first sensing unit provided in one embodiment of the present application.

[0021] Figure 5 This is a structural block diagram of a display device provided in another embodiment of the present application.

[0022] Figure 6 This is a structural block diagram of a control module provided in one embodiment of the present application.

[0023] Figure 7A flowchart of a display device control method provided in one embodiment of the present application.

[0024] Figure 8 A schematic diagram of the working process of dynamically adjusting the working parameters of the first sensing unit provided in one embodiment of the present application.

[0025] Figure 9 A schematic diagram of the working process of recognizing command gestures through a non-contact sensing method provided in one embodiment of the present application.

[0026] Figure 10 A schematic diagram of the working process of recognizing command gestures through a multimodal feature fusion sensing method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0028] It should be noted that, in this document, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0029] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the term "if" as used herein may be interpreted as "at the time of," "when," or "in response to a determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising" and "including" indicate the presence of the described features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, meaning any one or any combination. Thus, “A, B, or C” or “A, B, and / or C” means “any of: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition occurs only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0030] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and they can be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0031] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial restriction on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0032] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In the subsequent description, the use of suffixes such as "module", "component" or "unit" to represent elements is only for the purpose of facilitating the description of the present application and has no specific meaning. Therefore, "module", "component" or "unit" can be used interchangeably.

[0034] Figure 1 This is a structural block diagram of a display device provided in one embodiment of the present application. Figure 1 As shown, the display device provided by the embodiment of the present application includes a display module 110 , a control module 120 , a gesture sensing module 130 and a light sensing module 140 .

[0035] Among them, the display module 110, the gesture sensing module 130 and the light sensing module 140 are all communicatively connected to the control module 120; the light sensing module 140 is used to detect the brightness of the ambient light and generate an ambient light brightness signal; the control module 120 is used to adjust the working parameters of the gesture sensing module 130 according to the ambient light brightness signal; the gesture sensing module 130 is used to sense the command gesture made by the user and generate a corresponding sensing signal; the control module 120 is also used to generate a corresponding control signal based on the sensing signal, and control the display module 110 to display through the control signal.

[0036] Specifically, the display module 110 is used to display the operating system interface, application programs, multimedia content, etc. The gesture sensing module 130, the control module 120 and the display module 110 can be installed in the same component, such as Figure 2 As shown, in a laptop computer, the gesture sensing module 130 can be positioned above the display module 110, alongside the camera, while the control module 120 is mounted on a motherboard connected to the display module 110. In other embodiments, the display module 110 and the control module 120 can be assembled together, while the gesture sensing module 130 can be a separate component, positioned in an area convenient for user operation. This application does not limit the layout of the gesture sensing module 130, the control module 120, and the display module 110.

[0037] In one embodiment of the present application, the display device provided in the embodiment of the present application further includes a power module 150 , and the power module 150 provides a power supply voltage to the control module 120 .

[0038] In one embodiment of the present application, the gesture sensing module 130 includes a first sensing unit 131 for sensing a command gesture in a contactless manner and generating a corresponding sensing signal. Specifically, the first sensing unit 131 includes at least one of an infrared sensor, a visual sensor, a microwave sensor, a laser sensor, and an ultrasonic sensor.

[0039] Figure 3This is a structural block diagram of a first sensing unit provided in an embodiment of the present application; Figure 3 As shown, in a specific embodiment of the present application, the first sensing unit 131 is, for example, an infrared sensor, and the first sensing unit 131 includes a voltage stabilizing unit, an infrared driving module, a gesture recognition algorithm unit, a signal processing unit, a dynamic parameter configuration register, a system clock and power management unit, and an external interface.

[0040] The voltage stabilizing unit is used to convert the power supply voltage output by the control module 120 into the working voltage required by the first sensing unit 131. The voltage stabilizing unit can be implemented by a low dropout regulator (LDO), or other commonly used voltage regulators such as a DC-DC converter, a Buck-Boost converter, etc. Figure 4 As shown, the voltage stabilizing unit includes a voltage regulator U2 and a voltage regulator U3 , which convert 5V power into 3.3V power to power the 3.3V logic circuit in the first sensing unit 131 .

[0041] The infrared driver module uses an infrared LED array to emit infrared light into a pre-defined sensing area. When a user makes a command gesture within the sensing area, it blocks or reflects some of the infrared light. The photodiode array in the infrared driver module receives the reflected light signal and generates a corresponding electrical signal. The signal processing unit comprises an analog front-end and a digital logic unit. Because the electrical signal generated by the photodiode is generally weak, it is amplified by an operational amplifier in the analog front-end before signal processing. To improve signal quality and reduce interference from ambient light, a bandpass filter in the analog front-end is used to filter out interfering ambient light signals, retaining the portion that matches the infrared driver module's emission frequency. The amplified and filtered analog signal is converted to a digital signal by an analog-to-digital converter for further processing by the digital logic unit. The processed digital signal is then transmitted to the gesture recognition algorithm unit, which analyzes the digital signal's characteristics, such as time, frequency, and amplitude, according to a pre-defined algorithm. The algorithm then matches the digital signal with the standard feature information of a pre-defined gesture template to determine the user's gesture and generate a sensing signal corresponding to the command gesture.

[0042] The dynamic parameter configuration register stores environmental condition parameters and operating parameters for the operating mode corresponding to the environmental condition parameters. For example, the environmental condition parameters include an ambient light brightness threshold, and the operating parameters corresponding to the operating mode include the infrared emission frequency, the duty cycle of the pulse width modulation (PWM) signal, the drive current, the detection threshold, etc. The brightness of the ambient light detected by the light sensing module 140 is compared with the stored ambient light brightness threshold to determine the ambient light brightness threshold corresponding to the current ambient light brightness. The operating mode of the infrared driver module is determined based on the corresponding ambient light brightness threshold, and the corresponding operating parameters are configured for the infrared driver module based on the corresponding operating mode. Specifically, by adjusting the emission frequency of the infrared LED array, the duty cycle of the PWM signal, and the magnitude of the drive current, the energy density and energy intensity of the emitted infrared light can be changed. By adjusting the detection threshold, the signal strength threshold used for gesture recognition can be changed. In actual applications, when the ambient light is strong, the signal-to-noise ratio of the reflected signal can be increased by adjusting the operating parameters of the infrared driver module, thereby improving the quality of the infrared reflection signal and increasing the accuracy of gesture recognition.

[0043] The system clock and power management unit uses a built-in clock source to adjust the clock frequency based on the actual needs of the display device. This means it can increase the clock frequency when high-speed processing is required, and reduce it in low-power mode to optimize system energy consumption. Low-power mode is achieved through the built-in power management unit. When the display device has not detected gestures for a period of time or is inactive, it automatically switches to low-power mode. This mode reduces energy consumption by disabling the infrared driver module and other unnecessary modules. Once the display device's gesture recognition function is awakened by a gesture, it quickly switches back to normal operation, resuming infrared emission and signal processing functions.

[0044] In some other embodiments, such as when the first sensing unit 131 uses a visual sensor, when the ambient light is strong, the accuracy of command gesture recognition can also be improved by increasing the sensitivity of the visual sensor, shortening the image exposure time, adjusting the image grayscale, increasing the image contrast, and other technical means.

[0045] like Figure 4As shown, in this embodiment, functional modules such as the infrared driver module, gesture recognition algorithm unit, signal processing unit, dynamic parameter configuration register, and system clock and power management unit are integrated into a single integrated circuit U1. IIC communication is achieved with the control module 120 via the serial data interface SDA and the serial clock interface SCL. When the first sensing unit 131 detects a command gesture, an interrupt signal is sent to the control module 120 via the interrupt interface INT, causing the control module 120 to interrupt the current workflow and process the gesture sensing module's sensing signal. The external interface is implemented by connector J1, which connects to the external control module 120. Connector J1 provides a 5V power interface, a serial data interface SDA, a serial clock interface SCL, and an interrupt interface INT, enabling IIC, SPI, UART, GPIO, and other communication functions with external devices.

[0046] In another embodiment of the present application, Figure 5 As shown, to improve the accuracy of gesture recognition, the gesture sensing module 130 further includes a second sensing unit 132 for sensing the pressure generated by the instruction gesture and generating a corresponding piezoelectric signal as a sensing signal based on the pressure.

[0047] Specifically, the second sensing unit 132 can be implemented using a piezoelectric film. Specifically, a flexible piezoelectric film array is embedded on the surface of the first sensing unit 131 to implement the gesture sensing function of the second sensing unit 132. A user can make a command gesture in the sensing area corresponding to the second sensing unit 132. The second sensing unit 132 converts the pressure signal generated by the gesture on the piezoelectric film into a corresponding piezoelectric signal, which serves as the sensing signal output by the second sensing unit 132. The control module 120 uses the sensing signals from the first sensing unit 131 and the second sensing unit 134 as a basis for determining the user's command gesture, thereby further improving the accuracy of command gesture recognition.

[0048] Figure 6 This is a structural block diagram of a control module provided in one embodiment of the present application; Figure 6As shown, the control module 120 includes a microcontroller unit (MCU), a field programmable gate array (FPGA), an output interface, and a connector. The MCU, as the control center of the display device, is primarily responsible for receiving ambient light brightness data transmitted by the light sensing module 140, communicating with the gesture sensing module 130 via IIC via a connector, and configuring the operating parameters of the gesture sensing module. It also detects pulse changes on the gesture sensing module's interrupt interface INT to respond to sensing signals sent by the gesture sensing module in real time. Based on the gesture recognition results represented by the sensing signals, it configures corresponding control instructions and generates control signals corresponding to the control instructions. The power supply voltage from the power module 150 is provided to the gesture sensing module 130 via the power interface of the connector. The FPGA interacts with the MCU and, under the control of the control signals, generates signals related to the display driver, such as eDP and MIPI. This signals transmits pixel data and timing information to the display module 110 via the output interface, thereby controlling the display content and display effects of the display module 110.

[0049] In one embodiment of the present application, the user can customize the control instructions corresponding to different command gestures. Specifically, the command gestures can include at least one of a three-dimensional spiral gesture, a multi-finger collaborative gesture, a dynamic gesture sequence, and a multi-joint gesture. The control instructions can be operations on the displayed content, such as panning / rotating / zooming the display, capturing and saving the display, turning pages, etc., and can also be operations on the display effects, such as adjusting the backlight brightness, contrast, switching between wide / narrow display viewing angles, etc.

[0050] In a specific embodiment, the correspondence between the command gestures, the sensing signal characteristics, and the control commands is shown in Table 1 below.

[0051] Table 1

[0052] like Figure 5 As shown, in another embodiment of the present application, the display device further includes a user matching module 160 , which is communicatively connected with the control module 120 and is used to detect the user's operating authority over the display device.

[0053] Specifically, the user matching module 160 can confirm whether the current user has the authority to operate the display device by detecting the user's biometric information, for example, through face recognition, fingerprint recognition or voice recognition, to confirm whether the current user's command gestures are collected and recognized to avoid erroneous operations by unauthorized personnel.

[0054] In one embodiment of the present application, the user's biometric information can be stored in advance in a storage unit, which can be integrated into the user matching module 160. The user matching module 160 also includes a visual sensor, which can be, for example, a camera in an existing display device, and the camera automatically recognizes the user's face and voice. In other embodiments, the user matching module 160 also includes a fingerprint identifier, which automatically recognizes the user's fingerprint and confirms whether to collect and recognize the current user's command gestures, thereby preventing unauthorized personnel from erroneous operations.

[0055] In one embodiment of the present application, the display device further includes a status feedback module 170, which is communicatively coupled to the control module 120 and configured to provide feedback to the user on the current gesture recognition status. For example, the status feedback module 170 is communicatively coupled to the microcontroller unit (MCU) of the control module 120 and can provide feedback on the gesture recognition status to the user via an LED indicator light. For example, a gradual green transition indicates successful gesture recognition, while a flashing red light indicates failed gesture recognition, thereby prompting the user to repeat the gesture instruction.

[0056] The display device in the embodiment of the present application detects the brightness of ambient light through a light sensing module and generates an ambient light brightness signal. The control module adjusts the working parameters of the gesture sensing module according to the ambient light brightness signal. The gesture sensing module senses the command gesture made by the user and generates a corresponding sensing signal. The control module generates a corresponding control signal according to the sensing signal to control the display module to display. In this way, the display device can be controlled by recognizing the user's gestures, thereby meeting the diverse needs in different scenarios and improving the convenience of operation. The working parameters of the gesture sensing module are dynamically adjusted according to the ambient light brightness conditions, which can also improve the accuracy and reliability of command gesture recognition and reduce energy loss.

[0057] Figure 7 Schematic diagram of the flow of the display device control method provided in the embodiment of the present application. Figure 7 As shown, the display device control method provided in the embodiment of the present application includes the following steps: S1, detects the brightness of the ambient light through the light sensing module and generates an ambient light brightness signal.

[0058] S2, using the control module to adjust the working parameters of the gesture sensing module according to the ambient light brightness signal.

[0059] Specifically, because ambient light brightness can vary significantly at different times and locations, it can significantly impact the accuracy of command gesture recognition. Therefore, before triggering the gesture sensing module for gesture recognition, the light sensing module detects the ambient light brightness. This allows the control module to dynamically adjust the gesture sensing module's operating parameters based on the ambient light brightness, improving gesture recognition accuracy.

[0060] In a specific embodiment, when the command gesture is recognized by infrared sensing, the specific working process of dynamically adjusting the working parameters of the first sensing unit according to the brightness of the ambient light is as follows: Figure 8 As shown, the brightness of the ambient light is collected by the light sensing module, and the control module calculates the average brightness L within a preset time interval; if the average brightness L is greater than or equal to the strong light threshold L_high, the current ambient light is determined to be strong light, and the infrared sensor is controlled to operate in the first pulse mode; if the average brightness L is less than or equal to the weak light threshold L_low, the current ambient light is determined to be weak light, and the infrared sensor is controlled to operate in the third pulse mode; if the average brightness L is less than the strong light threshold L_high and greater than the weak light threshold L_low, the current ambient light is determined to be normal light, and the infrared sensor is controlled to operate in the second pulse mode. Among them, the frequency of the first pulse mode is greater than the frequency of the second pulse mode, the frequency of the second pulse mode is greater than the frequency of the third pulse mode, and the pulse width of the first pulse mode is less than the pulse width of the second pulse mode, and the pulse width of the second pulse mode is less than the pulse width of the third pulse mode. Therefore, in a strong light environment, the emission frequency and energy density of the first pulse mode are the largest, which can reduce the interference of ambient light and improve the quality of the infrared reflection signal. In a weak light environment, the infrared signal is easier to detect, and the reflection signal can be more accurately identified without the need for a high infrared emission frequency and energy density. In other embodiments, the system can also only detect strong light scenes and non-strong light scenes (i.e., low light and normal light scenes). In non-strong light scenes, the infrared sensor is controlled to operate in normal pulse mode; in strong light scenes, the infrared sensor is controlled to switch to high-frequency short pulse mode. By dynamically adjusting the infrared emission frequency to adapt to different ambient lighting conditions, the accuracy and reliability of command gesture recognition can be improved, and energy consumption can be reduced.

[0061] S3, using a gesture sensing module to sense the command gesture made by the user and generate a corresponding sensing signal.

[0062] In one embodiment of the present application, step S3 includes: sensing the instruction gesture in a non-contact sensing manner and generating a corresponding sensing signal.

[0063] Specifically, in some scenarios where it is inconvenient to directly contact the display device, gesture sensing modules such as infrared sensors, visual sensors, microwave sensors, laser sensors or ultrasonic sensors can be used to sense the command gestures made by the user in a non-contact manner. In this way, even if there is no contact with the display device, the control of the display device can be triggered.

[0064] In one embodiment of the present application, generating a corresponding sensing signal includes: matching the command gesture with a preset gesture template, and determining whether the command gesture complies with the operating specifications of the display device based on the matching result; if it is determined that the command gesture complies with the operating specifications of the display device, generating a corresponding sensing signal based on the command gesture; if it is determined that the command gesture does not comply with the operating specifications of the display device, executing a preset input auxiliary operation.

[0065] Specifically, the matching confidence level can be set to determine whether the command gesture matches the gesture template. When the matching confidence level between the command gesture and the preset gesture template is greater than the preset matching standard value (such as 90%), it indicates that the current signal feature is highly matched with the preset gesture template, and it can be determined that the command gesture meets the operating specifications of the display device. When the matching confidence level between the command gesture and the preset gesture template is less than or equal to the preset matching standard value, it indicates that the sensing signal is fuzzy or there is interference, and it is determined that the command gesture does not meet the operating specifications of the display device. When the command gesture does not meet the operating specifications of the display device, a status feedback module, such as a red flashing LED indicator, can be used to remind the user that the operation has failed and the command gesture needs to be made again. When the user makes the command gesture again, input auxiliary operations can be enabled, such as calling the camera to assist in the recognition of the command gesture, so as to improve the accuracy of gesture recognition and avoid multiple operation failures by the user, which affects the user experience.

[0066] In a specific embodiment, if Figure 9 As shown, the working process of identifying the command gesture by non-contact sensing includes: Step S311: Acquire first characteristic information of the gesture action through the first sensing unit.

[0067] Step S312: matching the first feature information with the first standard feature information in the preset gesture template, and calculating the matching confidence according to the matching result.

[0068] Step S313: Determine whether the matching confidence is greater than a preset matching standard value.

[0069] Step S314: When the matching confidence is greater than a preset matching standard value, the instruction gesture corresponding to the first feature information is determined, and a corresponding sensing signal is generated according to the instruction gesture.

[0070] Exemplarily, when the first sensing unit recognizes the command gesture through the infrared sensor, the first feature information is the signal feature of the gesture action sensed by the infrared matrix, and the first standard feature information is the infrared matrix signal feature corresponding to the gesture template. For example, when the user uses the two-finger pinch gesture for control, it is necessary to make a standard gesture action according to the operating specifications of the two-finger pinch gesture, that is, the distance between the two fingers in the gesture action must be less than the fixed distance, and the translation along the X-axis or Y-axis exceeds the preset distance. The first sensing unit obtains the first feature information by sensing the hand motion characteristics of the gesture action, and matches the first feature information with the first standard feature information of each gesture template. When the distance change rate of the two reflection points in the first feature information is greater than the distance change standard value (such as 10cm / s), the calculated matching confidence at this time will be greater than the preset matching standard value, then the gesture action made by the user is determined to be a two-finger pinch gesture, and a sensing signal representing the two-finger pinch gesture is generated.

[0071] Step S315: When the matching confidence is less than or equal to the preset matching standard value, it is determined that the instruction gesture does not comply with the operation specification of the display device, and a preset input auxiliary operation is performed.

[0072] Exemplarily, when the distance change rate of the double reflection points in the first feature information is less than the distance change standard value, and the first feature information does not match the first standard feature information of other gesture templates, the calculated matching confidence will be less than or equal to the preset matching standard value, and it is determined that the gesture action made by the user does not comply with the operating specifications of the display device.

[0073] In another embodiment of the present application, step S3 further includes: sensing the pressure generated by the instruction gesture, and generating a corresponding piezoelectric signal as a sensing signal based on the pressure.

[0074] Specifically, since a single sensing method is susceptible to interference or failure, the accuracy of the gesture sensing function decreases or even fails. Therefore, in order to further improve the sensing accuracy of command gestures, another embodiment of the present application uses a multimodal feature fusion method to identify command gestures. For example, a flexible piezoelectric film array is embedded on the surface of the infrared sensor module, and the infrared reflection signal (X / Y axis trajectory) and the piezoelectric signal (Z axis pressure) are used to jointly generate a sensing signal corresponding to the command gesture.

[0075] In another specific embodiment, Figure 10 As shown in FIG, the working process of recognizing command gestures through the sensing method of multimodal feature fusion specifically includes: Step S321: Acquire first characteristic information and second characteristic information of the gesture action through the first sensing unit and the second sensing unit respectively.

[0076] Step S322: Match the first feature information with the first standard feature information in the preset gesture template, match the second feature information with the second standard feature information in the preset gesture template, and calculate the matching confidence based on the matching results of the first standard feature information and the second feature information.

[0077] Step S323: Determine whether the matching confidence is greater than a preset matching standard value.

[0078] Step S324: When the matching confidence is greater than a preset matching standard value, the instruction gesture corresponding to the first feature information and the second feature information is determined, and a corresponding sensing signal is generated according to the instruction gesture.

[0079] Step S315: When the matching confidence is less than or equal to the preset matching standard value, it is determined that the instruction gesture does not comply with the operation specification of the display device, and a preset input auxiliary operation is performed.

[0080] Exemplarily, when the gesture sensing module recognizes a command gesture using an infrared sensor combined with a flexible piezoelectric film array, the first characteristic information is the signal characteristics of the gesture action sensed by the infrared matrix, the second characteristic information is the piezoelectric signal characteristics sensed by the flexible piezoelectric film array, the first standard characteristic information is the infrared matrix signal characteristics corresponding to the gesture template, and the second standard characteristic information is the piezoelectric signal characteristics corresponding to the gesture template. For example, when a user uses a three-dimensional spiral gesture for control, they need to perform a standard gesture action according to the operating specifications of the three-dimensional spiral gesture, namely, a clockwise or counterclockwise spiral motion of the hand along the Z axis, while generating pressure on the Z axis. The first sensing unit obtains the first characteristic information by sensing the hand motion characteristics of the gesture action, and the second sensing unit obtains the second characteristic information by sensing the pressure characteristics of the gesture action. The first characteristic information is matched with the first standard characteristic information of each gesture template, and the second characteristic information is matched with the second standard characteristic information of each gesture template. When the curvature of the spiral trajectory in the first feature information is greater than the curvature standard value (such as 0.8), and the Z-axis pressure gradient in the second feature information is greater than or equal to the pressure gradient standard value (such as Δ≥5kPa / s), the matching confidence calculated based on the matching results of the first feature information and the second feature information will be greater than the preset matching standard value, and the gesture performed by the user is determined to be a three-dimensional spiral gesture, and a sensing signal representing the three-dimensional spiral gesture is generated. When the curvature of the spiral trajectory in the first feature information is less than or equal to the curvature standard value, and / or the Z-axis pressure gradient in the second feature information is less than the pressure gradient standard value, and the first feature information and the second feature information do not match the first standard feature information and the second standard feature information of each gesture template, the matching confidence calculated based on the matching results of the first feature information and the second feature information will be less than or equal to the preset matching standard value, and the gesture performed by the user is determined to not comply with the operating specifications of the display device.

[0081] S4, using a control module to generate a corresponding control signal according to the sensing signal.

[0082] Specifically, users can customize the control signals corresponding to sensing signals. For example, users can define the sensing signals generated by a dynamic gesture sequence as functions to save and zoom the screen, or define the sensing signals generated by a dynamic gesture sequence as functions to delete the screen. Users can freely match preset command gestures with control functions to increase the flexibility of display control and further enhance the user experience.

[0083] S5, controlling the display module to display via the control signal.

[0084] Specifically, the display content of the display module can be controlled by the control signal, such as translating / rotating / zooming the display screen, capturing and saving the display screen, turning pages, deleting the screen, etc. The display effect of the display module can also be adjusted by the control signal, such as adjusting the backlight brightness, contrast, wide / narrow display viewing angle, etc.

[0085] In one embodiment of the present application, before step S1 , the display device control method further includes: step S0 , detecting the user's operation authority for the display device, and determining whether to activate the gesture sensing module according to the detection result.

[0086] Specifically, the system can verify the user's biometric information, such as face, fingerprint, and voice, to confirm whether the current user has permission to operate the display device. Only when the current user's biometric information matches the verification information will the user's gestures in the sensing area be captured. Otherwise, the user's operation permission will be re-verified after a preset delay. This prevents unauthorized personnel from arbitrarily operating the display device, providing additional security for the display system.

[0087] The display device control method provided in the present application detects the brightness of ambient light through a light sensing module and generates an ambient light brightness signal. According to the ambient light brightness signal, the working parameters of the gesture sensing module are adjusted. The gesture sensing module senses the command gesture made by the user and generates a corresponding sensing signal. According to the sensing signal, a corresponding control signal is generated to control the display module to display. In this way, the display device can be controlled through non-contact gesture operation, avoiding direct touching of the display device, improving the level of hygiene and safety, and can be applied to places with high hygiene requirements such as medical care, food processing, and public transportation, meeting diverse needs in different scenarios and improving the convenience of operation; the gesture sensing module parameter configuration can also be dynamically adjusted according to different ambient light conditions to improve the accuracy of gesture recognition.

[0088] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] As used herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion of elements other than the listed elements and may also include additional elements not specifically listed.

[0090] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A display device, characterized in that: The display device includes a display module, a control module, a gesture sensing module and a light sensing module, wherein the display module, the gesture sensing module and the light sensing module are all communicatively connected to the control module; The light sensing module is used to detect the brightness of the ambient light and generate an ambient light brightness signal; The control module is used to adjust the working parameters of the gesture sensing module according to the ambient light brightness signal; The gesture sensing module is used to sense the command gesture made by the user and generate a corresponding sensing signal; The control module is further configured to generate a corresponding control signal according to the sensing signal, and control the display module to perform display via the control signal.

2. The display device according to claim 1, wherein The gesture sensing module includes a first sensing unit for sensing the instruction gesture in a non-contact sensing manner and generating a corresponding sensing signal.

3. The display device according to claim 2, wherein The first sensing unit includes at least one of an infrared sensor, a visual sensor, a microwave sensor, a laser sensor, and an ultrasonic sensor.

4. The display device according to claim 2, wherein The gesture sensing module further includes a second sensing unit for sensing the pressure generated by the instruction gesture and generating a corresponding piezoelectric signal as the sensing signal based on the pressure.

5. The display device according to claim 1, wherein The display device further includes a user matching module, which is communicatively connected to the control module and is used to detect a user's operation authority over the display device.

6. A display device control method, used for the display device according to any one of claims 1 to 5, characterized in that: The method comprises the following steps: S1, detecting the brightness of ambient light through the light sensing module and generating an ambient light brightness signal; S2, using the control module to adjust the operating parameters of the gesture sensing module according to the ambient light brightness signal; S3, using the gesture sensing module to sense the instruction gesture made by the user and generate a corresponding sensing signal; S4, using the control module to generate a corresponding control signal according to the sensing signal; S5, controlling the display module to display via the control signal.

7. The display device control method according to claim 6, wherein: The step S3 includes: sensing the instruction gesture in a non-contact sensing manner and generating a corresponding sensing signal.

8. The display device control method according to claim 7, wherein: The step S3 further includes: sensing the pressure generated by the instruction gesture, and generating a corresponding piezoelectric signal based on the pressure as the sensing signal.

9. The display device control method according to claim 7 or 8, wherein: In step S3, generating the corresponding sensing signal includes: Matching the command gesture with a preset gesture template, and determining whether the command gesture complies with an operating specification of the display device according to a matching result; If it is determined that the instruction gesture complies with the operating specification of the display device, generating a corresponding sensing signal according to the instruction gesture; If it is determined that the instruction gesture does not comply with the operation specification of the display device, a preset input auxiliary operation is performed.

10. The display device control method according to claim 6, wherein: Before step S1, the method further includes the following steps: S0, detecting the user's operating authority for the display device, and determining whether to activate the gesture sensing module according to the detection result.

Images