Screen touch detection method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a screen touch detection method and device, electronic equipment and a medium. The method comprises the steps that in the process that a touch medium touches a screen, the touch medium is detected through an infrared touch frame on the outer layer of the screen, and a touch position is determined; determining a shielding area change rate corresponding to each refreshing moment of the infrared touch frame; the shielding area is the area, shielded by the touch medium, of an infrared wave beam of the infrared touch frame; the shielding area change rate is a ratio of a shielding area change value of two adjacent refreshing moments to a time difference of the two adjacent moments; in the time period when the touch medium touches the screen for one time, according to the relation between the refreshing moments and the shielding area change rate, the touch control moment when the touch medium touches the screen is determined, and the screen is controlled to respond according to the touch control position and the touch control moment. According to the scheme, the touch control moment can be accurately locked through the regular characteristics of the change rate of the shielding area, and the real-time performance of screen response is improved.

Description

Technical Field

[0001] This application relates to the field of touch control technology, and in particular to a screen touch detection method, device, electronic device and medium. Background Technology

[0002] As a new type of electronic device for display, teaching, and conference communication, smart interactive whiteboards are increasingly widely used in daily life. Smart interactive whiteboards typically have a large screen to display rich information, while also featuring touch control for convenient real-time interactive operation. Because their screen size is significantly larger than that of mobile touch electronic devices, capacitive touch methods commonly used in smaller devices are usually not feasible. Currently, the primary approach is to attach an infrared touch frame to the outside of the interactive whiteboard display screen for touch operation recognition.

[0003] Due to limitations in its sampling principle and physical structure, the infrared touch frame must be installed outside the display screen and at a certain distance from it. The infrared touch recognition plane formed by the infrared beam does not coincide with the screen display plane (i.e., the actual touch plane). This leads to situations where the infrared touch frame triggers recognition even when the screen is not actually touched, resulting in misidentification, low touch accuracy, and negatively impacting user operation and experience. Summary of the Invention

[0004] This application provides a screen touch detection method, apparatus, electronic device, and medium to accurately identify the actual touch moment of the touch medium touching the screen, thereby improving touch accuracy.

[0005] According to one aspect of this application, a screen touch detection method is provided, the method comprising:

[0006] During the process of touching the screen with a touch medium, the touch medium is detected through the infrared touch frame on the outer layer of the screen to determine the touch position;

[0007] Determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times.

[0008] During the time period of a single touch of the screen by the touch medium, the touch time of the touch medium touching the screen is determined according to the relationship between each refresh time and the rate of change of the occlusion area, so as to control the screen to respond according to the touch position and the touch time.

[0009] According to one aspect of this application, a screen touch detection device is provided, the device comprising:

[0010] The touch position determination module is used to detect the touch medium through the infrared touch frame on the outer layer of the screen during the process of touching the screen with a touch medium, and determine the touch position.

[0011] The correspondence determination module is used to determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times.

[0012] The touch timing determination module is used to determine the touch timing of the touch medium on the screen within the time period of a single touch on the screen by the touch medium, based on the relationship between each refresh time and the rate of change of the occlusion area, so as to control the screen to respond according to the touch position and the touch timing.

[0013] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] Memory connected to at least one processor for data processing; wherein,

[0016] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor can perform the screen touch detection method of any embodiment of this application.

[0017] According to another aspect of this application, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the screen touch detection method of any embodiment of this application.

[0018] The technical solution of this application embodiment detects the touch medium through an infrared touch frame on the outer layer of the screen during the touch process, determining the touch position; determining the occlusion area change rate corresponding to each refresh moment of the infrared touch frame; the occlusion area is the area of ​​the infrared beam of the infrared touch frame blocked by the touch medium; the occlusion area change rate is the ratio of the occlusion area change value between two adjacent refresh moments to the time difference between the two adjacent moments; within the time period of a single touch of the screen by the touch medium, the touch moment of the touch medium touching the screen is determined according to the relationship between each refresh moment and the occlusion area change rate, so as to control the screen to respond according to the touch position and the touch moment. The above solution can accurately lock the touch moment through the regular characteristics of the occlusion area change rate, so that the screen responds according to the touch moment, avoiding the problem of inaccurate touch time caused by the height difference between the infrared touch frame and the screen, and improving the real-time performance and touch accuracy of the screen response.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a screen touch detection method provided in this application embodiment;

[0022] Figure 2 This is a schematic diagram of the touch screen principle provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of the touch medium touch screen process provided in the embodiments of this application;

[0024] Figure 4 A flowchart illustrating a screen touch detection method provided in another embodiment of this application;

[0025] Figure 5 This is a schematic diagram illustrating the relationship between the rate of change of the occlusion area and the refresh time, provided in another embodiment of this application.

[0026] Figure 6 A flowchart illustrating a screen touch detection method provided in another embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the first sound sensor distribution provided in yet another embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the distribution of the second sound sensor provided in yet another embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of a screen touch detection device provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first," "second," "third," "fourth," "actual," "preset," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Figure 1 This is a flowchart illustrating a screen touch detection method provided in an embodiment of this application. This embodiment is applicable to situations involving touch detection of a screen. Typically, this embodiment is applicable to scenarios where touch detection is performed via an infrared touch frame. The method can be executed by a screen touch detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0034] S110. During the process of touching the screen with a touch medium, the touch medium is detected through the infrared touch frame on the outer layer of the screen to determine the touch position.

[0035] The touch medium is a medium that controls the screen by touching it, such as a user's finger, stylus, or touch glove. The screen is a touchscreen that can sense touch signals and generate a response. In this embodiment, the screen is an infrared touchscreen, and an infrared touch frame is provided on the outer layer of the screen, that is, the outer layer of the display surface of the screen, such as... Figure 2 As shown, one side of the infrared touch frame emits an infrared beam, and the other side receives the infrared beam, forming an infrared touch recognition plane. When a touch medium is present on the screen, the infrared light is blocked by the touch medium. The infrared touch frame recognizes the operation of the touch medium by the area and position of the blocked infrared light.

[0036] In this embodiment, during a single touch of the screen, the touch position generally does not change over time. Therefore, an infrared touch frame is used to detect the touch medium and determine the touch position. The moment the infrared touch frame detects the touch medium does not reflect the actual moment the touch medium touches the screen. Therefore, the touch position is determined first, and the touch moment is determined later.

[0037] S120. Determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times.

[0038] The occlusion area can be determined by detecting whether the infrared beam emitted by the infrared touch frame is blocked. For example, common touch media such as fingertips, finger pads, or the tip of a regular stylus are all cylindrical objects with non-uniform diameters. Therefore, in the operation recognition model, they can be simplified as having a cone-shaped tip and a cylindrical-like rear end. For example... Figure 3As shown, during the process of touching the screen with a touch medium, the touch medium gradually approaches the screen. The positional change of the touch medium relative to the infrared beam of the infrared touch frame is as follows: In the first stage, the touch medium is just detected by the infrared frame. Until the conical part of the touch medium passes through the infrared touch recognition plane, the area blocked by the touch medium rapidly increases from 0 to a certain area. In the second stage, the conical part of the touch medium passes through the infrared touch recognition plane, and the cylindrical part passes through the infrared touch recognition plane. Since the diameter of the cylindrical part is relatively small, the blocking area increases slowly. In the third stage, the cylindrical part begins to pass through the infrared touch recognition plane until the touch medium actually contacts the screen. Because the touch medium is blocked by the screen, it cannot move any further, and the blocking area increases slowly until it stops increasing.

[0039] In this embodiment, relying solely on the occlusion area is insufficient to determine the exact moment when the touch medium actually touches the screen. The occlusion area change rate more accurately reflects the relative positional change between the touch medium and the screen, allowing for the determination of the occlusion area change rate at each refresh moment of the infrared touch frame. Specifically, for the current refresh moment of the infrared touch frame, the difference between the occlusion area corresponding to the current refresh moment and the occlusion area corresponding to the previous refresh moment can be determined. The time difference between the current and previous refresh moments can then be calculated, and the ratio of this difference to the time difference can be used as the occlusion area change rate corresponding to the current refresh moment. Furthermore, since the infrared touch frame has a high refresh frequency and short intervals, it does not cause significant errors; therefore, the obtained occlusion area change rate can also be used as the occlusion area change rate corresponding to the previous refresh moment.

[0040] S130. During the time period of a single touch of the screen by the touch medium, the touch time of the touch medium touches the screen is determined according to the relationship between each refresh time and the rate of change of the occlusion area, so as to control the screen to respond according to the touch position and the touch time.

[0041] The time period for a single touch of the screen by the touch medium can be defined as the period from when the touch medium is detected by the infrared touch frame until it actually touches the screen. Within this time period, the relationship between each refresh time of the infrared touch frame and the rate of change of the occlusion area is determined. Based on the theoretical characteristics of the rate of change of the occlusion area during each stage of the touch medium's movement on the screen, the characteristics of the rate of change of the occlusion area when the touch medium touches the screen are determined. Therefore, in the relationship between each refresh time and the rate of change of the occlusion area, the refresh time corresponding to the theoretical rate of change of the occlusion area is taken as the touch time of the touch medium touching the screen. The screen responds according to the touch position and touch time, recognizing the touch operation of the touch medium.

[0042] For example, the characteristics of the rate of change of the occlusion area when a touch medium contacts the screen can be determined based on the actual situation, such as the shape of the touch medium. Assuming the touch medium is a cone-shaped object at the top and a columnar object at the bottom, the rate of change of the occlusion area increases rapidly at first, then slowly, and finally reaches zero. If the touch medium is a hemispherical object at the top and a columnar object at the bottom, the rate of change of the occlusion area increases extremely rapidly, then slowly. Therefore, based on the characteristic patterns of the rate of change of the occlusion area when the touch medium actually contacts the screen under different touch medium conditions, the judgment conditions for determining the rate of change of the occlusion area actually touched by the screen can be determined for different touch mediums.

[0043] It should be noted that if the touch medium is a flexible medium such as the fingertip, during the actual contact with the screen, the area of ​​the touch medium that continues to penetrate deeper may continue to change during the period when the touch medium is obstructed by the screen but continues to press the screen. However, due to the obstruction of the screen, the rate of change of the obstructed area is reduced. The above solution is also applicable and is not limited by the type of touch medium.

[0044] The technical solution of this application embodiment detects the touch medium through an infrared touch frame on the outer layer of the screen during the touch process, determining the touch position; determining the occlusion area change rate corresponding to each refresh moment of the infrared touch frame; the occlusion area is the area of ​​the infrared beam of the infrared touch frame blocked by the touch medium; the occlusion area change rate is the ratio of the occlusion area change value between two adjacent refresh moments to the time difference between the two adjacent moments; within the time period of a single touch of the screen by the touch medium, the touch moment of the touch medium touching the screen is determined according to the relationship between each refresh moment and the occlusion area change rate, so as to control the screen to respond according to the touch position and the touch moment. The above solution can accurately lock the touch moment through the regular characteristics of the occlusion area change rate, so that the screen responds according to the touch moment, avoiding the problem of inaccurate touch time caused by the height difference between the infrared touch frame and the screen, and improving the real-time performance and touch accuracy of the screen response.

[0045] In this embodiment of the application, before determining the touch moment of the touch medium touch screen, the method further includes:

[0046] If there is a negative rate of change of occlusion area in the relationship between each refresh time and the rate of change of occlusion area, then the rate of change of occlusion area and the corresponding refresh time will be removed.

[0047] For example, if the touch medium is a relatively soft object, when it is actually touched on the screen, it may jitter back and forth due to the obstruction of the screen. For example, at the moment of moving away from the screen, the rate of change of the obstruction area may be negative. However, this jitter phenomenon does not affect the determination of the actual screen touch time. It is considered a noise value in the touch time detection process. Therefore, if there is a negative rate of change of the obstruction area, the rate of change of the obstruction area and the corresponding refresh time are removed to eliminate noise values ​​and avoid affecting the detection of touch time.

[0048] Figure 4 This is a flowchart illustrating a screen touch detection method according to another embodiment of this application. This embodiment is an optimization based on the above embodiment; solutions not described in detail in this embodiment are found in the above embodiment. Figure 4 As shown, the method in this embodiment of the application specifically includes the following steps:

[0049] S210. During the process of touching the screen with a touch medium, the touch medium is detected by the infrared touch frame on the outer layer of the screen to determine the touch position.

[0050] S220. Determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times.

[0051] S230. During the time period of a single touch of the screen by the touch medium, find the interval in which the relationship between each refresh time and the rate of change of the occlusion area is inversely proportional.

[0052] For example, when the top of the touch medium is a conical object, or a columnar structure with varying diameters, the touch timing can be determined according to the scheme in the embodiments of this application. Specifically, for the touch medium with the above structure, the characteristic of the rate of change of the occlusion area during the time period of a single touch on the screen is: it first rapidly increases from 0 to a certain value, then rapidly decreases, and then slowly decreases. During the stage of decreasing occlusion area change rate, there is a touch timing when the touch medium contacts the screen. Therefore, in the relationship between each refresh time and the occlusion area change rate, an inversely proportional relationship interval is found. For example, as... Figure 5 As shown, in Figure 5 The ② and ③ stages in the diagram represent the inversely proportional relationship intervals.

[0053] S240. In the inversely proportional relationship interval, the target refresh time corresponding to the target occlusion area change rate is taken as the touch time of the touch medium touching the screen; wherein, the target occlusion area change rate is less than the preset occlusion area change rate, and the difference between the two is the smallest; the preset occlusion area change rate is the occlusion area change rate when the touch medium just touches the screen.

[0054] For example, a preset occlusion area change rate can be determined in advance. This preset occlusion area change rate can be the occlusion area change rate when the touch medium first touches the screen. Specifically, during the pre-testing process, the occlusion area change rate with a significant abrupt change in the inverse relationship interval can be used as the preset occlusion area change rate. During the actual testing process, the target occlusion area change rate, which is less than the preset occlusion area change rate and closest to the occlusion area change rate, can be found in the inverse relationship interval. The target refresh time corresponding to the target occlusion area change rate is then found in the relationship interval, which is the touch time when the touch medium touches the screen.

[0055] This application provides a screen touch detection method. It identifies an inversely proportional relationship between refresh times and the rate of change of occlusion area. Within this inversely proportional relationship, the target refresh time corresponding to the target rate of change of occlusion area is taken as the touch time when the touch medium touches the screen. The target rate of change of occlusion area is less than a preset rate of change of occlusion area, and the difference between the two is minimal. The preset rate of change of occlusion area is the rate of change of occlusion area when the touch medium just touches the screen. This method can determine the characteristics of the rate of change of occlusion area during the touch process based on the shape characteristics of a conventional touch medium. Therefore, it accurately identifies the actual touch time when the touch medium touches the screen, avoiding premature identification due to the screen and infrared touch frame not being on the same plane, thus improving the accuracy of touch detection.

[0056] It should be noted that the embodiments in this application are merely examples of a touch timing recognition method for a touch medium of one shape. The preset occlusion area change rate during touch recognition can be adjusted according to the shape of the touch medium. For example, if the entire portion of the touch medium from the moment it is detected by the infrared touch frame to the moment it actually touches the screen is a conical structure, then the preset occlusion area change rate should be the maximum value among the area occlusion change rates.

[0057] Figure 6 This is a flowchart illustrating a screen touch detection method according to another embodiment of this application. This embodiment is an optimization based on the above embodiments; solutions not described in detail in this embodiment are found in the above embodiments. Figure 6As shown, the method in this embodiment of the application specifically includes the following steps:

[0058] S310. During the process of touching the screen with a touch medium, the touch medium is detected through the infrared touch frame on the outer layer of the screen to determine the touch position.

[0059] S320. Determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times.

[0060] In this embodiment of the application, the process of determining the preset area change rate includes:

[0061] During the testing phase, the relationship between each refresh time and the rate of change of the occlusion area was determined in advance while touching the medium touch screen.

[0062] The touch sound signal is detected by the first sound sensor, and the test time when the touch sound signal is detected is determined.

[0063] In the relationship between refresh times and the rate of change of occlusion area, the rate of change of occlusion area corresponding to the refresh time with the smallest difference from the test time is taken as the preset rate of change of occlusion area; or...

[0064] The preset occlusion area change rate is determined based on the occlusion area change rate corresponding to the point where the slope of the inverse proportional relationship curve between each refresh time and the occlusion area change rate changes abruptly.

[0065] For example, a sound sensor can be placed at the edge of the screen to detect sound signals from the touch medium interacting with the screen. Figure 7As shown, first sound sensors are positioned at each edge of the screen. These first sound sensors can be directional, detecting only sounds generated within the screen area. The sound detection area of ​​all first sound sensors needs to cover the entire sub-area. During the testing phase, a touch medium can be allowed to touch the screen to determine the relationship between refresh times and the rate of change of the obstruction area. When the touch medium actually contacts the screen, sound is generated. The first sound sensors detect the touch sound signal, determining the test time at which the touch sound signal was detected. Among the refresh times and the rate of change of the obstruction area, the refresh time with the smallest difference from the test time is found. This refresh time is assumed to be the time when the touch medium actually contacts the screen, and the rate of change of the obstruction area corresponding to this refresh time is used as the preset rate of change of the obstruction area for subsequent searches of the target rate of change of the obstruction area and its corresponding refresh time. The advantage of this solution is that it eliminates the need for manual determination of the actual touch time; it only requires the first sound sensors to detect the moment when the touch medium touches the screen and generates sound, automatically determining the touch time during the testing phase and thus automatically determining the preset rate of change of the obstruction area corresponding to that touch time.

[0066] For example, one can also find the inverse relationship interval in the relationship between each refresh time and the rate of change of occlusion area to determine the inverse relationship curve. In the inverse relationship curve, the preset rate of change of occlusion area is determined based on the point where the slope changes abruptly. For example... Figure 5 As shown, the inverse proportionality curve corresponds to stages ② and ③. In this inverse proportionality curve, the point where the slope abruptly changes is... Figure 5 The point corresponding to the touch moment is used to determine the preset occlusion area and its rate of change. The rate of change of the occlusion area corresponding to this point can be directly used as the preset occlusion area rate of change, or the value where the difference between this rate of change and the rate of change of the occlusion area corresponding to this point is less than a preset threshold can be used as the preset occlusion area rate of change.

[0067] S330. During the time period of a single touch of the screen by the touch medium, the touch time of the touch screen by the touch medium is determined according to the relationship between each refresh time and the rate of change of the occlusion area.

[0068] S340. If the touch time of the touch medium touch screen cannot be determined based on the relationship between each refresh time and the rate of change of the occlusion area, then the target sound signal is obtained through the first sound sensor set at the edge of the screen.

[0069] For example, if the rate of change of the occlusion area does not change abruptly, or if the preset rate of change of the occlusion area is not selected appropriately, making it impossible to determine the touch moment, the target sound signal is acquired through the first sound sensor. When the infrared touch frame detects the presence of the touch medium, the first sound sensor detects the sound signal and acquires the target sound signal from the detected sound signal.

[0070] In this embodiment of the application, acquiring the target sound signal through a first sound sensor disposed at the edge of the screen includes:

[0071] Candidate sound signals are acquired through a first sound sensor located at the edge of the screen, and ambient sound signals are acquired through a second sound sensor located at the edge of the screen.

[0072] The target sound signal is obtained by removing the ambient sound signal from the candidate sound signal.

[0073] For example, since the screen is in an open environment, there may be noise present, so it is necessary to eliminate the effects of noise. A second sound sensor can be placed at the edge of the screen, such as... Figure 8 As shown. The second sound sensor can be a non-directional sound sensor, capable of detecting sound from all directions. The second sound sensor may detect ambient sound signals and candidate sound signals. A filtering algorithm can be used to filter out the instantaneous candidate sound signals to obtain the ambient sound signal. The ambient sound signal obtained by the second sound sensor is removed from the selected sound signal obtained by the first sound sensor to obtain the target sound signal, thus achieving noise reduction processing of the sound signal.

[0074] S350. Based on the matching result between the target sound signal and the preset touch sound signal, determine whether the target sound signal is generated by the touch medium touch screen.

[0075] For example, during the testing phase, different materials and structures of touch media can be used to generate sound signals by touching the screen. These sound signals are detected by a first sound sensor and used as preset touch sound signals, which are then recorded one-to-one with each touch media. During actual testing, if a target sound signal is acquired by the first sound sensor, it can be compared with the preset touch sound signals to determine if the acquired target sound signal is indeed generated by the touch media touching the screen. If the similarity reaches a preset similarity threshold, the target sound signal is considered to be generated by the touch media touching the screen; otherwise, it is considered not to be generated by the touch media touching the screen.

[0076] In this embodiment of the application, the process of determining the preset touch sound signal includes:

[0077] For each sub-region of the screen, a preset touch sound signal generated by the touch medium touching that sub-region is acquired in advance;

[0078] The preset gain is determined based on the relative position of the sub-region with respect to the first sound sensor that detected the preset touch sound signal;

[0079] The preset touch sound signal is processed based on the preset gain;

[0080] Accordingly, after acquiring the target sound signal through a first sound sensor positioned at the edge of the screen, the method further includes:

[0081] The sub-region for generating the target sound signal is determined based on the touch position;

[0082] Determine the preset gain corresponding to the sub-region, and process the target sound signal based on the preset gain.

[0083] For example, the detection range of each first sound sensor may vary, for instance, for... Figure 7 The third sound sensor at the lower edge of the screen may detect both sub-region 3-1 and sub-region 2-3. Since sub-region 3-1 is closer to the first sound sensor than sub-region 2-3, the intensity of the detected sound signals may differ. Different gains can be applied to the sound signals from detection areas at different distances to enhance the sound signals and improve detail. For example, during testing, each touch medium can be pre-programmed to generate a preset touch sound signal. The sub-region touched by the touch medium is determined, and a preset gain is determined based on the relative position of this sub-region and the first sound sensor that detected the preset touch sound signal. The preset touch sound signal is then processed based on the preset gain to enhance it.

[0084] In practical applications, if the target sound signal is acquired through the first sound sensor, the touch position detected by the infrared touch frame is obtained, the sub-region where the touch position is located is determined, and thus the preset gain corresponding to that sub-region is determined. Based on the preset gain, the target sound signal is processed to enhance it, achieving the same level of enhancement as the preset touch sound signal, which facilitates improved matching accuracy during subsequent similarity matching.

[0085] S360. If so, the time when the target sound signal is detected is taken as the touch moment of the touch medium touch screen, so as to control the screen to respond according to the touch position and the touch moment.

[0086] For example, if the target sound signal is generated by a touch-sensitive touchscreen, the time when the target sound signal is detected can be used as the touch moment of the touchscreen, and the screen can be controlled to respond based on the touch position and touch moment. If the target sound signal is not generated by a touchscreen, sound detection continues through the first sound sensor.

[0087] This application provides a screen touch detection method. If the touch time of the touch medium touch screen cannot be determined based on the relationship between each refresh time and the rate of change of the occlusion area, a target sound signal is acquired through a first sound sensor located at the edge of the screen. Based on the matching result of the target sound signal and a preset touch sound signal, it is determined whether the target sound signal was generated by the touch medium touch screen. If so, the time at which the target sound signal was detected is taken as the touch time of the touch medium touch screen. This application provides a supplementary solution when the touch time of the touch medium touch screen cannot be determined based on the relationship between each refresh time and the rate of change of the occlusion area. By detecting the touch time of the touch medium touch screen using a sound sensor, the success rate of touch time detection is improved. Dual detection ensures accurate detection of the touch time, thus improving the success rate and accuracy of screen touch detection.

[0088] Figure 9 This is a schematic diagram of a screen touch detection device provided in an embodiment of this application. This device can execute the screen touch detection method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Figure 9 As shown, the device includes:

[0089] The touch position determination module 410 is used to detect the touch medium through the infrared touch frame on the outer layer of the screen during the process of touching the screen with a touch medium, and determine the touch position.

[0090] The correspondence determination module 420 is used to determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times.

[0091] The touch timing determination module 430 is used to determine the touch timing of the touch medium touching the screen within the time period of a single touch of the touch medium, based on the relationship between each refresh time and the rate of change of the occlusion area, so as to control the screen to respond according to the touch position and the touch timing.

[0092] In this embodiment, the touch timing determination module 430 determines the touch timing of the touch medium touch screen based on the relationship between each refresh time and the rate of change of the occlusion area, including:

[0093] In the relationship between refresh time and the rate of change of occlusion area, find the intervals with an inverse relationship;

[0094] Within the inversely proportional range, the target refresh time corresponding to the target occlusion area change rate is taken as the touch time of the touch medium touching the screen; wherein, the target occlusion area change rate is less than the preset occlusion area change rate, and the difference between the two is the smallest; the preset occlusion area change rate is the occlusion area change rate when the touch medium just touches the screen.

[0095] In this embodiment of the application, the device further includes: a rejection module, configured to, before determining the touch time of the touch medium touch screen, if there is a negative rate of change of the occlusion area in the relationship between each refresh time and the rate of change of the occlusion area, then reject the rate of change of the occlusion area and the corresponding refresh time.

[0096] In this embodiment of the application, the device further includes:

[0097] The target sound signal acquisition module is used to acquire the target sound signal through a first sound sensor set at the edge of the screen if the touch time of the touch medium touch screen cannot be determined according to the relationship between each refresh time and the rate of change of the occlusion area.

[0098] The matching module is used to determine whether the target sound signal is generated by the touch medium touch screen based on the matching result between the target sound signal and the preset touch sound signal;

[0099] The sound detection time determination module is used to determine the touch time of the touch medium touch screen if the target sound signal is detected.

[0100] In this embodiment of the application, the device further includes:

[0101] A preset touch sound signal acquisition module is used to acquire preset touch sound signals generated by the touch medium touching the sub-area for each sub-area in the screen.

[0102] A preset gain determination module is used to determine a preset gain based on the relative positional relationship between the sub-region and the first sound sensor that detects the preset touch sound signal;

[0103] A gain processing module is used to process the preset touch sound signal based on the preset gain;

[0104] Accordingly, the device further includes: a target sound signal gain processing module, used to acquire the target sound signal through a first sound sensor set at the edge of the screen, and then determine the sub-region where the target sound signal is generated based on the touch position;

[0105] Determine the preset gain corresponding to the sub-region, and process the target sound signal based on the preset gain.

[0106] In this embodiment of the application, the target sound signal acquisition module acquires the target sound signal through a first sound sensor disposed at the edge of the screen, including:

[0107] Candidate sound signals are acquired through a first sound sensor located at the edge of the screen, and ambient sound signals are acquired through a second sound sensor located at the edge of the screen.

[0108] The target sound signal is obtained by removing the ambient sound signal from the candidate sound signal.

[0109] In this embodiment of the application, the device further includes: a preset area change rate determination module, used for:

[0110] During the testing phase, the relationship between each refresh time and the rate of change of the occlusion area was determined in advance while touching the medium touch screen.

[0111] The touch sound signal is detected by the first sound sensor, and the test time when the touch sound signal is detected is determined.

[0112] In the relationship between refresh times and the rate of change of occlusion area, the rate of change of occlusion area corresponding to the refresh time with the smallest difference from the test time is taken as the preset rate of change of occlusion area; or...

[0113] The preset occlusion area change rate is determined based on the occlusion area change rate corresponding to the point where the slope of the inverse proportional relationship curve between each refresh time and the occlusion area change rate changes abruptly.

[0114] The screen touch detection device provided in this application embodiment can execute a screen touch detection method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0115] Figure 10A schematic diagram of an electronic device 10, which can be used to implement embodiments of this application, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0116] like Figure 10 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, connected to the at least one processor 11 for data processing. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0117] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and data processing unit 19, such as network card, modem, wireless data processing transceiver, etc. Data processing unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0118] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as screen touch detection methods.

[0119] In some embodiments, the screen touch detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via ROM 12 and / or data processing unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the screen touch detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the screen touch detection method by any other suitable means (e.g., by means of firmware).

[0120] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable screen touch detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected through digital data processing (e.g., data processing networks) of any form or medium. Examples of data processing networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0125] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via data processing networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0126] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired information of the technical solution of this application can be achieved, and this is not limited herein.

[0127] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A screen touch detection method, characterized in that, The method includes: During the process of touching the screen with a touch medium, the touch medium is detected through the infrared touch frame on the outer layer of the screen to determine the touch position; Determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times. During the time period of a single touch of the screen by the touch medium, the touch time of the touch medium touching the screen is determined according to the relationship between each refresh time and the rate of change of the occlusion area, so as to control the screen to respond according to the touch position and the touch time.

2. The method according to claim 1, characterized in that, Based on the relationship between each refresh time and the rate of change of the occlusion area, the touch time of the touch medium touch screen is determined, including: In the relationship between refresh time and the rate of change of occlusion area, find the intervals with an inverse relationship; Within the inversely proportional range, the target refresh time corresponding to the target occlusion area change rate is taken as the touch time of the touch medium touching the screen; wherein, the target occlusion area change rate is less than the preset occlusion area change rate, and the difference between the two is the smallest; the preset occlusion area change rate is the occlusion area change rate when the touch medium just touches the screen.

3. The method according to claim 1, characterized in that, Before determining the touch moment of the touch medium touch screen, the method further includes: If there is a negative rate of change of occlusion area in the relationship between each refresh time and the rate of change of occlusion area, then the rate of change of occlusion area and the corresponding refresh time will be removed.

4. The method according to claim 1, characterized in that, The method further includes: If the touch time of the touch medium touch screen cannot be determined based on the relationship between each refresh time and the rate of change of the occlusion area, then the target sound signal is obtained through the first sound sensor set at the edge of the screen. Based on the matching result between the target sound signal and the preset touch sound signal, determine whether the target sound signal is generated by the touch medium touch screen; If so, the time when the target sound signal is detected is taken as the touch time of the touch medium touch screen.

5. The method according to claim 4, characterized in that, The process of determining the preset touch sound signal includes: For each sub-region of the screen, a preset touch sound signal generated by the touch medium touching that sub-region is acquired in advance; The preset gain is determined based on the relative position of the sub-region with respect to the first sound sensor that detected the preset touch sound signal; The preset touch sound signal is processed based on the preset gain; Accordingly, after acquiring the target sound signal through a first sound sensor positioned at the edge of the screen, the method further includes: The sub-region for generating the target sound signal is determined based on the touch position; Determine the preset gain corresponding to the sub-region, and process the target sound signal based on the preset gain.

6. The method according to claim 4, characterized in that, The target sound signal is acquired through a first sound sensor positioned at the edge of the screen, including: Candidate sound signals are acquired through a first sound sensor located at the edge of the screen, and ambient sound signals are acquired through a second sound sensor located at the edge of the screen. The target sound signal is obtained by removing the ambient sound signal from the candidate sound signal.

7. The method according to claim 4, characterized in that, The process of determining the preset area change rate includes: During the testing phase, the relationship between each refresh time and the rate of change of the occlusion area was determined in advance while touching the medium touch screen. The touch sound signal is detected by the first sound sensor, and the test time when the touch sound signal is detected is determined. In the relationship between refresh times and the rate of change of occlusion area, the rate of change of occlusion area corresponding to the refresh time with the smallest difference from the test time is taken as the preset rate of change of occlusion area; or... The preset occlusion area change rate is determined based on the occlusion area change rate corresponding to the point where the slope of the inverse proportional relationship curve between each refresh time and the occlusion area change rate changes abruptly.

8. A screen touch detection device, characterized in that, The device includes: The touch position determination module is used to detect the touch medium through the infrared touch frame on the outer layer of the screen during the process of touching the screen with a touch medium, and determine the touch position. The correspondence determination module is used to determine the rate of change of the occlusion area corresponding to each refresh time of the infrared touch frame; wherein, the occlusion area is the area of ​​the infrared beam of the infrared touch frame that is blocked by the touch medium; the rate of change of the occlusion area is the ratio of the change value of the occlusion area between two adjacent refresh times to the time difference between the two adjacent times. The touch timing determination module is used to determine the touch timing of the touch medium on the screen within the time period of a single touch on the screen by the touch medium, based on the relationship between each refresh time and the rate of change of the occlusion area, so as to control the screen to respond according to the touch position and the touch timing.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and The memory is connected to the at least one processor for data processing; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the screen touch detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the screen touch detection method according to any one of claims 1-7.

Images