Touch screen detection method and system of touch screen device, electronic device and storage medium

Abstract

This application provides a screen-click detection method, system, electronic device, and storage medium for a screen-clicking device. The method includes: repeatedly performing a screen-click judgment operation a preset number of times; determining the screen-clicking stability of the screen-clicking device based on the first successful screen-clicking count in the preset number of screen-clicking judgment operations; wherein the screen-clicking judgment operation includes: acquiring a screen-clicked image, wherein the screen-clicked image is an image of the display screen captured after the screen-clicking device clicks the display screen; determining the lit pixels in the preset area based on the grayscale value of each pixel in the screen-clicked image and a preset grayscale threshold, wherein the grayscale value of the lit pixels is greater than the preset grayscale threshold; counting the number of lit pixels; and determining whether the screen-clicking device has successfully clicked the screen based on the number of lit pixels. The above detection method has low detection cost, high detection efficiency, and flexible configuration of the control parameters for whether the display screen is lit, resulting in a better user experience.

Description

Technical Field

[0001] This application relates to the field of hardware testing technology, and more specifically to a screen detection method for a screen-dotting device, a screen detection system for a screen-dotting device, an electronic device, and a storage medium. Background Technology

[0002] A screen testing device is used to power on a connected monitor to test for malfunctions. Before using a screen testing device to test a monitor, the device itself is usually tested to ensure it is functioning correctly.

[0003] In existing technologies, some solutions employ manual inspection to test the stability of touchscreen devices. For example, after the touchscreen device under test performs a touchscreen operation on a normal display screen, a person observes and records whether the display screen lights up or the display data, and judges whether the touchscreen device is functioning well based on the recorded data. This solution requires significant manpower and is slow and inefficient. Other solutions use expensive testing equipment to test the touchscreen effect of the device. This solution is costly and has a fixed testing mode, making it difficult to adapt to different testing scenarios. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems. This application provides a screen dot detection method for a screen dot device, a screen dot detection system for a screen dot device, an electronic device, and a storage medium.

[0005] According to one aspect of this application, a screen-click detection method for a screen-clicking device is provided, used to detect whether the screen-clicking device lights up a normal display screen and displays a preset image. The screen-click detection method includes: repeatedly performing a screen-click judgment operation a preset number of times; and determining the screen-clicking stability of the screen-clicking device based on the first successful screen-clicking count in the preset number of screen-click judgment operations; wherein the screen-clicking judgment operation includes: acquiring a screen-clicked image, wherein the screen-clicked image is an image of the display screen captured after the screen-clicking device clicks the display screen; determining lit pixels in the preset area based on the grayscale value of each pixel in the preset area of ​​the screen-clicked image and a preset grayscale threshold, wherein the grayscale value of the lit pixels is greater than the preset grayscale threshold; counting the number of lit pixels; and determining whether the screen-clicking device has successfully clicked the screen based on the number of lit pixels.

[0006] For example, the preset image is a color image, and determining the lit pixels in the preset area includes: converting the color data of each pixel in the color image of the screen into grayscale values; and determining the pixels in the preset area whose grayscale values ​​are greater than a preset grayscale threshold as lit pixels.

[0007] For example, the method further includes: acquiring a first image of a display screen that has displayed a preset image; performing image segmentation on the first image to determine the location region of the display screen in the image, as the display screen region; and using the image of the display screen region as the clicked screen image.

[0008] For example, the method further includes: determining a region including the center of the image in the tapped screen image as a preset region, wherein the width of the preset region is less than the width of the tapped screen image, and the height of the preset region is less than the height of the tapped screen image.

[0009] For example, the method further includes the step of determining a preset grayscale threshold: if it is determined that the display screen has been successfully tapped, acquiring a tapped test image of the display screen; performing image segmentation on the tapped test image to determine the display area of ​​the location region of the display screen in the image and the background area formed by all pixels located outside the display area; determining a first overall grayscale value of the pixels in the display area and determining a second overall grayscale value of the pixels in the background area; and determining a preset grayscale threshold based on the first overall grayscale value and the second overall grayscale value, wherein the preset grayscale threshold is greater than or equal to the second overall grayscale value and less than or equal to the first overall grayscale value.

[0010] For example, the step of determining the preset grayscale threshold further includes: if it is determined that the display screen has been successfully tapped, acquiring a tapped test image of the display screen; determining the grayscale value of the pixels in a preset area of ​​the tapped test image; determining the lit pixels in the preset area based on the grayscale values ​​of the pixels in the preset area and the current preset grayscale threshold, wherein the pixel value of the lit pixels is greater than the current preset grayscale threshold; determining whether the tapping device has successfully tapped the screen based on the number of lit pixels, and if not, updating the current preset grayscale threshold; and performing the step of determining the lit pixels in the preset area according to the updated preset grayscale threshold until it is determined that the tapping device has successfully tapped the screen.

[0011] For example, determining whether a screen-triggering device has successfully triggered the screen based on the number of lit pixels includes: calculating the proportion of lit pixels in a preset area; if the proportion is less than a proportion threshold, determining that the screen-triggering device has failed to trigger the screen; if the proportion is greater than or equal to the proportion threshold, determining that the screen-triggering device has successfully triggered the screen.

[0012] For example, determining the screen-clicking stability of a screen-clicking device based on the first successful screen-clicking count includes: counting the first successful screen-clicking count determined in a preset number of screen-clicking judgment operations; and determining the screen-clicking stability of the screen-clicking device based on the first count and the preset number of counts.

[0013] According to another aspect of this application, a screen-click detection system for a screen-clicking device is also provided, used to detect whether the screen-clicking device lights up a normal display screen and displays a preset image, comprising: an execution module for repeatedly performing a screen-click judgment operation a preset number of times; and a determination module for determining the screen-clicking stability of the screen-clicking device based on the first successful screen-clicking count in the preset number of screen-click judgment operations; wherein the screen-click judgment operation includes: acquiring a screen-clicked image, wherein the screen-clicked image is an image of the display screen captured after the screen-clicking device clicks the display screen; determining the lit pixels in the preset area based on the grayscale value of each pixel in the preset area of ​​the screen-clicked image and a preset grayscale threshold, wherein the grayscale value of the lit pixels is greater than the preset grayscale threshold; counting the number of lit pixels; and determining whether the screen-clicking device has successfully clicked the screen based on the number of lit pixels.

[0014] According to another aspect of this application, an electronic device is also provided, including a processor and a memory. The memory stores computer program instructions, which are executed by the processor to perform the screen detection method of the aforementioned screen-dotting device.

[0015] According to another aspect of this application, a storage medium is also provided, on which program instructions are stored, which are used to execute the above-described screen detection method of the screen-dotting device when running.

[0016] In the above scheme, by performing image processing and judgment operations on the screen image acquired after each screen tapping operation, it is possible to determine whether each screen tapping operation of the screen tapping device was successful. By combining the number of successful taps across multiple screen tapping operations, the screen tapping stability of the screen tapping device can be quickly and accurately determined. This detection method is not only low-cost but also highly efficient, enabling accurate real-time detection of the screen tapping stability of the device. Furthermore, this scheme determines the success of each screen tapping operation based on the number of lit pixels with grayscale values ​​greater than a preset grayscale threshold in the screen image. Since the preset grayscale threshold is configurable, the control parameters used for stability determination can be flexibly configured, making it applicable to different types of screen tapping devices in different testing environments, thus broadening the scheme's applicability.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0018] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 A schematic flowchart of a screen detection method for a screen dotting device according to an embodiment of this application is shown;

[0020] Figure 2 A partial flowchart of a screen dot detection method for a screen dot device according to another embodiment of this application is shown;

[0021] Figure 3 A flowchart of a screen dot detection method for a screen dot device according to another embodiment of this application is shown;

[0022] Figure 4 A schematic block diagram of a dot-screen detection system for a dot-screen device according to an embodiment of this application is shown;

[0023] Figure 5 A schematic block diagram of an electronic device according to one embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.

[0025] As mentioned earlier, existing screen-pointing device testing solutions often require manually observing and recording whether the screen lights up or displays data after the device under test has been used to test a normal display screen. The recorded data is then used to determine if the device is functioning correctly. Alternatively, expensive testing equipment can be used to test the screen-pointing effect. Manual testing methods are labor-intensive, slow, and inefficient. Using expensive testing equipment is costly and has a fixed testing mode, making it difficult to adapt to different testing scenarios.

[0026] To at least partially solve the aforementioned technical problems, according to one aspect of this application, a screen-pointing detection method for a screen-pointing device is provided, used to detect whether the screen-pointing device illuminates a normal display screen and displays a preset image. The display screen can be any existing or future-developed display screen, including but not limited to mobile phone displays, computer displays, or television displays, etc., which are display devices capable of displaying images. In the embodiments of this application, the type, model, size, etc., of the display screen are not limited. However, in the embodiments of this application, the display screen is normal. In other words, in the embodiments of this application, a normal display screen is used to detect the screen-pointing stability of the screen-pointing device. The screen-pointing device can also be any existing or future-developed screen-pointing device, including but not limited to image signal generator devices (PG), etc. This application does not limit it, as long as it has the function of illuminating a normal display screen. The preset image can also be any suitable image input to the display screen, such as a monochrome image, a checkerboard image, a striped image, etc. This application does not limit it.

[0027] Figure 1 A schematic flowchart illustrating a screen-dotting detection method for a screen-dotting device according to an embodiment of this application is shown. Figure 1 As shown, the screen detection method of the screen dot device includes the following steps S110 and S120.

[0028] Step S120: Repeat the screen tap judgment operation a preset number of times. The preset number of times can be multiple times, and it can be set arbitrarily according to actual needs; this application does not limit it. Taking a preset number of 100 times as an example, the screen tap judgment operation can be performed 100 times in this step.

[0029] Specifically, step S120 includes steps S122, S124, S126, and S128, which are executed cyclically. In other words, each screen tap judgment operation includes the above four steps, and if the preset number of times is 100, these four steps can be executed 100 times.

[0030] Step S122: Acquire the screen-activated image. The screen-activated image is the image captured after the screen is activated using the screen-activation device. It should be noted that the screen-activated image may be an image showing the screen successfully activated or an image showing the screen not activated. In other words, the screen-activated image may be an image showing the screen successfully activated and displaying a preset image, or it may be an image showing the screen not activated and not displaying any image.

[0031] Exemplarily, the touch-screen device is connected to the display screen via a communication interface and can send a display signal to the display screen to display a preset image before each touch-screen operation. After the touch-screen device sends the display signal, a touch-screen image of the normal display screen can be acquired using various suitable image acquisition methods. Exemplarily, but not limitingly, the display screen can be photographed using any suitable image acquisition device such as a camera to obtain a touch-screen image including the display screen. In this example, the position and angle at which the image acquisition device photographs the display screen can be arbitrarily set according to actual needs, as long as the display screen can be completely captured. In a preferred embodiment, the image acquisition device can be placed at a preset position directly in front of the display screen to ensure that the display screen area is in the center of the touch-screen image acquired by the image acquisition device. It is understood that, for example, if the communication interface is good, the normal display screen can be lit up and display the preset image after receiving the display signal. In this case, the acquired touch-screen image can be a display screen image displaying the preset image. However, in the case of, for example, if the communication interface fails, the normal display screen may not receive the display signal, and therefore the normal display screen will not be lit up, thus not displaying any image. In this case, the obtained screen image can be an image of a screen that is not lit up (the screen area is a dark area).

[0032] The screen image can be a color image or a grayscale image; this application does not limit its application. The screen image can be the original image directly acquired by the image acquisition device, or an image based on the original image, or an image after simple preprocessing. This preprocessing may include image cropping, image scaling, overall image grayscale adjustment, and other processing operations that do not affect detection. The screen image can have any suitable image format, including but not limited to BMP, JPEG, RGB, or YUV formats.

[0033] Step S124: Based on the grayscale value of each pixel in a preset area of ​​the screen-lit image and a preset grayscale threshold, determine the lit pixels in the preset area. The lit pixels have grayscale values ​​greater than the preset grayscale threshold.

[0034] Optionally, the screen-intercepted image can be a color image. If the screen-intercepted image is a color image, it can first be converted to a grayscale image. Then, based on the grayscale values ​​of each pixel in the corresponding region of the converted grayscale image and a preset grayscale threshold, the illuminated pixels in the preset region can be determined. Alternatively, a preset grayscale conversion formula can be directly used to calculate the grayscale values ​​of each pixel in the preset region of the color screen-intercepted image. Then, based on the grayscale values ​​of each pixel and the preset grayscale threshold, the illuminated pixels in the preset region can be determined. Alternatively, the screen-intercepted image can also be a grayscale image. When the screen-intercepted image is a grayscale image, the illuminated pixels in the preset region can be directly determined based on the grayscale values ​​of each pixel in the preset region of the screen-intercepted image and a preset grayscale threshold.

[0035] The preset area can be the display area within the tapped image. If the tapped image only includes the display area, the preset area can be the entire or partial area of ​​the tapped image. If the tapped image includes both the display area and other areas, the preset area can be a partial area of ​​the tapped image. Specifically, the preset area can be the entire display area. Alternatively, the preset area can be a sub-region of the entire display area. For example, the preset area can be a sub-region of a preset size within the entire display area. The preset size can be arbitrarily set according to actual needs. For example, if the preset size is 100 pixels * 100 pixels, the preset area can be a 100-pixel * 100-pixel sub-region including the center pixel of the display area.

[0036] The preset grayscale threshold can be arbitrarily set according to actual detection needs. For example, a certain number of test images can be acquired in advance. These test images can be display screen images successfully displayed when the screen is manually confirmed to be successfully activated, captured by an image acquisition device. Then, the grayscale values ​​of the pixels in the display screen area of ​​these test images can be statistically analyzed to determine a suitable preset grayscale threshold. The specific implementation method of this scheme for determining the preset grayscale threshold will be explained later and will not be repeated here.

[0037] Taking a preset grayscale threshold of 150 as an example, in this step, for each pixel in the pre-defined area of ​​the screen, the grayscale value of each pixel is compared with the preset grayscale threshold of 150. Based on the comparison result, the pixels to be lit within the preset area are determined. For example, pixels in the preset area with a grayscale value greater than 150 can be directly identified as lit pixels.

[0038] Step S126: Count the number of lit pixels. Step S128: Based on the number of lit pixels, determine whether the screen-on device has successfully turned on the screen.

[0039] Taking a preset grayscale threshold of 150 and a preset area of ​​100 pixels * 100 pixels in the already illuminated image as an example, in step S126, any suitable pixel count acquisition method can be used to count the number of pixels in the preset area with a grayscale value greater than 150. Then, in step S128, based on the counted number of illuminated pixels, various suitable judgment logics can be used to determine whether the screen-illuminating device has successfully illuminated the screen.

[0040] In one example, step S128 determines whether the screen-touching device has successfully tapped the screen based on the number of lit pixels, including steps S128a, S128b, and S128c.

[0041] Step S128a: Calculate the proportion of lit pixels in the preset area.

[0042] Step S128b: If the percentage is less than the percentage threshold, determine that the screen-clicking device has failed to click.

[0043] Step S128c: If the percentage is greater than or equal to the percentage threshold, the screen-tapped device is confirmed to have successfully tapped the screen.

[0044] According to the embodiments of this application, the number of all pixels in a preset area of ​​the screen-tapped image acquired after the screen-tapped device is activated can be counted. The number of lit pixels in the area counted in step S126 can be taken as the number of lit pixels in that area. Then, the proportion of lit pixels to the total number of pixels is calculated, i.e., the proportion of lit pixels in the preset area. This proportion can be understood to represent the lighting rate of pixels in the preset area to a certain extent. Therefore, this proportion can be further compared with a pre-set proportion threshold to determine whether the screen-tapped device has successfully activated the screen. The proportion threshold can be arbitrarily set according to actual needs; different proportion thresholds can be set for different testing requirements. For example, the proportion threshold can be any value between 80% and 98%. If the proportion of lit pixels is greater than or equal to the preset proportion threshold, the screen-tapped operation is determined to be successful. If the proportion of lit pixels is less than the preset proportion threshold, the screen-tapped operation is determined to be unsuccessful.

[0045] Specifically, taking a 100-pixel * 100-pixel image area in the already tapped screen image as an example, the preset area can contain 10,000 pixels, and the percentage threshold can be 85%. In the tapped screen image obtained after the tapping device taps a normal display screen, if the number of lit pixels (activated pixels) in the preset area is greater than or equal to 8,500, the tapping operation is considered successful based on the percentage of lit pixels being greater than or equal to the 85% threshold. If the number of lit pixels (activated pixels) in the preset area is less than 8,500, the tapping operation is considered unsuccessful based on the percentage of lit pixels being less than the 85% threshold.

[0046] In the above scheme, the proportion of lit pixels with grayscale values ​​greater than a preset grayscale threshold in a preset region of the screen-triggered image is calculated. The success of the screen-triggered device is determined by comparing this proportion with a preset proportion threshold. This scheme involves low computational load and high accuracy, thereby improving the efficiency and precision of screen-triggered detection.

[0047] For example, the success of the current screen tapping device can also be determined directly based on the number of lit pixels and a preset threshold number.

[0048] In one example, the preset number of screen tapping judgment operations in step S120 can be executed sequentially. The execution order of the screen tapping judgment operations is consistent with the order of the screen tapping operations. After the screen tapping device performs the first screen tapping operation on a normal display screen, a tapped image of the display screen for the current screen tapping operation can be acquired as the first tapped image. Steps S124, S126, and S128 can then be executed for this first tapped image, and the result of whether the first screen tapping operation was successful can be recorded. Then, the screen tapping device continues to perform the second screen tapping operation, and a tapped image of the display screen after the second screen tapping operation is acquired as the second tapped image. Steps S124, S126, and S128 can then be executed for this second tapped image. This process is repeated until the number of screen taps performed by the screen tapping device reaches the preset number. For example, if the preset number is 100, then after the screen tapping device performs the 100th screen tapping operation, a tapped image of the display screen can be acquired as the 100th tapped image. Alternatively, the order of the screen tapping judgment operations can differ from the order of the screen tapping operations. For example, a screen tapping device can be used to continuously perform 100 screen tapping operations on a normal display screen. After each screen tapping operation, an image of the display screen is captured, thus collecting 100 tapped images at once. Then, the screen tapping judgment operations in steps S124, S126, and S128 can be performed on each of the 100 collected tapped images. The screen tapping judgment operations can be performed in the order of image capture or in a random order. Understandably, for the latter approach, the order of the screen tapping judgment operations may differ from the order of the screen tapping operations.

[0049] In another example, the screen-click judgment operation of the preset number of times in step S120 can also be executed approximately synchronously. For example, the method described above can be used to acquire a preset number of clicked images at once, where the preset number is the same as the preset number of clicks, and different clicked images correspond to screen-click operations at different times. Then, steps S124, S126, and S128 can be executed simultaneously on these clicked images. For example, these 100 clicked images can be simultaneously input into a program that loads the processing logic of steps S124, S126, and S128, and the processing result indicating whether the screen-click operation corresponding to each clicked image was successful can be obtained. For example, the clicked images corresponding to successful screen-click operations can be marked.

[0050] Step S140: Determine the first successful screen tapping by the screen tapping device during the operation based on the preset number of screen taps, and determine the screen tapping stability of the screen tapping device.

[0051] In this step, the first successful screen tap of the device can be determined based on a preset number of screen taps, and any appropriate determination logic can be used to determine the screen tapping stability of the device.

[0052] In one example, step S140 determines the screen-tapping stability of the screen-tapping device based on the first number of successful screen taps, including steps S141 and S142.

[0053] Step S141: Count the number of successful screen taps by the device during the preset number of screen tap judgment operations. Step S142: Based on the first successful tap count and the preset number of taps, determine the screen tapping stability of the device.

[0054] In step S141, any suitable method can be used to count the first number of times the screen-touch device successfully lights up the screen in 100 screen-touch judgment operations. For example, in the above embodiment where a preset number of screen-touched images are acquired at once, and the preset number is the same as the preset number of times, and different screen-touched images are for screen-touch operations at different times, the number of screen-touched images marked as successfully tapped can be counted as the first count. Then, in step S142, any suitable determination logic can be used to determine the screen-touch stability of the screen-touch device based on the first count and the preset number of times. For example, the ratio of the first count to the preset number of times can be calculated to obtain the screen-touch success rate ratio. And any suitable count ratio threshold can be preset. The screen-touch stability of the screen-touch device can be determined based on the screen-touch success rate ratio and the count ratio threshold. Optionally, a count ratio threshold can be set, and the screen-touch stability judgment result can be divided into two levels: good stability and poor stability. If the screen-touch success rate ratio is greater than or equal to the count ratio threshold, the screen-touch stability of the screen-touch device can be determined to be good; if the screen-touch success rate ratio is less than the count ratio threshold, the screen-touch stability of the screen-touch device can be determined to be poor. For example, if the preset number of attempts is 100, the first attempt count is 70, and the threshold for the number of attempts is 80%, then based on the above judgment logic, the screen-clicking stability of the device can be determined to be poor. Alternatively, multiple thresholds for the number of attempts can be set, and the screen-clicking stability judgment results can be divided into multiple levels. For example, two thresholds for the number of attempts can be set, and the screen-clicking stability judgment results can be divided into three levels: good stability, average stability, and poor stability. For example, if the two thresholds for the number of attempts are 60% and 80%, and the preset number of attempts is 100, then based on the case where the first attempt count is less than 60, the screen-clicking stability of the device can be determined to be poor; based on the case where the first attempt count is greater than or equal to 60 and less than 80, the screen-clicking stability of the device can be determined to be average; and based on the case where the first attempt count is greater than or equal to 80, the screen-clicking stability of the device can be determined to be good.

[0055] In the above scheme, the method of judging the screen-clicking stability of the screen-clicking device based on the first successful screen-clicking operation and the preset number of screen-clicking operations has low computational load, high computational accuracy, and simple and reasonable judgment logic, which can realize real-time and accurate detection of the screen-clicking stability of the screen-clicking device.

[0056] As mentioned earlier, existing screen tap detection schemes fall into two categories: manual detection, which is costly, inefficient, and prone to errors; and equipment-based detection, which is expensive and limited in its scope of application. The detection scheme described in this application, however, determines the success of each tap operation by performing simple image processing and judgment on the tapped image of the display screen after each tap operation. By combining the number of successful taps across multiple tap operations, the tap stability of the device can be quickly and accurately determined. This detection method is not only low-cost but also highly efficient, enabling real-time and accurate detection of the tap stability of the device. Furthermore, the scheme determines the success of each tap operation based on the number of lit pixels with grayscale values ​​greater than a preset grayscale threshold in the tapped image. Since the preset grayscale threshold is settable, the control parameters used for stability determination can be flexibly configured, making it applicable to different tap detection scenarios for different types of tap devices, thus broadening the scheme's applicability.

[0057] In some embodiments, before determining the lit pixels in the preset area in step S124, the method 100 further includes steps S121a, S121b and S121c.

[0058] Step S121a: Obtain the first image of the display screen that has already displayed the preset image.

[0059] Step S121b: Perform image segmentation on the first image to determine the location region of the display screen in the image, which is then used as the display screen area.

[0060] Step S121c: The image of the display area is used as the screen image.

[0061] The first image can be the original image captured by the image acquisition device. For example, after the touch-screen device touches a normal display screen, the original image of the display screen captured by the image acquisition device can include the imaging area of ​​the display screen as well as the imaging area of ​​objects near the display screen; that is, the original image includes both the display screen area and non-display screen areas. For example, a scenario where the image acquisition device captures the original image of the display screen could be: placing a camera directly in front of a white wall, with a distance of 2m between the camera and the white wall, and placing the display screen between the white wall and the camera, for example, with a distance of 20cm between the camera and the white wall. In this way, the first image of the display screen captured by the camera after the touch-screen device touches the display screen includes not only the imaging area of ​​the display screen but may also include the imaging area of ​​the white wall. To avoid the interference of pixels from non-display screen imaging areas such as the white wall in the first image with the judgment result of the touch-screen judgment operation, the first image can be processed to remove these potentially interfering pixels.

[0062] Specifically, in step S121b, various suitable image segmentation algorithms can be used to segment the first image to determine the display screen region in the first image. Image segmentation algorithms include, but are not limited to, thresholding algorithms, region growing algorithms, edge detection algorithms, and deep learning-based image segmentation algorithms. For example, the first image can be input into a trained image segmentation model, which outputs a mask image. The mask image includes pixels with a value of 0 and pixels with a value of 1. For example, in the mask image, the image region formed by pixels with a value of 1 can correspond to the display screen region in the first image. Thus, the display screen region in the first image can be determined. In step S121c, the first image can be cropped along the edge of the display screen region. It can be understood that the cropped image is composed of pixels from the display screen.

[0063] In the above scheme, the location of the display area in the first image of the acquired display screen is determined by image segmentation, and pixels in non-display areas of the first image are further removed to obtain the screen-detected image. This scheme can effectively reduce the adverse effects of interfering pixels in the image acquisition environment on screen detection, and can also reduce unnecessary computation to some extent. Furthermore, the image segmentation method for determining the display area is faster and more accurate. Therefore, it not only improves the detection efficiency of screen detection, but also improves the detection accuracy.

[0064] By way of example, and not limitation, the method 100 further includes step S123a. In step S123a, a region including the center of the image in the tapped screen image is determined as a preset region. The width of the preset region is less than the width of the tapped screen image, and the height of the preset region is less than the height of the tapped screen image.

[0065] In one example, after obtaining a dotted screen image containing only the display area using the methods described in steps S121a, S121b, and S121c, the center pixel of the dotted screen image can be determined using various suitable methods. Then, a preset area can be determined based on the position of the center pixel and the width and height of the preset area. Optionally, the center pixel coordinates can be calculated based on the image's width and height. Assuming the image width is W and the height is H, the center pixel coordinates are (W / 2, H / 2). Alternatively, the center pixel coordinates can be calculated based on the image array index. Assuming the image width is W and the height is H, and the pixel values ​​are stored in a one-dimensional array from top left to bottom right, the center pixel index can be calculated using the following formula: index = (H / 2) * W + (W / 2). Here, (H / 2) represents the vertical position of the center pixel, and (W / 2) represents the horizontal position of the center pixel. Thus, the coordinates or index of the center pixel of the dotted screen image can be determined. Furthermore, the position of the preset area can be determined based on the coordinates or index of the center pixel. The preset area can be any size and shape. Taking a rectangular area with width 'a' and height 'b' as an example, if the center pixel coordinates are (W / 2, H / 2), then the coordinates of the top left vertex of the preset area can be determined as (W / 2-a / 2, H / 2-b / 2).

[0066] In another example, after obtaining a dotted screen image that includes only the display area using the methods described in steps S121a, S121b, and S121c, the preset area can be determined directly based on the size of the dotted screen image and the shape and size of the preset area. Again, taking a rectangular area of ​​100 pixels * 100 pixels as an example, the dotted screen image can be pixelated to obtain a dot matrix graphic composed of pixels, which has x rows and y columns. The preset area can be determined using the following formula: (Start_x:Start_x+100, Start_y:Start_y+100); where Start_x = x / 2 - 50; Start_y = y / 2 - 50. Here, Start_x is the row containing the top edge of the preset area, and Start_y is the column containing the left edge of the preset area.

[0067] According to an embodiment of this application, the width of the preset region is smaller than the width of the dotted screen image, and the height of the preset region is smaller than the height of the dotted screen image. In other words, the preset region is a local area within the dotted screen image. Furthermore, the center of the preset region determined using the above method is the center of the display screen area; therefore, the preset region is the imaging area in the center of the display screen. Typically, the imaging effect in the center of the display screen can represent the overall imaging effect of the display screen. This scheme can significantly reduce the computational load and significantly improve detection efficiency while ensuring the accuracy of dotted screen detection.

[0068] In some embodiments, the method 100 further includes the step of determining a preset grayscale threshold. The step of determining the preset grayscale threshold includes the following steps S123.1 to S123.4.

[0069] Step S123.1: If it is determined that the display screen has been successfully activated, acquire the activated test image of the display screen. This step is similar to step S120, both involving acquiring the display screen image after the activation device performs an activation operation on the display screen. For example, the images acquired in both steps are captured after the activation device sends a display signal to the display screen driver via the communication interface, displaying the same preset image. However, the execution conditions for this step differ from those of step S120. This step acquires the test image only if it is determined that the display screen has been successfully activated, while step S120 acquires the display screen image regardless of whether the display screen has actually been successfully activated.

[0070] Step S123.2 involves performing image segmentation on the tested image to determine the display area (the region where the display is located in the image) and the background area (formed by all pixels outside the display area). This step is implemented similarly to step S121b above, and for simplicity, will not be described again here.

[0071] Step S123.3: Determine the first overall grayscale value of the pixels in the display area and the second overall grayscale value of the pixels in the background area. Any suitable calculation method can be used to determine the first and second overall grayscale values. In one example, the average grayscale value of all pixels in the image of the display area can be calculated and used as the first overall grayscale value. Similarly, the average grayscale value of all pixels in the image of the background area can be calculated and used as the second overall grayscale value. Taking the determination of the first overall grayscale value of the pixels in the display area as an example: First, all pixels in the display area can be traversed, and the grayscale value of each pixel can be obtained. Then, the grayscale values ​​of each pixel can be summed to obtain the total grayscale value. Then, the average grayscale value of each pixel in the display area can be calculated by dividing the total grayscale value by the total number of pixels in the display area. In other examples, the grayscale values ​​of all pixels in the display area can be sorted, and the median grayscale value after sorting can be found as the first overall grayscale value; alternatively, pixels at different locations in the display area can be assigned different weights, and then a weighted average method can be used to determine the first overall grayscale value. The method for determining the second overall grayscale value can be the same as the method for determining the first overall grayscale value, and will not be repeated here.

[0072] Step S123.4: Determine a preset grayscale threshold based on the first overall grayscale value and the second overall grayscale value. The preset grayscale threshold is greater than or equal to the second overall grayscale value and less than or equal to the first overall grayscale value.

[0073] Since the display area in the test image is the imaging area of ​​the illuminated display screen, its image color brightness is brighter than that of the background area. Therefore, the first overall grayscale value of the display area is larger than the second overall grayscale value of the background area. In this step, a preset grayscale threshold can be set to a range between the first and second overall grayscale values, which can effectively distinguish the display area from the background area.

[0074] For example, if the first overall grayscale value of pixels in the determined display area is 180, and the second overall grayscale value of pixels in the background area is 90, then a preset grayscale threshold can be set to any value between 90 and 180. Optionally, the preset grayscale value can be set to the average of the first and second overall grayscale values. For example, the preset grayscale threshold can be set to 135. Alternatively, the preset grayscale threshold can be set to any other suitable value. For example, a suitable preset grayscale threshold can be further set according to the characteristics of the preset image. For example, if the preset image is generally bright, the preset grayscale threshold can be set slightly larger; if the preset image is generally dark, the preset grayscale threshold can be set slightly smaller. Alternatively, the preset grayscale threshold can be determined by comparing the average of the first and second overall grayscale values ​​with the proportion of pixels in the display area whose grayscale values ​​are less than the average value. For example, if the proportion of pixels with gray values ​​less than the average value in the display area is less than a preset proportion, the average value can be directly used as the preset gray value threshold; if the proportion of pixels with gray values ​​greater than or equal to the average value in the display area is greater than the preset proportion, the preset gray value threshold can be determined to be greater than the average value and less than the larger of the first overall gray value and the second overall gray value.

[0075] For example, multiple screen-test images can be acquired continuously, with different screen-test images corresponding to different screen-tapping operations. The display signal sent by the screen-tapping device to the display can be the same, meaning the screen-tapping device drives the display to show the same preset image. Steps S123.1 to S123.4 can be performed on each screen-tested image to determine multiple preset grayscale thresholds corresponding to the multiple screen-tested images. Then, the average of the multiple preset grayscale thresholds can be used as the preset grayscale threshold for actual detection.

[0076] The preset grayscale threshold determined by the above scheme is relatively accurate, thus ensuring the accuracy of screen dot detection. Furthermore, the execution logic is simple, resulting in low computational cost.

[0077] Figure 2 A partial flowchart illustrating a screen dot detection method for a screen dot device according to another embodiment of this application is shown. Figure 2 As shown, the step of determining the preset grayscale threshold may further include steps S123.5 to S123.9. For example, the above-described steps S123.1 to S123.4 can be considered as a preliminary method for determining the initial grayscale threshold. After the initial grayscale threshold is preliminarily determined, the following calibration and update steps can be used to adjust and update the initial grayscale threshold to obtain a more reasonable and accurate preset grayscale threshold.

[0078] Step S123.5: If it is determined that the display screen has been successfully tapped, acquire a tapped test image of the display screen. Optionally, the tapped test image acquired in this step can be the same as the tapped test image acquired in step S123.1. In other words, if a tapped test image has already been acquired through step S123.1, this step can be skipped, and the tapped image acquired in step S123.1 can be used directly. Optionally, the tapped test image acquired in this step can be different from the tapped test image acquired in step S123.1. For example, after step S123.1, the tapping device can be used to perform one or more tapping operations on a normal display screen, and if the tapping is manually confirmed to be successful, an image of the display screen that has actually been successfully lit can be acquired.

[0079] Step S123.6: Determine the grayscale values ​​of pixels in a preset area of ​​the tested image. Any suitable method can be used to determine the grayscale values ​​of each pixel in the preset area.

[0080] Step S123.7: Based on the grayscale values ​​of pixels in the preset area and the current preset grayscale threshold, determine the illuminated pixels in the preset area. The illuminated pixels have grayscale values ​​greater than the current preset grayscale threshold. This step is implemented similarly to the method for determining illuminated pixels in step S124, and will not be described again here.

[0081] Step S123.8: Based on the number of lit pixels, determine whether the screen-tapping device has successfully tapped the screen. If not, update the current preset grayscale threshold. The method for determining whether the screen-tapping device has successfully tapped the screen is similar to step S128 and will not be repeated here. If it is determined that the screen-tapping device has failed to tap the screen, then...

[0082] Step S123.9: Based on the updated preset grayscale threshold, proceed to determine the illuminated pixels in the preset area until the screen-on device is confirmed to have successfully turned on the screen. That is, based on the updated preset grayscale threshold, execute steps S123.7 and S123.8 again until the screen-on device is confirmed to have successfully turned on the screen.

[0083] It is understandable that when acquiring a test image of the display screen, the brightness of the test image is easily affected by many factors such as underexposure or overexposure of the camera or the brightness of the surrounding environment. Therefore, under certain circumstances, the preset grayscale threshold set through steps S123.1 to S123.4 may be inaccurate. Therefore, the preset grayscale threshold can be calibrated and updated through verification methods to determine a more accurate preset grayscale threshold.

[0084] For simplicity, the specific implementation of the above steps is explained using an RGB image filled with red as an example. The current preset grayscale threshold is, for example, 76.245, and the preset area is, for example, a 100-pixel * 100-pixel rectangular area including the center of the display area. Exemplarily, after the display device sends a display signal to the display screen to display the preset image, a test image of the display screen can be acquired. Then, the display area in the test image can be determined using methods such as image segmentation, and the position of the 100-pixel * 100-pixel rectangular area including the center of the display area can be determined. In step S123.6, a grayscale conversion formula can be used to convert the color data of each pixel in the rectangular area of ​​the test image into grayscale values. Then, in step S123.7, the grayscale value of each pixel in the rectangular area can be compared with the current preset grayscale threshold 76.245, and the lit pixels in the rectangular area with grayscale values ​​greater than 76.245 can be determined.

[0085] In step S123.8, the number of lit pixels can be counted. If this number is greater than or equal to a preset threshold (e.g., 8500), it can be determined that the current screen-on device has successfully tapped the screen. This indicates successful verification, suggesting that the preset grayscale threshold setting is reasonable. In this step, if the number of lit pixels is less than 8500, it indicates that the screen-on device has failed to tap the screen according to the current preset grayscale threshold. Since the current test image was obtained under conditions where screen tapping was manually confirmed as successful, it can be determined that the verification result deviates from the actual value, i.e., verification failed. A suitable update method can then be used to update the preset grayscale threshold. In one example, the current preset grayscale threshold can be directly reduced by a preset value. This preset value can be a small value, which can be arbitrarily set according to actual needs. For example, the preset value can be 5, meaning the current preset grayscale threshold of 76.245 can be subtracted by 5 to obtain the updated preset grayscale threshold of 71.245. In another example, the preset grayscale threshold can also be adjusted based on camera exposure and ambient brightness. When both camera exposure and ambient brightness are relatively high, the preset grayscale threshold is increased accordingly. Conversely, when both camera exposure and ambient brightness are relatively low, the preset grayscale threshold is lowered accordingly. When one is high and the other is low, the preset grayscale threshold is adjusted flexibly.

[0086] In step S123.9, the updated preset grayscale threshold (e.g., 71.245) can be used as the current preset grayscale threshold. Then, step S123.7 is executed, comparing the grayscale value of each pixel in the previously determined rectangular region with the current preset grayscale threshold 71.245, and identifying the lit pixels in the rectangular region with grayscale values ​​greater than 71.245. Step S123.8 is then executed, counting the number of lit pixels in the rectangular region with grayscale values ​​greater than 71.245 and comparing this number with 8500. If the number is greater than 8500, then 71.245 is determined as the preset grayscale threshold for formal detection. Otherwise, the preset grayscale threshold is updated again, and steps S123.7 and S123.8 are executed based on the updated preset grayscale threshold…

[0087] The above-mentioned verification and update of the preset grayscale threshold can be used when the preset grayscale threshold of the actual test is in the most reasonable value range. In this way, it can more accurately determine whether the screen-on device has successfully lit up the screen during the test process, and further improve the accuracy of the screen-on detection of the screen-on device.

[0088] In some embodiments, the preset image is a color image. Exemplarily, step S124 determines the illuminated pixels in the preset area, including steps S124a and S124b. Step S124a converts the color data of each pixel in the color pre-painted image into grayscale values. Step S124b identifies pixels in the preset area whose grayscale values ​​are greater than a preset grayscale threshold as illuminated pixels.

[0089] The preset image is a color image, and the display screen can be a monitor capable of displaying color pixels, so the dotted screen image can also be a color image. For example, if the dotted screen image is an RGB image, in step S124a, the color data of the three channels of each pixel in the preset area of ​​the dotted screen image can be obtained. Then, any suitable grayscale conversion method can be used to convert the color data of each pixel into grayscale values. In one example, a cross-platform computer vision and machine learning software library algorithm (OPENCV algorithm) can be used to directly convert the color dotted screen image into a grayscale image, and the grayscale values ​​of each pixel in the preset area of ​​the grayscale image can be obtained according to the position of the preset area in the dotted screen image. In another example, a three-dimensional array composed of the brightness values ​​of the three RGB channels corresponding to each pixel in the preset area of ​​the color dotted screen image can be obtained. Then, any suitable grayscale value algorithm can be used to convert the three-dimensional array into a two-dimensional array composed of the grayscale values ​​of each pixel. For example, the weighted average method can be used to calculate the grayscale value of the pixel. Different color channels may contribute differently to the brightness of the image, and the grayscale value can be determined by weighted averaging. A common weighted average method is to set the weight of the red channel to 0.299, the green channel to 0.587, and the blue channel to 0.114. Specifically, the grayscale value of each pixel can be calculated using the following formula: Gray = R × 0.299 + G × 0.587 + B × 0.114. Here, Gray represents the grayscale value of the pixel, R is the brightness value of the pixel in the red channel, G is the brightness value of the pixel in the green channel, and B is the brightness value of the pixel in the blue channel. Alternatively, the maximum method can be used, taking the brightness value of the channel with the highest brightness among the three RGB channels as the grayscale value. Or, the minimum method can be used, taking the brightness value of the channel with the lowest brightness among the three RGB channels as the grayscale value. Or, the average method can be used, taking the average of the brightness values ​​of the pixel in the three RGB channels as the grayscale value.

[0090] The above-described scheme requires minimal computation and can quickly and accurately determine the grayscale values ​​of pixels in the displayed image regardless of its color mode. Furthermore, the scheme using grayscale values ​​to determine whether the display screen has been successfully lit requires even less computation and is more accurate. Therefore, the scheme has a wider range of applications and offers better detection efficiency and accuracy.

[0091] Figure 3 A flowchart illustrating a screen dot detection method for a screen dot device according to another embodiment of this application is shown. Figure 3As shown, before formal testing, one or more test images of a normal display screen can be acquired. These test images can be those capturing the display screen itself, obtained after the screen-on device has been manually confirmed to have successfully powered on. Then, image segmentation methods can be used to determine the positions of the display area and the background area in each test image. The display area is the imaging area of ​​the display screen, and the background area is the area outside the display area. The test image can be a color RGB image, and the grayscale value of each pixel in the test image can be determined using the following grayscale conversion formula: Gray = R × 0.299 + G × 0.587 + B × 0.114. The grayscale values ​​of each pixel in the display area can be obtained from this formula, and then the average of these pixel grayscale values ​​can be calculated to obtain a first overall grayscale value. Similarly, the grayscale values ​​of each pixel in the background area can be obtained, and the average of these pixel grayscale values ​​can be calculated to obtain a second overall grayscale value. Then, an initial grayscale threshold can be calculated based on the first and second overall grayscale values. The initial grayscale threshold can be any value between the first and second overall grayscale values. By way of example, but not limitation, the initial grayscale threshold can be the average of the first overall grayscale value and the second overall grayscale value. Then, using the initial grayscale threshold as the current preset grayscale threshold, the initial grayscale threshold can be calibrated and updated using the methods described in steps S123.5 to S123.9 above, and the updated preset grayscale threshold can be determined. The preset grayscale threshold is, for example, 150. Then, detection can begin based on the updated preset grayscale threshold. A preset number of screen-clicking judgment operations are sequentially performed on the display screen using a screen-clicking device. Specifically, after each screen click, a first image of the display screen can be captured using a camera positioned directly in front of the display screen. Then, the first image can be input into a trained image segmentation model to segment the display screen area in the first image. By way of example, the first image can be cropped based on the position of the display screen area to obtain a screen-clicked image that only includes the display screen area. Then, the grayscale value of each pixel in the currently tapped image can be calculated according to the grayscale conversion formula described above. The grayscale value of each pixel in a 100-pixel * 100-pixel rectangular region including the image center can be obtained, and the grayscale value of each pixel can be compared with a preset grayscale threshold (e.g., 150). Pixels with grayscale values ​​greater than 150 in this rectangular region can be identified as lit pixels. The number of lit pixels can then be counted, and the success of the tapping operation can be determined based on the number of lit pixels. Next, it can be determined whether the cumulative number of tapping judgment operations has reached a preset number. Taking a preset number of 100 as an example, if it has not reached 100, the tapping device is used again to perform a tapping operation on the display screen to perform another tapping judgment operation. This continues until the number of tapping judgment operations reaches 100.Then, we can count the number of successful first taps out of these 100 tap detection operations. We can then determine if this number exceeds a threshold. For example, if the number of successful first taps is 85, then the tap stability is good. If the number of successful first taps is less than or equal to 85, then the tap stability is poor.

[0092] According to another aspect of this application, a screen-pointing detection system for a screen-pointing device is also provided. This system is used to detect whether the screen-pointing device illuminates a normal display screen and displays a preset image. Figure 4 A schematic block diagram of a touch detection system 400 for a touch screen device according to an embodiment of this application is shown. Figure 4 As shown, the screen dot detection system 400 includes an execution module 410 and a determination module 420.

[0093] The execution module 410 is used to repeatedly perform the screen-touch judgment operation a preset number of times. The screen-touch judgment operation includes: acquiring a screen-touched image, wherein the screen-touched image is an image of the display screen captured after the screen is tapped using the screen-touch device; determining the lit pixels in the preset area based on the grayscale value of each pixel in the screen-touched image and a preset grayscale threshold, wherein the grayscale value of the lit pixels is greater than the preset grayscale threshold; counting the number of lit pixels; and determining whether the screen-touch device has successfully tapped the screen based on the number of lit pixels.

[0094] The determination module 420 is used to determine the first successful screen tap of the screen-tapping device during the operation based on a preset number of screen taps, and to determine the screen tapping stability of the screen-tapping device.

[0095] According to another aspect of this application, an electronic device is also provided. Figure 5 A schematic block diagram of an electronic device 500 according to one embodiment of this application is shown. Figure 5 As shown, the electronic device 500 includes a processor 510 and a memory 520. The memory 520 stores computer program instructions, which are executed by the processor 510 to perform the screen detection method 100 of the aforementioned screen dot device.

[0096] According to another aspect of this application, a storage medium is also provided, on which program instructions are stored. These program instructions, when executed, are used to perform the screen detection method of the aforementioned screen-pointing device. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0097] Those skilled in the art can understand the specific implementation scheme and beneficial effects of the above-mentioned screen detection system 400, electronic device 500 and storage medium by reading the relevant description of the screen detection method 100 for the screen detection device. For the sake of brevity, they will not be described in detail here.

[0098] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0099] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0101] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0102] Similarly, it should be understood that, in order to streamline this application and aid in understanding one or more of the various inventive aspects, features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with features fewer than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0103] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0104] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0105] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the dot-screen detection system of the dot-screen device according to embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0106] It should be noted that the above embodiments are illustrative of this application and not limiting of it, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0107] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A screen-touch detection method for a screen-touch device, used to detect whether the screen-touch device illuminates a normal display screen and displays a preset image, characterized in that, include: Repeat the screen tap detection operation a preset number of times; as well as Based on the preset number of successful screen taps during the screen tapping operation, the screen tapping stability of the screen tapping device is determined. The screen tapping judgment operation includes: Acquire a tapped screen image, wherein the tapped screen image is an image of the display screen captured after tapping the display screen using the tapping device; Based on the grayscale value of each pixel in a preset area of ​​the already-visible image and a preset grayscale threshold, the lit pixels in the preset area are determined, wherein the grayscale value of the lit pixels is greater than the preset grayscale threshold. Count the number of the lit pixels; Based on the number of lit pixels, determine whether the screen-touch device has successfully tapped the screen; The method further includes the step of determining the preset grayscale threshold: If it is determined that the display screen has been successfully tapped, acquire the tapped test image of the display screen; The tested image is segmented to determine the display area of ​​the display screen in the image and the background area formed by all pixels outside the display area. Determine a first overall grayscale value of pixels in the display screen area, and determine a second overall grayscale value of pixels in the background area; and The preset grayscale threshold is determined based on the first overall grayscale value and the second overall grayscale value.

2. The screen detection method for a screen-dotting device as described in claim 1, characterized in that, The preset image is a color image. Determining the lit pixels in the preset area includes: Convert the color data of each pixel in the colored dotted screen image into grayscale values; The pixels in the preset area whose grayscale value is greater than the preset grayscale threshold are identified as the lit pixels.

3. The screen detection method for a screen-dotting device as described in claim 1, characterized in that, The method further includes: Acquire a first image of the display screen that already displays the preset image; The first image is segmented to determine the location region of the display screen within the image, which is designated as the display screen area; and The image of the display screen area is used as the screen image that has been tapped.

4. The screen detection method for a screen dot-on device as described in claim 3, characterized in that, The method further includes: The region including the center of the image in the already tapped screen image is defined as the preset region; Wherein, the width of the preset area is less than the width of the tapped screen image, and the height of the preset area is less than the height of the tapped screen image.

5. The screen detection method for a screen-dotting device as described in any one of claims 1 to 4, characterized in that, The step of determining the preset grayscale threshold further includes: The preset grayscale threshold is greater than or equal to the second overall grayscale value and less than or equal to the first overall grayscale value.

6. The screen detection method for a screen-dotting device as described in claim 5, characterized in that, The step of determining the preset grayscale threshold further includes: If it is determined that the display screen has been successfully tapped, acquire the tapped test image of the display screen; Determine the grayscale value of the pixels in the preset area of ​​the tested image; Based on the grayscale values ​​of the pixels in the preset region and the current preset grayscale threshold, the lit pixels in the preset region are determined, wherein the pixel value of the lit pixel is greater than the current preset grayscale threshold. Based on the number of lit pixels, determine whether the screen-touch device has successfully tapped the screen; if not, update the current preset grayscale threshold. Based on the updated preset grayscale threshold, the step of determining the lit pixels in the preset area is then executed until it is determined that the screen-on device has successfully turned on the screen.

7. The screen detection method for a screen-dotting device as described in any one of claims 1 to 4, characterized in that, Determining whether the screen-touch device has successfully tapped the screen based on the number of lit pixels includes: Calculate the percentage of lit pixels in the preset area; If the percentage is less than the percentage threshold, it is determined that the screen-tapping device has failed to tap; and If the percentage is greater than or equal to the percentage threshold, it is determined that the screen-tapping device has successfully tapped the screen.

8. The screen detection method for a screen-dotting device as described in any one of claims 1 to 4, characterized in that, The step of determining the screen-tapping stability of the screen-tapping device based on the first number of successful screen taps includes: The first number of successful screen taps by the screen tapping device is determined in the preset number of screen tapping judgment operations. The screen-pointing stability of the screen-pointing device is determined based on the first number of times and the preset number of times.

9. A screen-pointing detection system for a screen-pointing device, used to detect whether the screen-pointing device illuminates a normal display screen and displays a preset image, characterized in that, include: The execution module is used to repeatedly perform the screen tap judgment operation a preset number of times; The determination module is used to determine the first successful screen tapping of the screen-tap device in the operation based on the preset number of screen taps, and to determine the screen tapping stability of the screen-tap device. The screen tapping judgment operation includes: Acquire a tapped screen image, wherein the tapped screen image is an image of the display screen captured after tapping the display screen using the tapping device; Based on the grayscale value of each pixel in a preset area of ​​the already-visible image and a preset grayscale threshold, the lit pixels in the preset area are determined, wherein the grayscale value of the lit pixels is greater than the preset grayscale threshold. Count the number of the lit pixels; Based on the number of illuminated pixels, determine whether the screen-touch device has successfully tapped the screen. The system is also used to determine the preset grayscale threshold, and the step of determining the preset grayscale threshold is as follows: If it is determined that the display screen has been successfully tapped, acquire the tapped test image of the display screen; The tested image is segmented to determine the display area of ​​the display screen in the image and the background area formed by all pixels outside the display area. Determine a first overall grayscale value of pixels in the display screen area, and determine a second overall grayscale value of pixels in the background area; and The preset grayscale threshold is determined based on the first overall grayscale value and the second overall grayscale value.

10. An electronic device comprising a processor and a memory, characterized in that, The memory stores computer program instructions, which, when executed by the processor, are used to perform the screen detection method of the screen-pointing device as described in any one of claims 1 to 8.

11. A storage medium on which program instructions are stored, characterized in that, The program instructions, when executed, are used to perform the screen detection method for the screen-pointing device as described in any one of claims 1 to 8.

Images