Non-intrusive method and device for detecting pop-up window button

Abstract

The present application provides a non-invasive detection method of a pop-up window button for identifying a pop-up window containing at least one pop-up window button in a display device. The method includes the following steps: capturing a screen image of the display device; comparing the screen image with a preset screen image and generating a difference image area according to the screen image and the preset screen image; when the difference image area is greater than an image area threshold, judging the difference image area as a pop-up window of the screen image; screening a plurality of contour lengths in the pop-up window that are greater than a contour length threshold by a Canny edge detection; and using a Douglas-Peucker algorithm and an endpoint number to interpret the plurality of contour lengths to generate a contour edge corresponding to the at least one pop-up window button.

Description

Technical Field

[0001] This invention relates to a non-intrusive detection method and apparatus, and more particularly, to a non-intrusive detection method and apparatus capable of automatically detecting pop-up window buttons. Background Technology

[0002] With the rapid development of industry, many factories are gradually automating their equipment. Typically, this equipment is equipped with displays that show the current production progress and various parameters, allowing operators to monitor the equipment's operational status. In recent years, with the development of industrial intelligence, many factories are also installing remotely controllable, non-invasive devices to connect to the equipment and monitor and collect data from the displays, thereby replacing manual labor.

[0003] However, during the monitoring of the equipment's display screen using a non-invasive device, unexpected pop-up windows, such as warning windows and selection windows, may appear on the peripheral devices or display screen installed in the equipment. Therefore, when a pop-up window appears, it may obscure the production parameters or data to be monitored on the screen, causing the non-invasive device to be unable to identify the data, leading to malfunctions and interrupting equipment operation, thus reducing efficiency.

[0004] Therefore, it is necessary to develop a new detection mechanism for pop-up windows and pop-up buttons to solve the problems of previous technologies. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a non-intrusive method and apparatus for detecting pop-up windows, thereby solving the problems of the prior art. This method can automatically detect pop-up windows on a device and identify the pop-up window buttons within them, effectively improving usability, increasing processing efficiency, and reducing labor and production costs.

[0006] To achieve the above objectives, the present invention discloses a non-intrusive detection method for a pop-up window button, used to identify at least one pop-up window button of a pop-up window on a display device, characterized in that the non-intrusive detection method for the pop-up window button includes the following steps:

[0007] S1: Capture an image from the display device;

[0008] S2: Compare the screen image with a preset screen image, and generate a difference image area based on the screen image and the preset screen image;

[0009] S3: Determine whether the difference in the image region is greater than a threshold for an image region;

[0010] S4: When the difference image area is greater than a threshold of an image area, the difference image area is determined as the pop-up window of the screen image;

[0011] S5: In this pop-up window, first preset a contour length threshold, then use Canney edge detection to filter out multiple contour lengths that are greater than the contour length threshold; and

[0012] S6: First, preset the number of endpoints of the at least one pop-up window button, and then use the Douglas-Puk algorithm and the number of endpoints to determine the length of the multiple contours to generate a contour edge corresponding to the at least one pop-up window button.

[0013] This further includes the following steps:

[0014] S11: Determine if the preset screen image exists. If the determination result is yes, proceed to step S2; and

[0015] S12: If the judgment result of step S11 is negative, replace the screen image with the preset screen image.

[0016] Step S2 further includes the following steps:

[0017] S21: Subtract the screen image from the preset screen image to generate a difference image; and

[0018] S22: The difference image is calculated by binarization and image erosion to generate the difference image region.

[0019] Step S6 further includes the following steps:

[0020] S61: Using the Douglas-Puk algorithm and the number of endpoints, determine the lengths of the multiple contours to generate a button contour that approximates the at least one pop-up window button; and

[0021] S62: Generate the outline edge corresponding to at least one pop-up window button based on the button outline and an outline correction accuracy, wherein the outline correction accuracy is between 1% and 10% of the outline length.

[0022] This further includes the following steps:

[0023] S7: Analyze the at least one pop-up window button of the pop-up window using optical character recognition and generate button text for one of the at least one pop-up window buttons.

[0024] The minimum number of endpoints is 4.

[0025] A non-invasive detection device for pop-up window buttons is also disclosed, for identifying at least one pop-up window button of a pop-up window on a display device, characterized in that the non-invasive detection device for pop-up window buttons comprises:

[0026] An image capturing module is used to capture a screen image of the display device;

[0027] An image analysis module is connected to the image acquisition module and stores a preset screen image and an image region threshold. The image analysis module compares the screen image and the preset screen image, and generates a difference image region based on the screen image and the preset screen image. When the difference image region is greater than the image region threshold, the image analysis module determines the difference image region as the pop-up window of the screen image; and

[0028] A calculation module is connected to the image analysis module and has a pre-stored contour length threshold and a number of endpoints. The calculation module uses Canney edge detection to filter out multiple contour lengths in the pop-up window that meet the contour length threshold, and uses the Douglas-Puk algorithm and the number of endpoints to determine the multiple contour lengths to generate a contour edge corresponding to at least one pop-up window button.

[0029] Wherein: when the preset screen image does not exist, the image analysis module replaces the preset screen image with the preset screen image.

[0030] Specifically, the image analysis module subtracts the screen image from the preset screen image to generate a difference image, and calculates the difference image using binarization and image erosion to generate the difference image region.

[0031] Wherein: the calculation module pre-stores a contour correction accuracy, the calculation module uses the Douglas-Puk algorithm and the number of endpoints to determine the length of the multiple contours to generate a button contour that approximates the at least one pop-up window button, and generates a contour edge corresponding to the at least one pop-up window button according to the button contour and the contour correction accuracy, wherein the contour correction accuracy is between 1% and 10% of the contour length.

[0032] The module further includes an optical character recognition module connected to the computing module. The optical character recognition module is used to analyze the at least one pop-up window button of the pop-up window and generate button text in the at least one pop-up window button.

[0033] The minimum number of endpoints is 4.

[0034] In summary, the non-intrusive detection method and apparatus for pop-up window buttons of the present invention can automatically detect pop-up windows of a device and identify pop-up window buttons within them, thereby improving usability. Furthermore, the non-intrusive detection method and apparatus for pop-up window buttons of the present invention can provide and identify complete pop-up window data, enabling operators to provide solutions based on all the data in the pop-up window, thereby improving processing efficiency and reducing labor and production costs. In addition, the non-intrusive detection method and apparatus for pop-up window buttons of the present invention can also effectively identify pop-up windows of display devices through various image processing methods and image recognition algorithms, thereby improving processing efficiency. Attached Figure Description

[0035] Figure 1 A flowchart illustrating the steps of a non-intrusive detection method for a pop-up window button according to a specific embodiment of the present invention is shown.

[0036] Figure 2 A functional block diagram illustrating a non-intrusive detection device with a pop-up window button according to a specific embodiment of the present invention is shown.

[0037] Figure 3A Draw Figure 2 A schematic diagram of the display device.

[0038] Figure 3B Draw Figure 2 The display screen of the display device includes a schematic diagram of a pop-up window.

[0039] Figure 4 Drawing based on Figure 1 The flowchart below shows the further steps of the non-intrusive detection method for pop-up window buttons in a specific embodiment.

[0040] Figure 5 Drawing based on Figure 1 The flowchart below shows the further steps of the non-intrusive detection method for pop-up window buttons in a specific embodiment.

[0041] Figure 6A and Figure 6B Drawing based on Figure 1 A schematic diagram illustrating the non-intrusive detection method for pop-up window buttons in a specific embodiment. Detailed Implementation

[0042] To make the advantages, spirit, and features of the present invention more easily and clearly understood, detailed descriptions and discussions will follow with reference to specific embodiments and the accompanying drawings. It is important to note that these specific embodiments are merely representative examples of the present invention, and the specific methods, apparatuses, conditions, materials, etc., exemplified are not intended to limit the present invention or the corresponding specific embodiments. Furthermore, the devices in the figures are only used to illustrate their relative positions and are not drawn to scale; this is to be stated beforehand.

[0043] In the description of this specification, references to terms such as "one embodiment," "another embodiment," or "part of an embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0044] Please refer to Figure 1 and Figure 2 . Figure 1 A flowchart illustrating the steps of a non-intrusive detection method for a pop-up window button according to a specific embodiment of the present invention is shown. Figure 2 A functional block diagram of a non-intrusive detection device 1 for a pop-up window button according to a specific embodiment of the present invention is shown. Figure 1 Non-invasive detection methods can be derived from Figure 2 The non-intrusive detection device 1 executes the pop-up window button. For example... Figure 2 As shown, the non-invasive detection device 1 for pop-up window buttons includes an image capturing module 11, an image analysis module 12, a calculation module 13, and an optical character recognition module 14. The non-invasive detection device 1 can be connected to the display device 5 of the device to capture and detect the screen image and data of the display device 5. The image analysis module 12 is connected to the image capturing module 11 and stores preset screen images and image area thresholds. The calculation module 13 is connected to the image analysis module 12 and pre-stores contour length thresholds and the number of endpoints. The optical character recognition module 14 is connected to the calculation module 13 and is used to analyze the pop-up window button and generate the button text in the pop-up window button. In practice, the image capturing module 11 can be an image capturing chip, the image analysis module 12 can be an image analysis chip, the computing module 13 can be a central processing chip (CPU), and the optical character recognition module 14 can be an optical character recognition chip. However, it is not limited to these. The image capturing module 11 and the image analysis module 12 can also be integrated into the same image processing chip, and the optical character recognition module 14 can also be other chips that can recognize text.

[0045] like Figure 1 and Figure 2As shown, in this specific embodiment, a non-intrusive detection method for pop-up window buttons is used to identify the pop-up window 51 of the display device 5 and at least one pop-up window button 52 within the pop-up window 51. The non-intrusive detection method for pop-up window buttons includes the following steps:

[0046] Step S1: The image capturing module 11 captures the screen image of the display device 5;

[0047] Step S2: The image analysis module 12 compares the screen image with the preset screen image and generates a difference image area based on the screen image and the preset screen image.

[0048] Step S3: The image analysis module 12 determines whether the difference image region is greater than the image region threshold;

[0049] Step S4: When the difference image area is greater than the image area threshold, the image analysis module 12 determines the difference image area as a pop-up window 51 of the screen image;

[0050] Step S5: Calculate the length of multiple contours in the pop-up window 51 that meet the contour length threshold by the Canny edge detector in module 13;

[0051] Step S6: Calculate the contour length of the module 13 by using the Douglas-Peucker algorithm and the number of endpoints to generate the contour edge of the corresponding pop-up window button 52;

[0052] And step S7: The optical character recognition module 14 analyzes the pop-up window button 52 of the pop-up window 51 using optical character recognition and generates the button text in the pop-up window button 52.

[0053] like Figure 1As shown, in addition to steps S1-S7 described above, the non-intrusive detection method for the pop-up window button in this specific embodiment also includes the following steps: After the image capturing module 11 captures the screen image of the display device 5 (as shown in step S1), in step S11, the image analysis module 12 first determines whether a preset screen image exists. If the preset screen image does not exist, in step S12, the image analysis module 12 replaces the screen image captured by the image capturing module 11 with the preset screen image; if the preset screen image already exists, the image analysis module 12 executes the subsequent step S2. In practice, the image capturing module 11 can capture the screen image of the display device 5 at multiple different times, and the time interval between the image capturing module 11 capturing the screen image can be preset or manually set. The preset screen image can be the screen image of the device during normal operation. When the non-intrusive detection device 1 for the pop-up window button is first started or restarted, the image analysis module 12 may not have or may clear the preset screen image. At this time, the image analysis module 12 uses the screen image captured by the image capturing module 11 for the first time as the preset screen image.

[0054] Furthermore, after the image analysis module 12 in step S12 uses the screen image captured by the image capturing module 11 for the first time as the preset screen image, in step S13, the operator can control whether the non-intrusive detection device 1 with the pop-up window button wants to end the detection. If the detection is to be ended, the non-intrusive detection device 1 with the pop-up window button will stop the detection of the pop-up window of the display device. If the detection is not ended, the process returns to step S1, and the image capturing module 11 captures the screen image of the display device 5 at the next time point.

[0055] Please refer to this as well. Figure 1 , Figure 2 , Figure 3A and Figure 3B . Figure 3A Draw Figure 2 A schematic diagram of display device 5. Figure 3B Draw Figure 2 The display screen of the display device 5 includes a schematic diagram of a pop-up window 51. For example... Figure 1 Step S2 Figure 3A and Figure 3B As shown, after the image analysis module 12 has stored a preset screen image and the image capturing module 11 has captured the screen image of the display device 5, the image analysis module 12 compares the screen image and the preset screen image, and generates a difference image area based on the screen image and the preset screen image. Figure 3A and Figure 3B For example, Figure 3A The screen image of display device 5 when the device is operating normally, and Figure 3B The display device 5 of the equipment displays a pop-up window 51. In practice, Figure 3A It can be a preset screen image stored in the image analysis module 12, and Figure 3B It can capture screen images at other points in time for the image capturing module 11. The image analysis module 12 can compare them. Figure 3A and Figure 3B and generate Figure 3A and Figure 3B The difference image region between them.

[0056] Please refer to Figure 1 , Figure 3B and Figure 4 . Figure 4 Drawing based on Figure 1 A flowchart further outlining the steps of a non-intrusive detection method for pop-up window buttons in a specific embodiment. (See attached flowchart.) Figure 1 and Figure 4 As shown, in this specific embodiment, the non-intrusive detection method for the pop-up window button further includes the following steps, which can be executed by the image analysis module 12 to generate a difference image area between the screen image and the preset screen image. These steps include: Step S21: The image analysis module 12 subtracts the screen image and the preset screen image to generate a difference image; and Step S22: The image analysis module 12 calculates the difference image using binarization and image erosion to generate a difference image area. In practice, the detailed process of step S21 can be that the image analysis module 12 first converts the screen image and the preset screen image into grayscale images, and then deletes screen images with the same grayscale value at the same position using image subtraction. When the display device 5 pops up the pop-up window 51, the pop-up window 51 will cover a portion of the display device 5, and the grayscale value of the area covered by the pop-up window 51 will change. Therefore, when the image analysis module 12 subtracts the screen image from the preset screen image, the grayscale image covered by the pop-up window 51 cannot be deleted, and the image analysis module 12 judges the grayscale image that cannot be deleted as a difference image.

[0057] As described above, in step S22, the image analysis module 12 performs binarization processing on the difference image. In practice, when the image analysis module 12 generates the difference image, each pixel of the difference image contains a grayscale difference. Further, the image analysis module 12 may include a grayscale threshold value, and the image analysis module 12 can highlight the difference image based on the grayscale difference and the grayscale threshold value. In practice, the grayscale threshold value can be determined based on the screen characteristics of the display device 5 or the grayscale value of the difference image. For example, the grayscale threshold value is 45. When the grayscale difference of one pixel in the difference image is less than 45, the image analysis module 12 adjusts the grayscale value of that pixel to 0; and when the grayscale difference of one pixel in the difference image is greater than or equal to 45, the image analysis module 12 adjusts the grayscale value of that pixel to 225. Therefore, the image analysis module 12 can optimize and highlight the difference image through image binarization.

[0058] Furthermore, in step S22, the image analysis module 12 also processes the difference image through image erosion. Image erosion uses structural elements to eliminate small variation areas and noise areas in the difference image. The structural elements can be preset areas or manually determined. In practice, when the equipment is operating, production data or numbers are displayed on the display device 5, and the production data or numbers change as the equipment operates. Therefore, the numbers in the screen image captured by the image capture module 11 may differ from the numbers in the preset screen image. That is, when the image analysis module 12 generates the difference image, the grayscale difference area caused by the change in production data or numbers will become the small variation area and noise area of ​​the difference image. Therefore, the image analysis module 12 then calculates the difference image and generates the difference image area through image erosion.

[0059] Please refer to it again. Figure 1 , Figure 2 and Figure 3B In this specific embodiment, the image analysis module 12 further stores image region thresholds. For example... Figure 1 As shown in step S3, after the difference image area is generated, the image analysis module 12 determines whether the difference image area is greater than the stored image area threshold. In practice, the image area threshold can be the size or area of ​​the pop-up window. The image area threshold can be the smallest size value among all pop-up windows that the display device will display, or it can be set manually. Therefore, when the image analysis module 12 analyzes and determines that the difference image area is greater than the stored image area threshold, it means that the difference image area is not an image difference that will occur during normal operation of the device. Therefore, the image analysis module 12 determines that the difference image area is the pop-up window 51 of the screen image of the display device 5, as shown in step S4.

[0060] On the other hand, if the image analysis module 12 analyzes and determines in step S3 that the difference image area is less than the threshold of the stored image area, it indicates that the display device may only be experiencing string changes rather than a pop-up window appearing. In this case, returning to step S13, the operator can control whether the non-intrusive detection device 1 of the pop-up window button should end the detection. If the detection is to be ended, the non-intrusive detection device 1 of the pop-up window button will stop the detection of the pop-up window on the display device. If the detection is not to be ended, the system returns to step S1, and the image capturing module 11 captures the screen image of the display device 5 at the next time point.

[0061] Please refer to the following: Figure 1 , Figure 2 , Figure 3B , Figure 5 And Figure 6. Figure 5 System drawing based on Figure 1 The flowchart below shows the further steps of the non-intrusive detection method for pop-up window buttons in a specific embodiment. Figure 6A and Figure 6B System drawing based on Figure 1 This is a schematic diagram illustrating the non-intrusive detection method for pop-up window buttons in a specific embodiment, used to identify pop-up window button 52. After the image analysis module 12 identifies the pop-up window 51, the calculation module 13 then further determines and identifies the pop-up window button 52 within the pop-up window 51. Figure 1 In step S5, the calculation module 13 first uses a Canny edge detector to filter out multiple contour lengths in the pop-up window 51 that meet the contour length threshold. In practice, the pop-up window 51 may contain text, button outlines, patterns, graphics, and other data. When the calculation module 13 calculates the pop-up window 51 using the Canny edge detector, the calculation module 13 draws and generates the contour of each data (such as characters, lines, etc.) in the pop-up window 51, and the contour of each data includes a contour length. The contour length can be the length of pixels. The contour length threshold can be the pixel length of the button outline in the pop-up window, and can be determined manually, but is not limited to this. For example, such as Figure 6A As shown, Figure 6A Each line segment A in the diagram represents a contour length that meets or exceeds the contour length threshold. Therefore, if the contour length threshold is 200 pixels, after the calculation module 13 calculates the contour length of all data in the pop-up window 51, the calculation module 13 will filter out the contour lengths in the pop-up window 51 that meet or exceed 200 pixels.

[0062] After the calculation module 13 generates the outline length of all the data in the pop-up window 51, the calculation module 13 then uses the Douglas-Peucker algorithm and the number of endpoints to determine the filtered outline length in order to generate the outline edge of the corresponding pop-up window button 52.

[0063] In this specific embodiment, the non-intrusive detection method for the pop-up window button further includes: step S61: the calculation module 13 uses the Douglas-Peucker algorithm and the number of endpoints to determine the contour length to generate an approximate button contour of the pop-up window button 52; and step S62: the calculation module 13 generates the contour edge of the corresponding pop-up window button 52 based on the button contour and the contour correction accuracy. In practice, the number of endpoints can be the same as the number of endpoints of the pop-up window 51. When the shape of the pop-up window is rectangular, the number of endpoints is 4, but it is not limited to this; the number of endpoints can be determined according to the shape of the pop-up window. Figure 6A As shown, when the calculation module 13 uses the Douglas-Peucker algorithm and sets the number of endpoints to 4 to determine the contour length, the calculation module 13 will find the four endpoints B among all contour lengths. At this time, the contour length in the rectangle enclosed by the four endpoints B forms an approximate button contour of the pop-up window button. Next, the calculation module 13 generates the contour edge of the corresponding pop-up window button 52 based on the button contour and the contour correction accuracy. The contour correction accuracy can be the distance dimension (ε) in the Douglas-Peucker algorithm, and the contour correction accuracy can be between 1% and 10% of the contour length, but is not limited to this. The contour correction accuracy can also be preset or determined manually. Figure 6B As shown, when the calculation module 13 calculates the button contour based on the contour correction accuracy, the calculation module 13 will find the line segment with the maximum distance based on endpoint B and delete the excess contour length (as shown by the dotted line in the figure), and then determine and generate the contour edge C of the corresponding pop-up window button. Please note that in this specific embodiment, Figure 3B The pop-up window 51 contains two pop-up window buttons 52 and 53, but in practice it is not limited to this, and the number of pop-up window buttons can also be one or more.

[0064] In summary, as Figures 1 to 5 As shown, the non-intrusive detection method for the pop-up window button of the present invention first captures screen images of the display device 5 at different points in time through the image capturing module 11, and then compares them with preset screen images (such as...) through the image analysis module 12. Figure 3A (as shown) and screen images (such as) Figure 3BAs shown in the figure, the image is further simplified and a difference image region is generated through image subtraction, binarization, and image erosion to determine the pop-up window 51 in the display device 5. Furthermore, the calculation module 13 calculates and identifies the pop-up button 52 of the pop-up window 51 using the Canny edge detector and the Douglas-Peucker algorithm, respectively. Finally, the optical character recognition module 14 identifies all the text in the pop-up window 51. Therefore, the non-intrusive detection method for the pop-up button of the present invention can automatically detect the pop-up window in the screen image of the display device and identify all the data of the pop-up button using the optical character recognition module.

[0065] In summary, the non-intrusive detection method and apparatus for pop-up window buttons of the present invention can automatically detect pop-up windows of a device and identify pop-up window buttons within them, thereby improving usability. Furthermore, the non-intrusive detection method and apparatus for pop-up window buttons of the present invention can provide and identify complete pop-up window data, enabling operators to provide solutions based on all the data in the pop-up window, thereby improving processing efficiency and reducing labor and production costs. In addition, the non-intrusive detection method and apparatus for pop-up window buttons of the present invention can also effectively identify pop-up windows of display devices through various image processing methods and image recognition algorithms, thereby improving processing efficiency.

[0066] The detailed description of the preferred embodiments above is intended to more clearly illustrate the features and spirit of the present invention, and is not intended to limit the scope of the invention to the preferred embodiments disclosed above. Rather, the aim is to cover various modifications and equivalent arrangements within the scope of the patent claims made by this invention. Therefore, the scope of the patent claims made by this invention should be interpreted in the broadest possible sense based on the foregoing description, so as to cover all possible modifications and equivalent arrangements.

Claims

1. A non-intrusive detection method for a pop-up window button, used to identify at least one pop-up window button of a pop-up window on a display device, characterized in that, The non-intrusive detection method for the pop-up window button includes the following steps: S1: Capture an image from the display device; S21: Subtract the screen image from a preset screen image to generate a difference image; S22: Calculate the difference image using binarization and image erosion to generate a difference image region; S3: Determine whether the difference in the image region is greater than a threshold for an image region; S4: When the difference image area is greater than a threshold of an image area, the difference image area is determined as the pop-up window of the screen image; S5: In this pop-up window, first preset a contour length threshold, then use Canney edge detection to filter out multiple contour lengths that are greater than the contour length threshold; and S6: First, preset the number of endpoints of the at least one pop-up window button, and then use the Douglas-Puk algorithm and the number of endpoints to determine the length of the multiple contours to generate a contour edge corresponding to the at least one pop-up window button.

2. The non-intrusive detection method for pop-up window buttons as described in claim 1, characterized in that, Further steps include: S11: Determine if the preset screen image exists. If the determination result is yes, proceed to step S2; and S12: If the judgment result of step S11 is negative, replace the screen image with the preset screen image.

3. The non-intrusive detection method for pop-up window buttons as described in claim 1, characterized in that, Step S6 further includes the following steps: S61: Using the Douglas-Puk algorithm and the number of endpoints, determine the lengths of the multiple contours to generate a button contour that approximates the at least one pop-up window button; and S62: Generate the outline edge corresponding to at least one pop-up window button based on the button outline and an outline correction accuracy, wherein the outline correction accuracy is between 1% and 10% of the outline length.

4. The non-intrusive detection method for pop-up window buttons as described in claim 1, characterized in that, Further steps include: S7: Analyze the at least one pop-up window button of the pop-up window using optical character recognition and generate button text for one of the at least one pop-up window buttons.

5. The non-intrusive detection method for pop-up window buttons as described in claim 1, characterized in that, The minimum number of endpoints is 4.

6. A non-intrusive detection device for a pop-up window button, used to identify at least one pop-up window button of a pop-up window on a display device, characterized in that, The non-intrusive detection device for the pop-up window button includes: An image capturing module is used to capture a screen image of the display device; An image analysis module is connected to the image acquisition module and stores a preset screen image and an image region threshold. The image analysis module subtracts the screen image from the preset screen image to generate a difference image, and calculates the difference image using binarization and image erosion to generate a difference image region. When the difference image region is greater than the image region threshold, the image analysis module determines the difference image region as the pop-up window of the screen image; and A calculation module is connected to the image analysis module and has a pre-stored contour length threshold and a number of endpoints. The calculation module uses Canney edge detection to filter out multiple contour lengths in the pop-up window that meet the contour length threshold, and uses the Douglas-Puk algorithm and the number of endpoints to determine the multiple contour lengths to generate a contour edge corresponding to at least one pop-up window button.

7. The non-intrusive detection device for a pop-up window button as described in claim 6, characterized in that, When the preset screen image does not exist, the image analysis module replaces the preset screen image with the original screen image.

8. The non-intrusive detection device for a pop-up window button as described in claim 6, characterized in that, The calculation module pre-stores a contour correction accuracy. The calculation module uses the Douglas-Puk algorithm and the number of endpoints to determine the length of the multiple contours to generate a button contour that approximates the at least one pop-up window button. Based on the button contour and the contour correction accuracy, a contour edge corresponding to the at least one pop-up window button is generated, wherein the contour correction accuracy is between 1% and 10% of the contour length.

9. The non-intrusive detection device for a pop-up window button as described in claim 6, characterized in that, It further includes an optical character recognition module connected to the computing module, the optical character recognition module being used to analyze the at least one pop-up window button of the pop-up window and generate a button text among the at least one pop-up window button.

10. The non-intrusive detection device for a pop-up window button as described in claim 6, characterized in that, The minimum number of endpoints is 4.

Images