Non-intrusive screen monitoring image-text dynamic recognition system and method

Abstract

The invention relates to the technical field of industrial equipment control, in particular to a non-intrusive screen monitoring image-text dynamic recognition system and method, the system comprises a region configuration module, a visual perception module, an intelligent decision module and a control execution module, and based on the system, the recognition method comprises the steps of determining a to-be-detected screen region in response to a configuration operation of a user; and identifying the preprocessed image through a screen self-capture technology running in a host operating system, outputting an instruction when an identification result is abnormal, and controlling the industrial equipment to start and stop. Equipment state information is obtained through a non-intrusive mode of a visual level, a non-intrusive relay series mechanism of a control level is adopted, and state monitoring and closed-loop control of an old industrial equipment man-machine interface can be achieved on the premise that hardware is not transformed and an internal communication protocol of the equipment is not accessed. The industrial problems of'interface false alarm 'and'out-of-control execution' are solved, and the method has the advantages of high compatibility, low cost and fast deployment.

Description

Technical Field

[0001] This invention relates to the field of industrial equipment control technology, and in particular to a non-invasive screen monitoring system and method for dynamic image and text recognition. Background Technology

[0002] In the industrial manufacturing sector, a large number of aging production equipment still in service undertake core production tasks, such as imported injection molding machines, stamping machines, and welding line equipment. These machines generally suffer from an inherent flaw: a disconnect between the human-machine interface (HMI) and the underlying execution system. Their front-end operations typically only possess basic monitoring functions, unable to provide real-time, accurate feedback on equipment status, and lacking the ability to proactively intervene in abnormal situations. For example, when the front-end interface displays "Equipment connection lost," the underlying equipment may still be executing current commands, leading to production out of control, material waste, or even safety accidents.

[0003] Currently, there are two main solutions in the industry: one is to upgrade the device's native control software to rebuild two-way communication. However, this method is highly dependent on the original equipment manufacturer and often fails to obtain the underlying protocol due to hardware closures or technological blockades, resulting in high modification costs and a lack of standardized reuse. The second solution is to add external sensors (such as photoelectric sensors and temperature sensors) to the device to indirectly infer the device's status. While this method avoids software interface issues, it involves modifying the device structure and may lead to the risk of security authentication failure.

[0004] To address the problems in existing technologies, this invention provides a non-invasive screen monitoring and image / text dynamic recognition system and method. Summary of the Invention

[0005] The purpose of this invention is to provide a non-invasive screen monitoring and image / text dynamic recognition system and method to solve the problem of the disconnect between "false alarms" and "execution failure" in existing equipment.

[0006] The technical solution of this invention is: a non-invasive screen monitoring and image / text dynamic recognition system, comprising: Region configuration module: used by users to configure the screen region to be detected, as well as the normal feature library and abnormal feature library corresponding to the screen region to be detected; Visual perception module: used to perform self-grabbing technology and image preprocessing of screen areas; Intelligent decision-making module: Recognizes the image information output by the visual perception module, and performs feature matching between the recognition results and the configured normal feature library and abnormal feature library to determine the operating status of industrial equipment; Control execution module: includes a signal conversion device and a signal relay; the signal relay is connected in series with the signal sensor of the industrial equipment; the signal relay receives a control command output by the host of the industrial equipment and drives the signal relay to operate based on the command, so as to control the signal on and off of the signal sensor, thereby controlling the operating status of the industrial equipment.

[0007] Preferably, the system is compatible with Windows XP and later Windows operating systems, and is matched with the Tesseract OCR engine for text recognition.

[0008] Preferably, the signal conversion device includes a PLC controller, a touch screen, a power supply, a USB to RS485 connector, and a buzzer; The PLC controller receives another control command output from the industrial equipment host and visualizes the real-time status of the industrial equipment host through a touch screen.

[0009] Preferably, the system software environment is compatible with one of the following Python versions: Python 2.7, Python 3.3, and Python 3.4, and integrates an algorithm library that is compatible with the corresponding Python version.

[0010] A non-invasive screen monitoring and image / text dynamic recognition method, applied to the aforementioned non-invasive screen monitoring and image / text dynamic recognition system, includes: Responding to the user's configuration operation, determine the screen area to be detected; Based on the determined screen area, the target area image is captured using screen capture technology running on the host operating system. By preprocessing the captured target area image, interference is removed to obtain the preprocessed image; Identify the text and image information of the processed image; The identification results are matched with a pre-determined database of normal and abnormal features. When the identification result successfully matches the abnormal state characteristics, a control command is generated and output to the outside through the host's communication interface. The operating status of the industrial equipment is controlled by connecting the signal sensors of the industrial equipment in series with an external non-intrusive relay.

[0011] Preferably, determining the screen area to be detected includes: using a method developed based on AutoIt to confirm screen pixel coordinates, with the position of the mouse tip as a reference, to acquire and determine the screen area to be detected; the screen area to be detected displays the device's working status.

[0012] Preferably, when the recognition confidence of the optical character recognition technology is lower than a preset threshold, a fuzzy matching algorithm is used to match the recognition result with a pre-determined normal feature library and abnormal feature library.

[0013] Preferably, the preprocessing operation includes at least grayscale processing and adaptive threshold binarization processing.

[0014] An electronic device includes a memory and a processor, the processor and the memory communicating with each other, the memory storing at least one instruction or at least one program executable by the processor, the at least one instruction or at least one program being loaded and executed by the processor to implement the non-intrusive screen monitoring and dynamic text recognition method.

[0015] A computer-readable storage medium includes: at least one instruction or at least one program stored in the storage medium, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the non-intrusive screen monitoring and dynamic text recognition method.

[0016] Compared with the prior art, the advantages of the present invention are: (1) By using highly compatible technologies such as AutoIt and an OCR engine adapted to older systems, the problem of deploying modern recognition technology on older systems such as Windows XP has been solved. The transformation cycle of a single device can be shortened from several weeks to less than 2 hours, and the cost is reduced by more than 90%.

[0017] (2) It adopts optical character recognition technology to directly identify complex text alarm information, and combines it with a fuzzy matching mechanism to overcome the shortcomings of physical sensors with limited functions. The accuracy of abnormal state recognition in typical industrial environments is as high as 99%.

[0018] (3) By intelligently analyzing the visual information of the HMI interface, a closed-loop control link between the operation front end and the execution device is constructed, fundamentally solving the problem of the disconnect between "false alarm of interface" and "runaway execution" of old equipment.

[0019] This invention is used for status monitoring and intelligent control of human-machine interfaces of old industrial equipment that lack open communication interfaces. Through the link of "visual recognition - signal conversion - equipment control", it acquires information through vision without the need for original manufacturer protocols or hardware modifications, thus completely avoiding compatibility issues and security authentication risks. It achieves intelligent closed-loop control of old equipment without modifying hardware or relying on underlying interfaces, with equipment response latency of less than 200ms, eliminating the risk of execution failure at the source. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the non-invasive screen monitoring and dynamic image recognition method described in this invention; Figure 2 This is a general principle block diagram of the non-invasive screen monitoring and image / text dynamic recognition system described in this invention; Figure 3 This is a partial hardware connection diagram of the non-intrusive screen monitoring and image / text dynamic recognition system described in this invention in a specific implementation scenario.

[0021] The module consists of: 1. Regional configuration module; 2. Visual perception module; 3. Intelligent decision-making module; and 4. Control execution module. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to specific embodiments: like Figure 1 As shown, a non-invasive screen monitoring and dynamic text recognition method is proposed. This method is implemented based on a hardware-configured recognition system, which includes: The region configuration module 1 is used by the user to configure the engine, determine the screen (pixel) region to be detected, and construct a feature library based on historical records; the visual perception module 2 is used to acquire images of specified pixel regions through screen self-capture technology and perform preprocessing; the intelligent decision module 3 is used to identify text or graphics in the preprocessed image and perform feature matching with a preset state feature library to determine the operating status of the industrial equipment; the control execution module 4 is used to output control signals to the industrial equipment through signal relays when the matching result is an abnormal state.

[0023] The system provided by this invention can realize the status monitoring and closed-loop control of the human-machine interface of old industrial equipment without modifying the hardware or relying on the underlying interface of the equipment. It solves the industry problems of "false alarms" and "execution failure" and has the advantages of high compatibility, low cost and fast deployment.

[0024] Computer equipment is updated and iterated rapidly, but in industrial production activities, the industrial equipment put into production is still the first generation or second generation (industrial control equipment from 2000 to 2005). In this invention, "old equipment" refers to the first generation or second generation (industrial control equipment from 2000 to 2005).

[0025] Based on a non-intrusive screen monitoring and image / text dynamic recognition system, and referring to its system principle block diagram, see attached... Figure 2 As shown, the specific steps of this non-invasive screen monitoring and dynamic text recognition method include: Configuration Phase: The user starts the region configuration engine, which monitors the screen area under test through the front-end human-computer interaction layer. It uses a coordinate tool developed based on AutoIt, which is compatible with most Windows operating systems. AutoIt has good system compatibility and stable operation. Running this pixel recognition tool, when the mouse is moved across the screen of the human-computer interaction layer, the pixel coordinates of the mouse tip will be displayed on the screen, and the range of pixel coordinates corresponding to the monitored screen area will be recorded.

[0026] The system is compatible with Windows XP as the minimum operating system version, and is also compatible with later Windows operating systems, such as Windows 7 or Windows 11. The language interpreter running on the system uses Python, including one of Python 2.7, Python 3.3, or Python 3.4, and integrates algorithm libraries compatible with the selected Python version.

[0027] Based on historical records, a normal feature library and an abnormal feature library for the region to be tested are constructed, and the feature libraries are configured during the configuration phase.

[0028] The area under test contains information reflecting the operating status of industrial equipment, including fonts and graphics. For example, an alarm information display box in abnormal conditions.

[0029] Normal characteristics are characterized by "running" and "ready"; abnormal characteristics are characterized by "fault" and "connection lost".

[0030] Monitoring Phase: The visual perception module 2 operates, detecting the designated test area. It calls the operating system API (such as Windows' GDI) at a set frequency (e.g., 1Hz) to capture the screen of the test area and preprocesses the captured image. This includes converting the captured image to grayscale using BGR, followed by binarization using an adaptive thresholding algorithm to remove background interference, highlight key graphics, enhance text contrast, and improve recognition accuracy. The processed image is then sent to the intelligent decision-making module 3.

[0031] The pixel range of the screen capture can be modified at will according to specific circumstances. That is, the area of ​​the screen (pixels) to be monitored can be arbitrarily defined, providing a feasibility guarantee for the standardized application of engines and application tools.

[0032] Information can be acquired visually without requiring original manufacturer protocols or hardware modifications, thus avoiding compatibility issues and security authentication risks associated with older equipment.

[0033] Decision-making phase: A trained database of languages ​​and characters from various countries, along with a customizable set of states, combined with OCR (Optical Character Recognition) technology, can handle various complex front-end production interfaces. The intelligent decision-making module 3 calls upon a Tesseract OCR engine adapted for Windows XP (e.g., the Tesseract 3.05 OCR engine) to perform numerical or font-based text recognition, or to perform image recognition using template matching algorithms. The recognition results are then matched with preset normal and abnormal feature libraries to determine the operating status of industrial equipment.

[0034] By running the reliable AutoIt algorithm tool on older Windows XP systems, combined with OCR engine versions and algorithm libraries that can run on older systems, a balance is achieved between performance, compatibility, and cost, solving practical pain points in the industrial field.

[0035] When the content to be identified is not complex, the identification results of the intelligent module can be directly added to the corresponding state set to make it reasonable. However, in more complex cases, a fuzzy matching mechanism can be added to the state set to address the issue of low accuracy when selecting modules adapted to older systems, and to improve robustness when identification accuracy is insufficient on older systems. For example, when the identification result is "connection broken" (65% confidence), the string similarity (e.g., edit distance) to "connection broken" is calculated; if a match is found, it is still considered an abnormal state.

[0036] Execution phase: If the matching result is consistent with the normal state database, the current state of the device is maintained.

[0037] If the matching result matches the abnormal status database, the host sends a signal to the signal relay to switch the signal relay status. This allows for the connection of external alarm devices such as buzzers and indicator lights, or direct connection to production equipment to control the start and stop of the production equipment.

[0038] For example, based on the above method, a specific implementation method for the corresponding execution stage is provided: See attached diagram for a partial hardware structure diagram of the system. Figure 3 As shown, it includes: an industrial equipment host that runs monitoring software, an industrial equipment HMI screen that is being monitored, and a control execution module 4.

[0039] The control execution module 4 includes a signal conversion device and a signal relay. The signal conversion device is powered by an external power supply and includes a USB to RS485 adapter, a PLC controller, a touch screen, a control switch, and a buzzer or indicator light.

[0040] The signal relay is equipped with two channels, one normally open and one normally closed, to meet different needs. Specifically, the signal relay is connected in series with the signal sensor of the industrial equipment. The signal relay receives control commands output by the host of the industrial equipment and drives the signal relay to operate based on the commands, thereby controlling the signal on and off of the signal sensor and indirectly controlling the start and stop of the industrial equipment.

[0041] The system runs on the Windows XP operating system. To implement this invention on the Windows XP system, the following software configuration is adopted as a preferred embodiment: Operating system: Windows XP SP3; programming language environment: Python interpreter version 3.4.4, with the following algorithm libraries: NumPy 1.15.4: Used for image array processing; OpenCV 3.2.0.8: Used for image grayscale conversion, binarization, and adaptive thresholding. pytesseract 0.3.7: The Python interface for Tesseract OCR; Pillow 4.2.1: Used for image format processing and screen capture support; pyserial 3.4: Used for serial communication and control signal relays; pywin32-218: Used to call the Windows GDI interface to implement screen capture; Tesseract OCR 3.05.01 is compatible with Windows XP and supports Chinese, English, and number recognition; it is a pixel coordinate positioning tool based on AutoIt v3.

[0042] In this environment, the PLC receives control commands output from the industrial equipment host and visualizes the real-time status of the industrial equipment host through a touchscreen. Specifically, in this embodiment, the system uses pywin32 to call the GDI interface to capture a specified screen area, uses OpenCV for image preprocessing, calls Tesseract 3.05.01 for text recognition, and sends commands to the USB-to-RS485 converter via pyserial to drive signal relays to execute equipment control.

[0043] Specifically, under normal operating conditions, signals are output externally via the host's USB interface or RS485 interface. The signal relays have built-in power supplies. After receiving signals from the industrial equipment's host, the signal relays logically control the opening and closing of two signal relays. For example, one signal relay controls the on / off state of an existing signal sensor (a signal that affects the equipment's start and stop) in series with the industrial equipment, indirectly controlling the operation and shutdown of the industrial equipment without altering its basic electrical circuit structure. The other signal relay connects to alarm devices such as indicator lights and buzzers.

[0044] When an abnormal operating status of industrial equipment is detected, the program sends a switching signal to the signal relays via the host's USB interface and a USB-to-RS485 converter. Upon receiving the signal, the normally closed contact of one signal relay opens, thereby cutting off the main circuit of the industrial equipment and achieving an emergency shutdown. The other signal relay triggers a buzzer to sound an audible and visual alarm.

[0045] This invention provides a non-invasive screen monitoring and image / text dynamic recognition system and method, whose "non-invasive" characteristic is reflected in two aspects: Information acquisition level: Through screen self-capture technology running on the host operating system, without intruding on the device's internal system or parsing the underlying communication protocol, device status information is obtained only from the visual level.

[0046] Control execution level: By connecting the signal relay in series to the equipment control loop, the start and stop control of external execution equipment can be realized without modifying the internal circuit or program of the equipment, thus avoiding the impact on the safety certification or structural integrity of the original equipment.

[0047] In summary, this invention enables the system to lock key visual information without parsing the underlying device protocol through screen self-grabbing operation, overcoming the compatibility barrier of closed interfaces of old devices and reducing adaptation costs by more than 90%. Intelligent feature recognition and matching (OCR recognition) enables accurate semantic parsing of HMI interface, directly solving the defect that physical sensors cannot recognize graphic and text alarms. The accuracy of abnormal state recognition in typical generated environments is as high as 99%.

[0048] By connecting signal relays in series with signal sensors in industrial equipment to control their on / off states, a closed-loop link of "visual recognition - equipment control" is formed without changing the overall structure of the original equipment. The feasibility and compatibility of this system on old operating systems enable low-cost, high-response closed-loop control, eliminating the risk of execution loss of control caused by "false alarms on the interface", and the equipment response delay is less than 200ms.

[0049] This invention achieves non-intrusive closed-loop control through the direct coupling of visual recognition and signal relays. Specifically, it utilizes the synergistic effects of screen self-grabbing, intelligent feature recognition and matching, and serial connection of execution devices to realize intelligent closed-loop control of aging industrial equipment without modifying the original hardware structure or relying on equipment interfaces. The transformation cycle for a single device is shortened from weeks or months to 2 hours, forming a reusable standardized solution. It provides a standardized, low-cost solution to the challenges of transforming aging equipment, aligning with the development trends of the Industrial Internet and intelligent manufacturing.

[0050] This invention is applicable to the following scenarios: Upgrading of closed-loop industrial control systems: Applied to older PLC and HMI systems that cannot obtain communication protocols. Examples include automotive welding lines and injection molding machine control consoles.

[0051] Multi-brand equipment coordination control: Solving the problem of coordinating the status of equipment from different eras and brands in a production line. For example, the linkage control between reactors and conveyor belts in a chemical production process.

[0052] Safety monitoring of high-risk equipment: Achieve visual-level emergency braking for equipment without safety interlock devices. For example, in the use of stamping presses and high-temperature kilns, the heating power supply will be cut off immediately if an "over-temperature alarm" is triggered.

[0053] Traditional equipment modification solutions are limited to specific hardware interfaces or communication protocols, while this invention applies a configurable visual rule engine to the field of industrial equipment control, achieving cross-generational compatibility, cross-industry migration, and cross-functional expansion in one go.

[0054] This invention also provides an electronic device, which includes a processor and a memory; the memory stores one or more instructions, which are adapted for the processor to load and execute, to implement a non-intrusive screen monitoring and dynamic text recognition method as described in the above method embodiments.

[0055] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory includes a program storage area and a data storage area. The program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. Furthermore, memory can include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory may also include a memory controller to provide the processor with access to the memory.

[0056] The internal structure of the electronic device provided in the embodiments of the present invention may include, but is not limited to, a processor, a memory, and a communication interface. The processor, memory, and communication interface in the electronic device may be connected by a bus or other means. In the embodiments of this specification, a connection via a bus is taken as an example.

[0057] The processor (or CPU, Central Processing Unit) is the computing and control core of the electronic device. A communication interface is used for communication between the memory and the processor. The memory stores programs and data. It is understood that the memory here can be a high-speed RAM storage device, or a non-volatile memory device, such as at least one disk storage device; optionally, it can also be at least one storage device located remotely from the aforementioned processor. The memory provides storage space, which stores the operating system of the electronic device, including but not limited to: Windows (an operating system), Linux (an operating system), etc., which are not limited in this invention; and the storage space also stores computer programs (including program code) suitable for being loaded and executed by the processor. In the embodiments of this specification, the processor loads and executes the computer program stored in the memory to implement the non-intrusive screen monitoring and dynamic text recognition method provided in the above method embodiments.

[0058] This invention also provides a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction, at least one program, code set, or instruction set related to the non-intrusive screen monitoring and dynamic text recognition method in the method embodiments. The at least one instruction, at least one program, code set, or instruction set can be loaded and executed by the processor of the electronic device to implement the non-intrusive screen monitoring and dynamic text recognition method provided in the above method embodiments.

[0059] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0060] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments, while other embodiments fall within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than those shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0062] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0063] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A non-intrusive screen monitoring system for dynamic recognition of graphics and text, characterized in that, The system comprises: a region configuration module for a user to configure a screen region to be detected, and a normal feature library and an abnormal feature library corresponding to the screen region to be detected; a visual perception module for performing a self-capture technique of the screen region and image preprocessing; an intelligent decision module for identifying image information output by the visual perception module, and performing feature matching between an identification result and the configured normal feature library and abnormal feature library to determine a running state of the industrial equipment; a control execution module comprising a signal conversion device and a signal relay; the signal relay is connected in series with a signal sensor of the industrial equipment; the signal relay receives a control instruction output by an industrial equipment host, and drives the signal relay to act based on the instruction to control signal on-off of the signal sensor, thereby controlling the running state of the industrial equipment.

2. A non-intrusive screen monitoring graphics and dynamic text recognition system according to claim 1, characterized in that, The system is applicable to Windows XP and Windows series operating systems after Windows XP, and matches a Tesseract OCR engine for character recognition.

3. The non-intrusive screen monitoring graphics dynamic identification system according to claim 1, wherein, The signal conversion device comprises a PLC controller, a touch screen, a power supply, a USB-to-RS485 connector, and a buzzer; the PLC controller receives a control instruction output by an industrial equipment host, and visualizes a real-time state of the industrial equipment host through the touch screen.

4. The non-intrusive screen monitoring graphics dynamic identification system according to claim 2, wherein, The Python version that can be adapted to run in the software environment of the system includes one of Python 2.7, Python 3.3, and Python 3.4, and an algorithm library corresponding to the Python version is integrated.

5. A non-invasive screen monitoring graphics and text dynamic identification method applied to a non-invasive screen monitoring graphics and text dynamic identification system according to any one of claims 1-4, characterized in that, The system comprises: in response to a configuration operation of a user, determining a screen region to be detected; based on the determined screen region, intercepting a target region image through a screen self-capture technique running on a host operating system; through preprocessing of the intercepted target region image, removing interference to obtain a preprocessed image; identifying character information and image information of the preprocessed image; performing feature matching between an identification result and a pre-determined normal feature library and abnormal feature library; when the identification result is successfully matched with the abnormal state, generating a control instruction and outputting the control instruction to the outside through a communication interface of the host, and controlling a running state of the industrial equipment through a non-invasive relay connected in series with a signal sensor of the industrial equipment.

6. The non-intrusive screen monitoring graphics and video dynamic identification method according to claim 5, characterized in that, Determining a screen region to be detected comprises: based on a method for confirming a screen pixel coordinate developed by AutoIt, taking a position of a mouse tip as a reference, acquiring and determining the screen region to be detected; and the screen region to be detected displays a device working state.

7. The non-intrusive screen monitoring graphics and video dynamic identification method according to claim 6, characterized in that, When a recognition confidence of the optical character recognition technology is lower than a pre-set threshold, a fuzzy matching algorithm is used to perform feature matching between the identification result and the pre-determined normal feature library and abnormal feature library.

8. The non-intrusive screen monitoring graphics and video dynamic identification method according to claim 5, characterized in that, The preprocessing operation at least comprises grayscale processing and adaptive threshold binarization processing.

9. An electronic device, comprising: The device comprises a memory and a processor, the processor and the memory communicate with each other, the memory stores at least one instruction or at least one program which can be executed by the processor, the at least one instruction or the at least one program is loaded and executed by the processor to realize the non-invasive screen monitoring graphic dynamic identification method as claimed in any one of claims 5-8.

10. A computer-readable storage medium, characterized in that, Comprise: The storage medium stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to realize the non-invasive screen monitoring graphic dynamic identification method as claimed in any one of claims 5-8.

Images