Multi-modal screen monitoring alarm method and system based on pixel difference detection

Abstract

The invention discloses a multi-mode screen monitoring and alarming method and system based on pixel difference detection, and the method comprises the steps: collecting the coordinates of a monitoring region selected by a user through a full-screen transparent top window through the dragging of a mouse, and receiving monitoring, alarming and automatic parameter configuration; after configuration verification is completed, an independent monitoring thread is started, and a monitoring area screen image is periodically intercepted; converting a front frame image and a rear frame image into a NumPy matrix, calculating an inter-frame pixel change quantized value through vectorization operation, and judging an alarm triggering condition in combination with a preset threshold value; when an alarm event is detected, calling a unified interface to trigger multi-mode alarms such as sound, pop-up windows, mails and the like, and synchronously starting an independent thread to keep an alarm screenshot; when it is detected that there is no effective pixel change within a preset duration, triggering an overtime alarm and starting a shutdown countdown thread, and if there is no change after countdown, executing system shutdown. According to the invention, on the premise of low system resource occupation, the accuracy, the real-time performance, the reliability and the function integrity of monitoring are synchronously realized.

Description

Technical Field

[0001] This invention relates to the field of computer image processing and screen monitoring technology, specifically to a multimodal screen monitoring alarm method and system based on pixel difference detection. Background Technology

[0002] With the widespread adoption of scenarios such as industrial supervisory control and data acquisition (SCADA) monitoring, real-time financial market monitoring, automated program operation and maintenance, and unattended task progress tracking, the industry has put forward an urgent need for all-time accurate monitoring of designated areas on the screen, real-time alarms for abnormal events, and automated handling. The detection accuracy, operating efficiency, alarm reliability, and functional completeness of screen monitoring and alarm technology directly determine the effectiveness of the monitoring scenario implementation.

[0003] Current mainstream screen monitoring solutions have many technical shortcomings. Traditional manual monitoring is costly in terms of manpower, and long-term monitoring can easily lead to missed detections and misjudgments due to visual fatigue. It is also unsuitable for unattended scenarios such as nighttime and holidays, and lacks standardized event recording and backtracking mechanisms, resulting in poor traceability.

[0004] Conventional screen capture and recording tools can only save screen content after the fact, and do not have the ability to detect and alarm real-time changes. They still require manual review and comparison. Full-screen recording mode consumes a lot of system resources, cannot achieve accurate monitoring of a specified area, and is prone to generating a lot of invalid data. At the same time, it lacks supporting automated processing functions and cannot meet the core needs of unattended scenarios.

[0005] Existing simple monitoring programs based on pixel comparison, while achieving basic automated change detection, still have significant shortcomings: First, the flexibility of monitoring area selection is insufficient, often requiring manual input of coordinates and lacking a visual drag-and-drop selection function, resulting in a high operational threshold; second, the pixel detection algorithm has inherent defects, failing to handle pixel value calculation overflow issues and lacking a scientific threshold filtering mechanism, leading to a high false alarm rate and low detection efficiency; third, the alarm methods are limited, mostly supporting only local pop-ups and sound alerts, lacking reliable remote notification capabilities, resulting in insufficient alarm reach; fourth, there is a lack of unattended automated handling functions such as no-change timeout detection and automatic shutdown; fifth, most adopt a single-threaded architecture, with monitoring logic and UI interface operation being coupled, easily causing interface lag and unresponsiveness; sixth, there is no comprehensive configuration persistence and log traceability mechanism, resulting in poor scenario adaptability.

[0006] In addition, professional-grade screen monitoring systems for industrial scenarios require dedicated hardware acquisition equipment, which results in high deployment and usage costs, poor versatility, and can only be adapted to specific industrial equipment and scenarios, failing to meet the lightweight monitoring needs of ordinary office and personal users.

[0007] In summary, existing screen monitoring technologies cannot simultaneously balance detection accuracy, operational efficiency, resource consumption, functional completeness, and ease of operation. They generally suffer from core pain points such as high false alarm rates, limited alarm methods, poor adaptability to unattended operation, and weak traceability, making it difficult to fully meet the screen monitoring and alarm needs of various scenarios. Summary of the Invention

[0008] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention provides a multimodal screen monitoring and alarm method and system based on pixel difference detection, to solve the core technical pain points commonly found in existing technologies, such as insufficient monitoring accuracy, high false alarm rate, single alarm mode, poor adaptability to unattended operation, weak system stability, and insufficient ease of operation and scenario versatility. Under the premise of low system resource consumption, it simultaneously achieves monitoring accuracy, real-time performance, reliability, and functional completeness, adapting to the screen monitoring and alarm needs of various scenarios, especially unattended scenarios.

[0009] To solve the above problems, the technical solution adopted by the present invention is as follows: A multimodal screen monitoring alarm method based on pixel difference detection includes the following steps: S1. Monitoring Area Configuration: Generates a full-screen transparent top-mounted overlay window, which collects the coordinates of the monitoring area selected by the user by dragging and dropping with the mouse, and receives the monitoring, alarm and automation parameters configured by the user. S2. Monitoring Initialization: Verify the validity of the configuration, create a directory for storing logs and screenshots, initialize the monitoring status flags, and start an independent monitoring thread; S3. Real-time monitoring and detection: The monitoring thread periodically captures screen images of the monitoring area, converts the two consecutive frames into NumPy matrices, calculates the quantized value of pixel changes between frames through vectorized matrix operations, and determines whether an alarm event is triggered by combining the preset threshold. S4. Multimodal Alarm: When an alarm event is detected, the unified alarm interface is called to trigger one or more alarm actions from sound, pop-up, and email according to the user configuration, and an independent screenshot thread is started simultaneously to save the alarm screenshot. S5. Unattended automated processing: When no valid pixel change is detected within a preset time period, a timeout alarm is triggered, and an independent shutdown countdown thread is started. The thread continuously detects screen changes during the countdown period. If no valid change is detected when the countdown ends, the system is shut down.

[0010] Preferably, in step S3, the method for calculating the inter-frame pixel difference is as follows: convert the two consecutive frames of PIL.Image format into NumPy matrices of type int16, calculate the absolute difference between the two matrices, sum all elements of the difference matrix to obtain the pixel change quantization value, and filter out false alarms caused by tiny pixel jitters through a preset threshold.

[0011] Preferably, in step S5, the execution logic of the shutdown countdown thread is as follows: save the reference screenshot at the time of shutdown and startup, periodically capture the current screen image and compare it with the reference screenshot; if a pixel change exceeding the threshold is detected, cancel the shutdown and resume normal monitoring; if no effective change is detected after the countdown ends, execute the system shutdown command.

[0012] Preferably, the email alarm action in step S4 has a built-in anti-duplicate sending mechanism. By constructing a unique identifier containing the recipient and email subject, it determines whether the same type of alarm email has been sent. If it has been sent, the sending operation is skipped. The method also includes a configuration persistence step: serializing all user-configured parameters into JSON format strings and saving them to a local file; when the program starts, it automatically reads and parses the configuration file to restore the user's historical configuration.

[0013] A multimodal screen monitoring and alarm system based on pixel difference detection that implements the above method includes a screen capture unit and an alarm notification unit, and further includes: The user interaction module is used to generate a full-screen transparent top-mounted overlay window, which enables the visual selection and coordinate acquisition of the monitoring area through mouse dragging, and also provides a parameter configuration interface and a running status display interface. The multi-threaded management module communicates and connects with the other modules and units respectively, and is used to independently run the monitoring thread, the screenshot saving thread and the shutdown countdown thread. The monitoring thread is used to periodically perform screen sampling and pixel change detection, the screenshot saving thread is used to automatically save screenshots when alarms are triggered, and the shutdown countdown thread is used to automatically shut down and manage the shutdown in the case of no change timeout. The pixel change detection unit is used to convert the captured screen frame image into a NumPy matrix, calculate the sum of the absolute differences between pixels between frames through vectorized matrix operations to obtain the pixel change quantization value, and combine it with a preset sensitivity threshold to determine whether to trigger an alarm. The multimodal alarm management module has a built-in unified alarm triggering interface that communicates with the pixel change detection unit. It is used to select one or simultaneously trigger multimodal alarm actions such as sound, pop-up window, and email according to user configuration. The persistent storage module communicates and connects with the other modules and units to persistently store user configurations in JSON format, operation logs in text format, and alarm screenshots in timestamp naming format.

[0014] Preferably, the full-screen transparent top-mounted overlay window of the user interaction module is built based on PyQt5's QWidget, and is set with borderless and always-on-top attributes, and the window transparency is 30%; The full-screen transparent top-mounted overlay window records the starting coordinates through mouse press events, draws the red rectangular border of the selected area in real time through drag events, and calculates and outputs the coordinates of the selected area through release events, thus completing the selection of the monitoring area.

[0015] Preferably, when the pixel change detection unit calculates the pixel difference between frames, it first converts the frame image in PIL.Image format into an int16 type NumPy matrix, then calculates the absolute difference between the two frame matrices, and sums the difference matrices to obtain the pixel change quantization value.

[0016] Preferably, the monitoring thread of the multi-threaded management module has a built-in dual monitoring mode, including a pixel change monitoring mode and a no-change timeout monitoring mode. The pixel change monitoring mode is used to trigger an alarm when the pixel change quantization value exceeds a preset threshold. The no-change timeout monitoring mode is used to trigger a timeout alarm and start a shutdown countdown thread when the pixel change quantization value is continuously lower than the threshold within a preset time period. After the shutdown countdown thread starts, it periodically captures screen images and compares them with reference screenshots for pixel differences. If a pixel change exceeding the threshold is detected before the countdown ends, the shutdown is canceled and normal monitoring is resumed. If no valid pixel change is detected before the countdown ends, the system shutdown command is executed.

[0017] Preferably, the unified alarm interface of the multimodal alarm management module is send_notification(title, message), and the interface internally executes the corresponding alarm action according to the user configuration, including: For sound alerts, user-defined audio files will be played first; if a user-defined audio file is invalid, the system default alert sound will be played. The pop-up notification action displays an alarm dialog box via QMessageBox, and simultaneously displays a bubble notification in the system tray; The email alert action sends alarm emails via the SMTP protocol, with a built-in anti-duplicate sending mechanism to prevent the same alarm event from being sent repeatedly.

[0018] Preferably, the configuration management logic of the persistent storage module is as follows: when the program starts, it automatically loads the JSON format configuration file, blocks the UI control signal triggering during loading, and restores the signal connection after loading is completed; after the user modifies the configuration, it automatically serializes the configuration into JSON format and saves it to ensure that the configuration can be restored after restarting.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention realizes the inter-frame pixel difference calculation through vectorized matrix operation based on NumPy. Compared with the traditional point-by-point pixel comparison method, the operation efficiency is significantly improved and the CPU usage is greatly reduced. Even on low-configuration devices, high-frequency accurate detection can be achieved. At the same time, by converting uint8 pixel data to int16 type for calculation, the numerical overflow problem of pixel value subtraction is avoided from the root, ensuring the authenticity and accuracy of the difference calculation results. Combined with the preset sensitivity threshold mechanism, it effectively filters out invalid interference such as small screen pixel jitter and mouse pointer movement, greatly reducing the false alarm rate and realizing the simultaneous improvement of detection accuracy and operation efficiency.

[0020] (2) The present invention realizes the visual drag-and-drop monitoring area delineation through a full-screen transparent top-cover window. Users do not need to manually input coordinate parameters. They can quickly select any rectangular monitoring area by simply dragging the mouse. It is suitable for multi-monitor scenarios. The operation process is simple and intuitive, which greatly reduces the threshold of use. At the same time, it can accurately focus on the core area of ​​interest on the screen, avoid the invalid calculation and data redundancy brought about by full-screen monitoring, and further improve the targeting and operating efficiency of monitoring.

[0021] (3) This invention realizes the free combination and flexible configuration of various alarm methods such as sound, pop-up window and email through a standardized unified alarm interface. It can realize real-time alarm at close range through local sound and pop-up window reminders, and can also realize remote alarm at different locations through email. It fully covers the alarm needs of different scenarios such as user presence and unattended operation, and effectively avoids the omission of alarm information. At the same time, the email alarm has a built-in anti-duplicate sending mechanism, which can prevent the repeated sending of invalid emails for the same alarm event, thus improving the standardization of alarm management while ensuring the alarm reach rate.

[0022] (4) This invention supports dual monitoring modes of pixel change alarm and no change timeout detection. It can identify abnormal changes in screen content and alarm in real time, and can also detect long-term no-change scenarios caused by task completion, program freeze, etc., trigger timeout alarm and link automatic shutdown function. Through an independent shutdown countdown thread, the shutdown process can be effectively controlled and dynamically canceled. If a valid change in the screen is detected within the countdown period, the shutdown process can be automatically terminated. This solves the problem of ineffective operation and resource waste of equipment in unattended scenarios, avoids the risk of accidental shutdown, and greatly improves the automation level of the system and the adaptability of unattended scenarios.

[0023] (5) This invention completely separates the logic of UI rendering interaction, monitoring and detection, screenshot saving, and shutdown countdown. The main thread is only responsible for UI response, and all time-consuming operations are run in independent sub-threads. Each thread does not interfere with each other and is independently managed, which fundamentally solves the problem of lag and unresponsiveness caused by monitoring logic blocking the UI interface in the traditional single-thread architecture. Even with continuous monitoring for 24 hours, the smoothness of interface operation can be guaranteed. Users can adjust monitoring parameters and start and stop the monitoring process at any time, which greatly improves the stability of system operation and user experience.

[0024] (6) This invention achieves full persistent storage of monitoring configuration through JSON format. Historical configuration can be automatically loaded when the program starts, eliminating the need for users to repeatedly set parameters after restarting. It also supports the import and export of configurations, allowing for quick switching of configuration schemes for different monitoring scenarios and adapting to the rapid reuse requirements of multiple scenarios. It is equipped with a standardized operation log recording and an automatic retention mechanism for alarm screenshots named by timestamp, which fully records the system operation status, alarm events and on-site images, enabling traceability of the entire monitoring process and facilitating post-event review and problem investigation of abnormal events.

[0025] (7) This invention is a purely software-based implementation solution, which does not require dedicated hardware acquisition equipment and has no additional hardware deployment costs. It is compatible with mainstream operating systems such as Windows and macOS and can be widely adapted to various scenarios such as industrial host computer monitoring, real-time financial market monitoring, automated program operation and maintenance, large file download / rendering task tracking, and remote operation and maintenance desktop monitoring. It can meet the needs of industrial-grade professional monitoring as well as the lightweight monitoring needs of ordinary office and personal users, which greatly expands the scope of application of the technology and lowers the threshold for use.

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0027] Figure 1 This is a block diagram illustrating the overall architecture principle of an embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall layered architecture of an embodiment of the present invention; Figure 3 This is a diagram illustrating the program page layout according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the startup monitoring process according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the pixel change alarm process according to an embodiment of the present invention. Figure 6 This is a pixel alarm page display diagram according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating the email configuration process according to an embodiment of the present invention. Figure 8 This is a display image of an alarm email according to an embodiment of the present invention. Detailed Implementation

[0028] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0029] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0030] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0031] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0032] Example 1: The present invention provides a multimodal screen monitoring and alarm system based on pixel difference detection, including a screen capture unit, an alarm notification unit, a user interaction module, a multi-threaded management module, a pixel change detection unit, a multimodal alarm management module, and a persistent storage module.

[0033] The user interaction module generates a full-screen transparent top-mounted overlay window, enabling the visual selection and coordinate acquisition of the monitoring area through mouse drag-and-drop operations, while also providing a parameter configuration interface and a running status display interface. The multi-threaded management module communicates with the user interaction module, the screenshot unit, and the pixel change detection unit respectively. It is used to independently run the monitoring thread, the screenshot saving thread, and the shutdown countdown thread. The monitoring thread is used to periodically perform screen sampling and pixel change detection, the screenshot saving thread is used to automatically save screenshots when an alarm is triggered, and the shutdown countdown thread is used for automatic shutdown management in scenarios with no change and timeout. The pixel change detection unit is used to convert the captured screen frame image into a NumPy matrix, calculate the sum of the absolute differences between pixels between frames through vectorized matrix operations, obtain the pixel change quantization value, and determine whether to trigger an alarm by combining it with a preset sensitivity threshold. The multimodal alarm management module has a built-in unified alarm triggering interface that communicates with the pixel change detection unit. It is used to trigger multimodal alarm actions such as sound alerts, pop-up alerts, and email alerts simultaneously or selectively according to user configuration. The persistent storage module communicates with each module to persistently store user configuration data in JSON format, store operation logs in text format, and store alarm screenshots in timestamp naming format.

[0034] Specifically, the multimodal screen monitoring and alarm system based on pixel difference detection provided in this embodiment is developed using Python 3.8 or later, and supports Windows 10 and later, and macOS 10.15 and later operating systems. It adopts a layered modular architecture design, divided from bottom to top into a system layer, data layer, monitoring layer, notification layer, and presentation layer. The component composition and collaborative logic of each layer are as follows: Figure 1 As shown, the business implementation architecture of this system is divided into four core layers: persistence layer, data processing layer, business logic layer, and user interface layer. The functional division and data interaction relationships of each layer are as follows: Figure 2 As shown.

[0035] The specific implementation methods of each unit and module are as follows: (1) User interaction module The user interaction module is built on the PyQt5 framework, and its core functions are divided into two parts: a visual selection component for the monitoring area and an interactive component for the main interface.

[0036] The specific implementation of the monitoring area visualization selection component is as follows: a full-screen transparent top-mounted overlay window based on QWidget is generated. The window is set with the Qt.FramelessWindowHint borderless property and the Qt.WindowStaysOnTopHint always on top property. The window transparency is set to 30%. It can automatically adapt to multi-monitor scenarios and completely cover all desktop display areas.

[0037] Implement region selection by overriding the window's mouse events: Override the mousePressEvent event, and when the left mouse button is pressed, record the global coordinates of the starting point: start_pos = event.globalPos(); Override the mouseMoveEvent event to update the current coordinates current_pos = event.globalPos() in real time during mouse dragging, and use QPainter to draw a red rectangle selection border with a line width of 2px and RGB value of (255,0,0) on the window; Override the mouseReleaseEvent event. When the left mouse button is released, take the minimum x and minimum y values ​​between the starting point and the release point as the coordinates of the top left corner of the region, and the maximum x and maximum y values ​​as the coordinates of the bottom right corner of the region, to obtain the complete coordinates of the monitored region (x1, y1, x2, y2). Send the coordinates to the main window through the custom region_selected signal, then close the full-screen transparent top overlay window, restore the main window display, and update the region coordinate display label.

[0038] The main window of this system is divided into six functional areas. The complete interface layout and control distribution are as follows: Figure 3 As shown.

[0039] The main interface interactive components are built based on QMainWindow and use QVBoxLayout as the overall layout, divided into 6 functional areas from top to bottom: Quick Actions Area: Contains two QPushButtons: "Start Monitoring" and "Stop Monitoring". The former has a green background and white text, and clicking it triggers the start_monitor() slot function; the latter has a red background and white text, is grayed out by default, and becomes active after monitoring starts, and clicking it triggers the stop_monitor() slot function.

[0040] Startup Options Area: Contains two QCheckBoxes: "Automatically start monitoring on startup" and "Minimize to tray on startup". The control state is automatically loaded and saved with the configuration file.

[0041] Screen region selection area: contains a QLabel coordinate display label and a "Grab screen region" QPushButton. Clicking the button triggers the region_select() slot function, hides the main window and starts a full-screen transparent top-mounted overlay window.

[0042] The monitoring options area is divided into three QGroupBox sub-areas: basic monitoring settings, screenshot function settings, and automatic shutdown settings. Each sub-area provides configuration controls for parameters such as monitoring mode selection, no-change timeout, check interval, screenshot save path, and shutdown countdown duration. All parameters support persistent configuration.

[0043] The alert method area is divided into three QGroupBox sub-areas: sound alert, pop-up alert, and email alert. Each sub-area provides an enable / disable switch and configuration control for the corresponding alert method, and supports functions such as custom audio file selection, SMTP server configuration, and email sending test.

[0044] Log Function Area: Includes a read-only QTextEdit log display control, as well as two function buttons: "View Log Folder" and "Clear Log", which display the system running status and event records in real time.

[0045] Meanwhile, the user interaction module implements the system tray function through QSystemTrayIcon, sets a custom tray icon and right-click shortcut menu, and supports functions such as minimizing to the tray, tray shortcut operations, and alarm bubble notifications. The default display duration of the bubble notification is 5 seconds.

[0046] (2) Multi-threaded management module After the user completes the selection of the monitoring area and parameter configuration, clicking the "Start Monitoring" button will trigger the system to execute a complete process of configuration verification, directory creation, thread initialization, and startup. This process is as follows: Figure 4 As shown.

[0047] The multi-threading management module is implemented based on Python's `threading` module and PyQt5's `QThread` class. It adopts an architecture that completely separates the main thread from the child threads. The main thread is only responsible for UI rendering and interactive responses, while all time-consuming operations run in independent child threads, avoiding UI blocking. The module includes three independent child thread management classes, all inheriting from `QThread`, with the following implementations: The MonitorThread: It has a built-in `is_running` boolean flag to control thread start and stop. After starting, the thread first captures the screen image of the current monitored area as the previous frame and records the current time as the last change time (`last_change_time`). Then it enters a loop, checking the `is_running` flag in each iteration; if it's False, the thread exits. After sleeping at a preset check interval, it captures the screen image of the current monitored area and passes it to the pixel change detection unit for difference calculation. If a valid pixel change is detected, it updates `last_change_time`, sends a `notification_triggered` signal to trigger an alarm, and simultaneously sends a `screenshot_triggered` signal to start the screenshot thread when automatic screenshotting is enabled. If no-change-time monitoring is enabled, it calculates the difference between the current time and `last_change_time`; if it exceeds a preset threshold, it sends a `timeout_notification_triggered` signal to trigger a timeout alarm, and simultaneously sends a `shutdown_triggered` signal to start the shutdown countdown thread when automatic shutdown is enabled.

[0048] ScreenshotThread: Upon receiving a screenshot trigger signal, it acquires the screen image of the current monitored area, saves the image losslessly to a preset path using the Pillow library according to the naming format of screenshot_YYYYMMDD_HHMMSS.png, sends the screenshot_saved signal after saving, and records the saving information in the log.

[0049] The ShutdownCountdownThread has a built-in shutdown_active boolean flag. After starting, it saves the current screen image as a reference frame, records the start time, the preset total countdown duration, and the check interval. Upon entering the loop, it first checks the shutdown_active boolean flag; if it is False, the thread exits. It periodically captures the current screen image and compares it with the reference frame for pixel differences. If a valid change is detected, it sends a shutdown_cancelled signal to cancel the shutdown and resume normal monitoring, records the log, and then exits the thread. If no valid change is detected, it calculates the remaining countdown duration in real time and sends a countdown_update signal to update the UI display. After the countdown ends, it sends a shutdown_execute signal, records the shutdown execution log, triggers a final alarm notification, and executes the corresponding shutdown command based on the operating system: os.system("shutdown / s / t 10") for Windows and os.system("sudo shutdown -h now") for macOS. After execution, the thread exits.

[0050] (3) Pixel change detection unit The pixel change detection unit is implemented based on NumPy version 1.21 and above. Its core is an inter-frame pixel difference quantization algorithm to prevent overflow. The specific execution steps are as follows: Image format conversion: The PIL.Image format RGB image captured by the Pillow library is first converted to a uint8 multidimensional array using np.array(), and then converted to a 16-bit integer array using astype(np.int16) to avoid underflow issues when subtracting uint8 data. For low-configuration devices, the color image can be converted to an 8-bit grayscale image first using PIL.Image.convert('L') before matrix conversion to reduce computation.

[0051] Inter-frame difference calculation: Subtract the previous frame image matrix from the current frame image matrix element by element, and calculate the absolute value of the difference using np.abs() to obtain the difference matrix diff_matrix = np.abs(previous_frame - current_frame).

[0052] Change quantization and threshold judgment: The scalar pixel change quantization value pixel_diff is obtained by summing all elements of the difference matrix using np.sum(). This value directly reflects the overall degree of change between two frames of images. The preset sensitivity threshold is 1000. If pixel_diff>1000, it is determined that a valid pixel change has been detected, and a trigger signal is output. If pixel_diff≤1000, it is determined that there is no valid change. This threshold can effectively filter out false alarms caused by minor screen pixel jitter and mouse pointer movement, while accurately capturing substantial changes in screen content.

[0053] Based on the aforementioned overflow-prevention pixel difference quantization algorithm, the system executes a complete process of periodic screen sampling, change judgment, and alarm triggering as follows: Figure 5 As shown.

[0054] (4) Multimodal alarm management module The multimodal alarm management module has a built-in unified alarm triggering interface, send_notification(title, message). This interface is a slot function. After receiving an alarm trigger signal, it reads the user configuration and executes the alarm modes enabled by the user in sequence. The specific implementation of each alarm mode is as follows: Sound alerts: If the user enables sound alerts, the system first checks if a custom audio file exists. If the file is valid, it is loaded and played in low-latency mode using PyQt5's QMediaPlayer, supporting mainstream audio formats such as .wav, .mp3, .ogg, and .flac. If the custom audio file is invalid, Windows systems play a system warning sound using winsound.MessageBeep(winsound.MB_ICONEXCLAMATION), while macOS systems play a system alert sound using the AppKit framework's NSSound.beep().

[0055] Pop-up notification: If the user enables pop-up notification, a modal information dialog box will pop up in the main thread using QMessageBox.information() to display the alarm title and details; at the same time, if the main window is minimized or hidden, the showMessage() method of the system tray will be called to display a warning bubble notification with a duration of 5 seconds in the lower right corner of the desktop.

[0056] When the system detects a valid pixel change and triggers a pop-up alert, the alarm notification effect on the main interface and system tray is as follows: Figure 6 As shown.

[0057] The system's email alert function requires SMTP service configuration, connectivity testing, and parameter verification to be completed first. The complete email configuration process is as follows: Figure 7 As shown.

[0058] Email Alerts: If the user enables email alerts, first construct a unique alert identifier `alert_key=f"{recipient email address}:{alert title}"`. Check this identifier in the global `email_sent_dict` dictionary. If it already exists, skip this send to avoid duplicate alerts. If it does not exist, construct a `MIMEMultipart` email object, set the email header information, and add a plain text alert body using `MIMEText`. Connect to the user-configured SMTP server via `smtplib.SMTP`, enable TLS encryption, log in with the sender's email account and authorization code, complete the email sending, close the connection, store the `alert_key` in the dictionary, and record the sending information in the log.

[0059] After the system triggers an alarm event, an alarm email is sent to the user's inbox via the SMTP protocol. Its standard format and content display are as follows: Figure 8 As shown.

[0060] (5) Persistent storage module The persistent storage module is responsible for the local storage management of all system data, and is divided into three parts: configuration storage, log storage, and screenshot storage. The specific implementation is as follows: Configuration storage: JSON format is used for persistence, with the default save path being `config / settings.json` in the program's running directory. When saving the configuration, all UI controls are traversed to collect the current configuration values, a hierarchical configuration dictionary is constructed, and the dictionary is serialized into a formatted JSON string using `json.dumps()`. The configuration directory is automatically created and written to a file. When loading the configuration, the program automatically reads the configuration file after startup, parses it into a configuration dictionary, and to avoid triggering unnecessary signals, `blockSignals(True)` is first called on all UI controls to block the signal. The signal connection is restored after parameter settings are completed. If loading fails, the default configuration is automatically used.

[0061] Log storage: The default save path is the `logs / ` directory under the program's running directory, and the file is named `log_YYYYMMDD.txt` by date. Log management is implemented through a custom `LogManager` class, which provides logging methods for three levels: info, warning, and error. Each log entry includes a timestamp, level, and details, and is synchronously output to the UI log display box and the local log file.

[0062] Screenshot storage: The default save path is the screenshots / directory under the program's running directory. Lossless screenshots in PNG format are saved with timestamps. When an alarm screenshot is triggered, the directory is automatically created, and the corresponding log is recorded after saving.

[0063] (6) Screenshot unit and alarm notification unit The screenshot unit is implemented based on the ImageGrab.grab() method of the Pillow library, supporting both Windows and macOS platforms. It can accurately capture images of a specified screen area based on the passed coordinate parameters and output frame data in PIL.Image format for use by the pixel change detection unit and the screenshot saving thread.

[0064] The alarm notification unit serves as the external adaptation layer for the multimodal alarm management module. It receives trigger signals from the pixel change detection unit, calls the unified alarm interface to distribute and execute multimodal alarm actions, and is also responsible for logging and synchronizing alarm events.

[0065] The overall operation flow of this system is as follows: After the program starts, it automatically loads the historical configuration. The user selects the monitoring area and configures the monitoring and alarm parameters through a full-screen transparent top-mounted overlay window. After clicking "Start Monitoring", the system completes the configuration validity verification, creates a storage directory, and starts the monitoring thread. The monitoring thread periodically performs screen sampling and pixel change detection, and triggers corresponding logic such as alarms, screenshots, and automatic shutdown based on the detection results. After clicking "Stop Monitoring", all sub-threads are safely exited, the current configuration is saved, and the monitoring process is completed.

[0066] In one possible embodiment, the full-screen transparent top-mounted overlay window of the user interaction module is built based on PyQt5's QWidget, and is set with borderless and always-on-top properties, and the window transparency is 30%; The full-screen transparent top-mounted overlay window records the starting coordinates through mouse press events, draws the red rectangular border of the selected area in real time through mouse drag events, and calculates and outputs the coordinates of the selected area through mouse release events, thus completing the selection of the monitoring area.

[0067] Specifically, in this embodiment, the full-screen transparent top-mounted overlay window of the user interaction module is implemented through an independent RegionSelectorWidget class, which inherits from PyQt5's QWidget. The specific implementation details are as follows: (1) Window initialization and property configuration In the constructor of the RegionSelectorWidget class, the basic properties of the window are set to ensure that it meets the requirements of full-screen, transparent, on-top, and borderless. The specific code logic is as follows: The parent class constructor super().__init__() is called to initialize the basic QWidget components.

[0068] Set window flags: Use self.setWindowFlags(Qt.FramelessWindowHint | Qt.WindowStaysOnTopHint), where Qt.FramelessWindowHint is used to remove window title bar, borders and other decorations to achieve a borderless effect; Qt.WindowStaysOnTopHint ensures that the window is always on top of all application windows to avoid being obscured.

[0069] Set window transparency: Set the overall window transparency to 30% using self.setWindowOpacity(0.3), which ensures that the user can see the screen content below the window while still being aware of the selected window's existence.

[0070] Adapting to full-screen display across multiple monitors: Obtain the total geometric dimensions of all monitors using QDesktopWidget().screenGeometry(), and call self.setGeometry(desktop_geometry) to set the window size to cover the entire area of ​​all screens, ensuring that arbitrary area selection can be achieved even in a multi-screen environment.

[0071] Initialize member variables: Define self.start_pos = QPoint() to record the starting coordinates when the mouse is pressed, self.current_pos = QPoint() to record the current coordinates during mouse dragging, and self.is_selecting = False to mark whether a region selection operation is in progress.

[0072] (2) Mouse event rewriting and region selection logic Interactive selection of the monitored area is achieved by overriding the three core events of QWidget: mouse press, mouse move, and mouse release. The specific implementation is as follows: Mouse press event (mousePressEvent): Override the mousePressEvent function to only respond to left mouse button presses: When event.button() == Qt.LeftButton, set self.is_selecting to True, indicating that area selection has begun; The global coordinates when the mouse is pressed are recorded using `self.start_pos = event.globalPos()` and used as the starting point of the selection area. Simultaneously initialize self.current_pos to the starting point coordinates to prepare for subsequent drag-and-drop drawing.

[0073] Mouse move event (mouseMoveEvent): Rewrite the mouseMoveEvent function to update the selected area in real time during mouse dragging: First, check if self.is_selecting is True; only execute subsequent logic if the selection state is selected. Update the current global coordinates of the mouse using `self.current_pos = event.globalPos()`; Calling self.update() triggers the window's redraw event, ensuring that the rectangular border of the selected area is updated in real time.

[0074] Mouse release event (mouseReleaseEvent): Rewrite the mouseReleaseEvent function to complete the area selection and output the coordinates: When event.button() == Qt.LeftButton and self.is_selecting is True, the region coordinates are calculated. Standardize the starting coordinates `self.start_pos` and the current coordinates `self.current_pos`: take the smaller of the two x-coordinates as the x-coordinate of the top-left corner of the region (x1 = min(self.start_pos.x(), self.current_pos.x())), and the larger of the two as the x-coordinate of the bottom-right corner of the region (x2 = max(self.start_pos.x(), self.current_pos.x())); similarly, take the smaller and larger of the y-coordinates as the y-coordinates of the top-left corner (y1) and the bottom-right corner (y2) of the region. Define and send a custom signal `self.region_selected.emit(x1, y1, x2, y2)` to pass the standardized region coordinates to the main window; Set self.is_selecting to False, call self.close() to close the full-screen transparent top-mounted overlay window, and complete the region selection process.

[0075] (3) Real-time drawing of selected areas By overriding the `paintEvent` function of `QWidget`, the real-time drawing of the red rectangular border of the selected area is achieved. The specific logic is as follows: In the paintEvent function, create a QPainter object and bind it to the current window; Set drawing parameters: Set the pen color to red (RGB value 255, 0, 0) and the line width to 2 pixels using painter.setPen(QPen(QColor(255, 0, 0), 2)) to ensure the rectangular border is clearly visible; Determine the state of self.is_selecting, and only draw the rectangle when the selection is active: Convert the starting and current coordinates to local window coordinates (since the window is full-screen, the global coordinates and local coordinates are consistent and can be used directly); The `painter.drawRect(QRect(self.start_pos, self.current_pos))` method draws a rectangular border with the starting and current coordinates as diagonal points, providing a visual feedback of the selected area.

[0076] (4) Connection between signal and main window In the main window, when the user clicks the "Capture Screen Region" button, the RegionSelectorWidget class is instantiated, and its region_selected signal is connected to the slot function of the main window: After receiving the area coordinates, the slot function of the main window updates the coordinate display label on the interface and saves the coordinates to the configuration; Meanwhile, the main window hides itself when the selection window is instantiated, and redisplays it after the selection window is closed, ensuring the continuity of the interaction process.

[0077] In one possible embodiment, when the pixel change detection unit calculates the pixel difference between frames, it first converts the frame image in PIL.Image format into an int16 type NumPy matrix to avoid the overflow problem when subtracting uint8 type data, then calculates the absolute difference between the two frame matrices, and sums the difference matrix to obtain the pixel change quantization value.

[0078] Specifically, in this embodiment, the pixel change detection unit is implemented through an independent PixelDiffDetector class. Its core function is to process the two consecutive PIL.Image format images output by the screenshot unit and calculate the pixel change quantization value. The specific implementation details are as follows: (1) Image preprocessing and format conversion First, the input PIL.Image format frame image is preprocessed to convert it into a NumPy array that can perform matrix operations. The specific steps are as follows: Receive input frame images: The calculate_diff method of the PixelDiffDetector class receives two parameters, namely the previous frame image prev_frame and the current frame image curr_frame, both of which are RGB format image objects from the PIL.Image library.

[0079] Optional grayscale conversion: To reduce computational load and improve detection efficiency, the color image can be converted to a grayscale image first. By calling the `convert('L')` method of `PIL.Image`, the RGB three-channel image is converted to an 8-bit single-channel grayscale image, where 'L' mode represents a grayscale level with pixel values ​​ranging from 0 to 255. This step is optional; users can choose whether to enable it through configuration. Enabling it can reduce computational load by approximately two-thirds.

[0080] Convert to NumPy array: The preprocessed PIL.Image image is converted to a NumPy multidimensional array using the np.array() function. For RGB color images, the conversion results in a three-dimensional array of shape (height, width, 3), where height is the image height, width is the image width, and 3 corresponds to the red, green, and blue channels. For grayscale images, the conversion results in a two-dimensional array of shape (height, width). The array data type defaults to uint8 (unsigned 8-bit integer), and the pixel value range is 0-255.

[0081] (2) Data type conversion to avoid overflow Because the value range of uint8 data type is only 0-255, directly subtracting the pixel values ​​of two frames of images will cause numerical overflow: for example, if a pixel value is 10 in the previous frame and 5 in the next frame, the result of 10-5 in uint8 type will become 251 due to underflow, causing serious distortion in the difference calculation. Therefore, it is necessary to convert the array data type. The specific steps are as follows: Convert to int16 type: Use the NumPy array's astype(np.int16) method to convert the uint8 type image array to int16 type (signed 16-bit integer). The int16 type has a value range of -32768 to 32767, which is sufficient to accommodate the positive and negative differences between the pixel values ​​of two frames, completely avoiding overflow issues.

[0082] Synchronously convert two frames of images: ensure that the array prev_arr of the previous frame and the array curr_arr of the current frame are both converted to int16 type, in preparation for subsequent matrix operations.

[0083] (3) Calculation of inter-frame pixel differences Based on the converted int16 type NumPy array, the inter-frame pixel differences are efficiently calculated through vectorized matrix operations. The specific steps are as follows: Calculate the pixel difference matrix: Subtract the current frame array from the previous frame array element by element to obtain the difference matrix diff_arr = curr_arr - prev_arr. Since the int16 type is used, the difference matrix will contain positive and negative differences, corresponding to increases and decreases in pixel values, respectively.

[0084] Calculate the absolute difference matrix: Take the absolute value of each element in the difference matrix using the np.abs(diff_arr) function to obtain the absolute difference matrix abs_diff_arr, ensuring that all pixel differences are represented in positive form, which is convenient for subsequent quantization and summation.

[0085] (4) Calculation of pixel change quantization value The absolute difference matrix is ​​aggregated to obtain scalar quantization values ​​that reflect the overall degree of change between the two frames. The specific steps are as follows: Summing the absolute difference matrix: The np.sum(abs_diff_arr) function sums all elements in the absolute difference matrix to obtain a scalar value pixel_diff, which is the pixel change quantization value.

[0086] The physical meaning of the quantization value: The size of pixel_diff directly reflects the degree of overall pixel change between two frames of images: the larger the value, the more obvious the change in screen content; the smaller the value, the more stable the screen content. For example, if two frames of images are exactly the same, pixel_diff is 0; if there is a significant change in screen content, pixel_diff will increase significantly.

[0087] (5) Connection with subsequent modules The calculated pixel_diff quantization value is transmitted to the multimodal alarm management module and compared with a preset sensitivity threshold (e.g., 1000). If pixel_diff > threshold, it is determined that a valid pixel change has been detected, and a multimodal alarm is triggered. If pixel_diff ≤ threshold, it is determined that there is no valid change (e.g., slight screen pixel jitter, mouse pointer movement, etc.), no alarm is triggered, and the monitoring continues for the next cycle.

[0088] In one possible embodiment, the monitoring thread of the multi-threaded management module has a built-in dual monitoring mode, including a pixel change monitoring mode and a no-change timeout monitoring mode. Among them, the pixel change monitoring mode is used to trigger an alarm when the detected pixel change quantization value exceeds a preset threshold; The no-change timeout monitoring mode is used to trigger a timeout alarm and start a shutdown countdown thread when the pixel change quantization value is consistently lower than the threshold within a preset time period.

[0089] Specifically, in this embodiment, the monitoring thread is implemented based on the QThread class of PyQt5 and named MonitorThread. The thread has built-in independent mode control logic, supporting both modes to be enabled individually or simultaneously. The start and stop status of each mode is controlled by boolean flags within the thread. The two modes share the same set of screen sampling data and pixel difference calculation results, avoiding the waste of system resources caused by repeated sampling and calculation. The specific implementation method is as follows: (1) Basic initialization implementation of monitoring thread and dual mode The constructor of the MonitorThread class receives the core input parameters required for dual-mode operation, and initializes the state variables required for thread management, providing basic support for dual-mode operation: Input parameter configuration: The core parameters received by the constructor include the monitoring area coordinates (x1, y1, x2, y2), the check interval parameter check_interval (unit: seconds, default value 5), the pixel change threshold diff_threshold (default value 1000), the no-change timeout duration no_change_timeout (unit: seconds, default value 60), the dual-mode enable flags pixel_monitor_enabled and timeout_monitor_enabled, as well as the function enable flags for automatic screenshot and automatic shutdown.

[0090] State variable initialization: Synchronous initialization of core control variables within the thread, including: The thread running flag is_running: a boolean value that controls the start and stop of the main loop of the monitoring thread. It is set to True when the thread starts and False when monitoring stops. Previous frame reference image: PIL.Image type, stores the screenshot of the previous cycle as the reference for pixel difference calculation; Last change timestamp last_change_time: datetime type, records the time when the last valid pixel change was detected, providing a time calculation benchmark for no-change timeout monitoring; The alarm trigger status flag, alert_triggered, is a boolean value that indicates whether an alarm has been triggered in the current period, preventing the same event from triggering an alarm repeatedly.

[0091] (2) Specific implementation of pixel change monitoring mode The pixel change monitoring mode is the basic mode enabled by default for the thread. Its execution logic is embedded in the main loop of the monitoring thread, and the specific execution steps are as follows: Baseline frame initialization: When the monitoring thread starts and enters the loop for the first time, it uses the ImageGrab.grab() method of the screenshot unit to capture a screen image based on the input monitoring area coordinates. This image is then assigned to previous_frame as the initial baseline frame, and last_change_time is initialized to the current system time, thus completing the baseline frame initialization.

[0092] Periodic screen sampling: The main thread loop sleeps at the interval set by check_interval. After the sleep ends, the screen capture unit is called again to capture the screen image of the current monitored area and obtain the current frame, current_frame, thus completing a single period of screen sampling.

[0093] Pixel difference calculation: Pass the previous_frame and current_frame into the pixel change detection unit and perform pixel difference quantization calculation: First, convert the two frames into NumPy matrices of type int16 to avoid overflow problems caused by subtracting uint8 types. Then, calculate the absolute difference between the two matrices. Sum all elements of the difference matrix to obtain the pixel change quantization value pixel_diff. Return the calculation result to the monitoring thread.

[0094] Change detection and alarm triggering: Compare the calculated pixel_diff with the preset diff_threshold. If pixel_diff > diff_threshold, a valid pixel change is detected. The last_change_time is updated to the current system time, and the previous_frame is updated to the current_frame as the reference frame for the next cycle. If alert_triggered is False, a custom notification_triggered signal is sent to the multimodal alarm management module, which calls the unified alarm interface to trigger the multimodal alarm action. If the user has enabled the automatic screenshot function, a screenshot_triggered signal is sent to the screenshot saving thread to start the automatic saving of alarm screenshots. After completion, alert_triggered is set to True to avoid the same change event triggering alarms repeatedly. If pixel_diff ≤ diff_threshold, it is determined that there is no effective pixel change. The previous_frame is retained as the reference frame, the last_change_time is not updated, alert_triggered is reset to False, and the cycle of the next period begins.

[0095] Mode start / stop control: If the user closes the pixel change monitoring mode during the monitoring process, the thread will skip the above pixel difference calculation and change judgment steps, and only retain the screen sampling logic to provide basic data for the no-change timeout monitoring mode and avoid thread interruption.

[0096] (3) Specific implementation of the no-change timeout monitoring mode The no-change timeout monitoring mode is an optional enabled mode. It shares the screen sampling and pixel difference calculation results with the pixel change monitoring mode, eliminating the need to repeat the sampling and calculation operations. The specific execution logic is as follows: Mode enable verification: When the monitoring thread starts, it first checks whether the timeout_monitor_enabled flag is True. If it is False, all execution logic of this mode is skipped. If it is True, the no_change_timeout parameter is loaded to complete the mode initialization.

[0097] Calculation of no change duration: After the pixel difference calculation is completed in each monitoring cycle, the judgment logic of this mode is executed: First, the time difference between the current system time and last_change_time is calculated to obtain the current no change duration no_change_duration = datetime.now() - last_change_time, and the duration is converted into a value in seconds.

[0098] Timeout detection and action triggering: Compare the calculated no_change_duration with the preset no_change_timeout. If no_change_duration ≥ no_change_timeout, it is determined that a no-change timeout event has been detected. First, a custom signal timeout_notification_triggered is sent to the multi-modal alarm management module to trigger a timeout alarm notification. Then, it is determined whether the user has enabled the no-change automatic shutdown function. If it has been enabled, a custom signal shutdown_triggered is sent to the multi-threaded control module to pause the main loop of the current monitoring thread, start an independent shutdown countdown thread, and set the shutdown_active flag to True to control the execution of the shutdown process. If no_change_duration < no_change_timeout, it is determined that the timeout condition has not been met, no action is triggered, and the cycle enters the next period.

[0099] Timeout status reset: If a new valid pixel change is detected in the pixel change monitoring mode during the timeout countdown, last_change_time will be updated immediately, the calculation base for the time without change will be reset, and a shutdown_cancelled signal will be sent to the shutdown countdown thread to terminate the shutdown process and resume the normal main loop of the monitoring thread to avoid accidental shutdown.

[0100] (4) Implementation of dual-mode collaborative operation and anomaly handling Dual-mode collaborative logic: When both modes are enabled simultaneously, the thread's main loop performs screen sampling and pixel difference calculation only once per cycle. The calculation results are simultaneously provided to both modes for judgment. The judgment logic of the two modes is executed serially. First, the judgment of the pixel change monitoring mode is executed, and then the judgment of the no-change timeout monitoring mode is executed to ensure data consistency and avoid CPU resource consumption caused by repeated calculations.

[0101] Dynamic mode switching is supported: The monitoring thread has a built-in signal and slot mechanism to receive mode switching signals sent by the main interface. Without interrupting the main loop of the monitoring thread, the pixel_monitor_enabled and timeout_monitor_enabled flags are dynamically modified to realize dynamic start and stop of dual modes and real-time update of parameters without restarting the monitoring process.

[0102] Exception handling mechanism: The monitoring thread has built-in try-except exception capture logic. For issues such as screen sampling failure, pixel difference calculation error, and parameter out of bounds, after capturing the exception, it records the error level log to the persistent storage module and sends an exception_notice signal to the main interface to display the exception information in the log area to avoid unexpected thread crashes. If sampling exceptions occur for 3 consecutive cycles, the monitoring thread will be automatically stopped and an exception alarm notification will be triggered.

[0103] In one possible embodiment, after the shutdown countdown thread starts, it periodically captures screen images and compares them with reference screenshots for pixel differences. If a pixel change exceeding a threshold is detected before the countdown ends, the shutdown operation is canceled and normal monitoring is resumed; if no valid pixel change is detected before the countdown ends, the system shutdown command is executed.

[0104] Specifically, in this embodiment, the shutdown countdown thread is implemented based on the QThread class of PyQt5 and named ShutdownCountdownThread. The thread has built-in independent countdown control logic and pixel difference comparison mechanism. It communicates with the main window and monitoring thread through custom signals to realize the visualization, dynamic cancellation and safe execution of the shutdown process. The specific implementation method is as follows: (1) Initialization of the shutdown countdown thread The constructor of the ShutdownCountdownThread class receives the core parameters required for the shutdown process, and initializes the state variables and reference data required for thread management, providing basic support for the countdown execution: Input parameter configuration: The core parameters received by the constructor include: Reference frame: PIL.Image type, which is the screen image during the shutdown countdown startup, used as the benchmark for pixel difference comparison; Shutdown countdown duration: in seconds, default value is 300, configured by the user on the main interface; The check interval (check_interval) is in seconds, with a default value of 5, which is the time interval for periodically capturing screen images. Pixel change threshold diff_threshold: Default value 1000, consistent with the threshold of pixel change monitoring mode; Monitoring area coordinates (x1, y1, x2, y2): Ensure that the captured screen image is consistent with the reference screenshot area.

[0105] State variable and signal initialization: Synchronous initialization of core control variables and custom signals within a thread: The thread running flag `shutdown_active` is a boolean value that controls the start and stop of the countdown main loop. It is set to `True` when the thread starts and to `False` when the shutdown is canceled or the countdown ends. The countdown start time (start_time) is of type datetime and records the time when the thread started. It is used to calculate the remaining countdown duration. Custom signal definitions include shutdown_cancelled (cancel shutdown signal), shutdown_execute (execute shutdown signal), and countdown_update (countdown update signal), used to communicate with the main window and monitoring thread.

[0106] (2) Implementation of periodic screen capture and pixel difference comparison After the shutdown countdown thread starts, it enters the main loop and periodically performs screen capture and pixel difference comparison according to the interval set by check_interval. The specific steps are as follows: Periodic screen capture: At the beginning of each loop, the ImageGrab.grab() method of the screen capture unit is called to capture the current screen image based on the coordinates of the passed-in monitoring area, and the current frame is obtained, ensuring that the captured area is completely consistent with the reference frame.

[0107] Pixel difference calculation: Pass the reference_frame and current_frame into the pixel change detection unit and perform the same difference quantization calculation as in the pixel change monitoring mode. First, convert the two PIL.Image frames into NumPy matrices of type int16 to avoid overflow issues when subtracting uint8 types; Calculate the absolute difference between the two matrices to obtain the difference matrix; Summing all elements of the difference matrix yields the pixel change quantization value pixel_diff.

[0108] Difference comparison and status determination: The calculated pixel_diff is compared with the preset diff_threshold, and the remaining time of the current countdown is calculated. Remaining time calculation: The elapsed time is obtained by subtracting start_time from datetime.now(), and the remaining time is obtained by subtracting the elapsed time from shutdown_duration, in seconds; Send countdown update signal: Send the remaining_time to the main window via the countdown_update signal to update the countdown display on the interface, allowing users to know the shutdown progress in real time.

[0109] (3) Implementation of canceling shutdown operation and restoring normal monitoring If a pixel change exceeding the threshold is detected before the countdown ends, the thread will execute the logic to cancel the shutdown. The specific steps are as follows: Valid change detection: When pixel_diff > diff_threshold, it is determined that a valid pixel change has been detected, indicating that the user may have returned to the computer or the task status has changed, and there is no need to continue the shutdown process.

[0110] Send a shutdown cancellation signal: Immediately send a custom shutdown_cancelled signal to the main window and the monitoring thread. After receiving the signal, the main window updates the interface state and removes the countdown display. After receiving the signal, the monitoring thread resumes the normal monitoring main loop and continues to perform screen sampling and change detection.

[0111] Log recording and thread exit: The message "Screen change detected, shutdown canceled" is recorded to the local log file via the log manager. At the same time, the shutdown_active flag is set to False, the main thread loop is exited, and the shutdown countdown process ends.

[0112] (4) Execution of system shutdown command If no valid pixel change is detected by the end of the countdown, the thread will execute a system shutdown command, with the following steps: Countdown end judgment: When remaining_time≤0, the countdown is judged to be over. At this time, the shutdown_active flag is still True, indicating that no valid pixel change has been detected and the shutdown operation needs to be performed.

[0113] Send shutdown signal: Send the custom shutdown_execute signal to the main window. After receiving the signal, the main window updates the interface status and displays the message "Shutting down".

[0114] Record shutdown logs: Record the message "Countdown ended, system shutdown" to a local log file using the log manager, and save the current screen image as the last screenshot before shutdown for easy traceability later.

[0115] Execute the system shutdown command: Execute the corresponding shutdown command based on the operating system type. Windows system: The shutdown command is executed via os.system("shutdown / s / t 10"), where " / s" means shutdown and " / t 10" means execution after 10 seconds, giving the user a short time to cancel. macOS: Execute the shutdown command using os.system("sudo shutdown -h now"), where "-h" means shutdown and "now" means execute immediately.

[0116] Exit Thread: After executing the shutdown command, set the shutdown_active flag to False, exit the main thread loop, and end the shutdown countdown process.

[0117] (5) Implementation of exception handling and manual cancellation Exception handling mechanism: The thread has built-in try-except exception capture logic. For issues such as screen capture failure and pixel difference calculation errors, after the exception is captured, a warning-level log is recorded to the local log file, and the next loop continues to be executed to avoid interruption of the shutdown process due to a single exception.

[0118] Manual cancellation support: The main interface provides a "Cancel shutdown" button. After the user clicks the button, the main window sets the shutdown_active flag to False. When the thread main loop detects that the flag is False, it immediately sends a shutdown_cancelled signal, executes the logic to cancel shutdown, and resumes normal monitoring.

[0119] In one possible embodiment, the unified alarm interface of the multimodal alarm management module is send_notification (title, message). The interface executes corresponding alarm actions according to user configuration, including: sound reminder action, pop-up reminder action, and email reminder action.

[0120] Among them, the sound reminder action will play the user-defined audio file first, and if the user-defined audio file is invalid, the system default prompt sound will be played. The pop-up notification action displays an alarm dialog box via QMessageBox, while simultaneously displaying a bubble notification in the system tray; The email alert action sends alarm emails via the SMTP protocol, and has a built-in anti-duplicate sending mechanism to prevent the same alarm event from being sent repeatedly.

[0121] Specifically, in this embodiment, the multimodal alarm management module is implemented through an independent NotificationManager class. This class is built on the PyQt5 framework and has a built-in unified alarm triggering interface and private implementation methods for each alarm action. It communicates with the main window, system tray, and log manager through custom signals to achieve flexible configuration and reliable execution of multimodal alarms. The specific implementation method is as follows: (1) Initialization and configuration loading of the NotificationManager class The constructor of the NotificationManager class completes the basic initialization of the module and the loading of user configuration, providing support for the execution of alarm actions: Dependency component binding: The constructor takes the main window object main_window, the system tray object tray_icon, and the log manager object log_manager as input parameters to establish communication connections with UI components and the logging module, ensuring that alarm actions can trigger UI updates and log recording.

[0122] Configuration loading: Reads the user's alarm configuration from the persistent storage module's JSON configuration file, including: Enable flags for each alarm mode: sound_enabled, popup_enabled, email_enabled; Sound notification configuration: Custom audio file path: custom_sound_path; Email notification configuration: recipient email address (recipient_email), SMTP server address (smtp_server), SMTP port (smtp_port), sender email address (sender_email), and sender authorization code (sender_auth_code).

[0123] Anti-duplicate sending mechanism initialization: Define a global dictionary email_sent_dict to store the unique identifier of sent emails. The key is a string in the format of "recipient email: alarm title" and the value is the sending timestamp to avoid sending emails for the same alarm event repeatedly.

[0124] (2) Specific implementation of the unified alarm interface send_notification `send_notification` is a public slot function of the `NotificationManager` class. It receives the alarm title and alarm message as input parameters and serves as the unified entry point for triggering multimodal alarms. The specific execution logic is as follows: Alarm event logging: First, the `info` method of `log_manager` is called to record the information "Alarm triggered: [title]-[message]" to the local log file, and at the same time output it to the log display area of ​​the main window for easy problem tracing.

[0125] Configuration validity check: Check if at least one of sound_enabled, popup_enabled, and email_enabled is True. If all are False, log a warning-level message "No alarm method enabled", return directly, and do not perform any further actions.

[0126] Alarm actions are executed sequentially: following the order of "sound alert → pop-up alert → email alert", the corresponding private implementation methods are called in turn. If an action fails, an error-level log is recorded and the next action is executed to ensure that other alarm methods are not affected.

[0127] (3) Specific implementation methods of sound reminder actions The sound notification action is implemented through the private method _play_sound, which prioritizes playing user-defined audio files and, if ineffective, falls back to the system default notification sound. The specific steps are as follows: Custom audio file verification: Check if sound_enabled is True. If it is False, return directly. If it is True, check if custom_sound_path is empty and if the file exists. Perform file existence verification using os.path.exists(custom_sound_path).

[0128] Custom audio file playback: If the file exists, playback is implemented using QMediaPlayer based on PyQt5. Initialize the QMediaPlayer object and set the playback mode to LowLatency to avoid playback delay; Audio files can be loaded using QMediaPlayer.setMedia(QUrl.fromLocalFile(custom_sound_path)), supporting mainstream formats such as .wav, .mp3, .ogg, .flac, and .m4a. Call the QMediaPlayer.play() method to play audio, while listening for the mediaStatusChanged signal. If playback fails (e.g., due to file corruption), log an error and automatically switch to the system default sound.

[0129] System default notification sound playback: If the custom audio file is invalid (path is empty, file does not exist, or playback fails), the default notification sound will be played according to the operating system type. Windows system: Play system warning sounds using winsound.MessageBeep(winsound.MB_ICONEXCLAMATION); macOS: Play the system default notification sound via NSSound.beep() of the AppKit framework. The AppKit module needs to be imported in advance.

[0130] (4) Specific implementation method of pop-up reminder action The pop-up notification action is implemented through the private method _show_popup, combined with a modal dialog box and system tray bubble notification to ensure that users can receive alarm information in a timely manner. The specific steps are as follows: Main thread UI operation validation: Since PyQt5 UI operations must be executed on the main thread, the _show_popup method uses QMetaObject.invokeMethod to ensure that the popup logic is executed on the main thread, thus avoiding thread conflicts.

[0131] Modal dialog box display: If popup_enabled is True, call QMessageBox.information() to display the modal information dialog box. Set the dialog box's parent object to the main window (main_window) to ensure the dialog box is always on top. Set the dialog box title to "alarm title" and the content to "alarm message". The dialog box icon is set to QMessageBox.Information, the button is set to QMessageBox.Ok, and the dialog box is closed when the user clicks the button.

[0132] System tray bubble notification display: Simultaneously check the status of the main window. If the main window is minimized or hidden, or if the user has not closed the dialog box in time, call the showMessage() method of the system tray object tray_icon to display the bubble notification. Set the bubble notification title to "title" and the content to "message". Set the bubble notification icon to the tray icon and set the display duration to 5000 milliseconds (5 seconds). During the display of the bubble notification, clicking the bubble will trigger the display and activation of the main window.

[0133] (5) Specific implementation methods of email notification actions The email notification action is implemented through the private method _send_email, which sends alert emails based on the SMTP protocol and has a built-in anti-duplicate sending mechanism. The specific steps are as follows: Email notification configuration verification: Check if email_enabled is True. If it is False, return directly. If it is True, check if the recipient email address, SMTP server address, port, sender email address, and authorization code are complete. If any are missing, record "Incomplete email configuration" in the error log and return.

[0134] Anti-duplicate sending verification: Construct a unique alarm identifier alert_key = f"{recipient_email}:{title}", and check if alert_key exists in email_sent_dict: If it exists, it means that this type of alarm email has been sent. Record "Avoid sending duplicate emails: [alert_key]" in the info log and return directly. If it does not exist, continue executing the email sending logic.

[0135] Email content construction: Building MIME formatted emails based on the email library: Initialize a MIMEMultipart object as an email container, and set the email headers From to sender's email address, To to recipient's email address, and Subject to alert title. Add a plain text email body using MIMEText, with the content being an alert message and the encoding format set to UTF-8; Add the MIMEText object to the MIMEMultipart container to complete the email content construction.

[0136] SMTP email sending: Connecting to an SMTP server and sending emails using the smtplib library: Initialize the smtplib.SMTP object, connect to the SMTP server address and port, and enable TLS encrypted transmission using the starttls() method; Call the login() method to authenticate your identity using the sender's email address and authorization code (most email addresses require an authorization code instead of a login password). Call the sendmail() method to send an email, with parameters being the sender's email address, the recipient's email address, and a MIMEMultipart object in as_string() format; After sending, the quit() method is called to close the SMTP connection, and the alert_key and the current timestamp are stored in email_sent_dict. The info log is recorded as "Email sent successfully: [alert_key]".

[0137] Exception handling: The built-in try-except block captures issues such as SMTP connection failure, authentication failure, and network exceptions. After capturing an exception, it records an error log "Email sending failed: [Exception information]", which does not affect the execution of other alarm methods.

[0138] Configuring dynamic updates and resetting the anti-duplicate sending mechanism Dynamic configuration updates: The NotificationManager class provides the update_settings() public method. When a user modifies the alarm configuration on the main interface, the main window calls this method to reload the configuration, ensuring that the alarm action is executed using the latest configuration.

[0139] Anti-duplicate sending mechanism reset: When the user clicks "Stop monitoring", the main window calls the reset_email_sent_dict() public method to clear the email_sent_dict dictionary, ensuring that the same type of alarm event can be sent again after monitoring is started next time.

[0140] In one possible embodiment, the configuration management of the persistent storage module includes: The program automatically loads a JSON-formatted configuration file when it starts up, blocking the signal triggering of UI controls during loading and restoring the signal connection after loading is complete. After the user modifies the configuration, it is automatically serialized and saved as a JSON-formatted configuration file to ensure that the configuration can be restored after a restart.

[0141] Specifically, in this embodiment, the configuration management of the persistent storage module is implemented through an independent ConfigManager class. This class is built based on Python's json module and the PyQt5 framework, and is responsible for configuration file path management, default configuration definition, configuration loading, configuration saving, configuration import and export, and exception handling. It works in conjunction with the UI controls of the main window through a signal and slot mechanism to achieve automatic configuration persistence. The specific implementation method is as follows: (1) Initialization and basic configuration implementation of the ConfigManager class The constructor of the ConfigManager class completes the basic initialization of the module, including defining the configuration file path, building the default configuration, and binding it to the main window. The specific steps are as follows: Configuration file path definition: Determine the program's running directory: Use `os.path.dirname(os.path.abspath(file))` to get the current script's location as the program's root directory; Define the configuration directory: Create a subdirectory named "config" under the program's root directory, with the path "config_dir = os.path.join(root_dir, "config"); Define the configuration file path: The configuration file is named settings.json, and the full path is config_path = os.path.join (config_dir, "settings.json").

[0142] Default configuration dictionary construction: Defines a hierarchical default configuration dictionary, default_settings, that covers all configurable items in the system.

[0143] Binding to the main window: The constructor takes the main window object main_window as input parameter, establishes the association with the UI control, and initializes the log manager object log_manager to record configuration operation logs.

[0144] (2) Specific implementation method for configuring automatic loading when the program starts The ConfigManager class provides the public method load_settings(), which is called by the main window when the application starts to automatically load the configuration. During the loading process, UI signals are blocked to prevent accidental triggering. The specific steps are as follows: Configuration file existence check: Call os.path.exists(config_path) to check if the configuration file exists; If it does not exist, log "Configuration file does not exist, use default configuration", set current_settings to a deep copy of default_settings (avoid modifying the default configuration by copy.deepcopy()), and proceed directly to the UI control settings step.

[0145] Configuration file reading and JSON parsing: If the configuration file exists, open the file using a try-except block; After successful parsing, check the structural integrity of current_settings. If some configuration items are missing, supplement them with the corresponding items in default_settings (achieved by recursively merging dictionaries) to ensure that the configuration items are complete.

[0146] UI control signal blocking: Collect all configuration-related UI controls in the main window, including QCheckBox, QSpinBox, QDoubleSpinBox, QLineEdit, QComboBox, etc. Call the blockSignals(True) method on each control to block its signal triggering, thus preventing the triggering of signals such as valueChanged and textChanged when setting control values, which would lead to unnecessary configuration saving operations.

[0147] UI control value settings: Set the values ​​of UI controls one by one according to the hierarchical structure of the configuration dictionary: For QCheckBox: setChecked() is set based on the boolean value of the configuration item; For QSpinBox / QDoubleSpinBox: setValue() according to the value of the configuration item; For QLineEdit: Set the text using setText() based on the string configured in the settings. For the coordinates of the monitored area: format the list [x1, y1, x2, y2] as a string and set it into the corresponding QLabel label.

[0148] To ensure that the order in which you set the settings is consistent with the order in which you save them, you should avoid misaligning the configuration items.

[0149] UI control signal recovery: After all control values ​​are set, call the blockSignals(False) method on each control to restore its signal connection; Re-establish the connection between the control signal and the configuration save slot function to ensure that the save is automatically triggered when the user modifies the configuration later. Record "Configuration loaded successfully" in the info log, completing the configuration loading process.

[0150] (3) Specific implementation method for automatic saving after user modification of configuration The ConfigManager class provides the public method save_settings(), which is automatically triggered when the user modifies the values ​​of UI controls, and implements the serialization and saving of the configuration. The specific steps are as follows: Collection of UI control values: Iterate through all configuration-related UI controls in the main window, collect the current control values ​​according to the hierarchy of default_settings, and construct a new configuration dictionary new_settings: For QCheckBox: Get the boolean value of isChecked(); For QSpinBox / QDoubleSpinBox: Get the value of value(); For QLineEdit: Get the string from text(); For the coordinates of the monitored area: parse the string from the QLabel label and convert it into a list format of [x1, y1, x2, y2].

[0151] Configure dictionary serialization: Use json.dumps() to serialize new_settings into a JSON string, and set indent=4 and ensure_ascii=False parameters to format the JSON file and support Chinese characters, making it easier for users to view and edit manually.

[0152] Configuration directory and file writing: Check if the configuration directory config_dir exists. If it does not exist, create the directory using os.makedirs(config_dir,exist_ok=True). Use a try-except block to open the configuration file and write a JSON string.

[0153] Automatic triggering mechanism implementation: During the initialization of the main window, connect the valueChanged, textChanged, stateChanged and other signals of all configuration-related UI controls to the save_settings() slot function of ConfigManager; When a user modifies any control value, the save_settings() method is automatically triggered, enabling real-time automatic saving of the configuration without requiring the user to manually click the save button.

[0154] (4) Configure extended implementation methods for import and export To facilitate user backup and switching of configurations, the ConfigManager class additionally provides the public methods import_settings() and export_settings() to implement configuration import and export functionality. Configuration export: The export_settings() method accepts the target file path as an input parameter (which is selected by the user via QFileDialog.getSaveFileName()). Serialize the current_settings dictionary into a JSON string and write it to the target file; The info log will show "Configuration exported successfully: [target path]".

[0155] Configuration import: The import_settings() method takes the source file path as an input parameter (which is selected by the user via QFileDialog.getOpenFileName()); Read the source file and parse the JSON to check the integrity of the configuration structure; If the structure is complete, block the UI signal, update current_settings and set the UI control values, and automatically save to the local configuration file after the signal is restored. Record the info log "Configuration import successful: [source path]"; if parsing fails, record the error log and prompt the user.

[0156] (5) Implementation of exception handling and fault tolerance mechanisms The ConfigManager class has built-in robust exception handling and fault tolerance mechanisms to ensure the stability of configuration management. File read / write exceptions: Capture file read / write exceptions such as PermissionError (insufficient permissions) and IsADirectoryError (path is a directory), log them, and then use the default configuration or prompt the user to check the path.

[0157] JSON parsing exception: Catch json.JSONDecodeError (format error), log it, and then use the default configuration to prevent the program from failing to start due to corrupted configuration files.

[0158] Configuration item missing fault tolerance: When loading configuration, if some configuration items are missing in current_settings, the missing items are supplemented by recursively merging default_settings to ensure the program functions completely.

[0159] UI control type fault tolerance: When setting UI control values, check whether the data type of the configuration item matches the control type. If they do not match, use the default value and log a warning to avoid program crashes.

[0160] Example 2: The present invention also provides a multimodal screen monitoring and alarm method based on pixel difference detection, comprising the following steps: S1. Monitoring area configuration steps: Generate a full-screen transparent top-mounted overlay window, collect the coordinates of the monitoring area selected by the user through mouse drag operation, and receive the monitoring parameters, alarm parameters and automation parameters configured by the user. S2. Monitoring initialization steps: Verify the validity of the configuration, create a directory for storing logs and screenshots, initialize monitoring status flags, and start an independent monitoring thread; S3. Real-time monitoring and detection steps: The monitoring thread periodically captures the screen image of the monitoring area, converts the two frames into NumPy matrices, calculates the quantization value of the pixel difference between the frames through vectorized matrix operations, and determines whether an alarm event is triggered by combining the preset threshold. S4. Multimodal alarm steps: When an alarm event is detected, the unified alarm interface is called to trigger one or more alarm actions from sound, pop-up, and email according to the user configuration. At the same time, an independent screenshot thread is started to save the alarm screenshot. S5. Unattended automated processing steps: When no valid pixel change is detected within a preset time period, a timeout alarm is triggered, and an independent shutdown countdown thread is started to continuously detect screen changes within the countdown period. If no valid change is detected when the countdown ends, the system is shut down.

[0161] In one possible embodiment, the specific method for calculating the inter-frame pixel difference in step S3 is as follows: convert the two consecutive frames of PIL.Image format into NumPy matrices of type int16, calculate the absolute difference between the two matrices, sum all elements of the difference matrix to obtain the pixel change quantization value, and filter out false alarms caused by tiny pixel jitters through a preset threshold.

[0162] In one possible embodiment, the shutdown countdown thread in step S5 executes as follows: save the reference screenshot at the time of shutdown startup, periodically capture the current screen image and compare it with the reference screenshot, if a pixel change exceeding the threshold is detected, cancel the shutdown and resume normal monitoring; if no valid change is detected after the countdown ends, execute the system shutdown command.

[0163] In one possible embodiment, the email alarm action in step S4 has a built-in anti-duplicate sending mechanism. By constructing a unique identifier containing the recipient and email subject, it determines whether the same type of alarm email has already been sent. If it has been sent, the current email sending operation is skipped.

[0164] In one possible embodiment, the multimodal screen monitoring alarm method based on pixel difference detection further includes a configuration persistence step: serializing all user-configured parameters into a JSON-formatted string and saving it to a local file; automatically reading and parsing the configuration file when the program starts to restore the user's historical configuration.

[0165] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal screen monitoring and alarm method based on pixel difference detection, characterized in that, Includes the following steps: S1. Monitoring Area Configuration: Generates a full-screen transparent top-mounted overlay window, which collects the coordinates of the monitoring area selected by the user by dragging and dropping with the mouse, and receives the monitoring, alarm and automation parameters configured by the user. S2. Monitoring Initialization: Verify the validity of the configuration, create a directory for storing logs and screenshots, initialize the monitoring status flags, and start an independent monitoring thread; S3. Real-time monitoring and detection: The monitoring thread periodically captures screen images of the monitoring area, converts the two consecutive frames into NumPy matrices, calculates the quantized value of pixel changes between frames through vectorized matrix operations, and determines whether an alarm event is triggered by combining the preset threshold. S4. Multimodal Alarm: When an alarm event is detected, the unified alarm interface is called to trigger one or more alarm actions from sound, pop-up, and email according to the user configuration, and an independent screenshot thread is started simultaneously to save the alarm screenshot. S5. Unattended automated processing: When no valid pixel change is detected within a preset time period, a timeout alarm is triggered, and an independent shutdown countdown thread is started. The thread continuously detects screen changes during the countdown period. If no valid change is detected when the countdown ends, the system is shut down.

2. The method according to claim 1, characterized in that, In step S3, the method for calculating the inter-frame pixel difference is as follows: convert the two consecutive frames of PIL.Image format into NumPy matrices of type int16, calculate the absolute difference between the two matrices, sum all elements of the difference matrix to obtain the pixel change quantization value, and filter out false alarms caused by tiny pixel jitters through a preset threshold.

3. The method according to claim 1, characterized in that, In step S5, the execution logic of the shutdown countdown thread is as follows: save the reference screenshot at the time of shutdown and startup, periodically capture the current screen image and compare it with the reference screenshot; if a pixel change exceeding the threshold is detected, cancel the shutdown and resume normal monitoring; if no valid change is detected after the countdown ends, execute the system shutdown command.

4. The method according to claim 1, characterized in that, The email alarm action in step S4 has a built-in anti-duplicate sending mechanism. By constructing a unique identifier containing the recipient and email subject, it determines whether the same type of alarm email has been sent. If it has been sent, the sending operation is skipped. The method also includes a configuration persistence step: serializing all user-configured parameters into JSON format strings and saving them to a local file; when the program starts, it automatically reads and parses the configuration file to restore the user's historical configuration.

5. A multimodal screen monitoring and alarm system based on pixel difference detection that implements the method of claim 1, comprising a screen capture unit and an alarm notification unit, characterized in that, Also includes: The user interaction module is used to generate a full-screen transparent top-mounted overlay window, which enables the visual selection and coordinate acquisition of the monitoring area through mouse dragging, and also provides a parameter configuration interface and a running status display interface. The multi-threaded management module communicates and connects with the other modules and units respectively, and is used to independently run the monitoring thread, the screenshot saving thread and the shutdown countdown thread. The monitoring thread is used to periodically perform screen sampling and pixel change detection, the screenshot saving thread is used to automatically save screenshots when alarms are triggered, and the shutdown countdown thread is used to automatically shut down and manage the shutdown in the case of no change timeout. The pixel change detection unit is used to convert the captured screen frame image into a NumPy matrix, calculate the sum of the absolute differences between pixels between frames through vectorized matrix operations to obtain the pixel change quantization value, and combine it with a preset sensitivity threshold to determine whether to trigger an alarm. The multimodal alarm management module has a built-in unified alarm triggering interface that communicates with the pixel change detection unit. It is used to select one or simultaneously trigger multimodal alarm actions such as sound, pop-up window, and email according to user configuration. The persistent storage module communicates and connects with the other modules and units to persistently store user configurations in JSON format, operation logs in text format, and alarm screenshots in timestamp naming format.

6. The system according to claim 5, characterized in that, The full-screen transparent top-mounted overlay window of the user interaction module is built based on PyQt5's QWidget, and is set to have no border, always on top, and a window transparency of 30%. The full-screen transparent top-mounted overlay window records the starting coordinates through mouse press events, draws the red rectangular border of the selected area in real time through drag events, and calculates and outputs the coordinates of the selected area through release events, thus completing the selection of the monitoring area.

7. The system according to claim 5, characterized in that, When the pixel change detection unit calculates the pixel difference between frames, it first converts the frame image in PIL.Image format into an int16 type NumPy matrix, then calculates the absolute difference between the two frame matrices, and sums the difference matrices to obtain the pixel change quantization value.

8. The system according to claim 5, characterized in that, The monitoring thread of the multi-threaded management module has a built-in dual monitoring mode, including a pixel change monitoring mode and a no-change timeout monitoring mode. The pixel change monitoring mode is used to trigger an alarm when the pixel change quantization value exceeds a preset threshold. The no-change timeout monitoring mode is used to trigger a timeout alarm and start a shutdown countdown thread when the pixel change quantization value is continuously lower than the threshold within a preset time period. After the shutdown countdown thread starts, it periodically captures screen images and compares them with reference screenshots for pixel differences. If a pixel change exceeding the threshold is detected before the countdown ends, the shutdown is canceled and normal monitoring is resumed. If no valid pixel change is detected before the countdown ends, the system shutdown command is executed.

9. The system according to claim 5, characterized in that, The unified alarm interface of the multimodal alarm management module is send_notification(title, message). Internally, the interface executes corresponding alarm actions according to user configuration, including: For sound alerts, user-defined audio files will be played first; if a user-defined audio file is invalid, the system default alert sound will be played. The pop-up notification action displays an alarm dialog box via QMessageBox, and simultaneously displays a bubble notification in the system tray; The email alert action sends alarm emails via the SMTP protocol, with a built-in anti-duplicate sending mechanism to prevent the same alarm event from being sent repeatedly.

10. The system according to claim 5, characterized in that, The configuration management logic of the persistent storage module is as follows: when the program starts, it automatically loads the JSON format configuration file, blocks the UI control signal triggering during loading, and restores the signal connection after loading is completed; after the user modifies the configuration, it automatically serializes the configuration into JSON format and saves it to ensure that the configuration can be restored after restarting.

Images