Digital display security warning method based on real-time content monitoring

Abstract

The application discloses a digital display security early warning method based on real-time content monitoring and belongs to the technical field of digital display system information security, wherein the method comprises the following steps: based on a content collection component embedded in a digital display, real-time content capture is performed; a content object dataset is generated; an embedded edge computing module of the digital display is activated, content recognition is performed on a text data sequence and the image data sequence, and behavior pattern recognition is performed on an operation data sequence; content security discrimination is performed in combination with preset content security early warning rules and content recognition results, and operation security discrimination is performed by comparing the behavior pattern recognition result with a standard behavior pattern; and a security early warning instruction is generated to perform a security early warning response. The application solves the technical problem that the digital display lacks real-time content monitoring and security early warning mechanism in the prior art, so that content violation, sensitive information leakage and operation abnormalities and other security risks cannot be discovered and responded in time.

Description

Technical Field

[0001] This invention relates to the field of information security technology for digital display systems, and specifically to a security early warning method for digital displays based on real-time content monitoring. Background Technology

[0002] With the rapid development of information technology, digital displays have been widely used in various application scenarios, including public advertising screens, information publishing systems, corporate display equipment, and smart home devices. These digital displays are not only used for information transmission and display, but also, to a certain extent, bear the responsibility of information security and content management.

[0003] Currently, most digital displays primarily focus on content display effects and user interaction experience, lacking effective real-time content monitoring and security early warning mechanisms. Existing content management systems often rely on backend servers for content review and management, resulting in issues such as response delays, insufficient data transmission security, and untimely detection of abnormal operations. Furthermore, with the continuous upgrading of cyberattack methods, digital displays face multiple security threats, including content tampering, sensitive information leakage, and malicious operations. Therefore, there is an urgent need for a solution capable of real-time monitoring of content and user behavior at the display end, promptly identifying and issuing early warnings of potential security risks to improve the overall security and reliability of digital display systems. Summary of the Invention

[0004] This application provides a digital display security early warning method based on real-time content monitoring, aiming to solve the technical problem that the lack of real-time content monitoring and security early warning mechanisms in existing digital displays leads to the inability to detect and respond to security risks such as content violations, leakage of sensitive information, and abnormal operation in a timely manner.

[0005] In view of the above problems, this application provides a digital display security early warning method based on real-time content monitoring.

[0006] This application provides a digital display security early warning method based on real-time content monitoring. The method includes: capturing real-time content using a content acquisition component embedded in the digital display to generate real-time display content; parsing the real-time display content to extract categorized objects and generate a content object dataset, wherein the content object dataset includes text data sequences, image data sequences, and operation data sequences; activating the embedded edge computing module of the digital display to perform content recognition on the text data sequences and image data sequences, and to perform behavior pattern recognition on the operation data sequences; combining preset content security early warning rules with the content recognition results to perform content security judgment, and comparing the behavior pattern recognition results with standard behavior patterns to perform operation security judgment; and generating a security early warning command to respond to the security early warning based on the content security judgment results and the operation security judgment results.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] By adopting a digital display security early warning method based on real-time content monitoring, including real-time content capture, classified object extraction, embedded edge computing for content and behavior recognition, security judgment based on preset rules, and generation of early warning instructions, this method solves the technical problem in existing technologies where digital displays lack real-time content monitoring and security early warning mechanisms, resulting in the inability to promptly detect and respond to security risks such as content violations, sensitive information leaks, and abnormal operations. This achieves the technical effect of improving the security and reliability of digital display systems.

[0009] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a digital display security early warning method based on real-time content monitoring is provided for embodiments of this application.

[0011] Figure 2 This application provides a flowchart illustrating the process of generating a content object dataset in a digital display security early warning method based on real-time content monitoring. Detailed Implementation

[0012] The overall concept of the technical solution provided in this application is as follows:

[0013] This application provides a digital display security early warning method based on real-time content monitoring. First, the content capture component embedded in the digital display captures the displayed content in real time and generates corresponding real-time display data. Then, this displayed content is parsed to extract text, images, and operation data, forming a content object dataset. Next, an embedded edge computing module is activated to perform content recognition on the text and images, and behavioral pattern recognition on the operation data. Then, based on preset content security early warning rules, the recognition results are used to make security judgments on the content and operations. Finally, a security early warning command is generated based on the judgment results, triggering corresponding security response measures. This achieves real-time monitoring and security early warning of the digital display during content display and operation, ensuring the legality and security of the displayed content and preventing potential security risks.

[0014] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0015] Examples, such as Figure 1 As shown in the embodiment of this application, a digital display security early warning method based on real-time content monitoring is provided. The method includes:

[0016] Step S100: Capture real-time content based on the content acquisition component embedded in the digital display and generate real-time display content.

[0017] Specifically, a digital display is a screen device that displays images or text via electronic devices, typically used in billboards, surveillance equipment, televisions, computer monitors, and other similar applications. It can display image or text information transmitted from signal sources such as computers, cameras, and video players. A content capture component refers to the hardware or software built into a digital display used to capture images and information displayed on the screen in real time. Content capture components can acquire displayed content through hardware interfaces (such as copying signal streams) or software interfaces (such as system monitoring). Real-time content capture refers to continuously and instantly acquiring the content currently displayed on the screen, ensuring that the captured content is up-to-date and without delay. The captured content is typically image or video data streams, but also includes text data streams and user-generated data streams.

[0018] The content capture component embedded in the digital display first activates the hardware-level replicator. The replicator's role is to replicate the signal stream transmitted from the signal source (such as a computer or video player) to the display in real time. The replicator captures data from the display signal stream in real time and transmits it to the subsequent decoding module. Specifically, the replicator does not directly display the image; instead, it extracts the original video stream signal for further processing, ensuring the timeliness and accuracy of information acquisition. Once the signal stream is captured, it is then transmitted to the hardware decoder. The decoder decodes the signal stream into a series of image frame sequences, which represent the continuously changing image on the display. Finally, the decoded image frame sequence is output as the real-time display content.

[0019] By using content capture components and hardware replicators to capture real-time display signal streams, and combining this with hardware decoders to generate image frame sequences, it is possible to ensure that the captured real-time content is not only timely but also accurate.

[0020] Step S200: Parse the real-time displayed content to extract classification objects and generate a content object dataset, wherein the content object dataset includes text data sequences, image data sequences and operation data sequences.

[0021] Specifically, the content object dataset is a collection of data obtained through the object extraction step of classification. This dataset includes different types of objects extracted from the real-time displayed content, typically including text data, image data, and action data. Text data sequences refer to sequences of text information extracted from the real-time displayed content. This sequence can include various types of text information, such as titles, prompts, menu items, and warning messages. Using text recognition technologies (such as OCR, Optical Character Recognition), text in images can be converted into machine-readable text format. Image data sequences refer to images or image frames extracted from the real-time displayed content. These image data can be static images or dynamic image sequences. Image data sequences can be used to extract valid image information from the displayed content using image recognition technologies (such as object detection and image segmentation). Action data sequences refer to action data related to user interaction, such as mouse clicks, scrolling, dragging, and keyboard input. Action data sequences record user interaction behaviors with the displayed content, typically obtained by monitoring input devices (such as mice and keyboards) or analyzing action elements within the displayed content.

[0022] First, it's necessary to acquire the content displayed in real-time on the digital display. Using an embedded content acquisition component, pre- or post-processor packet capture technology is used to obtain the real-time display signal from the display's signal stream and convert it into processable content data. After acquiring the real-time display content, the next step is object extraction. For areas containing text, OCR technology is used to extract the text information from the image. For image elements in the displayed content, image processing algorithms, such as image segmentation and object recognition, are used to extract the relevant images or image frames. For user input operations (such as mouse clicks, keyboard input, etc.), the operation data sequence is extracted by monitoring the input device operation or recognizing the operation area (such as button clicks or menu selections).

[0023] By classifying and extracting objects from the real-time displayed content, all extracted data is integrated into a structured content object dataset. This dataset provides data support for subsequent content analysis and security alerts.

[0024] This step enables the efficient extraction of structured data from complex display content. It not only improves the efficiency of display content processing but also lays the foundation for subsequent data analysis, behavior recognition, and security monitoring.

[0025] Step S300: Activate the embedded edge computing module of the digital display, perform content recognition on the text data sequence and the image data sequence, and perform behavior pattern recognition on the operation data sequence.

[0026] Specifically, embedded edge computing modules are computing modules integrated into a device. They typically reside between the data source and the cloud, processing data in the local environment where it is generated. The advantage of this type of module is reduced reliance on the cloud, enabling real-time data processing at the source of data generation, reducing latency, and improving processing efficiency. It is widely used in systems such as smart devices, IoT devices, and digital displays. Content recognition refers to analyzing real-time displayed content using technologies such as image recognition and text recognition to identify the elements contained within it. Behavioral pattern recognition refers to identifying user behavior patterns by analyzing user operation data sequences.

[0027] First, the embedded edge computing module of the digital display is activated, ready to receive real-time data streams acquired by the display. The edge computing module resides locally and processes the data directly to reduce data transmission latency and improve real-time responsiveness.

[0028] For text and image data sequences, the edge computing module processes them using integrated content recognition algorithms. For action data sequences, i.e., user interactions (such as cursor movements, mouse clicks, and keyboard input), the embedded edge computing module analyzes them using behavior pattern recognition technology. After content recognition and behavior pattern recognition, the edge computing module combines the results for comprehensive analysis. If any anomalies are detected in the content or behavior, an alert will be issued.

[0029] By activating the embedded edge computing module for real-time content recognition and behavior pattern recognition, and analyzing and processing real-time data locally, the digital display system becomes more intelligent and autonomous, enabling efficient processing and analysis of real-time data and improving its ability to cope with complex situations.

[0030] Step S400: Combine the preset content security warning rules with the content recognition results to make a content security judgment, and compare the behavior pattern recognition results with the standard behavior pattern to make an operation security judgment.

[0031] Specifically, content security warning rules refer to a set of predefined rules used to determine whether real-time displayed content contains potential security risks or violations. Content security warning rules typically include the following categories: sensitive keywords, sensitive semantics, sensitive patterns, and sensitive symbols. Content recognition results refer to the identification conclusions obtained after processing real-time displayed content through various technical means. Behavioral pattern recognition refers to monitoring and analyzing user interaction behavior, extracting their operation patterns, and judging whether there is abnormal or malicious behavior based on these operation patterns. Common behavioral patterns include mouse clicks, keyboard input, and swipe operations. Standard behavioral patterns refer to the general rules followed by user operations and interactions in normal usage scenarios. Operational security judgment refers to judging whether there are security risks or abnormal behaviors based on the analysis of user behavior patterns. If user behavior deviates significantly from standard behavioral patterns (e.g., frequently clicking the same link, quickly entering passwords, etc.), it is judged as a potential security risk.

[0032] First, the real-time displayed content is analyzed, and combined with content security warning rules, it is determined whether the content meets security requirements. If an advertisement or page contains political symbols or violent images, sensitive pattern recognition technology (such as convolutional neural networks) is used to analyze the image to determine whether it complies with security regulations. Natural Language Processing (NLP) technology is used to analyze the semantics of the content to determine whether there is any illegal or dangerous content, and further generate content security warnings.

[0033] Record user mouse clicks and swipes on the interface to determine if there are any abnormal behaviors such as frequent clicking of a link or rapid browsing of multiple pages. Record user input behavior, monitoring input frequency and character types. For example, if abnormally rapid input or repeated input of certain characters is detected, it will be identified as malicious script behavior or brute-force attack. Combine multiple behavior patterns, such as simultaneously monitoring mouse clicks and keyboard input, to detect if there are any deviations from standard behavior patterns.

[0034] The results of content security assessment and operational security assessment are comprehensively analyzed. For example, if an advertisement contains sensitive keywords and a user exhibits abnormal clicking behavior, the content and behavioral security risks are combined to generate a dual security warning. When both content and operational behavior trigger warnings simultaneously, a more urgent security response is generated, notifying the administrator or taking automated measures (such as blocking advertisements or restricting access).

[0035] By monitoring both content and operations, it can identify potential multi-dimensional security risks, flexibly adapt to different application scenarios and security requirements, and is widely used in multiple industries such as e-commerce, finance, and social media, thereby improving cross-domain security monitoring capabilities.

[0036] Step S500: Based on the content security judgment result and the operation security judgment result, generate a security warning instruction and respond to the security warning.

[0037] Specifically, content security assessment results refer to the judgment results derived from analyzing real-time displayed content based on preset content security warning rules. Operational security assessment results refer to determining whether user behavior conforms to normal operation by analyzing user behavior patterns and combining them with standard behavior patterns. Security warning instructions refer to response instructions generated based on content security and operational security assessment results, used to notify administrators or take automated measures to prevent further security risks. These instructions can be transmitted in various ways, including email notifications, SMS, push notifications, etc., or trigger automated processing, such as blocking suspicious content or locking accounts. Security warning response refers to the measures taken based on the warning instructions.

[0038] First, a comprehensive assessment of the current security situation will be conducted by combining the results of content security assessment and operational security assessment. If sensitive information is found during content analysis and abnormal user behavior is observed, a high level of security risk is considered, such as attacks, fraud, or malware.

[0039] Based on the comprehensive assessment results, one or more security alert directives are generated. These directives clearly identify the types of potential security threats and specify response measures according to the risk level.

[0040] For example: Low-risk warning: Sensitive keywords appear in the content, but user behavior is normal. In this case, a low-risk security warning is generated to remind the administrator to conduct a routine check. High-risk warning: The content contains sensitive images or illegal phrases, and the user's behavior pattern shows abnormalities. A high-risk security warning is generated, immediately notifying the administrator and initiating automated blocking measures. Triggering security warning response: Based on the generated security warning, automatic or manual response measures are taken. For example: If the content is determined to contain sensitive words or illegal advertisements, it can be automatically hidden, deleted, or blocked to prevent it from continuing to affect users. If user behavior is abnormal, the user's operations will be restricted, such as freezing the account or restricting access permissions. Furthermore, based on the warning, security personnel or administrators can be notified for further manual intervention and processing, especially in high-risk situations.

[0041] The notification and handling of alerts include: Automated response: for example, automatically locking accounts, automatically blocking advertisements or content, and prohibiting users from performing specific actions. Manual response: In some cases, notifying administrators or security personnel to review detailed security logs through the backend management system to confirm whether further manual inspection or intervention is needed.

[0042] By combining content security assessment results and operational security assessment results to generate security warning instructions, the platform's content and user behavior can be monitored in real time. When potential risks are detected, the platform can respond quickly and provide timely feedback to administrators, which helps to control and respond to possible security incidents in a timely manner.

[0043] Furthermore, based on the content acquisition component embedded in the digital display, real-time content capture is performed to generate real-time display content, including: activating the hardware layer replicator of the content acquisition component to perform pre-copy acquisition of the display signal transmitted from the signal source to obtain a real-time display signal stream; transmitting the real-time display signal stream to the hardware decoder for signal decoding to obtain a real-time image frame sequence, and outputting the real-time image frame sequence as the real-time display content.

[0044] Specifically, a hardware layer replicator is a specially designed hardware device or module used to "copy" display signals during signal transmission without affecting the normal operation of the signal source or monitor. It acts as an intermediate node, capable of acquiring the signal stream in real time and transmitting it to subsequent decoding modules. The replicator can extract signal data with very low latency. The display signal stream refers to the electronic signals transmitted from a signal source (such as a computer, video player, or other device) to a digital display. These signals contain all the information of the displayed content, including images, video, and text. The display signal stream can be a sequence of image frames or a video stream. A hardware decoder is a hardware device used to decode transmitted encoded signals (such as compressed video streams) into displayable image data. The decoder can parse and convert the encoded signal into a series of static or dynamic image frame sequences, which represent the real-time display content on the monitor. A real-time image frame sequence is a sequence of consecutive image frames decoded by the hardware decoder. Each frame represents the content displayed on the monitor at a particular moment. The continuity of the frame sequence ensures the real-time and dynamic nature of the displayed content, making it suitable for capturing video streams and dynamic content.

[0045] First, the content acquisition component embedded in the digital display activates the hardware-level replicator. The replicator's function is to copy the display signal stream from the signal source in real time via a special hardware interface, without affecting the normal operation of the signal source and the display. The replicator captures the electronic signals output from the signal source (such as a computer, video player, or monitoring system). By copying the signal stream, the content acquisition component can obtain the signal data in advance.

[0046] After acquiring the signal stream, the replicator transmits the real-time display signal stream to the subsequent decoding module. This pre-acquisition refers to acquiring the signal stream early in the transmission process to ensure the display content is obtained as early as possible, avoiding delays in subsequent processing. Once the replicator acquires the signal stream, it is transmitted to a hardware decoder for decoding. The hardware decoder's role is to decode the compressed or encoded signal stream into the original image or video content. The decoder decodes the video stream into consecutive image frames, forming a real-time image frame sequence. These image frames represent the content displayed on the screen at each moment. Assuming the signal stream originates from a playing video advertisement, the decoder converts the video signal into frame-by-frame images, thus forming a sequence of video image frames. The decoded image frame sequence is the obtained real-time display content. This sequence contains the image or text information displayed on the screen at each moment.

[0047] Through the collaboration of hardware replicators, decoders, and other tools, real-time content displayed on digital displays can be acquired efficiently and accurately. This ensures the stability and accuracy of digital displays in various complex environments, providing crucial data support, especially in scenarios with stringent requirements for real-time content monitoring and security detection.

[0048] Furthermore, the real-time display signal stream is transmitted to a hardware decoder for signal decoding to obtain a real-time image frame sequence. Then, the process further includes: interacting with the target display scene, obtaining the corresponding UI layout information, and identifying the corresponding key display areas based on the functional characteristics of the UI layout as a dividing constraint; fitting the key display areas to the real-time display signal stream according to the relative coordinates of the key display areas; and based on the fitting result, performing serialization extraction on the real-time display signal stream and outputting the serialization extraction result as a new real-time display signal stream.

[0049] Specifically, the interactive target display scenario refers to the overall structure of the interactive area or displayed content on a digital display. For example, a display may contain multiple modules, such as an advertising display area, information panels, and user interfaces; the interactive target display scenario is the collection of these modules. UI layout information refers to the position, size, and hierarchical relationship of each element in the digital display interface. UI layout information includes, but is not limited to, the position and distribution of elements such as buttons, text boxes, image display areas, and navigation bars. Functional characteristics refer to the functional nature of each display area in the UI layout; for example, one area may be used to display advertising content, another to display real-time data, and yet another for user interaction (such as buttons or input boxes). Functional characteristics provide a basis for identifying and dividing display areas. Key display areas refer to areas on the display screen that have important or sensitive functions. These areas typically involve the display of important information or user interaction interfaces, such as advertising display areas, alarm areas, or important data panels. Area relative coordinates refer to the position and size of a display area relative to the overall size of the display screen. This is usually represented by a coordinate system to accurately locate the size and position of the area on the screen. The coordinates can be absolute or relative to a parent area. Serialization extraction refers to extracting a specific sequence of images from a signal stream and formatting and organizing these images for subsequent processing and analysis. The extracted image frame sequence is output in a new format, typically used for further data analysis, storage, or display.

[0050] First, the current display scene needs to be obtained from the digital display. The interactive target display scene includes all interactive or important display areas on the screen, such as areas displaying advertisements, areas displaying real-time data, or interactive areas for user input. Based on UI layout information, these important areas are identified, usually through software interface extraction or image recognition.

[0051] Obtaining UI layout information typically involves analyzing the content displayed on a digital monitor to extract the position and function of all elements on the interface. UI layout information includes not only the positional coordinates of these elements (such as screen width and height) but also their functions (such as advertisements, information, buttons, etc.). This information can be obtained by software parsing the currently displayed interface or through the system's API. Based on the UI layout information, different areas on the screen are then divided according to their functional characteristics.

[0052] After obtaining the UI layout information and dividing it into functional features, the key display areas in the display scene are further identified. These key areas usually contain sensitive information or parts that require special monitoring, such as advertising content, alarm information, and real-time data display areas.

[0053] After identifying key display areas, these areas are fitted to image data in the real-time display signal stream based on their relative coordinates in the UI layout information. The fitting process uses algorithms to match the image area of ​​each key display area with the corresponding image area in the display signal stream, ensuring that the content of each image in the display signal stream aligns with the actual displayed content and avoiding image distortion or misalignment. After the fitting process is complete, a new image sequence is extracted from the real-time display signal stream based on the fitting results. Serialization extraction involves reorganizing and formatting the image frame sequence to ensure that the extracted image sequence accurately reflects the content of each key display area. Finally, the serialization extraction result is used to generate a new real-time display signal stream. This new signal stream contains the fitted and extracted image frame sequence, which more accurately reflects the content of key areas on the display for subsequent security analysis and content monitoring.

[0054] This step enables precise monitoring and capture of important display areas, ensuring the security and compliance of real-time displayed content.

[0055] Furthermore, real-time content capture based on the content acquisition component embedded in the digital display also includes: interactively acquiring the display driver component of the digital display, obtaining driver load information, and evaluating the available packet capture performance of the display driver component based on the driver load information; calculating and obtaining real-time packet capture parameters by combining the available packet capture performance with the display parameter information of the digital display; activating the software layer packet capture module of the content acquisition component, and performing post-packet capture of the digital display based on the real-time packet capture parameters to obtain real-time verified display content.

[0056] Specifically, the display driver is one of the core components of a digital display, responsible for controlling the generation and updating of the displayed content. It receives image signals from a computer or other signal sources, converts these signals into a format suitable for the display, and controls the brightness and color of each pixel on the screen through current and voltage. The display driver typically includes hardware components (such as a graphics card) and corresponding drivers. Driver load information refers to the load the display driver experiences during operation, typically manifested in the display's refresh rate, resolution, color depth, and the complexity of the displayed content. A higher driver load means a greater amount of image data to process and a higher computational burden on the display signals. Load information reflects the working status of the driver component and helps evaluate the display's processing capabilities. Available packet capture performance refers to the display driver's ability to effectively capture signals under normal operating load. Specifically, it is an indicator of whether the display driver can stably and accurately execute content capture tasks under high load. Display parameter information includes the display's technical specifications and settings, such as resolution, refresh rate, color depth, and display format. These parameters directly affect the display's image quality and content presentation, and are crucial for the accuracy and effectiveness of the packet capture process. Real-time packet capture parameters refer to the settings required when acquiring the display signal stream, such as packet capture frequency, sampling interval, and data extraction accuracy. These parameters are usually automatically adjusted by the system based on the current load of the display driver and the working status of the monitor to ensure efficient content acquisition without affecting display performance. A software-layer packet capture module is a tool that controls and manages the content capture process through software. Through the interface between the operating system and the display driver, it can extract display content in real time without affecting the display effect. Software-layer packet capture modules are typically used for content verification, performance monitoring, and data analysis. Post-processing packet capture refers to content acquisition after the display content has been generated and begun to be displayed on the monitor. Unlike pre-processing (signal source signal capture), post-processing packet capture focuses more on the actual output content of the monitor and is typically used to verify whether the content is displayed as expected and to check for errors or malfunctions. Real-time verification of display content refers to capturing and checking whether the content actually presented on the monitor meets expectations using a packet capture module. It is a crucial step in ensuring the correctness, accuracy, and security of display content, especially important in advertising monitoring, information display, and security alerts.

[0057] First, it's necessary to interact with the display driver component of the digital monitor to obtain its operating status and load information. The display driver component is responsible for processing image data on the monitor and driving it to display content. Interacting with the display driver component allows for real-time monitoring of the monitor's workload, including information such as refresh rate, resolution, and the complexity of the displayed content. Once a connection is established with the display driver component, its load information is extracted. This load information describes the current workload the monitor is under. For example, if the monitor is displaying dynamic video at high resolution, the display driver load will be high, while if static text is displayed, the load will be lower.

[0058] Based on the acquired driver load information, the available packet capture performance of the display driver component is evaluated. This is used to understand the driver component's ability to effectively support content capture under the current display load. For high-load scenarios, packet capture parameters need to be adjusted to avoid adverse effects on display performance. After evaluating the display driver's packet capture performance, real-time packet capture parameters are calculated based on the display's technical specifications (such as resolution, color depth, refresh rate, etc.). These parameters define the specific details of the packet capture operation, such as sampling frequency, image precision, and capture range. After calculating the real-time packet capture parameters, the software-layer packet capture module is activated. The software-layer packet capture module is the software tool in the system responsible for actually capturing the displayed content. Through the interface between the operating system and the display driver component, it extracts the content on the display in real time, captures each frame of image, and processes it according to the preset packet capture parameters.

[0059] After activating the packet capture module, post-capture packet acquisition begins. This means that the capture will start from the image content already generated on the monitor, rather than from the signal source. Based on real-time packet capture parameters, the content on the monitor is continuously captured and stored.

[0060] Finally, the captured display content is used for real-time verification to check its accuracy. This includes comparing the image content displayed on the monitor with the expected content to detect display errors, image distortion, color deviations, and other issues. The verification process can be implemented using image processing algorithms, data comparison tools, and other methods.

[0061] This step ensures that content capture and verification for digital displays are efficient and accurate, and provides reliable technical support for subsequent security monitoring and content verification.

[0062] Furthermore, such as Figure 2As shown, the real-time displayed content is parsed to extract classification objects and generate a content object dataset, wherein the content object dataset includes text data sequences, image data sequences, and operation data sequences, including: Step S210: extracting keyframes from the real-time displayed content and outputting the keyframe extraction results as the image data sequence; Step S220: performing content text recognition based on the image data sequence and outputting the text data sequence; Step S230: combining the image data sequence with an object detection method to extract the operation data sequence, wherein the operation data sequence includes cursor operation data and text operation data.

[0063] Specifically, keyframe extraction refers to extracting representative and informative frames from video or dynamic image sequences. Keyframes are typically frames with important information or changes within dynamic content, effectively representing a specific point in time or state. Keyframe extraction reduces the amount of image data processed while preserving the core information of the image content. Cursor operation data refers to interactive data such as cursor position changes, clicks, and scrolling controlled by the user via a mouse or touch device. By tracking the cursor's position and movements, we can understand the user's operations on the screen content and the areas of focus. Text operation data refers to the text input operations performed by the user on the screen, including keyboard key input, text selection, editing, and other behavioral data.

[0064] Upon receiving the real-time display content, keyframe extraction is performed first. The purpose of keyframe extraction is to select representative, information-rich frames from a continuous sequence of image frames. These frames are typically important content transition points in videos or moving images, effectively reducing the computational load of subsequent processing while preserving key image information. Tools used include inter-frame difference analysis and edge detection techniques. For the keyframes extracted from the real-time display content, Optical Character Recognition (OCR) technology is then used to extract the text information. OCR technology recognizes characters in an image and converts them into machine-processable text data.

[0065] Based on the image data sequence extracted from the displayed content, object detection and behavior recognition techniques are further used to extract the operation data sequence. The operation data sequence comprises two main parts: cursor operation data, which tracks the user's cursor movement and clicks, records user interaction behavior, and identifies the area of ​​user attention or the button being operated; and text operation data, which tracks the user's keyboard input behavior and identifies text editing, input, or selection operations on the screen. Object detection algorithms can be used to identify interactive elements in the displayed content (such as buttons, text boxes, etc.) and combine them with the cursor movement trajectory to determine the user's operation behavior. Through the above steps, the extracted text data, image data, and operation data are integrated into a structured content object dataset. This dataset includes: text data sequence: text information identified from images; image data sequence: image content extracted from keyframes; and operation data sequence: recorded user interaction data, including cursor operations and text input.

[0066] This step effectively extracts multi-dimensional data, including text, images, and operational information, from complex real-time displayed content, providing reliable data support for subsequent analysis, monitoring, and security alerts. It offers crucial technical support for realizing security alerts for digital displays based on real-time content monitoring, enhancing the system's real-time data processing capabilities and response speed.

[0067] Furthermore, by combining the preset content security warning rules and content recognition results, content security is judged, and the behavior pattern recognition results are compared with the standard behavior pattern to judge operation security. The content security warning rules include at least sensitive keywords, sensitive semantics, sensitive patterns, and sensitive symbols, and the standard behavior pattern includes at least mouse behavior pattern and keyboard behavior pattern.

[0068] Specifically, security alerts are issued by detecting the presence of sensitive keywords in real-time displayed content. Beyond simple keywords, sensitive semantics can be defined, which may indicate potential risks. Sensitive symbols may touch upon socially sensitive topics or violate certain platform rules. Sensitive symbols indicate content that should not be publicly displayed or requires special review and processing. Mouse behavior patterns refer to a user's normal mouse operation trajectory, click frequency, and dwell time. For example, users click buttons, move the cursor, and select menu items; these behavior patterns are used to determine if there are any anomalies. Keyboard behavior patterns refer to a user's keyboard operation habits, including the text content entered, the frequency and rhythm of keystrokes, etc. Standard keyboard behavior patterns can help the system identify potential malicious behaviors, such as brute-force attacks by automated scripts (bots).

[0069] First, the content is analyzed based on real-time display and content security alert rules. If sensitive words are identified, the content is marked as high-risk. Sensitive semantic rules can also be used to identify sensitive semantics, further enhancing content security assessment. For image content, sensitive pattern and symbol rules are used to identify whether images contain, for example, political symbols or inappropriate patterns, which will also trigger alerts. If the displayed content is determined to be non-compliant with security rules, a security alert signal is generated, prompting relevant personnel to conduct further inspections or take action.

[0070] When analyzing user action data, the system first analyzes whether user actions are abnormal based on preset standard behavior patterns: Mouse behavior patterns detect whether the user is performing normal clicks, drags, and selections. For example, a user might normally click an ad button on a webpage or browse information. Keyboard behavior patterns detect whether the user is typing text at a reasonable speed or frequently performing the same operation (such as entering the same character or password). If unusually high-frequency input is detected, it's suspected to be automated bot activity. The security of the operation is assessed by comparing the degree of match between user behavior and standard behavior patterns. If abnormal user behavior is detected, such as aggressive clicking or automated input, a security alert is generated and further analysis is performed.

[0071] The content and operational security assessments are interconnected. When displayed content is deemed unsafe, it will be further examined to see if it is accompanied by abnormal operating patterns. If both content and operational security assessments indicate a potential risk, a security warning will be generated.

[0072] By combining content security early warning rules and standard behavior patterns, more accurate content security monitoring and behavior pattern analysis can be achieved. Different content security rules and behavior patterns can be set according to actual application scenarios, making this method adaptable to various industries and needs.

[0073] Furthermore, the method also includes: extracting the content object dataset, the content security judgment result, and the operation security judgment result for temporary storage on the digital display; activating the integrated network module to call the temporarily stored results according to remote management instructions, and transmitting them to the remote management terminal, wherein the remote management instructions include periodic management instructions and dynamic management instructions; the remote management terminal uses the temporarily stored results as feedback information to update the content security warning rules and the standard behavior pattern, and sends the update results back to the integrated network module for remote control.

[0074] Specifically, internal temporary storage refers to temporarily storing the real-time collected content object dataset, content security judgment results, and operation security judgment results in the storage device inside the digital display. Remote management commands are instructions issued by a remote management terminal to control or manage the operation of the digital display. These commands can be periodic management commands (e.g., periodically querying or updating content security rules) or dynamic management commands (e.g., real-time adjustment of security warning rules, changing behavior monitoring standards, etc.). Periodic management commands are commands issued at predetermined time intervals, typically used for periodically updating, checking, or synchronizing data, such as periodically transferring internally stored data to the remote management system or periodically checking the system status. Dynamic management commands are commands issued for management operations required in real time. The integrated network module is a module inside the digital display responsible for connecting to the external network and transmitting data. It is typically used to transmit internal data to the remote management terminal or receive remote management commands and execute corresponding operations.

[0075] First, a dataset of real-time content is extracted, including content object datasets, content security assessment results, and operational security assessment results. This data reflects the currently displayed content and user behavior. All of this data is stored in the digital display's internal temporary storage, awaiting subsequent processing and use. In this way, the display can save and process necessary data without a remote connection.

[0076] When the remote management system needs to update the content security alert rules or behavior monitoring standards of the digital display, it sends a remote management command to the digital display. These commands can be periodic management commands for regularly synchronizing data and updating system configurations, or dynamic management commands for quickly adjusting system rules in the event of anomalies. Upon receiving the remote management command, the digital display transmits its internally stored temporary data (such as content object datasets and security assessment results) to the remote management terminal via its integrated network module. After receiving this data, the remote management terminal performs further analysis and updates the content security alert rules and standard behavior patterns based on the current situation. Once the update is complete, the remote management terminal feeds back the update results to the digital display's integrated network module. The digital display then adjusts its local content security rules and operation monitoring standards based on the returned update results. In this way, the digital display can adjust and optimize in real time according to external security conditions and management needs, ensuring it is always in the best security state.

[0077] By combining remote management commands with local data storage, more flexible device management and data updates can be achieved. Whether it's periodic safety checks or dynamic emergency responses, the remote management system can make precise adjustments through real-time data transmission and feedback mechanisms.

[0078] Furthermore, the method further includes: initializing the registration of the real-time verification display content and the real-time image frame sequence of the real-time display content based on the typical driving latency of the display driver component, and calculating image error features; adjusting the registration position of the real-time verification display content and the real-time display content based on the registration initialization result, and calculating the adjusted image error features; iteratively adjusting the registration position, calculating the image error features for each iteration, and obtaining an image error feature set; selecting the image error feature with the smallest feature value from the image error feature set as the verification image error feature, and comparing the verification image error feature with the verification error limit; if the verification image error feature does not meet the verification error limit, generating a hardware security warning command to issue a security warning.

[0079] Specifically, typical drive latency refers to the time delay that occurs from the time the display driver component receives the display signal to the time it completes signal transmission and display. This latency is caused by hardware, the optimization level of the driver, and various factors during signal processing. Registration initialization refers to the process of aligning the target image with the reference image during image processing to ensure they are spatially consistent. Verification error limit refers to a preset tolerance range used to measure whether image errors are within acceptable limits. If the error exceeds this limit, the displayed content is considered to have an unexpected error. Hardware safety warning commands are warning commands issued to the hardware when image errors in the displayed content exceed the tolerance range, indicating a display malfunction or safety hazard. This command can trigger further system protection or alarm measures.

[0080] First, the typical drive latency of the display driver components of a digital display is analyzed. To achieve more accurate image matching and verification, registration initialization is performed based on this latency. Registration initialization refers to the system setting an initial image alignment position by considering the impact of drive latency, and performing preliminary registration alignment between the real-time verification display content and the image frame sequence of the real-time display content. After initial registration, further adjustments are made to the registration positions of the real-time display content and the real-time verification display content. By repeatedly adjusting the registration positions, image errors caused by display drive latency, hardware differences, or signal transmission inconsistencies can be minimized. After each adjustment, the system calculates the image error characteristics, which reflect the gap between the displayed content and the reference content during the registration process.

[0081] The system iterates through multiple registration adjustments, calculating a new set of error features with each adjustment. All error features are collected to form an image error feature set. This set contains the error changes between the displayed content and the reference content during the multiple registration adjustments. The image error feature with the smallest feature value is selected from the error feature set; this feature represents the most accurate registration result. Then, this error feature is compared with a preset verification error limit. The verification error limit is a predefined threshold used to determine if the error is within an acceptable range. If the error feature exceeds the verification error limit, it indicates a significant problem in the registration process.

[0082] If the image error characteristics exceed the verification error limit, the system considers the displayed content to have a serious deviation or malfunction, which may affect the display effect or cause security risks. Therefore, a hardware security warning instruction is generated based on this result. This instruction can trigger hardware-level warning mechanisms, such as screen flickering, content locking, or issuing an alarm to maintenance personnel, prompting them to check or repair the issue.

[0083] By meticulously adjusting the image registration, the displayed content can be precisely calibrated to match the real-time image frame sequence, ensuring visual consistency of the displayed content and avoiding content errors caused by hardware latency or display deviations.

[0084] In summary, the digital display security early warning method based on real-time content monitoring provided in this application has the following technical effects:

[0085] 1. Through embedded content acquisition and real-time monitoring technology, comprehensive monitoring and real-time security warnings of digital display content are achieved. It can automatically identify and detect security risks in the displayed content, enhancing the system's ability to prevent malicious content or operations.

[0086] 2. Through hardware copying and decoding, efficient acquisition and accuracy of real-time display signals are ensured, providing high-quality video frame sequences, laying a solid foundation for subsequent content recognition and security judgment, and improving the accuracy and stability of content monitoring.

[0087] 3. By combining display driver load information, packet capture performance has been optimized, making the real-time content acquisition process more efficient without affecting display performance. This ensures the efficiency and accuracy of content security judgment and improves the precision of packet capture operations.

[0088] In summary, any step of the method described above can be stored as a computer instruction or program in an unrestricted computer memory, and can be called and identified by an unrestricted computer processor to implement any method in the embodiments of this application, without any additional restrictions.

[0089] Furthermore, the "first" or "second" mentioned above not only represents a sequential relationship but also a specific concept and / or refers to the possibility of selecting individual or all of multiple elements. Clearly, those skilled in the art can make various modifications and variations to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A digital display security early warning method based on real-time content monitoring, characterized in that, The method includes: Real-time content capture is performed based on the content acquisition component embedded in the digital display, generating real-time display content. The real-time displayed content is parsed and classified for object extraction to generate a content object dataset. This dataset includes text data sequences, image data sequences, and operation data sequences. The operation data sequences refer to user interaction-related data, including mouse clicks, scrolling, dragging, and keyboard input. The process includes: extracting keyframes from the real-time displayed content and outputting the keyframe extraction results as the image data sequences; performing text recognition based on the image data sequences and outputting the text data sequences; and combining the image data sequences with an object detection method to extract the operation data sequences, which include cursor operation data and text operation data. The embedded edge computing module of the digital display is activated to perform content recognition on the text data sequence and the image data sequence, and to perform behavior pattern recognition on the operation data sequence. By combining the preset content security warning rules and content recognition results, content security is judged, and the behavior pattern recognition results are compared with standard behavior patterns to judge operation security. Based on the content security assessment results and the operation security assessment results, a security warning command is generated to respond to the security warning.

2. The digital display security early warning method based on real-time content monitoring as described in claim 1, characterized in that, Based on the content acquisition component embedded in the digital display, real-time content capture is performed to generate real-time display content, including: The hardware layer replicator of the content acquisition component is activated to perform front-end replication and acquisition of the display signal transmitted from the signal source to obtain the real-time display signal stream; The real-time display signal stream is transmitted to a hardware decoder for signal decoding, a real-time image frame sequence is obtained, and the real-time image frame sequence is output as the real-time display content.

3. The digital display security early warning method based on real-time content monitoring as described in claim 2, characterized in that, The real-time display signal stream is transmitted to a hardware decoder for signal decoding to obtain a real-time image frame sequence. Afterwards, the process further includes: The interactive target display scene is identified, and the corresponding UI layout information is obtained. The functional characteristics of the UI layout are used as the dividing constraints to identify the corresponding key display areas. Based on the relative coordinates of the key display area, fit the key display area with the real-time display signal flow; Based on the fitting results, the real-time display signal stream is serialized and extracted, and the serialized extraction result is output as a new real-time display signal stream.

4. The digital display security early warning method based on real-time content monitoring as described in claim 2, characterized in that, Real-time content capture based on the content acquisition component embedded in the digital display also includes: Interactively acquire the display driver component of the digital display, obtain driver load information, and evaluate the available packet capture performance of the display driver component based on the driver load information; By combining the available packet capture performance with the display parameter information of the digital display, real-time packet capture parameters are calculated and obtained; The software layer packet capture module of the content acquisition component is activated, and the post-packet capture of the digital display is performed based on the real-time packet capture parameters to obtain the real-time verification and display content.

5. The digital display security early warning method based on real-time content monitoring as described in claim 1, characterized in that, Combining preset content security warning rules and content recognition results, content security is judged, and operation security is judged by comparing behavior pattern recognition results with standard behavior patterns. The content security warning rules include at least sensitive keywords, sensitive semantics, sensitive patterns, and sensitive symbols, and the standard behavior patterns include at least mouse behavior patterns and keyboard behavior patterns.

6. The digital display security early warning method based on real-time content monitoring as described in claim 1, characterized in that, The method further includes: Extract the content object dataset, the content security judgment result, and the operation security judgment result for temporary storage within the digital display. According to the remote management command, the integrated network module is activated to call the temporarily stored results in the machine and transmit them to the remote management terminal. The remote management command includes periodic management command and dynamic management command. The remote management terminal uses the temporarily stored results in the machine as feedback information to update the content security warning rules and the standard behavior pattern, and sends the update results back to the integrated network module for remote control.

7. The digital display security early warning method based on real-time content monitoring as described in claim 4, characterized in that, The method further includes: Based on the typical driving latency of the display driver component, the registration initialization of the real-time verification display content and the real-time image frame sequence of the real-time display content is performed, and the image error characteristics are calculated. Using the registration initialization result as the reference registration position, the registration position of the real-time verification display content and the real-time display content is adjusted, and the adjusted image error characteristics are calculated. The registration position is adjusted iteratively, and the image error features are calculated for each iteration to obtain the image error feature set. The image error feature with the smallest feature value is selected from the set of image error features as the verification image error feature, and the verification image error feature is compared with the verification error limit. If the verification image error features do not meet the verification error limit, a corresponding hardware security warning command will be generated to issue a security warning.

Images