Digital display safety early warning method based on real-time content monitoring

Abstract

The invention discloses a digital display safety early warning method based on real-time content monitoring, and belongs to the technical field of digital display system information safety, and the method comprises the steps: carrying out the real-time content capturing based on a content collection assembly embedded in a digital display; generating a content object data set; activating an embedded edge calculation module of the digital display, performing content identification on the character data sequence and the image data sequence, and performing behavior pattern identification on the operation data sequence; in combination with a preset content safety early warning rule and the content recognition result, content safety judgment is carried out, the behavior pattern recognition result is compared with a standard behavior pattern, and operation safety judgment is carried out; and generating a safety early warning instruction to perform safety early warning response. The technical problems that in the prior art, a digital display lacks a real-time content monitoring and safety early warning mechanism, and therefore potential safety hazards such as content violation, sensitive information leakage and operation abnormity cannot be found and responded in time are solved.

Description

Technical Field

[0001] The invention relates to the technical field of digital display system information security, and in particular to a digital display security early warning method based on real-time content monitoring. Background Art

[0002] With the rapid development of information technology, digital displays have been widely used in various application scenarios, including public advertising screens, information release systems, corporate display equipment, and smart home devices. These digital displays are not only used for information transmission and display, but also bear the responsibility of information security and content management to a certain extent.

[0003] At present, most digital displays focus on the display effect of content and user interaction experience, lacking effective real-time content monitoring and security warning mechanisms. Existing content management systems often rely on background servers for content review and management, and have problems such as response delays, insufficient data transmission security, and untimely detection of abnormal operations. In addition, with the continuous escalation of network attack methods, digital displays face multiple security threats such as content tampering, sensitive information leakage, and malicious operations. Therefore, there is an urgent need for a solution that can monitor content and operation behaviors in real time on the display side, and promptly identify and warn of potential security risks, so as to improve the overall security and reliability of digital display systems. Summary of the invention

[0004] This application provides a digital display security warning method based on real-time content monitoring, aiming to solve the technical problem that digital displays in the prior art lack real-time content monitoring and security warning mechanisms, resulting in the inability to promptly detect and respond to security risks such as content violations, sensitive information leakage and abnormal operations.

[0005] In view of the above problems, the present application provides a digital display security warning method based on real-time content monitoring.

[0006] The present application provides a digital display security warning method based on real-time content monitoring, the method comprising: based on a content acquisition component embedded in a digital display, performing real-time content capture to generate real-time display content; parsing the real-time display content to extract classified objects to generate a content object data set, wherein the content object data set comprises a text data sequence, an image data sequence and an operation data sequence; activating an embedded edge computing module of the digital display, performing content recognition on the text data sequence and the image data sequence, and performing behavior pattern recognition on the operation data sequence; combining preset content security warning rules with content recognition results, performing content security judgment, and comparing the behavior pattern recognition results with the standard behavior pattern to perform operation safety judgment; generating a security warning instruction according to the content security judgment result and the operation safety judgment result to perform a security warning response.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: By adopting a digital display security warning method based on real-time content monitoring, including real-time content capture, classified object extraction, embedded edge computing for content and behavior recognition, security judgment based on preset rules, and generation of warning instructions, the technical problem of the lack of real-time content monitoring and security warning mechanism for digital displays in the prior art, which leads to the inability to timely detect and respond to security risks such as content violations, sensitive information leakage and abnormal operation, is solved, thereby achieving the technical effect of improving the security and reliability of the digital display system.

[0008] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flowchart of a digital display security warning method based on real-time content monitoring is provided for an embodiment of the present application; Figure 2 A schematic flow chart of generating a content object data set in a digital display safety warning method based on real-time content monitoring is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0010] The overall idea of ​​the technical solution provided by this application is as follows: The embodiment of the present application provides a digital display security warning method based on real-time content monitoring. First, the display content is captured in real time by the content acquisition component embedded in the digital display, and the corresponding real-time display data is generated. Subsequently, the display content is parsed, and text, images and operation data are extracted to form a content object data set. Next, the embedded edge computing module is activated to perform content recognition on the text and images, and behavioral pattern recognition on the operation data. Then, combined with the preset content security warning rules, the recognition results are subjected to security judgment of the content and operation. Finally, a security warning instruction is generated based on the judgment result to trigger the corresponding security response measures. Real-time monitoring and security warning of the digital display during content display and operation are realized to ensure that the displayed content is legal and safe, and to prevent potential security risks.

[0011] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.

[0012] Examples, such as Figure 1As shown, the embodiment of the present application provides a digital display security warning method based on real-time content monitoring, the method comprising: Step S100: Based on the content acquisition component embedded in the digital display, real-time content is captured to generate real-time display content.

[0013] Specifically, a digital display is a screen device that displays images or text through an electronic device, and is usually used in scenes such as billboards, monitoring equipment, televisions, computer monitors, etc. It can display images or text information transmitted from a signal source (such as a computer, camera, video player, etc.). The content acquisition component refers to the hardware or software built into the digital display, which is used to capture the images and information presented on the display in real time. The content acquisition component can obtain the display content through a hardware interface (such as copying the signal stream) or a software interface (such as system monitoring). Real-time content capture refers to the continuous and instant acquisition of the content currently displayed on the display, ensuring that the captured content is the latest and there is no delay. The captured content is usually an image or video data stream, and also includes a text data stream and a user operation data stream.

[0014] The content acquisition component embedded in the digital display will first activate the replicator at the hardware layer. The role of the replicator is to replicate the signal stream transmitted from the signal source (such as a computer or video player) to the display in real time. The replicator will capture the data of the display signal stream in real time and transmit it to the subsequent decoding module. Specifically, the replicator will not display the image directly, but will extract the original video stream signal for subsequent processing. This ensures the timeliness and accuracy of information acquisition. Once the signal stream is captured, it will be transmitted to the hardware decoder. The decoder decodes the signal stream into a series of image frame sequences, which represent the continuously changing images on the display. Finally, the decoded image frame sequence is output as real-time display content.

[0015] By using content acquisition components and hardware copiers to capture real-time display signal streams, combined with hardware decoders to generate image frame sequences, it can be ensured that the captured real-time content is not only timely but also accurate.

[0016] Step S200: parsing the real-time display content to extract classified objects and generate a content object data set, wherein the content object data set includes a text data sequence, an image data sequence and an operation data sequence.

[0017] Specifically, the content object data set is a data set obtained through the classification object extraction step. The data set includes different types of objects extracted from the real-time display content, usually including text data, image data, and operation data. The text data sequence refers to the text information sequence extracted from the real-time display content. The sequence can include various types of text information, such as titles, prompts, menu items, warning information, etc. Through text recognition technology (such as OCR, Optical Character Recognition), the text in the image can be converted into a machine-readable text format. The image data sequence refers to the image or image frame extracted from the real-time display content. These image data can be static images or dynamic image sequences. The image data sequence can extract effective image information from the display content through image recognition technology (such as target detection, image segmentation, etc.). The operation data sequence refers to the operation data related to user interaction, such as mouse clicks, scrolling, dragging, keyboard input, etc. The operation data sequence records the user's interactive behavior on the display content, which is usually obtained by monitoring input devices (such as mouse, keyboard) or analyzing the operation elements in the display content.

[0018] First, it is necessary to obtain the real-time content displayed on the digital display. Using the embedded content acquisition component, through the pre- or post-packet capture technology, the real-time display signal is obtained from the signal stream of the display and converted into processable content data. After obtaining the real-time display content, the classification object extraction will be performed next. For areas containing text, OCR technology is used to extract the text information in the image. For image elements in the display content, image processing algorithms such as image segmentation and object recognition are used to extract related images or image frames. For user input operations (such as mouse clicks, keyboard input, etc.), the operation data sequence is extracted by monitoring the operation of the input device or identifying the operation area (such as button clicks or menu selections).

[0019] By extracting classified objects from real-time display content, all extracted data are integrated into a structured content object dataset, which provides data support for subsequent content analysis and security warnings.

[0020] Through this step, structured data can be efficiently extracted from complex display content, which not only improves the efficiency of display content processing, but also lays the foundation for subsequent data analysis, behavior recognition and security monitoring.

[0021] Step S300: activating the embedded edge computing module of the digital display, performing content recognition on the text data sequence and the image data sequence, and performing behavior pattern recognition on the operation data sequence.

[0022] Specifically, the embedded edge computing module is a computing module integrated into the device. It is usually located between the data source and the cloud and is responsible for processing data in the local environment where the data is generated. The advantage of this module is that it reduces dependence on the cloud and can process data in real time at the source of data generation, reducing latency and improving processing efficiency. It is widely used in systems such as smart devices, IoT devices, and digital displays. Content recognition refers to the analysis of real-time display content through image recognition, text recognition and other technologies to identify the elements contained therein. Behavioral pattern recognition refers to identifying the user's operating behavior pattern by analyzing the user's operating data sequence.

[0023] First, the embedded edge computing module of the digital display is activated to receive the real-time data stream collected from the display. The edge computing module is located locally and processes the data directly to reduce the delay of data transmission and improve the real-time response capability.

[0024] For text data sequences and image data sequences, the edge computing module will process them through an integrated content recognition algorithm. For operation data sequences, that is, user interaction behaviors (such as cursor operations, mouse clicks, keyboard input, etc.), the embedded edge computing module will use behavioral pattern recognition technology for analysis. After content recognition and behavioral pattern recognition, the edge computing module will combine the recognition results for comprehensive analysis. If some content or behavior is detected to be abnormal, an early warning will be issued.

[0025] By activating the embedded edge computing module to perform real-time content recognition and behavior pattern recognition, and analyzing and processing real-time data locally, the digital display system becomes more intelligent and autonomous, capable of efficiently processing and analyzing real-time data, and improving the ability to respond to complex situations.

[0026] Step S400: Combining the preset content security warning rules with the content recognition results, performing content security judgment, and comparing the behavior pattern recognition results with the standard behavior pattern to perform operation security judgment.

[0027] Specifically, content security warning rules refer to a set of predefined rules used to determine whether the real-time displayed content has potential security risks or illegal content. Content security warning rules usually include the following categories: sensitive keywords, sensitive semantics, sensitive patterns, sensitive signs, etc. Content recognition results refer to the recognition conclusions obtained after processing the real-time displayed content through various technical means. Behavior pattern recognition refers to the extraction of user operation patterns through monitoring and analysis of user interaction behaviors, and judging whether there are abnormal or malicious behaviors based on these operation patterns. Common behavior patterns include mouse click behavior, keyboard input behavior, sliding operation, etc. Standard behavior patterns refer to the general rules followed by user operations and interaction behaviors in normal use scenarios. Operation security judgment refers to judging whether there are security risks or abnormal behaviors in the operation based on the analysis of user behavior patterns. If the user behavior deviates greatly from the standard behavior pattern (for example, frequently clicking the same link, quickly entering passwords, etc.), it is judged as a potential security risk.

[0028] First, the real-time displayed content is analyzed and combined with the content security warning rules to determine whether the content meets the security requirements. If an advertisement or page contains political symbols or violent images, sensitive pattern recognition technology (such as convolutional neural networks) is used to analyze the image to determine whether it meets the security regulations. Natural language processing (NLP) technology is used to analyze the semantics of the content to determine whether there are any illegal or dangerous remarks, and further generate content security warnings.

[0029] Record the user's mouse click behavior and sliding track on the interface to determine whether there are abnormal behaviors such as frequent clicks on a link and rapid browsing of multiple pages. Record the user's input behavior and monitor the input frequency, character type, etc. For example, if abnormal rapid input or repeated input of certain characters is detected, it will be determined as malicious script behavior or brute force cracking. Combine multiple behavior patterns, such as monitoring mouse clicks and keyboard input at the same time, to detect whether there are any behaviors that deviate from the standard pattern.

[0030] Comprehensively analyze the results of content security identification and operation security identification. For example, if an advertisement contains sensitive keywords and the user shows abnormal clicking behavior, the content and behavior security risks will be combined to generate a double security warning. When both the content and the operation behavior trigger an early warning, a more urgent security response will be generated to notify the administrator or take automated processing measures (such as blocking advertisements, restricting access, etc.).

[0031] Through dual monitoring of content and operations, it can identify potential multi-dimensional security risks and can flexibly adapt to different application scenarios and security requirements. It is widely used in e-commerce, finance, social media and other industries, improving cross-domain security monitoring capabilities.

[0032] Step S500: Generate a security warning instruction to respond to the security warning based on the content security identification result and the operation security identification result.

[0033] Specifically, the content security judgment result refers to the judgment result obtained after analyzing the real-time displayed content according to the preset content security warning rules. The operation security judgment result refers to the judgment of whether the user's behavior conforms to normal operation through the analysis of the user's behavior pattern and the standard behavior pattern. The security warning instruction refers to the response instruction generated according to the content security judgment result and the operation security judgment result, which is used to notify the administrator or take automated measures to prevent further security risks. These instructions can be transmitted in a variety of ways, including email notification, SMS, message push, etc., or trigger automated processing, such as blocking suspicious content, locking accounts, etc. The security warning response refers to the measures taken according to the warning instruction.

[0034] First, we will combine the content security and operation security results to conduct a comprehensive assessment of the current security situation. If sensitive information is found in the content analysis and the user behavior is abnormal, it is considered that there is a high security risk, such as attack behavior, fraud, malicious program, etc.

[0035] Based on the results of the comprehensive assessment, one or more security warning instructions are generated. These instructions will clearly indicate the type of potential security threats and set response measures based on the risk level.

[0036] For example: Low-risk warning: Sensitive keywords appear in the content, but the user behavior is normal. At this time, a low-risk security warning instruction is generated to remind the administrator to conduct routine checks. High-risk warning: The content contains sensitive patterns or illegal sentences, and the user's behavior pattern shows abnormalities. A high-risk security warning instruction is generated, the administrator is notified immediately, and automated shielding measures are initiated. Triggering security warning response: According to the generated security warning instructions, response measures are taken automatically or manually. For example: If it is determined that the content contains sensitive words or illegal advertisements, the content can be automatically hidden, deleted or blocked to prevent it from continuing to affect users. If the user behavior is abnormal, the user's operations will be restricted, such as freezing the account, restricting access rights, etc. According to the warning instructions, security personnel or administrators can also be notified to conduct further manual intervention and processing, especially in high-risk situations.

[0037] The notification and processing of warning information include: Automated response: For example, automatically locking an account, automatically blocking ads or content, prohibiting users from performing specific operations, etc. Manual response: In some cases, notifying administrators or security personnel to view detailed security logs through the backend management system to confirm whether further manual inspection or intervention is required.

[0038] By combining the content security judgment results and the operation security judgment results to generate security warning instructions, it is possible to monitor platform content and user behavior in real time, respond quickly when potential risks are discovered, and provide timely feedback of warning instructions to administrators, which helps to promptly control and respond to possible security incidents.

[0039] Furthermore, based on the content acquisition component embedded in the digital display, real-time content capture is performed to generate real-time display content, including: activating the hardware layer copier of the content acquisition component, pre-copying and acquiring the display signal transmitted from the signal source, and obtaining a real-time display signal stream; transmitting the real-time display signal stream to the hardware decoder for signal decoding, obtaining a real-time image frame sequence, and outputting the real-time image frame sequence as the real-time display content.

[0040] Specifically, a hardware layer replicator is a specially designed hardware device or module used to "copy" display signals during signal transmission without affecting the normal operation of the signal source or display. It is equivalent to an intermediate node that can obtain the signal stream in real time and transmit it to the subsequent decoding module. The replicator can extract the signal data with very low latency. The display signal stream refers to the electronic signal transmitted from the signal source (such as a computer, video player or other device) to the digital display. These signals contain all the information of the display content, including images, videos, text, etc. The display signal stream can be an image frame sequence or a video stream. The hardware decoder is a hardware device used to decode the transmitted encoded signal (such as a compressed video stream) into displayable image data. The decoder can parse and convert the encoded signal into a series of static or dynamic image frame sequences, which represent the real-time display content on the display. The real-time image frame sequence is a sequence of continuous image frames decoded by the hardware decoder. Each frame of the image represents the content displayed on the display at a certain moment. The continuity of the frame sequence ensures the real-time and dynamic nature of the display content, and is suitable for capturing video streams and dynamic content.

[0041] First, the content acquisition component embedded in the digital display activates the hardware layer replicator. The role of the replicator is to copy the display signal stream from the signal source in real time through a special hardware interface without affecting the normal operation of the signal source and the display. The replicator captures the electronic signal output from the signal source (such as a computer, video player, monitoring system, etc.). By copying the signal stream, the content acquisition component can obtain the signal data in advance.

[0042] After acquiring the signal stream, the replicator will transmit the real-time display signal stream to the subsequent decoding module. The pre-capture here refers to the acquisition in the early stage of signal stream transmission to ensure that the display content is acquired as early as possible to avoid delays in the subsequent processing. After the replicator acquires the signal stream, these signals will be transmitted to the hardware decoder for decoding. The function of the hardware decoder is to decode the compressed or encoded signal stream into the original image or video content. The decoder will decode the video stream into continuous image frames to form a real-time image frame sequence. These image frames represent the display content on the display at every moment. Assuming that the signal stream comes from a video advertisement being played, the decoder will convert the video signal into frame-by-frame images to form a video image frame sequence. The decoded image frame sequence is the real-time display content obtained. This sequence contains the image or text information displayed on the display at every moment.

[0043] Through the collaboration of hardware copiers, decoders and other tools, real-time display content on digital displays can be obtained efficiently and accurately. This ensures the stability and accuracy of digital displays in various complex environments, and can provide important data support, especially in scenarios with strict requirements for real-time content monitoring and security detection.

[0044] Furthermore, the real-time display signal stream is transmitted to a hardware decoder for signal decoding to obtain a real-time image frame sequence, and then, it also includes: interacting with the target display scene, obtaining the corresponding UI layout information, and identifying the corresponding key display area based on the functional characteristics of the UI layout as a division constraint; fitting the key display area and the real-time display signal stream according to the area relative coordinates of the key display area; based on the fitting result, serializing and extracting the real-time display signal stream, and outputting the serialized extraction result as the new real-time display signal stream.

[0045] Specifically, the interactive target display scene refers to the interactive area or the overall structure of the display content presented on the digital display. For example, the display contains multiple modules, such as the advertising display area, information panel, operation interface, etc., and the interactive target display scene is a collection of these modules. UI layout information refers to the position, size, hierarchical relationship and other information of each element in the digital display interface. UI layout information includes but is not limited to the position and distribution of elements such as buttons, text boxes, picture display areas, and navigation bars. Among them, functional characteristics refer to the functional properties of each display area in the UI layout, for example, a certain area is used to display advertising content, a certain area is used to display real-time data, and another area is used for user interaction (such as buttons, input boxes, etc.). Functional characteristics provide a basis for identifying and dividing display areas. Key display areas refer to areas with important or sensitive functions on the display screen. These areas usually involve the display of important information or interfaces for interacting with users, such as advertising display areas, alarm areas, or important data panels. Area relative coordinates refer to the position and size of a display area relative to the overall size of the display screen. It is usually expressed by a coordinate system to accurately locate the size and position of the area on the screen. Coordinates can be absolute or relative to a parent area. Sequential extraction refers to extracting a specific sequence of images from a signal stream and formatting and organizing these images for subsequent processing and analysis. The extracted image frame sequence is output in a new format, usually for further data analysis, storage or display.

[0046] First, the current display scene needs to be obtained from the digital display. The interactive target display scene includes all interactive or important display areas on the screen, such as areas for displaying advertisements, areas for displaying real-time data, or interactive areas for user input. Based on the UI layout information, these important areas are identified, usually through software interface extraction or image recognition.

[0047] The process of obtaining UI layout information usually requires analyzing the display content of the digital display to extract the location and function of all elements of the display interface. UI layout information not only contains the location coordinates of these elements (such as the width and height of the screen), but also their functions (such as advertisements, information, buttons, etc.). This information can be obtained by software parsing the currently displayed interface or through the system's API interface. Based on the UI layout information, different areas on the screen are divided according to their functional characteristics.

[0048] After obtaining the UI layout information and dividing the functional features, the key display areas in the display scene are further identified. These key areas usually contain sensitive information or parts that need special monitoring, such as advertising content, alarm information, real-time data display areas, etc.

[0049] After identifying the key display areas, these key areas are fitted with the image data in the real-time display signal stream according to the relative coordinates in the UI layout information. The fitting process uses an algorithm to match the image area of ​​each key display area with the corresponding image area in the display signal stream, ensuring that the content of each image in the display signal stream is connected with the actual display content to avoid image distortion or misalignment. After the fitting process is completed, a new image sequence is extracted from the real-time display signal stream based on the fitting results. Serialized extraction is the reorganization and formatting of the image frame sequence to ensure that the extracted image sequence can accurately reflect the content of each key display area. Finally, the results of serialized extraction will be generated into a new real-time display signal stream. This new signal stream contains the fitted and extracted image frame sequence, which can more accurately reflect the content of the key areas on the display for subsequent security analysis and content monitoring.

[0050] This step enables accurate monitoring and capture of important display area content, ensuring the security and compliance of real-time display content.

[0051] Furthermore, real-time content capture is performed based on a content acquisition component embedded in a digital display, and the method also includes: interactively acquiring a display driver component of the digital display, acquiring driver load information, and evaluating available packet capture performance of the display driver component based on the driver load information; calculating and acquiring real-time packet capture parameters by combining the available packet capture performance with display parameter information of the digital display; activating a software layer packet capture module of the content acquisition component, and performing post-packet capture of the digital display based on the real-time packet capture parameters to acquire real-time verified display content.

[0052] Specifically, the display driver component is one of the core components of a digital display, responsible for controlling the generation and update of the display content. It receives image signals transmitted by a computer or other signal source, converts these signals into a format suitable for the display, and controls the brightness and color of each pixel of the display screen through current and voltage. The display driver component usually includes a hardware part (such as a graphics card) and a corresponding driver. The driver load information refers to the load that the display driver component bears during operation, which is usually manifested as the refresh rate, resolution, color depth, and complexity of the display content. The larger the driver load, the higher the amount of image data to be processed and the computing pressure of the display signal. The load information can reflect the working status of the driver component and help evaluate the processing capacity of the display. The available packet capture performance refers to the ability of the display driver component to effectively capture signals under normal working load. Specifically, it is an indicator to evaluate whether the display driver can stably and accurately perform content capture tasks under high load. Display parameter information includes the technical specifications and settings of the display, such as resolution, refresh rate, color depth, display format, etc. These parameters directly affect the image quality and content presentation effect of the display, and are crucial to the accuracy and effectiveness of the packet capture process. Real-time packet capture parameters refer to the relevant settings required when acquiring the display signal stream, such as packet capture frequency, sampling interval, data extraction accuracy, etc. These parameters are usually automatically adjusted by the system according to the current display driver load and the working status of the display to ensure that the display performance is not affected while the content is efficiently captured. The software layer packet capture module is a tool that controls and manages the content capture process through software means. Through the interface of the operating system and the display driver, the display content can be extracted in real time without affecting the display effect. The software layer packet capture module is usually used for content verification, performance monitoring and data analysis. Post-capture capture refers to the method of collecting content after the display content has been generated and started to be displayed on the display. Unlike pre-capture (signal source signal capture), post-capture capture focuses more on the actual output content of the display, and is usually used to verify whether the content is displayed as expected and check whether the display has errors or faults. Real-time verification of display content refers to capturing and checking whether the actual content presented on the display meets expectations through the packet capture module. It is a key link to ensure the correctness, accuracy and security of the display content, especially in advertising monitoring, information display and security warning.

[0053] First, it is necessary to interact with the display driver component of the digital display to obtain the operating status and load information of the display driver. The display driver component is responsible for processing the image data on the display and driving the display to display. By interacting with the display driver component, it is possible to monitor the workload of the display in real time and understand the current refresh rate, resolution, display content complexity and other information of the display. Once the connection with the display driver component is established, the load information of the display driver is extracted. The load information describes the current working pressure of the display. For example, if the display is displaying dynamic video at high resolution, the load of the display driver will be high, while if it is displaying static text, the load will be lower.

[0054] Based on the acquired driver load information, the available packet capture performance of the display driver component is evaluated. This is used to understand the driver component's ability to effectively support content capture under the current display load. For high load situations, the packet capture parameters need to be adjusted to avoid adverse effects on display performance. After evaluating the display driver's packet capture performance, the real-time packet capture parameters are calculated in combination with the display's technical specifications (such as resolution, color depth, refresh rate, etc.). These parameters define the specific details of the packet capture operation, such as sampling frequency, image accuracy, packet capture range, etc. After the real-time packet capture parameters are calculated, the software layer packet capture module is activated. The software layer packet capture module is the software tool in the system responsible for actually capturing the display content. Through the interface between the operating system and the display driver component, the content on the display is extracted in real time, each frame of the image is captured, and processed according to the preset packet capture parameters.

[0055] After activating the packet capture module, post-capture acquisition begins. This means that the acquisition will be performed from the image content already generated on the display, rather than from the signal source. According to the real-time packet capture parameters, the content on the display is continuously captured and stored.

[0056] Finally, the captured display content will be used for real-time verification to check whether the display content is correct. This includes comparing the image content displayed on the monitor with the expected content to detect whether there are display errors, image distortion, color deviation and other problems. The verification process can be achieved through image processing algorithms, data comparison tools, etc.

[0057] This step ensures that the content capture and verification of digital displays are efficient and accurate, and provides reliable technical support for subsequent security monitoring and content verification.

[0058] Further, such as Figure 2As shown, the real-time display content is parsed to extract classified objects and generate a content object data set, wherein the content object data set includes a text data sequence, an image data sequence and an operation data sequence, including: step S210: extracting key frames from the real-time display content, and outputting the key frame extraction result as the image data sequence; step S220: performing content text recognition based on the image data sequence, and outputting the text data sequence; step S230: combining the image data sequence with the target detection method to extract the operation data sequence, wherein the operation data sequence includes cursor operation data and text operation data.

[0059] Specifically, keyframe extraction refers to extracting representative and informative frames from a video or dynamic image sequence. Keyframes are usually frames with important information or changes in dynamic content, which can effectively represent a specific time point or state. Through keyframe extraction, the amount of image data processed can be reduced while maintaining the core information of the image content. Cursor operation data refers to interactive data such as position changes, click actions, scrolling, etc. of the cursor controlled by the user through a mouse or touch device on the screen. By tracking the position and movement of the cursor, the user's operations on the screen content and the area of ​​attention can be understood. Text operation data refers to the text input operations performed by the user on the screen, including keyboard key input, text selection, editing and other behavioral data.

[0060] After receiving the real-time display content, the key frame extraction is first performed on the content. The purpose of key frame extraction is to filter out representative frames with large amounts of information from a continuous sequence of image frames. These frames are usually important content switching points in videos or dynamic images, which can effectively reduce the amount of computation for subsequent processing and retain the key information of the image. The tools used include inter-frame difference analysis, edge detection and other technologies. For the key frames extracted from the real-time display content, the content text recognition technology (OCR) is then used to extract the text information therein. OCR technology recognizes the characters in the image and converts them into text data that can be processed by the machine.

[0061] According to the image data sequence extracted from the displayed content, the object detection and behavior recognition technology is further used to extract the operation data sequence. The operation data sequence includes two main parts: cursor operation data, which tracks the movement and clicks of the user's cursor, records the user's interactive behavior, and identifies the area of ​​user attention or the button being operated. Text operation data, which tracks the user's keyboard input behavior and identifies the user's text editing, input or selection operations on the screen. The object detection algorithm can be used to identify interactive elements (such as buttons, text boxes, etc.) in the displayed content, and determine the user's operation behavior in combination with the movement trajectory of the cursor. Through the above steps, the extracted text data, image data and operation data are integrated into a structured content object dataset. The dataset includes: text data sequence: text information recognized from the image. Image data sequence: image content extracted from the key frame. Operation data sequence: recorded user interaction data, including cursor operation and text input.

[0062] Through the implementation of this step, multi-dimensional data, including text, images and operation information, can be effectively extracted from complex real-time display content, thus providing reliable data support for subsequent analysis, monitoring and safety warning. It provides key technical support for the realization of digital display safety warning based on real-time content monitoring, and can improve the system's processing ability and response speed for real-time data.

[0063] Furthermore, the preset content security warning rules are combined with the content recognition results to perform content security judgment, and the behavior pattern recognition results are compared with the standard behavior patterns to perform operation security judgment, wherein the content security warning rules include at least sensitive keywords, sensitive semantics, sensitive patterns, and sensitive signs, and the standard behavior patterns include at least mouse behavior patterns and keyboard behavior patterns.

[0064] Specifically, security warnings are issued by detecting whether sensitive keywords are contained in the real-time displayed content. In addition to simple keywords, some sensitive semantics can also be defined, which may point to potential risks. Sensitive patterns may touch on socially sensitive topics or violate certain platform rules. Sensitive signs indicate that the content should not be displayed publicly or requires special review and processing. Mouse behavior patterns refer to the user's normal mouse operation trajectory, click frequency, dwell time, etc. For example, the user will click a button, move the cursor, select a menu item, etc., and these behavior patterns are used to determine whether there are abnormalities. Keyboard behavior patterns refer to the user's keyboard operation habits, including the text content entered, the frequency and rhythm of keystrokes, etc. Standard keyboard behavior patterns can help the system identify potential malicious behaviors, such as brute force cracking by automated scripts (robots).

[0065] First, the content is analyzed based on the real-time display content and combined with the content security warning rules. If sensitive words are identified, the content is marked as high-risk content. Using sensitive semantic rules, sensitive semantics can also be identified to further strengthen the security assessment of the content. For image content, sensitive pattern and sensitive symbol rules are used to identify whether the image contains, for example, political symbols or inappropriate patterns, which will also trigger warnings. If the displayed content is judged to be inconsistent with security rules, a security warning signal is generated to prompt relevant personnel to further check or take action.

[0066] When analyzing the user's operation data, we first analyze whether the user's operation is abnormal based on the preset standard behavior pattern: through the mouse behavior pattern, we detect whether the user performs normal click, drag, and selection operations. For example, the user normally clicks on the ad button on the web page, or browses information. Through the keyboard behavior pattern, we can detect whether the user enters text at a reasonable speed, or whether the same operation is frequently performed (such as entering the same character or password). If unconventional high-frequency input is detected, it will be suspected that it is an automatic operation by a robot. The safety of the operation is judged by comparing the degree of match between the user behavior and the standard behavior pattern. If there is an abnormality in the user behavior, such as violent clicks, automated input, etc., a security warning is generated and further analyzed.

[0067] Among them, content and operation safety judgment are interrelated. When the displayed content is judged to be unsafe, it will be further checked whether the content is accompanied by abnormal operation mode. If the results of content safety and operation safety judgment both indicate potential risks, a safety warning instruction is generated.

[0068] By combining content security warning rules and standard behavior patterns, more accurate content security monitoring and behavior pattern analysis can be achieved. Different content security rules and behavior patterns can be set according to actual application scenarios, making this method adaptable to a variety of industries and needs.

[0069] Furthermore, the method also includes: extracting the content object data set, the content security judgment result and the operation security judgment result for in-machine temporary storage of the digital display; according to the remote management instruction, activating the integrated network module to call the in-machine temporary storage result, and transmitting it to the remote management terminal, wherein the remote management instruction includes periodic management instructions and dynamic management instructions; the remote management terminal uses the in-machine temporary storage result as feedback information, updates the content security warning rules and the standard behavior mode, and transmits the update result back to the integrated network module for remote control.

[0070] Specifically, on-machine temporary storage refers to temporarily storing the content object data set, content security judgment results and operation security judgment results collected in real time in the storage device inside the digital display. Remote management instructions are instructions issued by the remote management terminal to control or manage the operation of the digital display. These instructions can be periodic management instructions (such as regular query, regular update of content security rules) or dynamic management instructions (such as real-time adjustment of security warning rules, change of behavior monitoring standards, etc.). Periodic management instructions refer to instructions issued at a predetermined time period, which are usually used to regularly update, check or synchronize data, such as regularly transmitting the data temporarily stored in the machine to the remote management system, or regularly checking the status of the system. Dynamic management instructions are instructions issued for management operations that need to be performed in real time. The integrated network module is a module inside the digital display, which is responsible for connecting to the external network and transmitting data. It is usually used to transmit on-machine data to the remote management terminal, or receive remote management instructions and perform corresponding operations.

[0071] First, the data set of real-time content is extracted, including content object data set, content security judgment results, and operation security judgment results. These data reflect the currently displayed content and user behavior. All of these data will be stored in the digital display's on-board temporary storage, waiting for subsequent processing and use. In this way, the display is able to save and process necessary data without a remote connection.

[0072] When the remote management system needs to update the content security warning rules or behavior monitoring standards of the digital display, it will send remote management instructions to the digital display. These instructions can be periodic management instructions for periodic data synchronization and system configuration update, or dynamic management instructions for rapid adjustment of system rules when abnormal events occur. Once the remote management instructions are received, the digital display transmits the temporary data stored in the machine (such as content object data sets, security identification results, etc.) to the remote management terminal through the integrated network module. After receiving this data, the remote management terminal will conduct further analysis and update the content security warning rules and standard behavior patterns based on the current situation. After the update is completed, the remote management terminal will feed back the update results to the integrated network module of the digital display. The digital display will adjust the local content security rules and operation monitoring standards based on the updated results transmitted back. In this way, the digital display can be adjusted and optimized in real time according to external security conditions and management needs to ensure that it is always in the best security state.

[0073] By combining remote management instructions and local data storage, more flexible equipment management and data updates can be achieved. Whether it is periodic safety inspections or dynamic emergency responses, the remote management system can make precise adjustments through real-time data transmission and feedback mechanisms.

[0074] Furthermore, the method also includes: initializing the alignment of the real-time verification display content and the real-time image frame sequence of the real-time display content in combination with the typical driving delay of the display driving component, and calculating the image error characteristics; adjusting the alignment position of the real-time verification display content and the real-time display content based on the alignment initialization result as the reference alignment position, and calculating the adjusted image error characteristics; iteratively adjusting the alignment position, calculating the image error characteristics of each iteration accordingly, and obtaining an image error feature set; selecting the image error feature with the smallest characteristic value from the image error feature set as the verification image error feature, and comparing the verification image error feature with the verification error limit; if the verification image error feature does not meet the verification error limit, generating a hardware safety warning instruction accordingly to issue a safety warning.

[0075] Specifically, typical drive delay refers to the time delay generated in the process from the display driver component receiving the display signal to the completion of signal transmission and display. This delay is caused by the degree of optimization of the hardware and driver and various factors in the signal processing process. Registration initialization refers to the process of aligning the target image with the reference image in the image processing process to ensure that they are in the same position in space. The verification error limit refers to a preset tolerance range used to measure whether the image error is within an acceptable range. If the error is greater than this limit, it is considered that the display content has an unexpected error. The hardware safety warning instruction is a warning instruction issued to the hardware when it detects that the image error of the displayed content exceeds the tolerance range, indicating that there is a display failure or safety hazard. This instruction can trigger further system protection or alarm measures.

[0076] First, the typical drive delay of the display driver component of the digital display is analyzed. In order to perform more accurate image matching and verification, the registration is first initialized based on this delay. Registration initialization means that the system sets an initial image alignment position by considering the impact of the drive delay, and preliminarily aligns the real-time verification display content with the image frame sequence of the real-time display content. After the initial registration is completed, the real-time display content and the real-time verification display content will be further adjusted. By repeatedly adjusting the registration position, image errors caused by display driver delays, hardware differences, or inconsistent signal transmission can be eliminated to the greatest extent. After each adjustment, the system calculates the error characteristics of the image, which reflect the gap between the display content and the reference content during the registration process.

[0077] The system will iterate multiple registration adjustments, and each adjustment will calculate a new set of error features. All error features will be collected and formed into an image error feature set. This error feature set contains the error changes between the displayed content and the reference content during the multiple registration adjustments. The image error feature with the smallest feature value is selected from the error feature set, which represents the most accurate registration result. Then, the error feature is compared with the preset verification error limit. The verification error limit is a defined threshold used to determine whether the error is within an acceptable range. If the error feature exceeds the verification error limit, it means that there is an obvious problem in the registration process.

[0078] If the image error feature exceeds the verification error limit, the system believes that there is a serious deviation or fault in the displayed content, which will affect the display effect or cause safety hazards. Therefore, a hardware safety warning instruction is generated based on this result. This instruction can trigger a warning mechanism at the hardware level, such as screen flickering, content locking, or alerting maintenance personnel to the need for inspection or repair.

[0079] Through careful adjustment of image registration, the displayed content and the real-time image frame sequence can be accurately calibrated to ensure the visual consistency of the displayed content and avoid content errors caused by hardware delays or display deviations.

[0080] In summary, the digital display security warning method based on real-time content monitoring provided by the embodiment of the present application has the following technical effects: 1. Through embedded content acquisition and real-time monitoring technology, comprehensive monitoring and real-time security warning of digital display content are achieved, which can automatically identify and detect security risks in displayed content and enhance the system's ability to prevent malicious content or operational behaviors.

[0081] 2. Through hardware replication and decoding processing, the efficient acquisition and accuracy of real-time display signals are ensured, providing high-quality video frame sequences, laying a solid foundation for subsequent content recognition and security judgment, and improving the accuracy and stability of content monitoring.

[0082] 3. The packet capture performance is optimized by combining the display driver load information, making the real-time content acquisition process more efficient without affecting the display performance, ensuring the efficiency and accuracy of the content security judgment process, and improving the accuracy of the packet capture operation.

[0083] Any step of the method described above can be stored as a computer instruction or program in an unlimited computer memory, and can be called and recognized by an unlimited computer processor to implement any method in the embodiments of the present application, without any unnecessary restrictions.

[0084] Furthermore, the first or second mentioned above not only represents an order relationship, but also represents a specific concept, and / or refers to the selection of multiple elements individually or in whole. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application intends to include these modifications and variations.

Claims

1. A digital display safety early warning method based on real-time content monitoring, characterized in that: The method comprises: Based on the content acquisition component embedded in the digital display, real-time content capture is performed to generate real-time display content; Parsing the real-time display content to extract classified objects and generate a content object data set, wherein the content object data set includes a text data sequence, an image data sequence and an operation data sequence; activating an embedded edge computing module of the digital display, performing content recognition on the text data sequence and the image data sequence, and performing behavior pattern recognition on the operation data sequence; Combine the preset content security warning rules with the content recognition results to make content security judgments, and compare the behavior pattern recognition results with the standard behavior patterns to make operation security judgments; Based on the content security judgment results and the operation security judgment results, a security warning instruction is generated to respond to the security warning.

2. The digital display safety early warning method based on real-time content monitoring according to claim 1, characterized in that: Based on the content acquisition component embedded in the digital display, real-time content is captured to generate real-time display content, including: activating the hardware layer replicator of the content acquisition component to perform pre-copy acquisition of the display signal transmitted from the signal source to obtain a real-time display signal stream; The real-time display signal stream is transmitted to a hardware decoder for signal decoding, a real-time image frame sequence is acquired, and the real-time image frame sequence is output as the real-time display content.

3. The digital display safety warning method based on real-time content monitoring according to claim 2, characterized in that: The real-time display signal stream is transmitted to a hardware decoder for signal decoding to obtain a real-time image frame sequence, and then the method further includes: Interact with the target display scene, obtain the corresponding UI layout information, and identify the corresponding key display area based on the functional characteristics of the UI layout as the division constraint; fitting the key display area and the real-time display signal stream according to the relative coordinates of the key display area; Based on the fitting result, serialization extraction is performed on the real-time display signal stream, and the serialization extraction result is output as a new real-time display signal stream.

4. The digital display safety warning method based on real-time content monitoring as claimed in claim 2, characterized in that: Real-time content capture based on the content acquisition components embedded in the digital display, including: Interactively acquiring a display driver component of a digital display, acquiring driver load information, and evaluating available packet capture performance of the display driver component according to the driver load information; Combining the available packet capture performance with the display parameter information of the digital display, calculating and obtaining real-time packet capture parameters; The software layer packet capture module of the content acquisition component is activated, and post-packet capture of the digital display is performed based on the real-time packet capture parameters to obtain real-time verification display content.

5. The digital display safety warning method based on real-time content monitoring as claimed in claim 3, characterized in that: Parse the real-time display content to extract classified objects and generate a content object data set, wherein the content object data set includes a text data sequence, an image data sequence and an operation data sequence, including: Extracting key frames from the real-time display content, and outputting the key frame extraction results as the image data sequence; Performing content text recognition based on the image data sequence, and outputting the text data sequence; The operation data sequence is extracted by combining the image data sequence with the target detection method, wherein the operation data sequence includes cursor operation data and text operation data.

6. The digital display safety warning method based on real-time content monitoring according to claim 1, characterized in that: Combining the preset content security warning rules with the content recognition results, content security judgment is performed, and the behavior pattern recognition results are compared with the standard behavior patterns to perform operation security judgment, wherein the content security warning rules include at least sensitive keywords, sensitive semantics, sensitive patterns, and sensitive signs, and the standard behavior patterns include at least mouse behavior patterns and keyboard behavior patterns.

7. The digital display safety warning method based on real-time content monitoring according to claim 1, characterized in that: The method further comprises: Extracting the content object data set, the content safety identification result and the operation safety identification result for temporary storage in the digital display; According to the remote management instruction, the integrated network module is activated to call the temporary storage result in the machine and transmit it to the remote management terminal, wherein the remote management instruction includes a periodic management instruction and a dynamic management instruction; The remote management terminal uses the temporarily stored results in the machine as feedback information, updates the content security warning rules and the standard behavior mode, and transmits the updated results back to the integrated network module for remote control.

8. The digital display safety warning method based on real-time content monitoring as claimed in claim 4, characterized in that: The method further comprises: In combination with a typical driving delay of the display driving component, initializing the registration of the real-time verification display content with the real-time image frame sequence of the real-time display content, and calculating image error characteristics; Taking the registration initialization result as the reference registration position, adjusting the registration position of the real-time verification display content and the real-time display content, and calculating the adjusted image error characteristics; Iteratively adjust the registration position, calculate the image error features of each iteration accordingly, and obtain the image error feature set; Selecting an image error feature with the smallest eigenvalue from the image error feature set as a verification image error feature, and comparing the verification image error feature with a verification error limit; If the verification image error feature does not meet the verification error limit, a hardware safety warning instruction is generated to issue a safety warning.

Images