A subway PIS security auditing system

Abstract

The application discloses a subway PIS security audit system and relates to the technical field of information security; the system acquires display buffer information and window information of a broadcast terminal, determines a service playing window and an effective playing area, and extracts an effective playing area picture through display buffer mapping; the picture is subjected to text area positioning and text recognition, combined with sensitive word screening and local text semantic auditing to form a text auditing result, and meanwhile, a picture auditing result is formed through a local visual auditing model; a risk warning, a risk record or a no-risk audit log are generated according to the text auditing result and the picture auditing result, and the picture, recognized text, auditing result and processing result are associated and saved as local audit records. The application can improve the graphic and text security auditing and risk tracing of subway PIS playing content.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to a subway PIS security audit system. Background Technology

[0002] The subway passenger information system is used to broadcast arrival information, operational announcements, emergency notices, promotional videos, scrolling text, and other passenger-oriented information in stations, trains, control centers, and other locations. The content it broadcasts is characterized by its wide reach, high update frequency, and significant public impact, thus requiring continuous security auditing of the text and video footage. Existing security auditing methods typically involve periodically taking full-screen screenshots or polling the desktop to obtain the broadcast control terminal's image, then identifying abnormal content through text recognition, sensitive word matching, or image classification. However, full-screen screenshots simultaneously capture the desktop background, taskbar, system windows, and other areas unrelated to passenger information playback, resulting in a large amount of invalid data for subsequent text recognition and image review processing. This can easily increase processor and storage resource consumption and may lead to issues such as acquisition delays, image omissions, or inaccurate correlation between audit results and actual playback content during continuous video playback.

[0003] Existing text review methods typically rely primarily on direct string matching using sensitive word databases. This can easily lead to omissions of text content that does not directly contain sensitive words but has semantic ambiguity, implicit violations, or disguised negative connotations. Text review and video review are usually conducted separately, lacking a unified processing mechanism to link the review results of the same playback screen with the text. Some review solutions rely on external cloud interfaces to upload playback screens or recognize text, which is unsuitable for operating environments where subway internal networks are isolated, data cannot be transmitted externally, or external networks are unavailable. Existing solutions lack a complete mechanism for linking and saving data between captured images, recognized text, risk types, review results, abnormal snapshots, and alarm information, making it difficult to accurately reconstruct the risk content, review basis, and processing process after a risk is detected. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of inaccurate image capture range, insufficient identification of graphic and textual risks, reliance on external networks for the review process, and difficulty in fully tracing risky content in the existing technology, and to propose a subway PIS security audit system.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution: A subway PIS safety audit system includes: The information acquisition module is used to acquire the display buffer information and window information of the broadcast control terminal of the subway passenger information system, determine the service playback window based on the window information, and determine the effective playback area based on the service playback window. The image extraction module is used to establish a display buffer mapping relationship based on the display buffer information, and extract the image of the valid playback area from the display buffer according to the valid playback area; The text recognition module is used to locate and recognize text regions in the effective playback area to obtain structured text data. The text security review module is used to match the text content in the structured text data with sensitive words in the sensitive word library of subway operation scenarios to obtain sensitive word screening results. The structured text data that does not match sensitive words is input into the local text semantic review model to obtain semantic review results. The text review results are formed based on the sensitive word screening results and the semantic review results. The screen security review module is used to input the screen of the effective playback area into the local visual review model to obtain the screen review result; The audit processing module is used to generate an audit processing result based on the text review result and the screen review result, and to associate and save the effective playback area screen, the structured text data, the text review result, the screen review result and the audit processing result according to the acquisition frame identifier to form a local audit record.

[0006] Further, determining the service playback window based on the window information includes: Access the graphics card device of the broadcast control terminal of the subway passenger information system through the direct rendering infrastructure interface to obtain the display buffer information corresponding to the current display screen; The window identifier, window coordinates, window size, and window level of the current window are read through the window management interface, and the window identifier, window coordinates, window size, and window level are combined to form window information; The service playback window corresponding to the broadcast control program of the subway passenger information system is determined from the window information based on the window identifier.

[0007] Further, determining the effective playback area based on the service playback window includes: The coverage area of ​​the service playback window in the current display screen is determined based on the window coordinates and window size of the service playback window; The covered area is defined as the effective playback area, so that desktop backgrounds, taskbars and non-business windows located outside the effective playback area are not included in the extraction of the effective playback area screen.

[0008] Further, establishing a display buffer mapping relationship based on the display buffer information, and extracting the valid playback area image from the display buffer according to the valid playback area, includes: The display buffer is mapped to the local address space via the direct rendering infrastructure interface; Based on the region coordinates and region size of the effective playback region, determine the range of pixel data in the display buffer corresponding to the effective playback region; Pixel data within the specified pixel data range is read to form the effective playback area image.

[0009] Furthermore, text region localization and text recognition are performed on the effective playback area to obtain structured text data, including: The locally deployed text recognition model is invoked to locate the text regions in the subtitles, scrolling announcements, pop-up texts, and interface prompts in the effective playback area, thereby obtaining the text region positions. The text content is obtained by performing text recognition on the image content corresponding to the text region. The text region location and the text content are associated to form the structured text data.

[0010] Furthermore, the structured text data is subjected to sensitive word screening, including: The text content in the structured text data is matched with sensitive words in the sensitive word library of subway operation scenarios. The matched text content, the position of the text content in the structured text data, and the corresponding risk type are recorded as the sensitive word screening results.

[0011] Furthermore, the structured text data that did not match sensitive words is input into the local text semantic review model, including: The local text semantic review model performs semantic analysis on the structured text data that does not match sensitive words, identifies semantic ambiguity, implicit violations and disguised misleading content, and obtains the semantic review result. The sensitive word screening results and the semantic review results are correlated to form the text review results.

[0012] Further, the effective playback area is input into the local visual review model to obtain the image review result, including: The continuously extracted valid playback area is sampled to obtain the video frames to be reviewed; The video frame to be reviewed is input into the local visual review model to identify inappropriate images, illegal scenes and unsuitable visual content in the video frame to be reviewed. The identified image risk types are associated with the corresponding video frames to be reviewed, thus forming the image review results.

[0013] Furthermore, an audit processing result is generated based on the text review result and the image review result, including: When the text review result or the screen review result indicates that there is a risk, the corresponding valid playback area screen is saved as an abnormal snapshot, a risk record including risk type and confidence level data is generated, and a risk alarm including the risk type is triggered. The risk record and the risk alarm are used as the audit processing result. When neither the text review result nor the screen review result indicates the presence of risk, a risk-free audit log is generated, and the risk-free audit log is used as the audit processing result. The local audit records are stored in the broadcast control terminal of the subway passenger information system.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention determines the service playback window and effective playback area by acquiring the display buffer information and window information of the broadcast control terminal, and extracts the effective playback area image based on the display buffer mapping. This can limit the image acquisition range to the area where the actual playback content is located, reducing the interference of irrelevant images such as desktop background, taskbar and non-service windows on subsequent processing. By performing text recognition and image review only on the pixel data corresponding to the effective playback area, it is beneficial to reduce the transmission and processing of invalid image data and improve the targeting of playback content acquisition and security review.

[0015] 2. This invention performs hierarchical review of played text through sensitive word screening and local text semantic review, and combines a local visual review model to identify screen risks in the same valid playback area. It can simultaneously detect explicit sensitive text, implicit semantic risks, and inappropriate visual content. By collecting frame identifiers and associating them with the valid playback area screen, structured text data, text review results, screen review results, and audit processing results, risk records and risk alarms are generated when risks are detected, and risk-free audit logs are generated when no risks are detected, thus forming a queryable and verifiable audit link. All identification and review processes are completed locally on the broadcast control terminal, which can adapt to the operating environment of the subway's internal network isolation and the unavailability of external networks. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a functional module diagram of a subway PIS safety audit system provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] This embodiment uses a subway passenger information system broadcast control terminal with a Linux operating system, a windowed graphical display environment, and a local offline auditing program as an example for illustration. The operating system, graphical interface, model framework, and data storage method used are only used to illustrate one possible implementation of the present invention and do not constitute a limitation on the scope of protection of the present invention. The technical solution of the present invention includes business playback window positioning, effective playback area determination, display buffer mapping, effective playback area image extraction, text area positioning, text recognition, sensitive word screening, local text semantic auditing, local video image auditing, risk alarm, and local audit record storage, forming a local offline graphic security audit of the content played by the subway passenger information system. In this embodiment, the subway passenger information system broadcast control terminal is installed in subway stations, control centers, depots, or trains. It is used to play operation announcements, arrival information, emergency notices, promotional videos, scrolling announcements, pictures, and other passenger information through display devices. The subway passenger information system broadcast control terminal includes a processor, a graphics card, a local storage device, and a display output interface. The processor runs the subway passenger information system broadcast control program and the security audit program, and the graphics card generates the current display screen based on the playback content output by the subway passenger information system broadcast control program. The security audit program and the subway passenger information system broadcast control program are deployed within the same subway passenger information system broadcast control terminal. The text recognition model, local text semantic review model, local visual review model, subway operation scenario sensitive word library, model configuration file and tag mapping table are all pre-stored in the local storage device of the subway passenger information system broadcast control terminal. The effective playback area screen, structured text data, text review results, screen review results and audit processing results generated during the security audit process are all transmitted and saved within the subway passenger information system broadcast control terminal and are not sent to external servers.

[0019] After the security audit program is started, it reads the local configuration file that is pre-stored in the local storage device. The local configuration file records at least the playback window identifier, graphics card device node, target display output identifier, supported pixel format, storage location of text recognition model, storage location of local text semantic review model, storage location of local visual review model, storage location of sensitive word library for subway operation scenario, video frame sampling interval, scrolling announcement area, and storage location of local audit records for the broadcast control program of the subway passenger information system. The local configuration file also records the version identifiers of the text recognition model, the local text semantic review model, the local visual review model, and the sensitive word library for subway operation scenarios. The security audit program loads the above models, model configuration files, tag mapping tables, and sensitive word libraries for subway operation scenarios, and establishes text recognition queues, text review queues, image review queues, and audit record queues in local memory. The sensitive word database for subway operation scenarios consists of multiple sensitive word records. Each sensitive word record includes at least the sensitive word text and its corresponding risk type. Risk types include illegal speech, inappropriate slogans, public opinion sensitive words, and other text risk types pre-configured according to subway operation specifications.

[0020] Playback window positioning and effective playback area determination; Specifically, in the information acquisition module, the security audit program acquires the display buffer information and window information of the broadcast control terminal of the subway passenger information system, determines the business playback window based on the window information, and determines the effective playback area based on the business playback window; The security audit program reads the graphics card device node recorded in the local configuration file and opens the graphics card device node in read-only mode; in this embodiment, the graphics card device node is the direct rendering infrastructure device node that undertakes the output of the display screen of the subway passenger information system. The security audit program reads the display connection information of the graphics card device through the display resource query interface; for each display connection, it reads the corresponding connection status, display output area and its associated display controller identifier; and identifies the display connections that are in the connected state and are currently outputting a display screen as candidate display connections. When the target display output identifier is recorded in the local configuration file, the security audit program compares the target display output identifier with the display output identifiers of each candidate display connection, and determines the candidate display connection with the same identifier as the target display connection. When the target display output identifier is not recorded in the local configuration file, or when the broadcast control terminal of the subway passenger information system is connected to multiple display devices, the security audit program calculates the overlap range between the service playback window and the display output area corresponding to each candidate display connection after determining the service playback window, and determines the candidate display connection with the largest overlap range as the target display connection.

[0021] After determining the target display connection, the security audit program identifies the target display controller based on the display controller identifier associated with the target display connection, and reads the main display plane associated with the target display controller; the main display plane refers to the display plane that provides the current desktop composite image to the target display controller; The security audit program reads the frame buffer identifier currently associated with the main display plane, and obtains the frame buffer's screen width, screen height, pixel format, byte span per line, data start offset, and buffer object identifier based on the frame buffer identifier; In this embodiment, the display buffer refers to the pixel storage area corresponding to the frame buffer currently associated with the main display plane; the display buffer information consists of the graphics card device node identifier, target display connection identifier, target display controller identifier, main display plane identifier, frame buffer identifier, screen width, screen height, pixel format, byte span per line, data start offset, and buffer object identifier; Here, the byte span of each row represents the number of bytes between the starting positions of two adjacent rows of pixel data in the display buffer; the pixel format represents the arrangement of each color channel within a single pixel and the number of bytes occupied by a single pixel; the data start offset represents the offset of the effective pixel data in the frame buffer relative to the starting position of the buffer object; and the buffer object identifier is used to request the mapping information of the buffer object from the graphics card driver. If the graphics card driver supports reading the current frame buffer, the security audit program requests buffer object mapping information from the graphics card driver based on the buffer object identifier, obtains the mapping handle and mapping offset, and writes the mapping handle and mapping offset into the display buffer information.

[0022] This implementation adopts a single data plane continuous pixel format. The security audit program pre-registers the allowed pixel formats in the local configuration file. The allowed pixel formats include red-green-blue continuous pixel format without alpha channel and red-green-blue continuous pixel format with alpha channel. When the target display connection to the currently output screen cannot be obtained, the target display controller cannot be obtained, the main display plane cannot be obtained, the frame buffer identifier cannot be obtained, the current frame buffer uses an unregistered pixel format, or the buffer object mapping information cannot be obtained, the security audit program terminates the current round of screen acquisition and generates an acquisition anomaly record. The collection of anomaly records shall include at least the identifier of the broadcast control terminal of the subway passenger information system, the time of the anomaly, the anomaly process, and the cause of the anomaly; the anomaly process record shall be the location of display resources, the reading of frame buffers, or the mapping of buffer objects; the anomaly cause record shall be the failure to obtain a target display connection, the failure to obtain a frame buffer, the unsupported pixel format, or the unmappable buffer object; the collection of anomaly records shall be written into the local audit log in subsequent processing, and the current round shall not be considered as having completed a risk-free audit; While acquiring display buffer information, the security audit program reads the window tree in the current graphics display environment through the window management interface and retrieves each top-level window starting from the root window.

[0023] For each top-level window, the security audit program reads the window identifier, window name, process identifier, visibility status, window coordinates, window size, window hierarchy, and client area information; In this embodiment, window information refers to data used to represent the current window's identity, position, size, and display order. Window information includes at least window identifier, window coordinates, window size, and window hierarchy. Window coordinates represent the window's position relative to its parent window, window size includes window width and window height, and window hierarchy represents the window's position in the current window stacking order. The security audit program compares the window name or process identifier with the playback window identifier recorded in the local configuration file; when the window name or process identifier matches the playback window identifier and the corresponding window is visible, the window is identified as a candidate service playback window. When the subway passenger information system broadcast control program includes both a control window and a playback window, the local configuration file records the control window identifier and the playback window identifier respectively. The security audit program only uses the candidate service playback window corresponding to the playback window identifier for subsequent processing. When there are multiple candidate service playback windows with the same window identifier, the security audit program reads the window level and window size of each candidate service playback window, and determines the candidate service playback window that is visible, has a higher window level, and has the largest display area as the service playback window. The service playback window refers to the window in the subway passenger information system's broadcast control program that actually displays operation announcements, promotional videos, subtitles, pictures, or other passenger information.

[0024] Since the window coordinates read by the window management interface may be relative to the parent window, the security audit program transforms the coordinates of the top left corner of the business playback window from the parent window coordinate system to the root window coordinate system to obtain the absolute coordinates of the top left corner of the business playback window relative to the top left corner of the root window. When the business playback window has a window border, title bar, or other window decoration area, the security audit program reads the client area position and client area size of the business playback window and determines the client area as the business playback area that carries the actual playback content. When the window management interface cannot directly obtain customer area information, the security audit program reads the window decoration boundary data pre-recorded in the local configuration file, and corrects the window coordinates and window size of the business playback window according to the window decoration boundary data, thereby excluding the title bar and window border. When the broadcast control terminal of the subway passenger information system is connected to only one display device, the absolute coordinates of the service playback area in the root window coordinate system are used as the coordinates relative to the target display output. When the broadcast control terminal of the subway passenger information system is connected to multiple display devices, the root window forms a unified coordinate space covering multiple display outputs; the security audit program calculates the overlap range between the business playback area and each display output area, and determines the display output with the largest overlap range as the target display output; After determining the target display output, the security audit program subtracts the starting coordinates of the target display output in the root window coordinate system from the absolute coordinates of the service playback area in the root window coordinate system to obtain the pixel coordinates of the service playback area relative to the upper left corner of the target display output. The security audit procedure re-verifies the corresponding target display connection, target display controller, main display plane, and display buffer based on the target display output, ensuring that the service playback area and the display buffer correspond to the same display device.

[0025] The security audit procedure determines the coverage area of ​​the service playback area in the display buffer based on the pixel coordinates of the service playback area relative to the target display output and the size of the service playback area; The coverage area is cropped based on the display buffer screen boundary; the rightmost boundary between the left boundary of the service playback area and the left boundary of the display buffer is determined as the left boundary of the effective playback area; the lowermost boundary between the upper boundary of the service playback area and the upper boundary of the display buffer is determined as the upper boundary of the effective playback area; the leftmost boundary between the right boundary of the service playback area and the right boundary of the display buffer is determined as the right boundary of the effective playback area; and the uppermost boundary between the lower boundary of the service playback area and the lower boundary of the display buffer is determined as the lower boundary of the effective playback area. Based on the left boundary, upper boundary, right boundary, and lower boundary of the effective playback area, the region coordinates and region size of the effective playback area are obtained; The effective playback area refers to the business screen area that needs to be extracted and security audited in the future. Since the effective playback area is determined by the client area of ​​the business playback window, the desktop background, taskbar and non-business windows located outside the effective playback area do not participate in the subsequent pixel extraction, text recognition and screen auditing. When the business playback window uses full-screen playback mode, the effective playback area covers all or most of the target display output display area; when the business playback window uses non-full-screen playback mode, the effective playback area only covers the display area actually occupied by the client area of ​​the business playback window. When there is a parent window whose window level is higher than the business playback window and overlaps with the effective playback area, the security audit program reads the window coordinates and window size of the parent window and converts the window coordinates of the parent window to the target display output coordinate system.

[0026] The security audit procedure calculates the overlapping area between the upper-level window and the effective playback area, and records the overlapping area as the occlusion area; the occlusion area includes at least the occlusion window identifier, the coordinates of the occlusion area, and the size of the occlusion area. The security audit procedure collects data based on the actual visible screen in the current display buffer, without restoring or presuming that the content played by the service below the obscured area is not displayed. When the obstructed area covers the entire effective playback area, the security audit program generates a window obstruction anomaly record and does not generate a risk-free audit conclusion for the business playback content that is completely obstructed. When the obstruction area only covers part of the effective playback area, the security audit program continues to audit the currently visible screen and records the partial obstruction status, obstruction window identifier, and obstruction area location; the partial obstruction status is written to the local audit log when a risk record, risk-free audit log, or audit anomaly record is subsequently generated.

[0027] The output displays buffer information, service playback window information, effective playback area, and window occlusion status. The display buffer information and effective playback area are used by the image extraction module to extract the image of the effective playback area, while the service playback window information and window occlusion status are used to generate subsequent audit processing results and local audit records.

[0028] Display buffer mapping and extraction of the effective playback area; Specifically, in the image extraction module, the security audit program establishes a display buffer mapping relationship based on the display buffer information, and extracts the image of the valid playback area from the display buffer according to the valid playback area; The security audit program establishes a memory mapping relationship between the buffer object and the security audit program's local address space in read-only mode based on the mapping handle and mapping offset in the display buffer information; After successful mapping, the security audit program adds the mapping start address to the data start offset in the display buffer information to obtain the start address of the valid pixel data in the current frame buffer.

[0029] Before reading the valid playback area, the security audit program determines the pixel data range corresponding to the valid playback area based on the valid playback area coordinates, the valid playback area size, the byte span of each line, and the number of bytes occupied by a single pixel, and checks whether the pixel data range is within the mapping space; The security audit program determines the starting position of the first row of the effective playback area in the display buffer based on the number of pixel rows corresponding to the upper boundary of the effective playback area and the byte span of each row; then it determines the starting reading position of the first pixel of the first row of the effective playback area based on the number of pixels corresponding to the left boundary of the effective playback area and the number of bytes occupied by a single pixel. Starting from the initial reading position, read the pixel data in the current line according to the width of the effective playback area; after the current line is read, move to the starting position of the next line according to the byte span of each line, and continue to read the effective pixel data in the next line; repeat the above process until the number of pixel lines read reaches the height of the effective playback area; When each row of pixel data in the display buffer contains an alignment byte, the security audit program copies the valid pixel data in each row according to the width of the valid playback area, skips the alignment byte at the end of each row, and writes the valid pixel data of each row into the continuous image buffer in sequence. When the text recognition model and the local visual auditing model are able to receive image data with each line spanning bytes, the security auditing program establishes a read-only image view pointing to the mapped pixel range corresponding to the effective playback area, and uses the read-only image view as the effective playback area screen, without copying pixel data outside the effective playback area; When the pixel format of the display buffer is different from the input pixel format required by the character recognition model or the local visual auditing model, and the pixel format belongs to the convertible pixel format registered in the local configuration file, the security auditing program only performs color channel order adjustment or pixel format conversion on the pixel data in the effective playback area. The effective playback area refers to the image data extracted from the display buffer according to the area coordinates and area size of the effective playback area; the width and height of the effective playback area correspond to the area width and area height of the effective playback area, respectively.

[0030] When the pixel data corresponding to any row of the effective playback area exceeds the mapping space, memory mapping fails to be established, mapping space reading fails, or pixel format conversion fails, the security audit program stops extracting the current effective playback area, removes the current memory mapping, and generates a pixel extraction exception record. The pixel extraction anomaly record should include at least the time of the anomaly, the display buffer identifier, the effective playback area, the anomaly stage, and the cause of the anomaly; the pixel extraction anomaly record is passed to the audit record queue, but no corresponding risk-free audit log is generated; After the current valid playback area is extracted, the security audit program removes the read-only memory mapping for this round; if the graphics card driver allows continuous mapping and the frame buffer identifier currently associated with the main display plane has not changed, the mapping relationship can also be maintained for use in the next round of image extraction. When the playback content of the broadcast control terminal of the subway passenger information system is switched, the window size changes, or the display mode changes, the frame buffer associated with the main display plane may be switched; the security audit program reads the frame buffer identifier currently associated with the main display plane during continuous image extraction and compares it with the frame buffer identifier with which an established mapping relationship has been established. When the two frame buffer identifiers match, continue to use the current display buffer mapping relationship to read the pixel data corresponding to the valid playback area; When the two frame buffer identifiers are inconsistent, the original display buffer mapping relationship is stopped, the original memory mapping is released, and the screen width, screen height, pixel format, byte span per line, data start offset, buffer object identifier, mapping handle and mapping offset of the new frame buffer are obtained again. The security audit program re-establishes the read-only mapping relationship based on the updated display buffer information and re-checks the pixel data range corresponding to the effective playback area; when the screen size of the new frame buffer changes, it re-executes the effective playback area boundary clipping to ensure that the effective playback area is within the effective screen range of the new frame buffer.

[0031] When a new frame buffer cannot be mapped, a display buffer switching exception record is generated, and playback content that has not been reviewed during the switching period is not considered as risk-free content. The security audit procedure selects the valid playback area frames that will be entered into the subsequent text and image review process from the continuously extracted valid playback area frames according to the video frame sampling interval recorded in the local configuration file; the video frame sampling interval is represented by the number of continuously extracted valid playback area frames. The security audit procedure sets a sampling counter; the sampling counter increments once for each valid playback area frame extracted; when the sampling counter reaches the video frame sampling interval, the current valid playback area frame is simultaneously sent to the text recognition queue and the image review queue, and the sampling counter is reset to zero. For the valid playback area images sent to the text recognition queue and the image review queue, the security audit program assigns capture frame identifiers according to the image generation order and records the capture time, business playback window information, valid playback area and window occlusion status. The captured frame identifier is used to associate structured text data, text review results, screen review results, and audit processing results formed from the same valid playback area.

[0032] The security audit program creates an audit task to be completed for the valid playback area. The audit task to be completed uses the capture frame identifier as the task identifier and sets the text review status and the screen review status respectively. When the audit task to be completed is created, both the text review status and the screen review status are recorded as pending processing. The same valid playback area simultaneously enters the text recognition queue and the screen review queue, thereby ensuring that each complete audit task generates corresponding text review results and screen review results; Frames in the effective playback area that do not reach the video frame sampling interval will not enter the text recognition queue and the image review queue, and will not generate structured text data, text review results, image review results, or complete local audit records. In this embodiment, the video frame sampling interval is set to one, meaning that each extracted valid playback area image enters the text recognition process and the image review process. In another implementation, when the video frame sampling interval is greater than one, only the valid playback area that reaches the video frame sampling interval is sent to the text recognition queue and the screen review queue at the same time. The screens that are not selected do not form a complete audit task, thus avoiding the situation where the same captured frame only has text review results but not screen review results.

[0033] Character recognition and structured text data generation; Specifically, in the text recognition module, the security audit program reads the valid playback area from the text recognition queue, performs text region localization and text recognition on the valid playback area, and obtains structured text data; This implementation uses a text recognition model pre-deployed locally on the broadcast control terminal of the subway passenger information system; the text recognition model includes text region positioning processing and text content recognition processing; the model parameter file, character dictionary, model configuration file and inference program of the text recognition model are all stored on the local storage device; In one specific implementation, the text recognition model can be the locally deployed Baidu PaddlePaddle text recognition model, or it can be other local text detection and recognition models with the same input and output capabilities. The security audit procedure will perform text region location processing on the effective playback area screen; the text region location processing will analyze the pixel distribution in the effective playback area screen, identify the areas where subtitles, scrolling announcements, pop-up text and interface prompts are located in the effective playback area screen, and output one or more text region positions; Text region position refers to the position data of text content within the effective playback area of ​​the screen; for a rectangular text region, the text region position includes the coordinates of the top left corner of the text region, the width of the text region, and the height of the text region; for an slanted text region, the text region position includes the coordinates of multiple vertices surrounding the text region. The security audit procedure extracts corresponding partial frames from the effective playback area based on the location of each text area, and inputs each partial frame into the text content recognition and processing. The text content recognition process outputs the text content according to the arrangement order of characters in the local image. When the text content recognition process outputs the text recognition score at the same time, the text recognition score is associated with the corresponding text content and saved, but the text recognition score is not used as the text risk confidence level.

[0034] When the subway passenger information system plays fixed subtitles, the text region positioning process locates the area where the fixed subtitles are located, and the text content recognition process outputs the text content of the current fixed subtitles. When the subway passenger information system plays scrolling announcements, the text area positioning process locates the currently visible scrolling announcement area in the valid playback area screen, and the text content recognition process outputs the announcement text that has now entered the valid playback area. When the subway passenger information system plays pop-up text located within the business playback window, the text area positioning processing locates the area where the pop-up text is located, and the text content recognition processing outputs the corresponding text content. When the subway passenger information system displays interface prompts, the text area location and text content of the interface prompts are obtained in the same way. After text recognition is completed, the security audit program assigns text region identifiers to each text region according to its position in the effective playback area of ​​the screen, and associates the text region position and text content corresponding to the same text region. Structured text data refers to data that organizes the text positions and text content in the effective playback area according to a preset data structure; each piece of structured text data includes at least the capture frame identifier, text region identifier, text region position, and text content.

[0035] When multiple text regions exist within the same valid playback area, corresponding structured text data are generated for each region. Each structured text data has the same capture frame identifier but different text region identifiers. When no text area is detected in the effective playback area, the security audit program generates a structured text data with empty text content and retains the acquisition frame identifier in the structured text data to indicate that the text recognition processing of the effective playback area has been completed. When the text recognition model fails to return a valid text region location, the text content output format is incorrect, or the text recognition program fails to run, the text review status of the corresponding audit task to be completed will be updated to "processing failed," and the failed step will be recorded as text recognition. A text review result that does not contain any text risk must not be generated directly.

[0036] Normally generated structured text data is written into the text review queue as input for sensitive word screening and local text semantic review in the text security review module; For scrolling announcements, the local configuration file pre-records the scrolling announcement area; the scrolling announcement area is determined by the area coordinates and area size of the scrolling announcement in the business playback window. After the text region is located, when the text region is located within the scrolling announcement area, the corresponding text content is marked as scrolling announcement text and written into the scrolling text sequence corresponding to the scrolling announcement area in the order of collection time. The scrolling text sequence is used to save the text content that has actually been displayed in the continuous effective playback area of ​​the same scrolling announcement, and to record the capture frame identifier that constitutes the scrolling text sequence. For the scrolling announcement text obtained from the current valid playback area screen, the security audit program reads the last text corresponding to the previous valid playback area screen in the scrolling text sequence, and finds the longest consecutive character segment that is the same between the last text of the previous valid playback area screen and the first text of the current valid playback area screen. When the same consecutive character segment is found, the newly added text following the same consecutive character segment in the current scrolling announcement text is appended to the scrolling text sequence to avoid repeatedly saving text that appears repeatedly in adjacent valid playback areas. When no identical consecutive character segment is found, the current scrolling announcement text is appended to the scrolling text sequence as a new text segment, and a text separator is written between the previous text segment and the current text segment. Each time new text is added to the scrolling text sequence, the updated scrolling text sequence is used as the text content in the structured text data corresponding to the current acquisition frame, and subsequent sensitive word screening and local text semantic review are performed.

[0037] When risky words appear in adjacent valid playback areas, but form a complete and continuous text in the updated scrolling text sequence, the subsequent sensitive word screening or local text semantic review will form a text review result based on the updated scrolling text sequence. The scrolling text sequence simultaneously records the identifiers of each capture frame that constitutes the scrolling text sequence; when the scrolling text sequence is determined to have text risk, the identifiers of each capture frame that forms the risky text, the corresponding valid playback area screen, the updated scrolling text sequence, and the position of the risky text are all written into the risk record. When no text appears in multiple consecutive valid playback areas of the scrolling announcement area, the scrolling announcement playback task ends, or the business playback window switches playback content, the current scrolling text sequence ends and a new scrolling text sequence is established for subsequent scrolling announcements. For fixed subtitles, pop-up text, and interface prompts that are not located in the scrolling announcement area, the scrolling text sequence appending process will not be performed. Instead, the structured text data corresponding to the current valid playback area screen will be used directly for subsequent review.

[0038] Text security audit; Specifically, in the text security audit module, the security audit program reads structured text data from the text audit queue, performs sensitive word screening on the structured text data, obtains sensitive word screening results, and inputs the structured text data that did not hit sensitive words into the local text semantic audit model to obtain semantic audit results, and then forms the text audit results; The security audit program reads the text content from the structured text data and performs string matching between the text content and sensitive words in the sensitive word library of subway operation scenarios that has been loaded into local memory. String matching uses consecutive identical characters as the matching condition; security auditing programs search for consecutive strings in text content that are exactly the same as sensitive words; When a continuous string that is exactly the same as a sensitive word exists in the text content, the corresponding structured text data is identified as the hit sensitive word, and the matched text content, the matching position of the sensitive word in the text content, and the corresponding risk type are recorded. Because structured text data includes the location of text regions, it is possible to associate the hit sensitive words with the corresponding text regions in the effective playback area. The results of sensitive word screening should include at least the frame identifier, text region identifier, text region location, matched text content, matching location, risk type, source of risk basis, and string matching confidence level. The risk is recorded as a sensitive word string matching source. For sensitive word matching results where consecutive strings are completely identical, the string matching confidence score is recorded as a value of one. A string matching confidence score of one only indicates that consecutive strings in the structured text data are completely identical to sensitive words in the sensitive word library. It does not indicate the model's predicted probability, nor is it compared with the category score output by the local text semantic review model.

[0039] When the same text content matches multiple sensitive words, the matching position, risk type and string matching confidence of each sensitive word are recorded respectively, and each matching record is included in the sensitive word screening result corresponding to the same collection frame identifier; When the text content in the structured text data does not match any sensitive words, the structured text data is marked as not matching any sensitive words and then passed to the local text semantic review process. When the text content in the structured text data is empty, string matching and local text semantic auditing are not performed, and a text auditing result with no text risk is generated; For structured text data that contains sensitive words, there is no requirement for the local text semantic review model to reconfirm; the corresponding text risk information is directly generated based on the sensitive word screening results. The local text semantic auditing model is a text security classification model pre-deployed on the local broadcast control terminal of the subway passenger information system. It can output the category identifiers and category scores corresponding to normal, semantic ambiguity, implicit violations and disguised bad guidance. In one specific implementation, the locally deployed JoySafety security auditing model can be used, or other local text security classification models with the same input and output capabilities can be used.

[0040] When the same acquisition frame identifier corresponds to only one piece of structured text data, the text content in that structured text data is directly used as the text to be reviewed. When the same capture frame identifier corresponds to multiple structured text data, the security audit program arranges the text content according to the position of each text area in the effective playback area screen. First, the text areas are arranged in order from top to bottom, and then the text areas at the same horizontal position are arranged in order from left to right. The arranged text content is connected with a line break separator to form the text to be audited. At the same time, a mapping relationship is established between each text fragment in the text to be reviewed and the corresponding text area identifier, so that the risky text discovered by the local text semantic review model can be traced back to the text area in the original valid playback area; The input to the local text semantic review model includes fixed review instructions and text to be reviewed. The fixed review instructions are pre-stored in the local configuration file and are used to limit the local text semantic review model to output review categories from normal, semantically ambiguous, implicit violations and disguised bad guidance, and output the category score of the corresponding review category. When the length of the text to be reviewed does not exceed the maximum input length of the local text semantic review model, the complete text to be reviewed will be input into the local text semantic review model. When the length of the text to be reviewed exceeds the maximum input length of the local text semantic review model, the security audit program divides the text to be reviewed into multiple text segments according to the text region boundaries, so that each text segment does not exceed the maximum input length. The security audit procedure does not merge text regions where the correspondence between two different text regions is unclear, and does not arbitrarily delete characters; when the text content in a single text region exceeds the maximum input length, it is divided into multiple text segments in character order. The local text semantic review model outputs the original category identifier and the corresponding category score for each text or text segment to be reviewed.

[0041] The security audit program reads the text label mapping table in the local configuration file and maps the original category identifiers of the model to normal, semantically ambiguous, implicit violation or disguised bad guidance respectively; The category with the highest category score is identified as the review category of the current text or text segment to be reviewed, and the corresponding category score is recorded as the text review confidence level; When the same collection frame contains multiple text segments, as long as the review category of any text segment is semantic ambiguity, implicit violation or disguised bad guidance, the semantic review result corresponding to the collection frame will be determined to have text risk, and the text risk type, text review confidence and text region mapping relationship corresponding to all risky text segments will be retained. When all text segments are classified as normal, the semantic audit result is determined to be that there is no text risk. The semantic audit results should include at least the capture frame identifier, text region location, presence of text risk, text risk type, text audit confidence level, and source of risk basis; the source of risk basis is recorded as the local text semantic audit model, and the data type of the text audit confidence level is recorded as the model category confidence level; The model parameter file of the local text semantic auditing model is loaded from the local storage device into the local memory when the security auditing program starts; the text to be audited is passed to the local text semantic auditing model through local function calls or local inter-process interfaces, and the semantic auditing result is returned to the security auditing program through local memory without sending text content to external servers. When the local text semantic auditing model fails to return a category identifier, fails to return a valid category score, returns a category identifier that cannot be converted through the text label mapping table, or the local model inference program fails to run, the text auditing status of the corresponding audit task to be completed will be updated to processing failure, and the failed step will be recorded as local text semantic auditing. No conclusion that there is no text risk will be generated.

[0042] After completing the sensitive word screening and local text semantic review, the security audit program generates text review results according to the collection frame identifier; The text review results are data generated by associating the sensitive word screening results and semantic review results according to the collection frame identifier; For structured text data that contains sensitive words, the risk type, matched text content, matching position, text region position, string matching confidence and risk basis source from the sensitive word screening results are written into the text review results, and the existence of text risk is recorded in the text review results. For structured text data that does not contain any sensitive words but whose semantic audit results indicate text risks, the text risk type, text audit confidence level, text region location, and source of risk basis from the semantic audit results are written into the text audit results, and the existence of text risks is recorded. For structured text data that does not contain any sensitive words and whose semantic audit results indicate no text risk, record "no text risk" in the text audit results. When the same collection frame corresponds to multiple structured text data, the sensitive word screening results and semantic review results corresponding to each structured text data will be grouped into the same text review result; If any structured text data in the same acquisition frame has a text risk, the text review result corresponding to that acquisition frame will record that there is a text risk, and retain all risky text, text area location, text risk type, confidence level data, confidence level data type, and source of risk basis; For the string matching confidence score generated by sensitive word screening and the model category confidence score generated by the local text semantic review model, the corresponding confidence score data types are recorded separately in the text review results. The two are not compared numerically as model probabilities of the same nature. Once the text review results are generated, the text review status of the corresponding audit task to be completed will be updated to "processed and completed", and the text review results will be written to the audit task to be completed and the audit record queue.

[0043] Video footage security audit; Specifically, in the video security review module, the security audit program reads the valid playback area image that is the same as the valid playback area image processed by the text recognition module from the video review queue, and uses the valid playback area image as the video frame to be reviewed; The video frames to be reviewed and the effective playback area screens for text region positioning and text recognition by the text recognition module have the same acquisition frame identifier, so the screen review process will no longer perform secondary sampling of the effective playback area screens that have entered the complete audit task. The local visual review model is pre-stored in the local storage device of the broadcast control terminal of the subway passenger information system; the local visual review model is a local image classification model that can output the corresponding category labels and category scores of normal, bad images, illegal scenes and inappropriate visual content; In one specific implementation, a locally deployed Falcons.ai visual auditing model can be used, or other local image security classification models with the same input and output capabilities can be used.

[0044] The security audit program reads the model configuration file that is stored together with the local visual audit model; the model configuration file records at least the mapping relationship between the model input width, model input height, color channel arrangement order, pixel value processing method, model output label and image risk type; Adjust the size of the video frames to be reviewed according to the model input width and model input height, and adjust the pixel channels according to the color channel arrangement order recorded in the model configuration file; When the model configuration file specifies that pixel values ​​should be normalized, the values ​​of each pixel channel will be converted to the range specified in the model configuration file. When the model configuration file specifies that standardization should be performed based on the channel mean and channel standard deviation, the channel mean and channel standard deviation in the model configuration file are read, and standardization is performed on each pixel channel. After size adjustment, channel arrangement, and pixel value processing, the video frames to be reviewed form visual model input data and are input into the local visual review model; The model parameter file and inference program of the local visual review model are all run locally on the broadcast control terminal of the subway passenger information system, and the video frames to be reviewed are not sent to external servers. The local visual review model performs inference on the input data of the visual model and outputs one or more original category labels and their corresponding category scores. The security audit program reads the visual label mapping table in the local configuration file and maps the original category identifiers to normal, bad images, illegal scenes or inappropriate visual content; The mapping category corresponding to the original category identifier with the highest category score is determined as the image review category of the current video frame to be reviewed, and the corresponding category score is recorded as the image review confidence level; When the mapping category is normal, the image review result is recorded as no image risk exists; When the mapping category is inappropriate image, illegal scene, or unsuitable visual content, the image review result is recorded as having image risk, and the corresponding image risk type and image review confidence level are recorded; The video review results should include at least the capture frame identifier, whether there is a video risk, the type of video risk, the video review confidence level, the source of the risk basis, and the local visual review model version identifier; the source of the risk basis is recorded as the local visual review model, and the data type of the video review confidence level is recorded as the model category confidence level.

[0045] The model configuration file, visual label mapping table, and local visual audit model use the same model version identifier; the model version identifier is written into the screen audit results and the subsequent local audit records. When the local visual audit model fails to return the original category identifier, fails to return a valid category score, the original category identifier cannot be converted through the visual label mapping table, or the local model inference program fails to run, the screen audit status of the corresponding audit task to be completed will be updated to processing failure, and the failed step will be recorded as local visual audit, without generating a conclusion that there is no screen risk. Once the video review result is generated normally, the video review status of the corresponding audit task to be completed will be updated to "processed and completed", and the video review result will be written to the audit task to be completed and the audit record queue.

[0046] Audit processing and local audit record storage; Specifically, in the audit processing module, the security audit program determines whether to generate an audit processing result based on the text review status and screen review status of the audit task to be completed. The security audit procedure uses the captured frame identifier as the basis for association, linking the text review results and the image review results to the same audit task to be completed; Risk records, risk alerts, or risk-free audit logs are generated only when both the text review status and the video review status are completed. When any of the following processes fails: text recognition, sensitive word screening, local text semantic review, or local visual review, the corresponding review status will be updated to "processing failed," and the failed step and the reason for failure will be written into the audit task to be completed. When the text review status or the screen review status is "processing failed", no risk-free audit log will be generated. Instead, an audit exception record will be generated.

[0047] The audit exception record should include at least the capture frame identifier, capture time, processing failure step, failure reason, completed audit result, business playback window information, effective playback area, window occlusion status, and the local storage location of the corresponding effective playback area screen. Security audit procedures must not assume that an audit process that has not yet returned an audit result, is in a pending processing status, or has returned a processing failure status is free of risk; Risk records, risk alerts, risk-free audit logs, or audit anomaly records are all considered audit processing results. When a text audit result record contains text risks, the security audit program will save the valid playback area of ​​the text audit result as an abnormal snapshot. Read the risk text, text area location, text risk type, confidence level data, confidence level data type, and risk basis source from the text review results, and generate the risk record corresponding to the text risk; Risk records are data used to record discovered risks and the basis for their review; risk records corresponding to text risks should include at least the collection frame identifier, collection time, risk source, risk type, risk text, text area location, confidence level data, confidence level data type, risk basis source, and local storage location of the abnormal snapshot.

[0048] When the source of risk assessment is string matching of sensitive words, the confidence score data is the string matching confidence score, and the data type of the confidence score record is the string matching confidence score. When the source of risk is a local text semantic review model, the confidence score data is the text review confidence score, and the data type of the confidence score data record is the model category confidence score; The security audit process also generates risk alerts; the risk alerts corresponding to text risks include at least the identifier of the broadcast control terminal of the subway passenger information system, the identifier of the acquisition frame, the acquisition time, the text risk type, the risk text, and the local storage location of the abnormal snapshot; Risk alarms are displayed through the local operation and maintenance interface of the subway passenger information system broadcast control terminal, enabling operation and maintenance personnel to retrieve the corresponding abnormal snapshots, risk text, and text area locations. The risk record and risk alarm corresponding to the text risk together constitute the audit processing result corresponding to the collection frame; When the video review result record shows a video risk, the video frame to be reviewed that generated the video review result will be saved as an abnormal snapshot. Read the image risk type, image review confidence level, confidence level data type, and risk basis source from the image review results, and generate the risk record corresponding to the image risk.

[0049] The risk record corresponding to the image risk shall include at least the capture frame identifier, capture time, risk source, image risk type, image review confidence level, confidence level data type, risk basis source, local visual review model version identifier, and local storage location of abnormal snapshot; The security audit procedure also generates risk alarms that include the type of video risk; the risk alarms include at least the identifier of the broadcast control terminal of the subway passenger information system, the identifier of the acquisition frame, the acquisition time, the type of video risk, and the local storage location of the abnormal snapshot; The risk record and risk alarm corresponding to the risk in the image together constitute the audit processing result corresponding to the captured frame; When both the text review result and the image review result corresponding to the same captured frame are recorded as having risks, the text risk information and the image risk information are recorded separately in the same risk record. The risk record includes at least the capture frame identifier, capture time, text risk type, risk text, text area location, text review confidence or string matching confidence, text risk basis source, image risk type, image review confidence, image risk basis source, and local storage location of abnormal snapshots; Simultaneously, risk alarms containing text risk types and image risk types are generated; this implementation method does not use one type of normal review result to offset another type of already discovered risk. As long as either the text review result or the image review result indicates the presence of risk, a risk record and risk alarm are generated.

[0050] When both the text review status and the video review status are completed, and the text review result record shows no text risk, and the video review result record shows no video risk, a risk-free audit log is generated. The risk-free audit log is used to record data that the currently valid playback area has undergone text and image review and no risks have been found; The risk-free audit log should include at least the collection frame identifier, collection time, subway passenger information system broadcast control terminal identifier, text review status, screen review status, text recognition model version identifier, local text semantic review model version identifier, local visual review model version identifier, subway operation scenario sensitive word library version identifier, and risk-free audit conclusion. When the current effective playback area is partially obscured, the risk-free audit log records the window obscuration status, obscured window identifier, and obscured area location. The risk-free audit conclusion is limited to the effective playback area that is actually visible in the display buffer. The content that is not visible below the obscured area is not considered as having completed the risk-free audit. The risk-free audit log serves as the audit processing result corresponding to this acquisition frame.

[0051] After generating the audit processing results, the security audit program will effectively associate the playback area screen, structured text data, text review results, screen review results, and audit processing results according to the acquisition frame identifier to form a local audit record; Local audit logs refer to a complete record of the process of capturing images of a valid playback area, text recognition, text review, image review, and risk handling. Local audit logs should include at least the following: the identifier of the broadcast control terminal of the subway passenger information system, the identifier of the acquisition frame, the acquisition time, the identifier of the target display connection, the identifier of the display buffer, the identifier of the business playback window, the window coordinates and window size, the area coordinates and area size of the effective playback area, the window occlusion status, the identifier of the occluded window and the location of the occluded area, the effective playback area image or the local storage location of the effective playback area image, the structured text data, the text review status, the text review result, the image review status, the image review result, the risk record and risk alarm, the risk-free audit log or the audit anomaly record, the character recognition model version identifier, the local text semantic review model version identifier, the local visual review model version identifier, and the subway operation scenario sensitive word library version identifier; When the audit processing result is a risk record and a risk alarm, the local audit record saves the local storage location of the abnormal snapshot, the source of the risk, the type of risk, the confidence level data, the data type of the confidence level data, the source of the risk basis, and the content of the risk alarm. When the audit processing result is a risk-free audit log, the local audit record saves the text audit result and the screen audit result on which the risk-free audit conclusion was based; When the audit result is an audit anomaly record, the local audit record saves the failed processing step, the reason for the failure, the completed audit result, and the local storage location of the corresponding valid playback area screen, and does not generate a risk-free audit conclusion. The security audit procedure writes local audit records to the local storage device of the broadcast control terminal of the subway passenger information system; various data are linked together through the collection frame identifier, enabling operation and maintenance personnel to query the corresponding valid playback area screen, identify text, audit results and risk handling process based on the collection time, collection frame identifier, risk type, risk basis source or model version identifier; For valid playback areas that do not pose a risk, local audit records and representative valid playback areas can be saved according to a preset retention period; for valid playback areas that pose a risk or have audit anomalies, complete anomaly snapshots, structured text data, audit results, risk basis, and anomaly information should be saved.

[0052] During continuous operation, when the subway passenger information system broadcast control program switches from window playback mode to full-screen playback mode, or from full-screen playback mode to window playback mode, the window coordinates or window size of the service playback window change. Before extracting the valid playback area image each time, the security audit program rereads the current window coordinates, window size, and window layer of the business playback window through the window management interface; If the current window coordinates, window size, and window level are the same as the previous reading result, continue to use the previously determined valid playback area; When the current window coordinates, window size, or window hierarchy changes, re-execute window coordinate transformation, multi-display device target display output selection, client area determination, effective playback area boundary clipping, and window occlusion area calculation. After redefining the effective playback area, the positions of subsequent text areas are all based on the upper left corner of the updated effective playback area as the coordinate starting point; each local audit record saves the business playback window information and effective playback area information of the corresponding acquisition frame, so that the audit data formed before and after the window change will not be confused in position; When the business playback window is closed or no longer visible, the security audit program generates an abnormal record of the business playback window, stops the extraction of the current screen, and does not generate a risk-free audit log.

[0053] In this embodiment, the subway passenger information system broadcast control terminal operates in the subway's internal network environment, and the security audit program does not have an external cloud audit interface configured. During system initialization, the text recognition model, local text semantic review model, local visual review model, model configuration file, tag mapping table, and subway operation scenario sensitive word library are loaded from the local storage device. The effective playback area is extracted from the local display buffer of the broadcast control terminal of the subway passenger information system; the structured text data is generated by the local character recognition model; sensitive word screening is performed by the local string matching program; the semantic review results are generated by the local text semantic review model; the screen review results are generated by the local visual review model; risk alarms are output through the local operation and maintenance interface; and local audit records are written to the local storage device of the broadcast control terminal of the subway passenger information system. When the broadcast control terminal of the subway passenger information system is disconnected from the external network, the following processes continue to be executed: display resource positioning, business playback window positioning, effective playback area determination, display buffer mapping, effective playback area image extraction, text recognition, sensitive word screening, local text semantic review, local visual review, risk alarm, and local audit record saving.

[0054] Throughout the entire security audit process, the effective playback area screen, structured text data, text to be reviewed, text review results, video frames to be reviewed, and screen review results are not sent to external servers.

[0055] In a complete operation, after the subway passenger information system broadcast control program starts and displays the service playback window, the security audit program reads the display connection, display controller, main display plane and frame buffer information in the graphics card device to obtain the current display buffer information; reads the current window tree, determines the service playback window according to the playback window identifier in the local configuration file; transforms the client area coordinates of the service playback window to the target display output coordinate system, determines the effective playback area, and records the window occlusion status; The security audit program establishes a read-only mapping relationship based on the mapping handle and mapping offset corresponding to the frame buffer. It reads the pixel data in the effective playback area according to the coordinates of the effective playback area, the byte span of each line, and the number of bytes occupied by a single pixel to form the effective playback area screen and assigns a capture frame identifier to the effective playback area screen that enters the image and text review process. The same valid playback area is simultaneously sent to the text recognition queue and the screen review queue; the text recognition model performs text region localization and text recognition on the valid playback area screen to obtain structured text data; for the text content in the scrolling announcement area, the security audit program forms a scrolling text sequence based on the overlapping characters in the consecutive valid playback area screens; The text content in the structured text data is matched against a sensitive word database for subway operation scenarios. When a sensitive word is matched, text risk information is generated based on the sensitive word screening results. When no sensitive word is matched, the text to be reviewed is input into a local text semantic review model, and a semantic review result is generated based on the category identifier and category score output by the model. The text review result is generated based on the sensitive word screening result and the semantic review result, and the text review status is updated to "processing completed". According to the model configuration file of the local visual review model, the size adjustment, color channel adjustment and pixel value processing are performed on the video frames to be reviewed. The resulting visual model input data is input into the local visual review model, and the screen review result is formed according to the category identifier and category score output by the model. The screen review status is then updated to "processing complete".

[0056] When both text review and video review status are completed, an audit processing result is generated based on the text review result and video review result; when either review result indicates that there is a risk, an anomaly snapshot is saved, and a risk record and risk alarm are generated; when both review results indicate that there is no risk, a risk-free audit log is generated; when either processing step fails, an audit anomaly record is generated. Finally, the effective playback area screen, structured text data, text review results, screen review results, and audit processing results are associated and saved according to the acquisition frame identifier to form a local audit record.

[0057] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A subway PIS safety audit system, characterized in that, include: The information acquisition module is used to acquire the display buffer information and window information of the broadcast control terminal of the subway passenger information system, determine the service playback window based on the window information, and determine the effective playback area based on the service playback window. The image extraction module is used to establish a display buffer mapping relationship based on the display buffer information, and extract the image of the valid playback area from the display buffer according to the valid playback area; The text recognition module is used to locate and recognize text regions in the effective playback area to obtain structured text data. The text security review module is used to match the text content in the structured text data with sensitive words in the sensitive word library of subway operation scenarios to obtain sensitive word screening results. The structured text data that does not match sensitive words is input into the local text semantic review model to obtain semantic review results. The text review results are formed based on the sensitive word screening results and the semantic review results. The screen security review module is used to input the screen of the effective playback area into the local visual review model to obtain the screen review result; The audit processing module is used to generate an audit processing result based on the text review result and the screen review result, and to associate and save the effective playback area screen, the structured text data, the text review result, the screen review result and the audit processing result according to the acquisition frame identifier to form a local audit record.

2. The subway PIS safety audit system according to claim 1, characterized in that, The service playback window is determined based on the window information, including: Access the graphics card device of the broadcast control terminal of the subway passenger information system through the direct rendering infrastructure interface to obtain the display buffer information corresponding to the current display screen; The window identifier, window coordinates, window size, and window level of the current window are read through the window management interface, and the window identifier, window coordinates, window size, and window level are combined to form window information; The service playback window corresponding to the broadcast control program of the subway passenger information system is determined from the window information based on the window identifier.

3. A subway PIS safety audit system according to claim 2, characterized in that, Determining the valid playback area based on the aforementioned service playback window includes: The coverage area of ​​the service playback window in the current display screen is determined based on the window coordinates and window size of the service playback window; The covered area is defined as the effective playback area, so that desktop backgrounds, taskbars and non-business windows located outside the effective playback area are not included in the extraction of the effective playback area screen.

4. A subway PIS safety audit system according to claim 3, characterized in that, Establishing a display buffer mapping relationship based on the display buffer information, and extracting the valid playback area image from the display buffer according to the valid playback area, including: The display buffer is mapped to the local address space via the direct rendering infrastructure interface; Based on the region coordinates and region size of the effective playback region, determine the range of pixel data in the display buffer corresponding to the effective playback region; Pixel data within the specified pixel data range is read to form the effective playback area image.

5. A subway PIS safety audit system according to claim 4, characterized in that, The effective playback area is used to locate and recognize text regions to obtain structured text data, including: The locally deployed text recognition model is invoked to locate the text regions in the subtitles, scrolling announcements, pop-up texts, and interface prompts in the effective playback area, thereby obtaining the text region positions. The text content is obtained by performing text recognition on the image content corresponding to the text region. The text region location and the text content are associated to form the structured text data.

6. A subway PIS safety audit system according to claim 5, characterized in that, Sensitive word screening is performed on the structured text data, including: The text content in the structured text data is matched with sensitive words in the sensitive word library of subway operation scenarios. The matched text content, the position of the text content in the structured text data, and the corresponding risk type are recorded as the sensitive word screening results.

7. A subway PIS safety audit system according to claim 1, characterized in that, Input the structured text data that did not match sensitive words into the local text semantic review model, including: The local text semantic review model performs semantic analysis on the structured text data that does not match sensitive words, identifies semantic ambiguity, implicit violations and disguised misleading content, and obtains the semantic review result. The sensitive word screening results and the semantic review results are correlated to form the text review results.

8. A subway PIS safety audit system according to claim 4, characterized in that, The valid playback area is input into the local visual review model to obtain the image review result, including: The continuously extracted valid playback area is sampled to obtain the video frames to be reviewed; The video frame to be reviewed is input into the local visual review model to identify inappropriate images, illegal scenes and unsuitable visual content in the video frame to be reviewed. The identified image risk types are associated with the corresponding video frames to be reviewed, thus forming the image review results.

9. A subway PIS safety audit system according to claim 1, characterized in that, An audit processing result is generated based on the text review result and the image review result, including: When the text review result or the screen review result indicates that there is a risk, the corresponding valid playback area screen is saved as an abnormal snapshot, a risk record including risk type and confidence level data is generated, and a risk alarm including the risk type is triggered. The risk record and the risk alarm are used as the audit processing result. When neither the text review result nor the screen review result indicates the presence of risk, a risk-free audit log is generated, and the risk-free audit log is used as the audit processing result. The local audit records are stored in the broadcast control terminal of the subway passenger information system.

Images