Operation and maintenance safety auditing method and system based on OmniParser and computer equipment

Abstract

The invention provides an operation and maintenance security auditing method based on OmniParser, and the method comprises the steps: a data collection step: collecting the operation process of operation and maintenance personnel, and obtaining an operation and maintenance video stream; a key operation identification step: converting interaction elements in the operation and maintenance video stream into structured data based on OmniParser, identifying key operations of operation and maintenance personnel, and converting the key operations into description texts; a timestamp marking step: synchronizing the description text with a timestamp in the operation and maintenance video stream, and associating the description text with the operation and maintenance video stream to obtain a marked operation and maintenance video stream; a retrieval step: according to a retrieval condition input at an auditing end, outputting a corresponding retrieval result when obtaining text information meeting the retrieval condition, and selectively skipping to a corresponding position for marking the operation and maintenance video stream; and a monitoring and abnormal operation identification step: monitoring the operation and maintenance video stream and the corresponding description text in real time based on the constructed user behavior analysis model so as to identify the abnormal operation and carry out alarm and blocking.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and more specifically, to an OmniParser-based operation and maintenance security audit method, system and computer equipment. Background Art

[0002] In the traditional field of network security auditing, auditing methods mainly rely on log records and text auditing, which have problems such as large log volumes, poor readability, and difficulty in associating security events. With the development of technology, screen recording auditing methods have emerged to improve the intuitiveness and accuracy of auditing by recording the user's operation process. However, these methods are time-consuming and labor-intensive in event tracking, and are prone to missing event details. At the same time, a large amount of resources are required to store related files, resulting in slow audit response and reduced audit efficiency.

[0003] Therefore, providing an OmniParser-based operation and maintenance security audit method to improve audit efficiency and enhance audit accuracy is an urgent problem to be solved. Summary of the invention

[0004] The present invention provides an operation and maintenance safety audit method, system and computer equipment based on OmniParser, so as to at least solve the problems of low audit efficiency and poor audit accuracy of the existing operation and maintenance safety audit method.

[0005] The present invention aims to provide an operation and maintenance security audit method based on OmniParser. By introducing OmniParser technology to analyze screen recording videos, the key operations of operation and maintenance personnel are identified and described. At the same time, combined with the operation and maintenance personnel behavior analysis model, high-risk operations are monitored and identified in real time, thereby achieving an operation and maintenance security audit based on OmniParser with accuracy, efficiency and security.

[0006] To achieve the above object, the present invention provides an operation and maintenance security audit method based on OmniParser, comprising:

[0007] Data collection steps: collect the operation process of the operation and maintenance personnel and obtain the operation and maintenance video stream;

[0008] Key operation identification step: based on OmniParser, the interactive elements in the operation and maintenance video stream are converted into structured data, and the key operations of the operation and maintenance personnel are identified, and the key operations are converted into description texts;

[0009] Timestamp marking step: synchronizing the description text with the timestamp in the operation and maintenance video stream, and associating the description text with the operation and maintenance video stream to obtain a marked operation and maintenance video stream;

[0010] Retrieval step: according to the retrieval conditions inputted at the audit end, when the text information meeting the retrieval conditions is obtained, the corresponding retrieval results are outputted, and the corresponding position of the marked operation and maintenance video stream can be selectively jumped;

[0011] Monitoring and abnormal operation identification step: Based on the constructed user behavior analysis model, the operation and maintenance video stream and the corresponding description text are monitored in real time to identify abnormal operations and issue alarms and block them.

[0012] Furthermore, the key operation identification step includes:

[0013] Based on OmniParser, the operation and maintenance video stream is analyzed frame by frame, the interactive elements in the operation and maintenance video stream are identified and converted into structured data, and the user behavior of the operation and maintenance personnel is identified, the user behavior is mapped to key operations, and the key operations are converted into description text, which includes the timestamp of the operation, the type of operation and the operation target.

[0014] Furthermore, the timestamp marking step further comprises:

[0015] The timestamp mark in the marked operation and maintenance video stream can be edited on the audit end, and the timestamp mark can be deleted and added.

[0016] Furthermore, the retrieval step includes:

[0017] Enter the search conditions on the audit end, search the description text, and output the corresponding search results when the text information that meets the search conditions is found. The search results include a description text summary and a corresponding timestamp. Click on the description text summary or the corresponding timestamp to jump to the corresponding position of the marked operation and maintenance video stream.

[0018] Furthermore, the search condition includes at least one of the operation type, the operation target and the operation time range.

[0019] Furthermore, the monitoring and abnormal operation identification step includes:

[0020] Based on the pre-trained random forest model of historical operation and maintenance data, multiple user behavior decision trees are constructed to monitor the operation and maintenance video stream and the corresponding description text in real time to identify abnormal operations;

[0021] If a potential abnormal operation is identified, the audit end will trigger an alarm; if an abnormal operation is identified, the audit end will block it according to the preset security policy.

[0022] The present invention provides an OmniParser-based operation and maintenance security audit system, which is applied to the above-mentioned OmniParser-based operation and maintenance security audit method, comprising:

[0023] Data collection module: collects the operation process of operation and maintenance personnel and obtains the operation and maintenance video stream;

[0024] Key operation identification module: based on OmniParser, the interactive elements in the operation and maintenance video stream are converted into structured data, and the key operations of the operation and maintenance personnel are identified, and the key operations are converted into description texts;

[0025] Timestamp marking module: synchronizes the description text with the timestamp in the operation and maintenance video stream, and associates the description text with the operation and maintenance video stream to obtain a marked operation and maintenance video stream;

[0026] Retrieval module: according to the retrieval conditions inputted at the audit end, when obtaining text information meeting the retrieval conditions, the corresponding retrieval results are outputted, and the corresponding position of the marked operation and maintenance video stream can be selectively jumped;

[0027] Monitoring and abnormal operation identification module: Based on the constructed user behavior analysis model, the operation and maintenance video stream and the corresponding description text are monitored in real time to identify abnormal operations and issue alarms and block them.

[0028] Furthermore, the key operation identification module includes:

[0029] Based on OmniParser, the operation and maintenance video stream is analyzed frame by frame, the interactive elements in the operation and maintenance video stream are identified and converted into structured data, and the user behavior of the operation and maintenance personnel is identified, the user behavior is mapped to key operations, and the key operations are converted into description text, which includes the timestamp of the operation, the type of operation and the operation target.

[0030] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation and maintenance security audit method based on OmniParser as described above is implemented.

[0031] The present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the operation and maintenance security audit method based on OmniParser as described above is implemented.

[0032] Compared with the prior art, the advantages and positive effects of the present invention are:

[0033] The present invention ensures the transparency of the audit process by recording the operation behaviors of the operation and maintenance personnel in real time, so that each operation can be traced. By introducing OmniParser technology, the interactive elements in the screen recording video stream can be converted into structured data, and key operations can be identified to generate detailed description text and timestamp tags. This enables auditors to quickly locate key operations without having to view the video frame by frame, thereby greatly improving audit efficiency.

[0034] Through the application of real-time monitoring and user behavior analysis models, the present invention can identify and issue warnings before potential security incidents occur, effectively preventing security risks. For identified high-risk operations that have occurred, the system can immediately respond and take measures, such as terminating the session or restricting permissions, thereby preventing potential security threats in the first place.

[0035] In addition, the retrieval function provided by the present invention is simple and easy to use, and supports full-text search and advanced search based on multiple search conditions, so that auditors can quickly locate relevant video clips according to specific conditions, further improving the efficiency of operation and maintenance security audits. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the operation and maintenance security audit method based on OmniParser of the present invention;

[0037] Figure 2 Schematic diagram of an interactive element conversion interface based on OmniParser in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the structure of the operation and maintenance security audit system based on OmniParser of the present invention;

[0039] Figure 4 A schematic diagram of a computer device provided in an embodiment of the present invention.

[0040] In the above picture:

[0041] 40. bus; 41. processor; 42. memory; 43. communication interface; 100. data acquisition module; 200. key operation identification module; 300. timestamp marking module; 400. retrieval module; 500. monitoring and abnormal operation identification module. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] In the description of the present application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "liquid level", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. The terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, "multiple" means two or more.

[0044] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0045] like Figure 1-Figure 4 As shown, the present invention provides an operation and maintenance security audit method, system and computer equipment based on OmniParser to achieve an operation and maintenance security audit with accuracy, efficiency and security. Figure 1-Figure 4 The OmniParser-based operation and maintenance security audit method, system and computer equipment provided by the present invention are described in detail.

[0046] Embodiment 1:

[0047] like Figure 1 As shown in the figure, an operation and maintenance security audit method based on OmniParser includes:

[0048] Data collection step S100: collecting the operation process of the operation and maintenance personnel to obtain the operation and maintenance video stream;

[0049] In some embodiments, data acquisition agent software is deployed on the workstation of the operation and maintenance personnel to capture screen activities, including but not limited to mouse movement, clicks, keyboard input, application window switching, etc. When the operation and maintenance personnel log in to the system and start working, the data acquisition agent software automatically starts the screen recording function and starts recording the operation process. The screen recording can be set to be triggered under specific conditions, such as when the user logs in to a specific system or performs a specific task. According to audit requirements and storage resources, the resolution, frame rate and encoding format of the screen recording are configured to balance video quality and storage efficiency. During the screen recording process, metadata related to the video stream is also recorded, including operation time, operator identity information, login session information, applications used, and system environment parameters. The operation and maintenance video stream is transmitted to the audit end in real time and stored.

[0050] Key operation identification step S200: based on OmniParser, the interactive elements in the operation and maintenance video stream are converted into structured data, and the key operations of the operation and maintenance personnel are identified, and the key operations are converted into description texts;

[0051] In some embodiments, OmniParser is a screen parsing tool for converting screenshots of user interfaces into structured data. OmniParser combines interactive area detection models, icon description models, and OCR modules to analyze each frame in the operation and maintenance video stream frame by frame.

[0052] Preferably, the key operation identification step S200 includes:

[0053] Based on OmniParser, the operation and maintenance video stream is analyzed frame by frame, the interactive elements in the operation and maintenance video stream are identified and converted into structured data, and the user behavior of the operation and maintenance personnel is identified, the user behavior is mapped to key operations, and the key operations are converted into descriptive text. The descriptive text includes the timestamp, operation type and operation target of the operation.

[0054] In some embodiments, Figure 2 As shown in Figure 1, OmniParser can analyze the operation and maintenance video stream frame by frame to capture the text information on the screen, including user input commands, system output, text on the application interface, etc. In addition to text, OmniParser can also identify graphical interface elements on the screen, such as icons, buttons, menu options, etc., and convert these interactive elements into Figure 2The structured data shown is easy to store and retrieve. At the same time, all user behaviors on the screen are captured through OmniParser technology, including mouse clicks, keyboard input, etc. The captured behavior data is parsed to identify user intentions. For example, mouse clicks may be associated with opening files, selecting menu items or confirming dialog boxes. The parsed behaviors are mapped to key operations, including but not limited to file operations (such as open, edit, save, delete), command execution, application startup, system setting changes, etc. For file upload and download operations, relevant file data is captured by monitoring file system events and network traffic. File system events include file creation, modification, deletion, and detailed information of file transfer. Relevant communication protocols such as HTTP, FTP, SMB, etc. are parsed to extract detailed information of file transfer, including file name, size, source and target path, transfer status, etc. By analyzing network traffic, file upload and download activities can be identified and recorded. Based on the identified information, the system automatically generates description text, for example: "The user opened the configuration file config.xml by mouse click at 14:30:23." The description text includes the timestamp of the operation, the operation type, and the operation target.

[0055] Timestamp marking step S300: synchronizing the description text with the timestamp in the operation and maintenance video stream, and associating the description text with the operation and maintenance video stream to obtain the marked operation and maintenance video stream;

[0056] In some embodiments, the description text contains timestamp information of the operation, the description text is synchronized with the timestamp in the operation and maintenance video stream, and the description text is associated with the operation and maintenance video stream to obtain a marked operation and maintenance video stream, and the description text will be displayed synchronously with the video as marking information of the operation and maintenance video.

[0057] Preferably, the timestamp marking step S300 further includes:

[0058] Edit the timestamp in the operation and maintenance video stream on the audit side, and delete and add timestamps.

[0059] In some embodiments, the audit terminal has a user editing interface that allows auditors to select, edit or delete existing timestamp tags. Auditors can select and modify the description text of the tag or adjust its association with the timestamp in the video stream or delete the timestamp tag by clicking on the tag on the interface, and can add new timestamp tags at any position in the video stream. In order to improve efficiency, the system can provide a batch editing function, allowing auditors to select multiple timestamp tags for editing or deletion at the same time.

[0060] Retrieval step S400: according to the retrieval condition inputted at the audit end, when the text information meeting the retrieval condition is obtained, the corresponding retrieval result is outputted, and the corresponding position of the marked operation and maintenance video stream can be selectively jumped;

[0061] Preferably, the searching step S400 includes:

[0062] Enter the search conditions on the audit end to search the description text. When the text information that meets the search conditions is found, the corresponding search results are output. The search results include the description text summary and the corresponding timestamp. Click the description text summary or the corresponding timestamp to jump to the corresponding position of the marked operation and maintenance video stream.

[0063] Preferably, the search condition includes at least one of an operation type, an operation target, and an operation time range.

[0064] In some embodiments, auditors can search based on multiple dimensions, such as combining operation type, operation target, and operation time range, to narrow the search scope and quickly locate specific events. The present invention also supports fuzzy retrieval so that relevant records can still be retrieved when the description text does not completely match. In the search result list, matching keywords or phrases can be highlighted to facilitate auditors to quickly identify relevant information. Auditors can sort the search results as needed, such as by timestamp, operation type, or relevance. Click on the description text summary or timestamp in the search results to jump to the corresponding position of the marked operation and maintenance video stream.

[0065] Monitoring and abnormal operation identification step S500: Based on the constructed user behavior analysis model, the operation and maintenance video stream and the corresponding description text are monitored in real time to identify abnormal operations and issue alarms and block them.

[0066] Preferably, the monitoring and abnormal operation identification step S500 includes:

[0067] Based on the pre-trained random forest model of historical operation and maintenance data, multiple user behavior decision trees are constructed to monitor the operation and maintenance video stream and the corresponding description text in real time to identify abnormal operations;

[0068] If a potential abnormal operation is identified, the audit end will trigger an alarm; if an abnormal operation is identified, the audit end will block it according to the preset security policy.

[0069] In some embodiments, before building a user behavior analysis model, the system needs to collect a large amount of historical operation and maintenance data, and perform preprocessing work such as cleaning, normalization, and feature extraction. A random forest model is trained based on historical operation and maintenance data. The model consists of multiple decision trees and can process high-dimensional data and recognize complex patterns. Behavioral features such as operation frequency, operation sequence, operation duration, etc. are extracted from the operation and maintenance video stream and description text as input to the model, and the extracted features are compared with the pre-trained model to identify abnormal behavior. When the system identifies potential abnormal operations, it immediately notifies the auditor through an alarm mechanism. The alarm method can be sound, email, SMS or system notification. For confirmed abnormal operations, the system automatically executes blocking measures according to preset security policies, such as terminating sessions, restricting user permissions or locking accounts.

[0070] Embodiment 2:

[0071] like Figure 3 As shown, the present invention provides an OmniParser-based operation and maintenance security audit system, which is applied to the above-mentioned OmniParser-based operation and maintenance security audit method, including:

[0072] Data collection module 100: collects the operation process of the operation and maintenance personnel and obtains the operation and maintenance video stream;

[0073] Key operation identification module 200: based on OmniParser, the interactive elements in the operation and maintenance video stream are converted into structured data, and the key operations of the operation and maintenance personnel are identified, and the key operations are converted into description texts;

[0074] The timestamp marking module 300: synchronizes the description text with the timestamp in the operation and maintenance video stream, and associates the description text with the operation and maintenance video stream to obtain the marked operation and maintenance video stream;

[0075] Retrieval module 400: according to the retrieval conditions inputted at the audit end, when obtaining text information meeting the retrieval conditions, outputting corresponding retrieval results, and selectively jumping to the corresponding position of the marked operation and maintenance video stream;

[0076] Monitoring and abnormal operation identification module 500: Based on the constructed user behavior analysis model, the operation and maintenance video stream and the corresponding description text are monitored in real time to identify abnormal operations and issue alarms and block them.

[0077] Preferably, the key operation identification module 200 includes:

[0078] Based on OmniParser, the operation and maintenance video stream is analyzed frame by frame, the interactive elements in the operation and maintenance video stream are identified and converted into structured data, and the user behavior of the operation and maintenance personnel is identified, the user behavior is mapped to key operations, and the key operations are converted into descriptive text. The descriptive text includes the timestamp, operation type and operation target of the operation.

[0079] Embodiment three:

[0080] Combination Figure 4 As shown, this embodiment discloses a specific implementation of a computer device. The computer device may include a processor 41 and a memory 42 storing computer program instructions.

[0081] Specifically, the processor 41 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0082] Among them, the memory 42 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 42 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 42 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 42 may be inside or outside the data processing device. In a specific embodiment, the memory 42 is a non-volatile memory. In a specific embodiment, the memory 42 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, referred to as PROM), an erasable PROM (Erasable ProgrammableRead-Only Memory, referred to as EPROM), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, referred to as EEPROM), an electrically alterable ROM (Electrically Alterable Read-Only

[0083] The RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0084] The memory 42 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 41 .

[0085] The processor 41 implements the OmniParser-based operation and maintenance security audit method in the above embodiment by reading and executing computer program instructions stored in the memory 42 .

[0086] In some of the embodiments, the computer device may further include a communication interface 43 and a bus 40. Figure 4 As shown, the processor 41, the memory 42, the communication interface 43 and the bus 40 are connected and communicate with each other.

[0087] The communication interface 43 is used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0088] The communication interface 43 can also realize data communication with other components such as external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0089] The bus 40 includes hardware, software or both, and couples the components of the computer device to each other. The bus 40 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 40 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SAT) bus, or a Serial Advanced Technology Attachment (SAT) bus.

[0090] The bus 40 may include a SATA bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable bus, or a combination of two or more of these. Where appropriate, the bus 40 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0091] In addition, in combination with the OmniParser-based operation and maintenance security audit method in the above embodiment, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the OmniParser-based operation and maintenance security audit methods in the above embodiment is implemented.

[0092] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An operation and maintenance security audit method based on OmniParser, characterized in that: include: Data collection steps: collect the operation process of the operation and maintenance personnel and obtain the operation and maintenance video stream; Key operation identification step: based on OmniParser, the interactive elements in the operation and maintenance video stream are converted into structured data, and the key operations of the operation and maintenance personnel are identified, and the key operations are converted into description texts; Timestamp marking step: synchronizing the description text with the timestamp in the operation and maintenance video stream, and associating the description text with the operation and maintenance video stream to obtain a marked operation and maintenance video stream; Retrieval step: according to the retrieval conditions inputted at the audit end, when the text information meeting the retrieval conditions is obtained, the corresponding retrieval results are outputted, and the corresponding position of the marked operation and maintenance video stream can be selectively jumped; Monitoring and abnormal operation identification step: Based on the constructed user behavior analysis model, the operation and maintenance video stream and the corresponding description text are monitored in real time to identify abnormal operations and issue alarms and block them.

2. The OmniParser-based operation and maintenance security audit method according to claim 1, characterized in that: The key operation identification step includes: Based on OmniParser, the operation and maintenance video stream is analyzed frame by frame, the interactive elements in the operation and maintenance video stream are identified and converted into structured data, and the user behavior of the operation and maintenance personnel is identified, the user behavior is mapped to key operations, and the key operations are converted into description text, which includes the timestamp of the operation, the type of operation and the operation target.

3. The OmniParser-based operation and maintenance security audit method according to claim 1, characterized in that: The timestamping step further comprises: The timestamp mark in the marked operation and maintenance video stream can be edited on the audit end, and the timestamp mark can be deleted and added.

4. The OmniParser-based operation and maintenance security audit method according to claim 1, characterized in that: The retrieval step comprises: Enter the search conditions on the audit end, search the description text, and output the corresponding search results when the text information that meets the search conditions is found. The search results include a description text summary and a corresponding timestamp. Click on the description text summary or the corresponding timestamp to jump to the corresponding position of the marked operation and maintenance video stream.

5. The OmniParser-based operation and maintenance security audit method according to claim 2, characterized in that: The search condition includes at least one of the operation type, the operation target, and the operation time range.

6. The OmniParser-based operation and maintenance security audit method according to claim 1, characterized in that: The monitoring and abnormal operation identification steps include: Based on the pre-trained random forest model of historical operation and maintenance data, multiple user behavior decision trees are constructed to monitor the operation and maintenance video stream and the corresponding description text in real time to identify abnormal operations; If a potential abnormal operation is identified, the audit end will trigger an alarm; if an abnormal operation is identified, the audit end will block it according to the preset security policy.

7. An OmniParser-based operation and maintenance security audit system, applied to the OmniParser-based operation and maintenance security audit method according to any one of claims 1 to 6, characterized in that: include: Data collection module: collects the operation process of operation and maintenance personnel and obtains the operation and maintenance video stream; Key operation identification module: based on OmniParser, the interactive elements in the operation and maintenance video stream are converted into structured data, and the key operations of the operation and maintenance personnel are identified, and the key operations are converted into description texts; Timestamp marking module: synchronizes the description text with the timestamp in the operation and maintenance video stream, and associates the description text with the operation and maintenance video stream to obtain a marked operation and maintenance video stream; Retrieval module: according to the retrieval conditions inputted at the audit end, when obtaining text information meeting the retrieval conditions, the corresponding retrieval results are outputted, and the corresponding position of the marked operation and maintenance video stream can be selectively jumped; Monitoring and abnormal operation identification module: Based on the constructed user behavior analysis model, the operation and maintenance video stream and the corresponding description text are monitored in real time to identify abnormal operations and issue alarms and block them.

8. The OmniParser-based operation and maintenance security audit system according to claim 7, characterized in that: The key operation identification module includes: Based on OmniParser, the operation and maintenance video stream is analyzed frame by frame, the interactive elements in the operation and maintenance video stream are identified and converted into structured data, and the user behavior of the operation and maintenance personnel is identified, the user behavior is mapped to key operations, and the key operations are converted into description text, which includes the timestamp of the operation, the type of operation and the operation target.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the OmniParser-based operation and maintenance security audit method as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an OmniParser-based operation and maintenance security audit method as described in any one of claims 1 to 6 is implemented.

Images