Monitoring method and related device

Abstract

The embodiment of the invention provides a monitoring method and a related device, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a monitoring image displayed on a display interface of second monitoring equipment in an internal network; the monitoring image displays monitoring information of an intranet; under the condition that the image features of the monitoring image are inconsistent with the image features of the previous frame of monitoring image, performing position identification on preset characters in the monitoring image, and determining a target area image in the monitoring image; performing content identification on the target area image to obtain monitoring information in the target area image; and when determining that a safety accident occurs in the intranet according to the monitoring information, outputting alarm information. Through the automatic monitoring mode, the monitoring personnel do not need to pay attention to the monitoring image in real time, the working pressure of the monitoring personnel is reduced, the monitoring personnel can put more energy into equipment or a system with problems, and the working efficiency of the monitoring personnel is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a monitoring method and related devices. Background Art

[0002] Isolating the internal and external networks effectively improves intranet security. This requires on-duty personnel to monitor the internal network's monitoring system and promptly contact relevant personnel for resolution when the monitoring system indicates a security incident. However, this approach can consume significant staff attention, especially during night shifts, which can lead to delayed monitoring due to staff lack of energy. Summary of the Invention

[0003] The embodiments of the present application provide a monitoring method and related devices for at least reducing the probability of untimely monitoring.

[0004] In a first aspect, an embodiment of the present application provides a monitoring method, comprising:

[0005] Acquire a monitoring image displayed on a display interface of a second monitoring device in the intranet; the monitoring image displays monitoring information of the intranet;

[0006] In the case where the image features of the surveillance image are inconsistent with those of the last acquired surveillance image, position recognition is performed on the preset text in the surveillance image to determine the target area image in the surveillance image;

[0007] Performing content recognition on the target area image to obtain monitoring information in the target area image;

[0008] When a security incident is determined to have occurred in the intranet according to the monitoring information, an alarm message is output.

[0009] Through the above method, the monitoring personnel do not have to pay attention to the monitoring images in real time, which reduces the work pressure of the monitoring personnel and allows the monitoring personnel to devote more energy to the equipment or system with problems, thereby improving the work efficiency of the monitoring personnel.

[0010] In some embodiments, the step of performing position recognition on the preset text in the surveillance image to determine the target area image in the surveillance image includes:

[0011] Performing content recognition on the surveillance image to determine the position of the preset text in the surveillance image;

[0012] Determining the position and size of the target area image in the surveillance image according to the structure of the surveillance image and the size of the surveillance image; the structure of the surveillance image is used to indicate the position and size ratio of different information in the surveillance image;

[0013] Taking the position as a starting point, the target area image is determined in the monitoring image according to the orientation and size of the target area image in the monitoring image.

[0014] Through the above method, redundant information in the surveillance image can be eliminated to improve the accuracy of subsequent content recognition.

[0015] In some embodiments, performing content recognition on the target area image to obtain monitoring information in the target area image includes:

[0016] Dividing the target area image into a plurality of cells and recording the row and column positions of each cell;

[0017] Performing content recognition on each cell to obtain monitoring data corresponding to each cell;

[0018] The monitoring data are combined according to the row-column relationship to obtain the monitoring information.

[0019] By using the above method, the target area is divided into multiple cells for identification, which reduces the identification area and improves the accuracy of content identification.

[0020] In some embodiments, dividing the target area image into a plurality of cells comprises:

[0021] Drawing row and column dividing lines in the target area image according to the size of the target area image and the structure of the target area image; the structure of the target area image is used to indicate the number of rows and columns included in the target area image and the size ratio of each row and column;

[0022] The target area image is segmented according to the row and column segmentation lines, and the target area image is segmented into a plurality of cells.

[0023] In some embodiments, before performing content recognition on each cell, the method further includes:

[0024] Preprocessing is performed on each of the cells, and the preprocessing includes at least one of increasing resolution, improving contrast, and strengthening cell borders.

[0025] In some embodiments, when a security incident is determined to have occurred in the intranet based on the monitoring information, outputting an alarm message includes:

[0026] Inserting the monitoring data in the same row in the monitoring information into the database in sequence;

[0027] If the monitoring data is not newly added data, it is determined that no security incident has occurred in the intranet, and the monitoring data is discarded;

[0028] If the monitoring data is newly added data, it is determined that a security incident occurs in the intranet, and the alarm information is output according to the monitoring data.

[0029] In some embodiments, outputting the alarm information according to the monitoring data includes:

[0030] According to the blank information in the preset alarm template, the corresponding text extracted from the monitoring data is filled in to generate the alarm information;

[0031] Call the information sending platform interface to push the alarm information to the client of the target person.

[0032] In some embodiments, obtaining a monitoring image displayed on a display interface of a second monitoring device in the intranet includes:

[0033] Using the image acquisition device corresponding to the first monitoring device, photographing the monitoring image displayed on the display interface of the second monitoring device;

[0034] or,

[0035] The monitoring image stream outputted from the second monitoring device is received via a data transmission interface between the monitoring device and the second monitoring device, and a screenshot of the monitoring image stream is taken to obtain the monitoring image.

[0036] In some embodiments, the method further comprises:

[0037] A hash value of the surveillance image is obtained. If the hash value is different from the hash value of the previous surveillance image, it is determined that the image features of the surveillance image are inconsistent with those of the previous surveillance image.

[0038] In a second aspect, an embodiment of the present application provides a monitoring device, comprising:

[0039] An acquisition module, configured to acquire a monitoring image displayed on a display interface of a second monitoring device in the intranet; the monitoring image displays monitoring information of the intranet;

[0040] a determination module, configured to, when the image features of the surveillance image are inconsistent with those of the last acquired surveillance image, perform position recognition on the preset text in the surveillance image and determine the target area image in the surveillance image;

[0041] a processing module, configured to perform content recognition on the target area image and obtain monitoring information from the target area image;

[0042] The alarm module is used to output alarm information when it is determined that a security incident occurs in the intranet based on the monitoring information.

[0043] In a third aspect, the present application provides an electronic device, comprising: a memory and a processor;

[0044] The memory is used to store computer instructions; the processor is used to execute the computer instructions stored in the memory to implement any method in the first aspect.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement any one of the methods in the first aspect.

[0046] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements any one of the methods in the first aspect when executed by a processor.

[0047] The embodiment of the present application provides a monitoring method and related devices, the monitoring method comprising: obtaining a monitoring image displayed on a display interface of a second monitoring device in an intranet; the monitoring image displays monitoring information of the intranet; in the case where the monitoring image is inconsistent with the image features of the previous monitoring image obtained, performing position recognition on the preset text in the monitoring image to determine the target area image in the monitoring image; performing content recognition on the target area image to obtain monitoring information in the target area image; outputting an alarm message when it is determined that a security incident has occurred in the intranet based on the monitoring information. Through automated monitoring, monitoring personnel do not have to pay attention to monitoring images in real time, reducing their work pressure and allowing them to devote more energy to the equipment or system with problems, thereby improving their work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a scenario provided in an embodiment of the present application;

[0049] Figure 2 A flow chart of a monitoring method provided in an embodiment of the present application Figure 1 ;

[0050] Figure 3 A schematic diagram of a surveillance image provided in an embodiment of the present application;

[0051] Figure 4 A flow chart of a monitoring method provided in an embodiment of the present application Figure 2 ;

[0052] Figure 5 A schematic diagram of the structure of a monitoring device provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0055] In the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects, and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or execution order, and words such as "first" and "second" do not necessarily mean different.

[0056] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0057] First, a brief introduction to the technical terms involved in the embodiments of the present application is given.

[0058] 1. Computer Vision: Computer vision is the study of how to "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Further image processing is performed, and the computer generates images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies with the goal of building artificial intelligence systems that can extract information from images or multidimensional data.

[0059] 2. Artificial Intelligence: Artificial intelligence (AI) is a key driving force behind the new scientific, technological, and industrial revolutions. It is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A key component of the intelligence discipline, AI aims to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI is a broad discipline encompassing robotics, speech recognition, image recognition, natural language processing, expert systems, machine learning, and computer vision.

[0060] 3. Optical Character Recognition (OCR) refers to the process by which electronic devices examine printed characters on paper, determine their shapes by detecting patterns of light and dark areas, and then use character recognition methods to translate these shapes into computer text. Specifically, for printed characters, this technology uses optical methods to convert the text in a paper document into a black-and-white dot matrix image file. Recognition software then converts the text in the image into text format for further editing and processing in word processing software.

[0061] As mentioned previously, for network security reasons, most companies currently employ a model of isolating their internal and external networks, effectively enhancing intranet security. This model requires on-duty personnel to monitor the intranet's monitoring system in real time and promptly contact relevant personnel for resolution when the monitoring system indicates a security incident.

[0062] However, the above processing method will occupy a lot of energy of the on-duty personnel, especially in the case of night shift, the problem of untimely monitoring may occur due to insufficient energy of the on-duty personnel.

[0063] In view of this, an embodiment of the present application provides a monitoring method and related devices, which collect the images displayed by the monitoring system and perform image recognition on the images displayed by the monitoring system. When it is determined that the monitoring system indicates a safety accident based on the image recognition results, an alarm message is generated, and the alarm information is sent to relevant personnel through a third-party platform to prompt the relevant personnel to deal with it in a timely manner. Through the above method, the monitoring efficiency can be greatly improved, the work pressure of the on-duty personnel can be reduced, and the work efficiency can be improved.

[0064] It should be noted that the monitoring method and related devices provided in the embodiments of the present application can be used in the field of artificial intelligence, and can also be used in any field other than artificial intelligence. The embodiments of the present application do not limit the application fields of the provided monitoring method and related devices.

[0065] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] Figure 1 A schematic diagram of a monitoring method according to an embodiment of the present invention is shown in FIG. Figure 1 As shown, it includes a first monitoring device 1 and a second monitoring device 2.

[0067] The second monitoring device 2 may include a server 21 and a display device 22 . The server 21 may be provided with a monitoring system, which may monitor various devices and / or systems in the intranet and display monitoring images through the display device 22 .

[0068] The first monitoring device 1 may include an image acquisition device 11 and a processing device 12. The image acquisition device 11 is used to acquire the monitoring image, and the processing device 12 is used to identify the monitoring image, determine whether a security incident occurs in the intranet, and output an alarm message after a security incident occurs.

[0069] In some embodiments, the image acquisition device 11 can obtain the monitoring image by acquiring the image displayed by the display device 22. For example, the image acquisition device 11 can be a camera with a viewing angle facing the display device 22, which regularly acquires the image displayed by the display device 22 to obtain the monitoring image generated by the monitoring system.

[0070] In some embodiments, the image acquisition device 11 can be connected to the server 21 via a data transmission interface (e.g., a High Definition Multimedia Interface (HDMI)). The server 21 can output a monitoring image stream to the image acquisition device 11 via the data transmission interface, and the image acquisition device 11 can obtain the monitoring image based on the monitoring image stream. For example, the image acquisition device 11 can be a video collector that periodically takes screenshots of the monitoring image stream output by the server 21 to obtain the monitoring image. For example, the video collector can use the cap.read() method to take screenshots of the monitoring image stream output by the server 21 every 30 seconds to obtain the monitoring image.

[0071] In some embodiments, the processing device 12 may also be connected to other devices. Therefore, after receiving an image, the processing device 12 needs to determine whether the image is from the image acquisition device 11. For example, the cv2 class library of OpenCV-Python can be used to set parameters of the VideoCapture method to determine whether the acquired image is from the image acquisition device 11.

[0072] In some embodiments, because the surveillance images are captured by the image acquisition device 11 through screenshots or photos, the display device 22 cannot be turned off, and the window displaying the surveillance images on the display device 22 is always at the forefront and cannot be obscured by other windows. To achieve optimal recognition, the monitored window is also maximized. To achieve the above functions, the server 21 is also equipped with a control device.

[0073] In some embodiments, the control device includes an anti-screensaver module, which can simulate the keyboard to trigger key events through an independent process, so that the display device is always in the display state. For example, the keyboard class library is used to implement the function of simulating the pressing of the ctrl key, which is triggered once every preset time period (for example, 30 seconds).

[0074] In some embodiments, the control device further includes a window control module, which is used to control the window displaying the monitoring image on the display device 22 to always be at the forefront and maximized. For example, the pywin32 class library is used to obtain the window control handle by window name, thereby controlling the window displaying the monitoring image to achieve the top and maximized display functions, thereby ensuring that the monitored window is in a monitorable state.

[0075] Below, in Figure 1 Based on the scenario shown, the monitoring method provided in the embodiment of the present application is introduced with the first monitoring device as the execution entity.

[0076] Figure 2 Schematic diagram of the monitoring method provided in this embodiment Figure 1 ,like Figure 2 Shown, including:

[0077] S201. Acquire a monitoring image displayed on a display interface of a second monitoring device in an intranet; the monitoring image displays monitoring information of the intranet.

[0078] In some embodiments, the monitoring information may include status information, operation information, etc. of each device and / or system in the intranet. That is, the monitoring information is composed of status information, operation information, etc. of each device and / or system.

[0079] In some embodiments, the corresponding relevant information of each device or system in the monitoring information may be referred to as monitoring data of the device or system, that is, the monitoring information includes monitoring data of multiple devices and / or systems.

[0080] In some embodiments, the monitoring data of each device or system may include the name of the device or system, data collection time, monitoring content, etc.

[0081] In some embodiments, the first monitoring device may be configured as follows: Figure 1 The monitoring image displayed on the display interface of the second monitoring device is obtained in the manner shown. Figure 3 As shown, the monitoring image may include monitoring images of various devices and / or systems, as well as corresponding monitoring information.

[0082] S202: When the image features of the surveillance image are inconsistent with those of the last acquired surveillance image, position recognition is performed on the preset text in the surveillance image to determine a target area image in the surveillance image.

[0083] In some embodiments, when the current surveillance image is acquired, the current surveillance image can be compared with the surveillance image of the previous frame acquired to determine whether the image features of the current surveillance image are consistent with the previous frame image. If the image features are consistent, it means that the current surveillance image is the same as the surveillance image of the previous frame acquired, that is, the surveillance information in the image has not changed and no further processing is required. If the image features are inconsistent, it means that the surveillance information in the image has changed and further processing is required. For example, the hash value of the current image can be obtained and the hash value of the current image can be compared with the hash value of the previous frame image. If the hash values ​​are consistent, it means that the image features are consistent. If the hash values ​​are inconsistent, it means that the image features are inconsistent.

[0084] In some embodiments, the target area image in the monitoring image may refer to an area image displaying monitoring information. Figure 3 The area shown in the figure displays monitoring data for each device and system.

[0085] In some embodiments, preset text may be used to indicate the location of the monitoring information, for example, Figure 3 "Monitoring Information" in the

[0086] In some embodiments, the monitoring device 1 can perform content recognition on the monitoring image, extract preset text from the monitoring image, determine the location of the preset text, and then delineate the target area image in the monitoring image based on the range of the preset text. Content recognition of the monitoring image can be performed using an artificial intelligence model using OCR technology.

[0087] S203: Perform content recognition on the target area image to obtain monitoring information in the target area image.

[0088] In some embodiments, after the target area is determined, secondary content recognition may be performed on the target area image using OCR technology to extract monitoring information from the target area image.

[0089] S204: When it is determined according to the monitoring information that a security incident occurs in the intranet, an alarm message is output.

[0090] In some embodiments, after obtaining the monitoring information, the monitoring data corresponding to each device or system in the monitoring information can be matched with the preset content. If the match with the preset content is successful, when it is determined that a security incident has occurred in the intranet, an alarm information is generated or output based on the monitoring data.

[0091] In some embodiments, after obtaining the monitoring information, the monitoring data corresponding to each device or system in the monitoring information can be inserted into the corresponding database, and during the insertion process, it is determined whether the monitoring data already exists in the database. If so, the monitoring data is discarded; if not, the monitoring data is written to the database, and alarm information is generated and output based on the monitoring data.

[0092] In some embodiments, the first monitoring device can generate alarm data to the client of relevant personnel (e.g., maintenance personnel, monitoring personnel, etc.) by calling the interface of the third-party data generation platform. For example, the alarm data can be sent to the mobile phone of the relevant personnel via SMS or phone call.

[0093] The monitoring method provided by the embodiment of the present application obtains a monitoring image displayed on the display interface of a second monitoring device in an intranet; the monitoring image displays monitoring information of the intranet; when the image features of the monitoring image are inconsistent with the image features of the previous monitoring image obtained, the position of the preset text in the monitoring image is identified to determine the target area image in the monitoring image; the content of the target area image is identified to obtain the monitoring information in the target area image; when it is determined according to the monitoring information that a safety accident has occurred in the intranet, an alarm message is output. The above method can improve the accuracy of image recognition by using the first monitoring device to collect the monitoring image and perform a second recognition on the monitoring image; and can generate alarm information based on the monitoring image, so that the monitoring personnel do not have to pay attention to the monitoring image in real time, reduce the work pressure of the monitoring personnel, and allow the monitoring personnel to devote more energy to the equipment or system with problems, thereby improving the work efficiency of the monitoring personnel.

[0094] Based on the above embodiments, Figure 4 The monitoring method provided in the embodiments of the present application is further explained.

[0095] Figure 4 Schematic diagram of the monitoring method provided in this embodiment Figure 2 ,like Figure 4 As shown, including:

[0096] S401: Acquire a monitoring image displayed on a display interface of a second monitoring device in an intranet.

[0097] S402: Determine whether the image features of the surveillance image are consistent with the image features of the previous surveillance image acquired based on the surveillance image hash value. If not, execute step S403; if consistent, end the process.

[0098] The specific implementation of steps S401-S402 in the embodiment of the present application is as follows: Figure 2 The specific implementation methods of steps S201-S202 in the illustrated embodiment are similar and will not be repeated here.

[0099] S403: Perform content recognition on the surveillance image to determine the position of the preset text in the surveillance image.

[0100] In some embodiments, the position of the preset text in the monitoring image may be the coordinates of the image block where the preset text is located in the monitoring image.

[0101] For example, the monitoring image can be subjected to content recognition through OCR technology to extract the text included in the monitoring image, and the preset text can be compared with the text included in the monitoring image to determine the preset text. After determining the preset text, the coordinates of the boundary of the image block where the preset text is located can be obtained. For example, the data structure returned after OCR recognition is a dictionary containing multiple lists, each list corresponding to a detected text block. The text list (`text` list) is the recognized text, and the coordinate list (`left`, `top`, `width`, `height`) are the upper left corner coordinates and the width and height of the bounding box, respectively. Traverse each recognition result, find the target text by comparing the text content, and then obtain its position coordinates.

[0102] For example, Figure 3 As shown, after the "monitoring information" is extracted from the monitoring image, the coordinates of the image block where the "monitoring information" is located can be calculated based on its upper left corner coordinates and width and height.

[0103] S404: Determine the target area image in the surveillance image according to the position of the preset text in the surveillance image.

[0104] In some embodiments, after obtaining the position of the preset text, the position and size of the target area image in the surveillance image can be determined based on the structure of the surveillance image and the size of the surveillance image; the structure of the surveillance image is used to indicate the position and size ratio of different information in the surveillance image.

[0105] For example, for Figure 3The structure of the surveillance image can indicate that the time is directly above the surveillance image, and its size occupies A% of the surveillance image; the surveillance image is below the time, in the upper middle part of the surveillance image, and its size occupies B% of the surveillance image; the monitoring information is in the lower left corner of the surveillance image, and its size occupies C% of the surveillance image; the specific monitoring content is in the lower right corner of the surveillance image, and its size occupies D% of the surveillance image. Optionally, the structure of the surveillance image can also indicate the distribution of each part of the information in the surveillance image in more detail, for example, the coordinates of the image corresponding to each part in the surveillance image, the aspect ratio, etc.

[0106] In some embodiments, the structure of the monitoring image may be pre-stored in the first monitoring device. The first monitoring device may determine the orientation and size of the target area based on the structure of the monitoring image and the size of the collected monitoring image. For example, Figure 3 As shown, the target area is located to the right of the image block where the monitoring information is located and at the lower right corner of the entire monitoring image. Its size accounts for x% of the monitoring image, and the aspect ratio is C%. Therefore, the corresponding length and width of the target area can be determined based on the acquired monitoring image size.

[0107] In some embodiments, after determining the position and size of the target area image in the monitoring image, the target area image can be determined in the monitoring image based on the position and size of the target area image in the monitoring image, taking the preset text position as the starting point.

[0108] For example, Figure 3 As shown, the target area is to the right of the image block where the "monitoring information" is located. The coordinates of the right fixed point of the image block where the "monitoring information" is located can be used as the starting point, and the area can be delineated to the right and downward according to the corresponding length and width to determine the target area image.

[0109] S405: Divide the target area image into multiple cells.

[0110] In some embodiments, after determining the target area image, row and column dividing lines can be drawn in the target area image according to the size of the target area image and the structure of the target area image; the structure of the target area image is used to indicate the number of rows and columns included in the target area image and the size ratio of each row and column; the target area image is divided according to the row and column dividing lines, and the target area image is divided into multiple cells.

[0111] For example, Figure 3As shown, the monitoring information in the target area is displayed in a table-like format (different information is distributed in different cells). To improve the recognition of the monitoring information, row and column dividing lines can be drawn in the target area (i.e., the borders of each cell are drawn) based on the number of rows and columns included in the target area, the size ratio of each row and column, and the size of the target area. The target area is then divided into multiple cells based on the drawn row and column dividing lines.

[0112] In some embodiments, after the target area is divided into multiple cells, the row and column positions of each cell may be recorded: cell 1 is in the first row and first column, and cell 2 is in the first row and second column.

[0113] S406: Perform content recognition on each cell to obtain the monitoring information.

[0114] In some embodiments, after obtaining each cell, OCR technology can be used to identify the content of each cell to obtain the content corresponding to each cell (i.e., the monitoring data in each cell). The identified content is then combined according to the row and column positions of each cell to obtain the monitoring information.

[0115] In some embodiments, in order to further improve the recognition effect, each cell may be preprocessed, and the preprocessing includes at least one of increasing resolution, improving contrast, and strengthening cell borders.

[0116] For example, super-resolution technology and image interpolation methods can be used to improve the resolution of cells; histogram equalization, contrast stretching, gamma correction and other methods can be used to improve the contrast of cells; Canny edge detection and Sobel operator can be used to detect edges, and then morphological operations (such as dilation) can be used to thicken the edges.

[0117] S407. Determine whether there is a security incident in the intranet based on the monitoring information. If so, execute the step shown in S408. If not, end the process.

[0118] In some embodiments, after obtaining the monitoring information, the monitoring data in the same row of the monitoring information can be inserted into the database in sequence; if the monitoring data is not new data, it is determined that no security incident has occurred in the intranet, and the monitoring data is discarded; if the monitoring data is new data, it is determined that a security incident has occurred in the intranet.

[0119] For example, Figure 3As shown, first, the monitoring data corresponding to the first line of the monitoring information "Device A, XX year XX month XX day XX hour XX minute XX second, device..." is inserted into the database, and it is determined whether there is data identical to the monitoring data in the database. If they are identical, it is determined that the monitoring data is not new data, and it is determined that no security incident has occurred in the intranet, and the monitoring data is discarded. If they are not identical, it is determined that the monitoring data is new data, the monitoring data is written into the database, and it is determined that a security incident has occurred in the intranet.

[0120] Then, the monitoring data corresponding to the second and third rows are processed in turn according to the processing method of the monitoring data corresponding to the first row.

[0121] S408: Output the alarm information according to the monitoring data.

[0122] In some embodiments, when it is determined based on the monitoring data that a security incident occurs in the intranet, alarm information can be generated based on the monitoring data and output.

[0123] Exemplarily, according to the blank information in the preset alarm template, the corresponding text is extracted from the monitoring data and filled in to generate the alarm information; the information sending platform interface is called to push the alarm information to the client of the target person.

[0124] For example, after an alarm is generated, the system can call a third-party messaging platform interface to send the information to a device convenient for on-duty personnel to receive it, reminding them to handle the alarm in a timely manner. Common messaging methods include: WeChat, SMS modem, email, mobile phone, or landline.

[0125] In summary, the monitoring method provided by the embodiment of the present application can reduce the number of on-duty personnel in scenarios where manual monitoring is required. Previously, multiple monitoring personnel may be needed to monitor multiple devices, and the information that each person can monitor at the same time is limited. The technical solution of the embodiment of the present application can only push the information that has changed to the monitoring personnel, so that the monitoring personnel can focus on the equipment with problems and can monitor multiple devices at the same time. It reduces the work pressure of the monitoring personnel. Previously, monitoring important equipment information required full attention, which was stressful. The technical solution of the embodiment of the present application can assist the monitoring personnel and remind them again of information changes that were not discovered in time. It reduces the work pressure of the monitoring personnel.

[0126] Based on the above embodiment, an embodiment of the present application further provides an alarm system, which includes a first monitoring device and a second monitoring device.

[0127] The second monitoring device is deployed with a monitoring system, which can monitor various devices and / or systems in the intranet and display monitoring images.

[0128] The second monitoring device includes an anti-screensaver module and a window control module. The anti-screensaver module is used to periodically trigger keyboard events to prevent the monitored computer from entering a screensaver or hibernation state. The window control module is used to bring the window displaying the monitoring image to the forefront and maximize it when it is not already in the foreground or maximized. The specific implementation methods for this module can be referenced to those described in the previous embodiments and will not be further elaborated here.

[0129] The first monitoring device is used to collect the monitoring image, identify the monitoring image, determine whether a security incident occurs in the intranet, and output an alarm message after a security incident occurs. The specific implementation method can refer to the implementation method in the above embodiment and will not be repeated here.

[0130] On the basis of the above embodiments, an embodiment of the present application further provides a monitoring device, which is applied to the first monitoring device in the above embodiments.

[0131] Figure 5 A schematic diagram of the structure of a monitoring device 50 provided in an embodiment of the present application is shown in FIG. Figure 5 Shown, including:

[0132] The acquisition module 501 is used to acquire a monitoring image displayed on a display interface of a second monitoring device in the intranet; the monitoring image displays monitoring information of the intranet.

[0133] The determination module 502 is configured to perform position recognition on the preset text in the surveillance image and determine the target area image in the surveillance image when the image features of the surveillance image are inconsistent with those of the last acquired surveillance image.

[0134] The processing module 503 is configured to perform content recognition on the target area image and obtain monitoring information from the target area image.

[0135] The alarm module 504 is configured to output alarm information when it is determined based on the monitoring information that a security incident occurs in the intranet.

[0136] In some embodiments, the determination module 502 is also used to perform content recognition on the surveillance image to determine the position of the preset text in the surveillance image; determine the position and size of the target area image in the surveillance image based on the structure of the surveillance image and the size of the surveillance image; the structure of the surveillance image is used to indicate the position and size ratio of different information in the surveillance image; and take the position as the starting point to determine the target area image in the surveillance image based on the position and size of the target area image in the surveillance image.

[0137] In some embodiments, the processing module 503 is also used to divide the target area image into multiple cells and record the row and column positions of each cell; perform content recognition on each cell to obtain the monitoring data corresponding to each cell; and combine each monitoring data according to the row and column relationship to obtain the monitoring information.

[0138] In some embodiments, the processing module 503 is further used to draw row and column dividing lines in the target area image according to the size of the target area image and the structure of the target area image; the structure of the target area image is used to indicate the number of rows and columns included in the target area image and the size ratio of each row and column; the target area image is divided according to the row and column dividing lines, and the target area image is divided into multiple cells.

[0139] In some embodiments, the processing module 503 is further configured to pre-process the cells, wherein the pre-processing includes at least one of increasing resolution, improving contrast, and strengthening cell borders.

[0140] In some embodiments, the alarm module 504 is also used to insert the monitoring data in the same row of the monitoring information into the database in sequence; if the monitoring data is not new data, it is determined that no security incident has occurred in the intranet, and the monitoring data is discarded; if the monitoring data is new data, it is determined that a security incident has occurred in the intranet, and the alarm information is output based on the monitoring data.

[0141] In some embodiments, the alarm module 504 is also used to extract corresponding text from the monitoring data and fill in the blank information in the preset alarm template to generate the alarm information; call the information sending platform interface to push the alarm information to the client of the target person.

[0142] In some embodiments, the acquisition module 501 is also used to capture the monitoring image displayed on the display interface of the second monitoring device through the image acquisition device corresponding to the first monitoring device; or, through the data transmission interface between the second monitoring device and the second monitoring device, receive the monitoring image stream output from the second monitoring device, and take a screenshot of the monitoring image stream to obtain the monitoring image.

[0143] In some embodiments, the acquisition module 501 is further used to obtain a hash value of the surveillance image. If the hash value is different from the hash value of the previous surveillance image, it is determined that the image features of the surveillance image are inconsistent with those of the previous surveillance image.

[0144] The monitoring device provided in the embodiments of the present application can execute the monitoring method described in any of the above embodiments. Its principles and technical effects are similar and will not be repeated here.

[0145] An embodiment of the present application also provides an electronic device.

[0146] Figure 6 This is a schematic diagram of the structure of the electronic device 60 provided in the embodiment of the present application, as shown in FIG. Figure 6 As shown, the electronic device may include: a transceiver 601, a processor 602, and a memory 603. The electronic device may be the first monitoring device in any of the above embodiments.

[0147] The processor 602 executes the computer-executable instructions stored in the memory, so that the processor 602 implements the solution in the above embodiment. The processor 602 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0148] The memory 603 is connected to the processor 602 via a system bus and implements communication therebetween. The memory 603 is used to store computer program instructions.

[0149] The transceiver 601 can receive and transmit data and instructions.

[0150] Optionally, the electronic device 60 may further include a communication interface 604 to communicate with an external or internal device via the communication interface 603. The external device may be, for example, a client (e.g., a mobile phone or tablet). In a specific implementation, if the communication interface 604, the memory 603, and the processor 602 are implemented independently, the communication interface 604, the memory 603, and the processor 602 may be interconnected via a bus to enable communication between them.

[0151] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. System buses can be divided into address buses, data buses, and control buses. For ease of illustration, the diagram uses only a single thick line, but this does not imply a single bus or type of bus. Transceivers enable communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and non-volatile memory.

[0152] Optionally, in a specific implementation, if the communication interface 604, the memory 603 and the processor 602 are integrated on a chip, the communication interface 604, the memory 603 and the processor 602 can complete communication through an internal interface.

[0153] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the monitoring method in the above embodiment.

[0154] In an embodiment of the present application, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the technical solution of the above-mentioned monitoring method embodiment is implemented. Its implementation principle and technical effect are similar and will not be repeated here.

[0155] In one possible implementation, a computer-readable medium may include random access memory (RAM), read-only memory (ROM), compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium designed to carry or store desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include optical disc, laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above are also intended to be included within the scope of computer-readable media.

[0156] A computer program product is also provided in an embodiment of the present application, including a computer program. When the computer program is executed by a processor, the technical solution of the above-mentioned monitoring method embodiment is implemented. Its implementation principle and technical effect are similar and will not be repeated here.

[0157] In the specific implementation of the above-mentioned terminal device or server, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0158] Those skilled in the art will appreciate that all or part of the steps of any of the above method embodiments may be accomplished by hardware associated with program instructions. The aforementioned program may be stored in a computer-readable storage medium, and when the program is executed, all or part of the steps of the above method embodiments are executed.

[0159] If the technical solution of this application is implemented in the form of software and sold or used as a product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of this application can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a computer program or several instructions. This computer software product enables a computer device (which can be a personal computer, server, network device, or similar electronic device) to perform all or part of the steps of the method described in the embodiments of this application.

[0160] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0161] It should be further noted that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0162] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0163] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.

[0164] If an integrated unit / module is implemented in hardware, the hardware may be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0165] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disk, etc., various media that can store program code.

[0166] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A monitoring method, characterized in that: Applied to a first monitoring device located in an external network, the method includes: Acquire a monitoring image displayed on a display interface of a second monitoring device in the intranet; the monitoring image displays monitoring information of the intranet; In the case where the image features of the surveillance image are inconsistent with those of the last acquired surveillance image, position recognition is performed on the preset text in the surveillance image to determine the target area image in the surveillance image; Performing content recognition on the target area image to obtain monitoring information in the target area image; When a security incident is determined to have occurred in the intranet according to the monitoring information, an alarm message is output.

2. The method according to claim 1, characterized in that The step of performing position recognition on the preset text in the surveillance image to determine the target area image in the surveillance image includes: Performing content recognition on the surveillance image to determine the position of the preset text in the surveillance image; Determining the position and size of the target area image in the surveillance image according to the structure of the surveillance image and the size of the surveillance image; the structure of the surveillance image is used to indicate the position and size ratio of different information in the surveillance image; Taking the position as a starting point, the target area image is determined in the monitoring image according to the orientation and size of the target area image in the monitoring image.

3. The method according to claim 1 or 2, characterized in that The performing content recognition on the target area image to obtain monitoring information in the target area image includes: Dividing the target area image into a plurality of cells and recording the row and column positions of each cell; Performing content recognition on each cell to obtain monitoring data corresponding to each cell; The monitoring data are combined according to the row-column relationship to obtain the monitoring information.

4. The method according to claim 3, characterized in that The step of dividing the target area image into a plurality of cells comprises: Drawing row and column dividing lines in the target area image according to the size of the target area image and the structure of the target area image; the structure of the target area image is used to indicate the number of rows and columns included in the target area image and the size ratio of each row and column; The target area image is segmented according to the row and column segmentation lines, and the target area image is segmented into a plurality of cells.

5. The method according to claim 3, characterized in that Before performing content recognition on each cell, the method further includes: Preprocessing is performed on each of the cells, and the preprocessing includes at least one of increasing resolution, improving contrast, and strengthening cell borders.

6. The method according to claim 1 or 2, characterized in that When a security incident is determined to have occurred in the intranet according to the monitoring information, outputting alarm information includes: Inserting the monitoring data in the same row in the monitoring information into the database in sequence; If the monitoring data is not newly added data, it is determined that no security incident has occurred in the intranet, and the monitoring data is discarded; If the monitoring data is newly added data, it is determined that a security incident occurs in the intranet, and the alarm information is output according to the monitoring data.

7. The method according to claim 6, characterized in that Outputting the alarm information according to the monitoring data includes: According to the blank information in the preset alarm template, the corresponding text extracted from the monitoring data is filled in to generate the alarm information; Call the information sending platform interface to push the alarm information to the client of the target person.

8. The method according to claim 1, characterized in that The acquiring of a monitoring image displayed on a display interface of a second monitoring device in the intranet includes: Using the image acquisition device corresponding to the first monitoring device, photographing the monitoring image displayed on the display interface of the second monitoring device; or, The monitoring image stream outputted from the second monitoring device is received via a data transmission interface between the monitoring device and the second monitoring device, and a screenshot of the monitoring image stream is taken to obtain the monitoring image.

9. The method according to claim 1, characterized in that The method further comprises: A hash value of the surveillance image is obtained. If the hash value is different from the hash value of the previous surveillance image, it is determined that the image features of the surveillance image are inconsistent with those of the previous surveillance image.

10. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The computer program comprises a computer program, which implements the method according to any one of claims 1 to 9 when executed by a controller.

Images