Abnormal alarm method, device, equipment and storage medium of remote monitoring system

Abstract

The present application provides an abnormal alarm method, device, equipment and storage medium for a remote monitoring system; the method includes obtaining real-time monitoring images collected by two adjacent cameras to the monitoring area; performing area division processing on the monitoring image to obtain multiple sub-image areas of the first monitoring image and the second monitoring image; obtaining position feature information of the sub-image area of ​​the first monitoring image in the monitoring image, and performing feature extraction processing on the multiple sub-image areas to obtain image feature information of each sub-image area; constructing an image structure of the monitoring image according to the position feature information and image feature information of the sub-image area in the first monitoring image; performing image processing on the first monitoring image according to the image structure, and performing abnormal behavior detection processing on the processed monitoring image; if abnormal behavior is detected, a security alarm is triggered. This solution can improve the timeliness and accuracy of abnormal alarms for security monitoring.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of remote monitoring, and in particular to an abnormal alarm method, device, equipment and storage medium for a remote monitoring system. Background Art

[0002] With the development of society and the advancement of science and technology, image processing technology plays an increasingly important role in remote monitoring systems. The security of financial institutions is of vital importance. At present, traditional remote monitoring systems usually rely on cameras to collect images of the monitoring area, and then use manual monitoring or simple motion monitoring technology to identify abnormal behaviors. These methods have many limitations. For example, manual monitoring is inefficient and prone to errors, while simple motion detection technology cannot accurately distinguish between real hidden security threats and harmless environmental changes. Therefore, the timeliness and accuracy of abnormal alarms in traditional remote monitoring systems are low. Summary of the invention

[0003] The present application provides an abnormal alarm method, device, equipment and storage medium for a remote monitoring system, which can improve the timeliness and accuracy of remote monitoring abnormal alarms.

[0004] In a first aspect, the present application provides an abnormal alarm method for a remote monitoring system, comprising:

[0005] Acquire real-time monitoring images of the monitoring area collected by two adjacent cameras to obtain a first monitoring image and a second monitoring image, wherein the geographical area is divided into a plurality of monitoring areas, each monitoring area is correspondingly provided with a camera, and the two adjacent cameras are two cameras provided for two adjacent monitoring areas in the geographical area, and the monitoring areas of the two adjacent cameras partially overlap;

[0006] For each monitoring screen, based on the screen objects contained in the monitoring screen, the monitoring screen is divided into regions to obtain a plurality of sub-screen regions of a first monitoring screen and a plurality of sub-screen regions of a second monitoring screen;

[0007] Acquire position feature information of a sub-picture area of ​​the first monitoring screen in the monitoring screen, and perform feature extraction processing on a plurality of the sub-picture areas to obtain image feature information of each sub-picture area, wherein the position feature information includes position information of the sub-picture area in the monitoring screen and associated position information of the sub-picture area and an associated sub-picture area, wherein the associated sub-picture area is a sub-picture area in the second monitoring screen that is positionally associated with the sub-picture area;

[0008] constructing an image structure of the monitoring screen according to the position feature information and image feature information of the sub-screen area in the first monitoring screen, wherein the image structure includes a plurality of area nodes, one node corresponds to one sub-screen area, and the node information of the area node indicates the position feature information and image feature information of the corresponding sub-screen area;

[0009] Performing image processing on the first monitoring picture according to the image structure, and performing abnormal behavior detection processing on the processed monitoring picture;

[0010] If abnormal behavior is detected, a security alert is triggered.

[0011] In a second aspect, the present application provides an abnormal alarm device, applicable to a security monitoring system, comprising:

[0012] A picture acquisition unit is used to acquire real-time monitoring pictures of a monitoring area collected by two adjacent cameras to obtain a first monitoring picture and a second monitoring picture, wherein the geographical area is divided into a plurality of monitoring areas, each of which corresponds to a camera, and the two adjacent cameras are two cameras set in two adjacent monitoring areas of the geographical area;

[0013] A division unit, for performing region division processing on each monitoring screen based on screen objects contained in the monitoring screen, to obtain a plurality of sub-screen regions of a first monitoring screen and a plurality of sub-screen regions of a second monitoring screen;

[0014] a feature processing unit, configured to obtain position feature information of a sub-picture area of ​​the first monitoring picture in the monitoring picture, and perform feature extraction processing on the multiple sub-picture areas to obtain image feature information of each sub-picture area, wherein the position feature information includes position information of the sub-picture area in the monitoring picture and associated position information of the sub-picture area and an associated sub-picture area, wherein the associated sub-picture area is a sub-picture area in the second monitoring picture that is positionally associated with the sub-picture area;

[0015] A graph structure processing unit, configured to construct an image structure of the monitoring screen according to the position information and image feature information of the sub-screen area in the first monitoring screen, wherein the image structure includes a plurality of area nodes, one node corresponds to one sub-screen area, and the node information of the area node indicates the position information and image feature information of the corresponding sub-screen area;

[0016] A detection unit is used to perform image processing on the first monitoring picture according to the image structure, and perform abnormal behavior detection processing based on the processed monitoring picture; if abnormal behavior is detected, a security alarm is triggered.

[0017] According to a third aspect, an electronic device is provided, which is suitable for a remote monitoring system and includes a processor and a memory; the memory is used to store a computer program; and the processor is used to execute the computer program to implement any of the methods described above.

[0018] In a fourth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method in the above-mentioned first aspect and its various implementations.

[0019] In summary, the embodiment of the present application obtains real-time monitoring screens of a monitoring area captured by two adjacent cameras to obtain a first monitoring screen and a second monitoring screen, the geographical area is divided into multiple monitoring areas, each monitoring area corresponds to a camera, the two adjacent cameras are two cameras set for two adjacent monitoring areas in the geographical area, and the monitoring areas of the two adjacent cameras partially overlap; for each monitoring screen, based on the screen objects contained in the monitoring screen, the monitoring screen is divided into regions to obtain multiple sub-screen areas of the first monitoring screen and multiple sub-screen areas of the second monitoring screen; the position feature information of the sub-screen area of ​​the first monitoring screen in the monitoring screen is obtained, and feature extraction processing is performed on the multiple sub-screen areas to obtain the position feature information of each sub-screen area Image feature information, the position feature information includes the position information of the sub-screen area in the monitoring screen and the associated position information of the sub-screen area and the associated sub-screen area, the associated sub-screen area is the sub-screen area in the second monitoring screen that has a position association with the sub-screen area; constructing the image structure of the monitoring screen according to the position information and image feature information of the sub-screen area in the first monitoring screen, the image structure includes a number of area nodes, one node corresponds to a sub-screen area, and the node information of the area node indicates the position information and image feature information of the corresponding sub-screen area; performing image processing on the first monitoring screen according to the image structure, and performing abnormal behavior detection processing on the processed monitoring screen; if abnormal behavior is detected, triggering a security alarm. This solution provides an abnormal alarm method for a remote monitoring system, which uses artificial intelligence and other technologies to improve the timeliness and accuracy of abnormal alarms for security monitoring.

[0020] In addition, this solution can improve the image processing effect through a series of processing, including image structure construction, to obtain high-quality monitoring screen images, and perform safe abnormal behavior detection based on high-quality monitoring screen images, thereby improving the accuracy of abnormal behavior detection and abnormal alarms in the remote monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of an implementation environment of an abnormal alarm method for a remote monitoring system provided in an embodiment of the present application;

[0023] Figure 2a A flowchart of an abnormal alarm method for a remote monitoring system provided in one embodiment of the present application;

[0024] Figure 2b A schematic diagram of monitoring screen division provided in one embodiment of the present application;

[0025] Figure 3 A schematic block diagram of an abnormality alarm device provided in an embodiment of the present application;

[0026] Figure 4 Another schematic block diagram of the abnormality alarm device provided in the embodiment of the present application;

[0027] Figure 5 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here can be implemented in a sequence other than those illustrated or described here. In an embodiment of the present invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, but B can also be determined according to A and / or other information. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units does not have to be limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the description of the present application, unless otherwise specified, "multiple" refers to two or more than two.

[0030] Artificial intelligence (AI) is a comprehensive technical science that replicates and expands human intelligence through digital computers and computer-controlled machines to perceive the environment, acquire knowledge, and apply this knowledge to achieve the best results. In short, AI is committed to exploring the nature of intelligence and creating new intelligent machines that can mimic human intelligent responses. As a branch of computer science, AI covers a wide range of technical fields from hardware to software, including but not limited to sensor technology, dedicated AI chips, cloud technology, data storage, big data processing, pre-trained model technology, operating systems, and mechatronics. In particular, pre-trained models, after fine-tuning, can play a role in multiple application areas of artificial intelligence. In terms of software technology, AI also includes key areas such as computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0031] Machine learning is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and practical learning. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0032] In the field of computer technology, especially in the field of remote monitoring or security technology, image processing technology plays a vital role. With the development of society and the advancement of science and technology, traditional remote monitoring systems, which rely on manual monitoring or simple motion detection technology, have gradually failed to meet the needs of efficient and accurate security monitoring. The embodiment of the present application provides an abnormal alarm method for a remote monitoring system, which uses artificial intelligence and other technologies to improve the timeliness and accuracy of abnormal alarms for security monitoring.

[0033] In addition, these traditional methods have the limitations of being inefficient, error-prone, and unable to accurately distinguish between security threats and harmless environmental changes. Although the development of artificial intelligence and deep learning technology has brought new image processing methods to remote monitoring systems, which can analyze image content to identify and predict potential security incidents, the existing technology does not pay attention to the local features of the image, such as texture, color and shape, as well as the overall composition information of the image, resulting in low image quality of the monitoring screen, inaccurate detection of abnormal behavior, and further resulting in low accuracy and timeliness of abnormal alarms of the remote monitoring system. In an embodiment of the present application, applying artificial intelligence technology to the remote monitoring system can also improve the accuracy of abnormal behavior detection in the hall, thereby further improving the accuracy of abnormal alarms.

[0034] The implementation environment of the abnormal alarm method provided in the embodiment of the present application is introduced below.

[0035] Figure 1 A schematic diagram of an implementation environment of the abnormal alarm method of the remote monitoring system provided in the embodiment of the present application includes a camera device 11 and a server 12. The camera device 11 and the server 12 can be connected to each other by wired or wireless communication.

[0036] In one embodiment, the hall is divided into multiple monitoring areas, each of which is provided with a camera device 11, which is used to collect monitoring screen data in real time and upload it to the server 12. Each camera device uses the screen data of the corresponding monitoring area to upload it to the server 12 for image processing to realize abnormal behavior detection.

[0037] The server 12 is used to process the monitoring images uploaded by the camera device 11, and then perform abnormal behavior detection based on the processed images. If abnormal behavior is detected, a security alarm is triggered. Specifically, a security alarm message can be sent to the terminal device 13 to indicate that there are security problems.

[0038] Specifically, the server 12 obtains real-time monitoring images of the monitoring area collected by two adjacent cameras to obtain a first monitoring image and a second monitoring image. A geographical area such as an indoor space is divided into multiple monitoring areas, and each monitoring area corresponds to a camera. The two adjacent cameras are two cameras set in two adjacent monitoring areas of the geographical area, and the monitoring areas of the two adjacent cameras partially overlap. For each monitoring image, based on the image objects contained in the monitoring image, the monitoring image is divided into regions to obtain multiple sub-image areas of the first monitoring image and multiple sub-image areas of the second monitoring image; the position feature information of the sub-image area of ​​the first monitoring image in the monitoring image is obtained, and the feature extraction process is performed on the multiple sub-image areas to obtain each sub-image area. domain, the position feature information including the position information of the sub-picture area in the monitoring screen and the associated position information of the sub-picture area and the associated sub-picture area, the associated sub-picture area being a sub-picture area in the second monitoring screen that has a position association with the sub-picture area; constructing an image structure of the monitoring screen according to the position feature and image feature information of the sub-picture area in the first monitoring screen, the image structure including a plurality of area nodes, one node corresponding to one sub-picture area, the node information of the area node indicating the position feature information and image feature information of the corresponding sub-picture area; performing image processing on the first monitoring screen according to the image structure, and performing abnormal behavior detection processing on the processed monitoring screen; if abnormal behavior is detected, triggering a security alarm.

[0039] The present application embodiment does not limit the specific type of the terminal device 13. In some embodiments, the terminal device 13 may include but is not limited to: mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, wearable smart devices, medical devices, etc. The device is often equipped with a display device, which may also be a display, a display screen, a touch screen, etc., and the touch screen may also be a touch screen, a touch panel, etc.

[0040] In some embodiments, the server may be one or more. When there are multiple servers, there are at least two servers for providing different services, and / or there are at least two servers for providing the same service, such as providing the same service in a load balancing manner, which is not limited in the embodiments of the present application. Among them, the above-mentioned server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can also become a node of the blockchain.

[0041] In the embodiment of the present application, the terminal device 13 and the server 12 may be directly or indirectly connected via wired communication or wireless communication, and the present application does not impose any limitation thereto.

[0042] It should be noted that the implementation environment of the embodiments of the present application includes but is not limited to Figure 1 shown.

[0043] The technical solutions of the embodiments of the present application are described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0044] Figure 2a Schematic diagram of the flow of the image processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a device with an abnormal alarm function of a remote monitoring system, for example, an abnormal alarm device. In some embodiments, the execution subject of the embodiment of the present application can be Figure 1 For the convenience of description, the embodiment of the present application is described by taking the execution subject as an electronic device as an example.

[0045] like Figure 2a As shown, the abnormal alarm process of the remote monitoring system of the embodiment of the present application includes:

[0046] S101. Acquire real-time monitoring images of a monitoring area captured by two adjacent cameras to obtain a first monitoring image and a second monitoring image. The geographical area is divided into multiple monitoring areas, each of which corresponds to a camera. The two adjacent cameras are two cameras set in two adjacent monitoring areas of the geographical area, and the monitoring areas of the two adjacent cameras partially overlap.

[0047] In an example, the first monitoring screen or the second monitoring screen is any captured image, for example, a frame of image in video data captured by a camera.

[0048] S102 . For each monitoring screen, based on the screen objects contained in the monitoring screen, perform area division processing on the monitoring screen to obtain a plurality of sub-screen areas of a first monitoring screen and a plurality of sub-screen areas of a second monitoring screen.

[0049] The screen objects of the embodiments of the present application may include elements in the monitoring screen. For example, the monitoring screen may include at least one of background elements, foreground elements, object elements (such as objects or character elements), etc., where the foreground elements may be one or more and the background elements may also be one or more.

[0050] In the embodiment of the present application, in order to accurately process the monitoring screen, after acquiring the monitoring screen, the electronic device divides the monitoring screen into regions based on a series of screen elements included in the monitoring screen to obtain multiple sub-screen regions, wherein each of the sub-screen sub-regions includes at least one complete or independent element.

[0051] In a possible implementation, the electronic device divides each element in the monitoring screen into a sub-screen area based on the number of screen elements included in the monitoring screen. Figure 2b , Assuming that the foreground elements of the surveillance screen include three people or objects, the surveillance screen can be divided into three sub-screen areas a, b, and c.

[0052] In some embodiments, the electronic device uses a target monitoring method or semantic segmentation to divide the monitoring screen into regions. For example, the electronic device can use a segmentation algorithm based on deep learning, such as hierarchical attention semantic segmentation object detection, to semantically segment the screen elements included in the monitoring screen to obtain multiple sub-screen regions.

[0053] S103. Obtain position feature information of a sub-screen area of ​​the first monitoring screen in the monitoring screen, and perform feature extraction processing on the multiple sub-screen areas to obtain image feature information of each sub-screen area, wherein the position feature information includes position information of the sub-screen area in the monitoring screen and associated position information of the sub-screen area and an associated sub-screen area, wherein the associated sub-screen area is a sub-screen area in the second monitoring screen that has a positional association with the sub-screen area.

[0054] The electronic device obtains position information of multiple sub-screen areas in the first monitoring screen and associated position information relative to associated sub-screen areas in the second monitoring screen, and extracts image features of the sub-screen areas to obtain image feature information of the sub-screen areas.

[0055] Among them, the associated sub-screen area is a sub-screen area in the second monitoring screen captured by the adjacent camera that has a position association with the sub-screen area in the first monitoring screen, including the same position, relative position, etc. In one embodiment, since the monitoring areas of adjacent cameras have overlapping parts, there are sub-screen areas with corresponding positions in the monitoring screens captured by two adjacent cameras. In the embodiment of the present application, the relative position information between these sub-screen areas is considered when constructing the position feature information of the sub-screen area, which can accurately characterize the position attribute of the sub-screen area, thereby more accurately constructing the image structure of the monitoring screen, thereby improving the accuracy of abnormal behavior detection.

[0056] In an embodiment of the present application, the position information of the sub-picture area may include the geometric coordinate information of the sub-picture area in the monitoring screen. In some embodiments, the position information of the sub-picture area in an embodiment of the present application may include the coordinate information of the sub-picture area in the monitoring screen, and may also include the size of the sub-picture area, the shape of the image area and other information.

[0057] In a possible implementation, the electronic device simultaneously inputs multiple sub-picture areas into a feature extraction model for feature extraction, and obtains feature information of multiple sub-picture areas. The feature information may include information such as texture and color of the sub-picture area, and may also include contextual relationships of the sub-picture. In one embodiment, a ViT (Vision Transformer) model may be used, which captures the complex relationship between sub-picture areas through a self-attention mechanism, and then extracts rich image feature representations.

[0058] S104. Construct an image structure of the monitoring screen according to the position feature information and image feature information of the sub-screen area in the first monitoring screen, wherein the image structure includes a plurality of area nodes, one node corresponds to a sub-screen area, and the node information of the area node indicates the position information and image feature information of the corresponding sub-monitoring screen area.

[0059] In one embodiment, the sub-picture area may be used as a node of the image structure, and based on the image feature information and position feature information of the sub-picture area, the node information of the node is determined to obtain the image structure of the monitoring screen.

[0060] Among them, the image structure, also known as the graph structure, is a mathematical model used to represent the complex relationship between objects, where objects are abstracted as nodes and the relationship between objects is abstracted as the edges connecting the nodes. In graph theory, the graph structure consists of a set of nodes and a set of edges, which can describe the connection relationship and order between nodes. Applying the image structure can improve the efficiency and accuracy of image processing.

[0061] In an embodiment of the present application, in order to improve the processing effect of the monitoring screen, when processing the monitoring screen, the monitoring screen is divided into multiple sub-screen areas based on the screen elements included in the monitoring screen, and the multiple sub-screen areas are used as a node in the image structure. Based on the feature information and position feature information of the sub-screen area, the node information of the node corresponding to the sub-screen area is determined. In this way, the image structure of the monitoring screen can be constructed. The image structure not only includes the image feature information of each element in the monitoring screen but also includes the position information (that is, the overall composition information of the monitoring screen). In this way, when performing subsequent monitoring screen processing based on the image structure, not only the image information of the elements in the monitoring screen is paid attention to, but also the position information of the elements in the monitoring screen is paid attention to, thereby improving the processing effect of the monitoring screen and improving the accuracy of abnormal behavior alarms.

[0062] The following introduces the specific process of the electronic device constructing the image structure of the monitoring screen based on the feature information and position information of multiple electronic screen areas.

[0063] In an embodiment of the present application, for each of the multiple sub-picture areas, the sub-picture area is used as a node of the image structure, so that an image structure including multiple nodes can be obtained, and the multiple nodes in the image structure correspond one-to-one to the multiple sub-picture areas of the target image. Then, based on the feature information and position information of each of the multiple sub-picture areas, the node information of each of the multiple nodes is determined. In an embodiment of the present application, for each of the multiple sub-picture areas, based on the feature information and position information of the sub-picture area, the specific method of determining the node information of the node corresponding to the sub-picture area is basically the same. For the convenience of description, one of the multiple sub-picture areas is used as an example for explanation.

[0064] In one possible implementation, the electronic device uses the feature information and position feature information of the sub-screen area of ​​the hall as the node information of the corresponding node. At this time, the node information includes the image feature information and position feature information of the corresponding sub-screen area, that is, the node information of the node is diversity feature information.

[0065] In one possible implementation, the electronic device may perform feature fusion processing on the image feature information and position feature information of the sub-picture area, and generate node information of the corresponding node based on the fused feature information. The embodiment of the present application does not limit the specific manner in which the electronic device performs feature fusion on the feature information and position information of the sub-picture area. In one example, the electronic device performs processing such as adding, multiplying, or splicing the feature information and position information of the sub-picture area.

[0066] In some embodiments, where the node information also indicates the pixel attribute information in order to improve the effect of image processing and the accuracy of security abnormal behavior alarms, the embodiment of the present application can also obtain the pixel attribute information of the sub-screen area in the first monitoring screen, and construct the image structure of the monitoring screen according to the position information, image feature information and pixel attribute information of the sub-screen area in the first monitoring screen.

[0067] The pixel attribute information may include pixel distribution information, pixel RGB information, and the like.

[0068] In one example, the electronic device uses the image feature information, position feature information and pixel attribute information of the sub-picture area as the node information of the corresponding node. At this time, the node information includes the feature information, position information and pixel attribute information of the corresponding sub-picture area, that is, the node information of the node is diversity feature information.

[0069] In an example, similarly, the electronic device may perform feature fusion on the image feature information, position feature information and pixel attribute information of the sub-picture area, and generate node information of the corresponding node according to the fused feature information.

[0070] S105: Perform image processing on the first monitoring picture according to the image structure, and perform abnormal behavior detection processing on the processed monitoring picture.

[0071] In some embodiments, the process of performing image processing on the monitoring screen according to the image structure may include:

[0072] Acquire a security monitoring screen database, wherein the security monitoring screen database includes a plurality of sample monitoring screens and a sample graph structure corresponding to each sample monitoring screen;

[0073] Calculating similarity information between the first monitoring picture and the sample monitoring picture according to the image structure and the sample image structure corresponding to the sample monitoring picture;

[0074] Selecting a target sample monitoring picture from the security monitoring picture database according to the similarity information;

[0075] Image processing is performed on the first monitoring picture according to the image parameters of the target sample monitoring picture.

[0076] In one embodiment, there are multiple similarities between monitoring pictures. Specifically, the step of "calculating similarity information between the first monitoring picture and the sample monitoring picture according to the image structure and the sample image structure corresponding to the sample monitoring picture" may include:

[0077] Constructing a first feature vector of the first monitoring picture according to node information of nodes in the image structure;

[0078] Constructing a second feature vector of the sample monitoring picture according to node information of nodes in the sample image structure;

[0079] A similarity calculation process is performed based on the first feature vector and the second feature vector to obtain similarity information between the first monitoring screen and the sample monitoring screen.

[0080] Among them, the similarity of monitoring images refers to: an indicator that measures the similarity between two graph structures in terms of topological structure, node attributes or edge attributes. Some common graph similarity measurement methods include: structural similarity (SSIM), pixel similarity, cosine similarity, edit distance and feature similarity.

[0081] In one embodiment, in order to accurately realize the accuracy of abnormal alarm of the remote monitoring system, the specific process of selecting the target sample monitoring screen from the security monitoring screen database according to the similarity information may include:

[0082] Selecting a sample monitoring screen image with a similarity greater than a first preset threshold from the security monitoring screen database as a target sample monitoring image;

[0083] If there is no sample monitoring screen image whose value is greater than the first preset threshold, the associated position information in the node information of the image structure is adjusted, and the first feature vector of the first sample monitoring screen is updated again according to the adjusted node information in the image structure; the similarity calculation is re-executed until there is a sample monitoring screen image whose value is greater than the first preset threshold.

[0084] In one embodiment, in order to improve the accuracy of abnormal behavior, if there is no sample monitoring screen image greater than the first preset threshold, the pixel points contained in the associated sub-screen area are positioned at the sub-pixel level to obtain the position information of the sub-pixel points, and the position information of the pixel points is reconstructed according to the position information of the sub-pixel points to update the associated position information of the associated sub-screen area, thereby updating the position feature information indicated by the node. After the node information is updated and adjusted, the first feature vector of the first sample monitoring screen can be regenerated according to the adjusted node information, and then similarity calculation, matching and other processes can be performed.

[0085] In one embodiment, in order to improve the accuracy of abnormal behavior, if there is no sample monitoring screen image greater than the first preset threshold, the first monitoring screen and the second monitoring screen can also be registered to obtain the position deviation between the first monitoring screen and the second monitoring screen, and the associated position information is updated and adjusted according to the position deviation, thereby updating the position feature information indicated by the node. After the node information is updated and adjusted, the first feature vector of the first sample monitoring screen can be regenerated according to the adjusted node information, and then similarity calculation, matching and other processes can be performed.

[0086] The first preset threshold may be set according to actual needs, for example, between 50% and 80%.

[0087] In one embodiment, there are multiple ways to perform image processing on the first monitoring screen according to the image parameters of the target sample monitoring screen. For example, in order to improve the overall image quality, key image parameters are extracted from the target sample monitoring screen, and the key image parameters include at least one of color distribution, texture features, brightness, contrast, and object size; image processing is performed on the first monitoring screen according to the key image parameters.

[0088] For example, the image parameters of the first monitoring screen are compared with the image parameters of the target sample to identify the difference between the two. This step can use statistical methods or machine learning models to quantify the difference between the parameters. Based on the difference, the first monitoring screen is enhanced to make its parameters closer to the target sample. This may include adjusting brightness, contrast, color balance, etc. to improve image quality.

[0089] In one embodiment, the specific implementation is as follows:

[0090] Image parameter extraction and comparison

[0091] First, extract key image parameters, including brightness, contrast, and color balance, from the first surveillance image and the target sample surveillance image. These parameters can be obtained through image processing libraries such as OpenCV. Subsequently, statistical methods or machine learning models can be used to quantify the differences between the two sets of image parameters. The purpose of this step is to identify the main differences in the visual features of the two images and provide a basis for subsequent image enhancement.

[0092] Quantifying Differences and Image Enhancement

[0093] Based on the above quantified differences, it is determined whether image enhancement processing needs to be performed on the first monitoring picture. If the difference exceeds a preset threshold, image parameters such as brightness, contrast, and color balance will be automatically adjusted to make the parameters of the first monitoring picture closer to the target sample monitoring picture. For example, image processing methods or strategies can be based on histogram equalization, linear transformation, and color space conversion. It should be understood that the enhancement strategy is intended to improve image quality to make it more suitable for monitoring and analysis.

[0094] Implement image enhancement and effect evaluation

[0095] After the enhancement strategy is determined, image enhancement is performed on the first monitoring screen. This can include automatically adjusting parameters such as brightness, contrast, and color balance. The enhanced image will be saved or displayed for visual inspection or re-evaluation using the machine learning model. If the enhancement effect does not meet expectations, fine-tuning can be performed based on feedback until the image quality meets the requirements. Multiple iterations are required to ensure that the image enhancement process is both effective and accurate. The improved first monitoring screen has higher image quality, which is more compatible with abnormal behavior detection, and improves the accuracy of subsequent abnormal behavior detection.

[0096] In one embodiment, performing abnormal behavior monitoring processing based on the processed monitoring screen includes: extracting object behavior features of the processed monitoring screen using an abnormal behavior detection model, and performing abnormal behavior detection processing based on the object behavior features.

[0097] For example, a deep learning network-based abnormal behavior detection model is pre-trained, and the processed monitoring images are input into the model. The model extracts behavioral features from the monitoring images, performs abnormal behavior recognition and detection based on the behavioral features, and finally outputs the detection results.

[0098] S106. If abnormal behavior is detected, a security alarm is triggered.

[0099] For example, if abnormal behavior is detected, an alarm light or a bell will be triggered, and an alarm message will be sent to the terminal devices of certain personnel. The specific alarm method can be set according to actual needs.

[0100] From the above, it can be seen that the embodiment of the present application uses the above-mentioned series of processing, including image structure construction, etc., in order to accurately match high-quality sample images from a high-quality sample monitoring image database, improve the image quality of the current monitoring image based on the high-quality sample image, and perform safe abnormal behavior detection based on the high-quality monitoring image, thereby improving the accuracy of abnormal alarms.

[0101] The above describes in detail the embodiment of the abnormal alarm method of the remote monitoring system of the present application. Figure 3, describe in detail the device embodiments of the present application.

[0102] Figure 3 It is a schematic block diagram of an abnormal alarm device provided in one embodiment of the present application.

[0103] like Figure 3 As shown, the abnormal alarm device 30 is suitable for a remote monitoring system, and specifically includes:

[0104] The picture acquisition unit 301 is used to acquire real-time monitoring pictures of the monitoring area collected by two adjacent cameras to obtain a first monitoring picture and a second monitoring picture, wherein the geographical area is divided into a plurality of monitoring areas, each of which corresponds to a camera, and the two adjacent cameras are two cameras set in two adjacent monitoring areas of the geographical area;

[0105] A division unit 302 is used to perform region division processing on each monitoring screen based on the screen objects contained in the monitoring screen to obtain multiple sub-screen regions of the first monitoring screen and multiple sub-screen regions of the second monitoring screen;

[0106] a feature processing unit 303, configured to obtain position feature information of a sub-picture area of ​​the first monitoring picture in the monitoring picture, and perform feature extraction processing on the multiple sub-picture areas to obtain image feature information of each sub-picture area, wherein the position feature information includes position information of the sub-picture area in the monitoring picture and associated position information of the sub-picture area and an associated sub-picture area, wherein the associated sub-picture area is a sub-picture area in the second monitoring picture that is positionally associated with the sub-picture area;

[0107] A graph structure processing unit 304, configured to construct an image structure of the monitoring picture according to the position information and image feature information of the sub-picture area in the first monitoring picture, wherein the image structure includes a plurality of area nodes, one node corresponds to one sub-picture area, and the node information of the area node indicates the position information and image feature information of the corresponding sub-picture area;

[0108] The detection unit 305 is used to perform image processing on the first monitoring picture according to the image structure, and perform abnormal behavior detection processing based on the processed monitoring picture; if abnormal behavior is detected, a security alarm is triggered.

[0109] In some embodiments, Figure 4 As shown, the abnormal alarm device 30 further includes: a pixel acquisition unit 306, which is specifically used to: acquire pixel attribute information of the sub-picture area in the first monitoring picture;

[0110] The graph structure processing unit 304 is used to construct the image structure of the monitoring screen according to the position information, image feature information and pixel attribute information of the sub-screen area in the first monitoring screen, wherein the node information also indicates the pixel attribute information.

[0111] In some embodiments, the detection unit 305 is used to obtain a security monitoring screen database, wherein the security monitoring screen database includes a plurality of sample monitoring screens and a sample graph structure corresponding to each sample monitoring screen;

[0112] Calculating feature similarity information between the first monitoring picture and the sample monitoring picture according to the image structure and the sample image structure corresponding to the sample monitoring picture;

[0113] Selecting a target sample monitoring picture from the security monitoring picture database according to the similarity information;

[0114] Image processing is performed on the first monitoring picture according to the image parameters of the target sample monitoring picture.

[0115] In some embodiments, the detection unit 305 is used to:

[0116] Constructing a first feature vector of the first monitoring picture according to node information of nodes in the image structure;

[0117] Constructing a second feature vector of the sample monitoring picture according to node information of nodes in the sample image structure;

[0118] A similarity calculation process is performed based on the first feature vector and the second feature vector to obtain similarity information between the first monitoring screen and the sample monitoring screen.

[0119] In some embodiments, the detection unit 305 is used to: select a sample monitoring screen image with a similarity greater than a first preset threshold from the security monitoring screen database as a target sample monitoring image;

[0120] If there is no sample monitoring screen image whose value is greater than the first preset threshold, the associated position information in the node information of the image structure is adjusted, and the first feature vector of the first sample monitoring screen image is updated based on the adjusted node information in the image structure; the similarity calculation is re-executed until there is a sample monitoring screen image whose value is greater than the first preset threshold.

[0121] In some embodiments, the detection unit 305 is used to:

[0122] Extracting key image parameters from the target sample monitoring picture, wherein the key image parameters include at least one of color distribution, texture features, brightness, contrast, and object size;

[0123] Image processing is performed on the first monitoring picture according to the key image parameters.

[0124] In some embodiments, the detection unit 305 is used to: use the abnormal behavior detection model to extract the object behavior features of the monitored image, and perform abnormal behavior detection according to the object behavior features.

[0125] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 3 The device shown can execute the embodiments of the above method, and the aforementioned and other operations and / or functions of each module in the device are respectively for implementing the corresponding method embodiments of the electronic device, which will not be described in detail here for the sake of brevity.

[0126] The above describes the device of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0127] Figure 5 is a schematic block diagram of an electronic device provided in an embodiment of the present application, Figure 5 The electronic device can be used to execute the above method embodiments.

[0128] like Figure 5 As shown, the electronic device 30 may include:

[0129] The memory 31 and the processor 32, the memory 31 is used to store the computer program and transmit the program code to the processor 32. In other words, the processor 32 can call and run the computer program from the memory 31 to implement the method in the embodiment of the present application.

[0130] For example, the processor 32 may be configured to execute the steps in the above method according to the instructions in the computer program.

[0131] In some embodiments of the present application, the processor 32 may include but is not limited to:

[0132] General purpose processors, digital signal processors, application specific integrated circuits (ASICs), field programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0133] In some embodiments of the present application, the memory 31 includes but is not limited to:

[0134] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory or a flash memory. The volatile memory can be a random access memory, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory, dynamic random access memory, synchronous dynamic random access memory, double data rate synchronous dynamic random access memory, enhanced synchronous dynamic random access memory, synchronously linked dynamic random access memory and direct memory bus random access memory.

[0135] In some embodiments of the present application, the computer program may be divided into one or N modules, which are stored in the memory 31 and executed by the processor 32 to complete the method for recording pages provided by the present application. The one or N modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0136] According to one aspect of the present application, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0137] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of the above method embodiment.

[0138] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0139] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0141] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0142] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An abnormal alarm method for a remote monitoring system, characterized in that: The method comprises: Acquire real-time monitoring images of the monitoring area collected by two adjacent cameras to obtain a first monitoring image and a second monitoring image, wherein the geographical area is divided into a plurality of monitoring areas, each of the monitoring areas is correspondingly provided with a camera, and the two adjacent cameras are two cameras provided for two adjacent monitoring areas in the geographical area, and the monitoring areas of the two adjacent cameras partially overlap; For each monitoring screen, based on the screen objects contained in the monitoring screen, the monitoring screen is divided into regions to obtain a plurality of sub-screen regions of a first monitoring screen and a plurality of sub-screen regions of a second monitoring screen; Acquire position feature information of a sub-picture area of ​​the first monitoring screen in the monitoring screen, and perform feature extraction processing on a plurality of the sub-picture areas to obtain image feature information of each of the sub-picture areas, wherein the position feature information includes position information of the sub-picture area in the monitoring screen and associated position information of the sub-picture area and an associated sub-picture area, wherein the associated sub-picture area is a sub-picture area in the second monitoring screen that is positionally associated with the sub-picture area; constructing an image structure of the monitoring screen according to the position feature information and image feature information of the sub-screen area in the first monitoring screen, wherein the image structure includes a plurality of area nodes, one node corresponds to one sub-screen area, and the node information of the area node indicates the position feature information and image feature information of the corresponding sub-screen area; Acquire a security monitoring screen database, the security monitoring screen database including a plurality of sample monitoring screens and a sample graph structure corresponding to each sample monitoring screen; calculate similarity information between the first monitoring screen and the sample monitoring screen according to the image structure and the sample image structure corresponding to the sample monitoring screen; select a target sample monitoring screen from the security monitoring screen database according to the similarity information; perform image processing on the first monitoring screen according to image parameters of the target sample monitoring screen, and perform abnormal behavior detection processing on the processed monitoring screen; If abnormal behavior is detected, a security alert is triggered.

2. The method according to claim 1, characterized in that in, The node information further indicates pixel attribute information, and the method further comprises: Obtaining pixel attribute information of a sub-picture area in the first monitoring picture; Constructing the image structure of the monitoring screen according to the position information and image feature information of the sub-screen area in the first monitoring screen, including: constructing the image structure of the monitoring screen according to the position feature information, image feature information and pixel attribute information of the sub-screen area in the first monitoring screen.

3. The method according to claim 1, characterized in that The calculating the similarity information between the first monitoring picture and the sample monitoring picture according to the image structure and the sample image structure corresponding to the sample monitoring picture includes: Constructing a first feature vector of the first monitoring picture according to node information of nodes in the image structure; Constructing a second feature vector of the sample monitoring picture according to node information of nodes in the sample image structure; A similarity calculation process is performed based on the first feature vector and the second feature vector to obtain similarity information between the first monitoring screen and the sample monitoring screen.

4. The method according to claim 3, characterized in that: The selecting a target sample monitoring picture from the security monitoring picture database according to the similarity information includes: Selecting a sample monitoring screen image with a similarity greater than a first preset threshold from the security monitoring screen database as a target sample monitoring image; If there is no sample monitoring screen image whose value is greater than the first preset threshold, the associated position information in the node information of the image structure is adjusted, and the first feature vector of the first monitoring screen is updated again according to the adjusted node information in the image structure; the similarity calculation is re-executed until there is a sample monitoring screen image whose value is greater than the first preset threshold.

5. The method according to claim 1, characterized in that The performing image processing on the first monitoring picture according to the image parameter of the target sample monitoring picture includes: Extracting key image parameters from the target sample monitoring picture, wherein the key image parameters include at least one of color distribution, texture features, brightness, contrast, and object size; Image processing is performed on the first monitoring picture according to the key image parameters.

6. The method according to claim 1, characterized in that The abnormal behavior detection process is performed according to the processed monitoring screen, including: using an abnormal behavior detection model to extract object behavior features of the processed monitoring screen, and performing abnormal behavior detection process according to the object behavior features.

7. An abnormal alarm device, characterized in that: Applicable to remote monitoring systems, including: A picture acquisition unit is used to acquire real-time monitoring pictures of a monitoring area collected by two adjacent cameras to obtain a first monitoring picture and a second monitoring picture, wherein the geographical area is divided into a plurality of monitoring areas, each of which corresponds to a camera, and the two adjacent cameras are two cameras set in two adjacent monitoring areas of the geographical area; A division unit, for performing region division processing on each monitoring screen based on screen objects contained in the monitoring screen, to obtain a plurality of sub-screen regions of a first monitoring screen and a plurality of sub-screen regions of a second monitoring screen; a feature processing unit, configured to obtain position feature information of a sub-picture area of ​​the first monitoring screen in the monitoring screen, and perform feature extraction processing on a plurality of the sub-picture areas to obtain image feature information of each of the sub-picture areas, wherein the position feature information includes position information of the sub-picture area in the monitoring screen and associated position information of the sub-picture area and an associated sub-picture area, wherein the associated sub-picture area is a sub-picture area in the second monitoring screen that is positionally associated with the sub-picture area; A graph structure processing unit, configured to construct an image structure of the monitoring screen according to the position feature information and image feature information of the sub-screen area in the first monitoring screen, wherein the image structure includes a plurality of area nodes, one node corresponds to one sub-screen area, and the node information of the area node indicates the position information and image feature information of the corresponding sub-screen area; A detection unit is used to obtain a security monitoring screen database, wherein the security monitoring screen database includes multiple sample monitoring screens and a sample image structure corresponding to each sample monitoring screen; calculate similarity information between the first monitoring screen and the sample monitoring screen according to the image structure and the sample image structure corresponding to the sample monitoring screen; select a target sample monitoring screen from the security monitoring screen database according to the similarity information; perform image processing on the first monitoring screen according to image parameters of the target sample monitoring screen, and perform abnormal behavior detection processing based on the processed monitoring screen; if abnormal behavior is detected, trigger a security alarm.

8. An electronic device, suitable for use in a remote monitoring system, comprising a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: For storing computer programs; The computer program enables a computer to execute the method according to any one of claims 1 to 6.

Images