Intelligent video recognition linkage system and method based on multi-channel camera physical topology

Abstract

The present invention discloses an intelligent video recognition linkage system and method based on the physical topology of multiple cameras. The intelligent video recognition linkage method includes the following steps: dividing the multiple cameras into camera groups with multiple viewing angles according to the topological relationship of the multiple cameras; when a camera recognizes an abnormal event within the viewing angle, the multiple cameras in the group are linked to automatically adjust the viewing angle to track and monitor the maximum viewing angle of the abnormal event; when the abnormal event moves and disappears from the maximum viewing angle range of the camera in a group, the cameras between groups are linked to automatically adjust the viewing angle to track the abnormal event according to the moving direction of the abnormal event. Through communication between cameras and sharing of intelligent recognition result information, continuous tracking is achieved after the abnormal event occurs, and there is no blind spot in the video security linkage.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent monitoring technology, and in particular relates to an intelligent video recognition linkage system and method based on the physical topology of multiple cameras. Background Art

[0002] Object detection, tracking, and recognition technology is an active branch of research in the field of computer vision and has been widely used in many security-related applications, such as suspicious intrusions and abnormal events.

[0003] The video linkage technology on the market emphasizes the linkage between the video system and other external systems. For example, the intelligent inspection system based on video surveillance linkage technology on the market can remotely control lights, fans and air conditioners by collecting substation equipment operating status data and video surveillance image information, and interact with fire and theft prevention systems and access control systems to achieve mutual coordination and linkage.

[0004] However, the current video linkage system can only use a single camera to monitor the monitoring area, and lacks the ability to continue tracking after an abnormal event occurs, resulting in loopholes and blind spots in the monitoring area. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent video recognition linkage system and method based on the physical topology of multiple cameras, so as to solve the problem that in the prior art, monitoring can only use a single camera to monitor the monitoring area, resulting in loopholes and blind spots.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides an intelligent video recognition linkage system based on a multi-channel camera physical topology, comprising a camera, an internal local area network, a video decoding server, an intelligent recognition server, and a finite state machine decision server based on the recognition result;

[0008] The camera is used to collect video data from the station room;

[0009] The internal LAN is used to transmit video stream data collected by the camera;

[0010] The video decoding server is used to decode the video stream;

[0011] The intelligent recognition server is equipped with a deep neural network model, which is used to detect and recognize the decoded image sequence, and pass the detection and recognition results to the finite state machine decision server. The recognition results include two conclusions: abnormal events have occurred or no abnormal events have occurred.

[0012] The finite state machine decision server based on the recognition result is used to control the camera to track the behavior trajectory of abnormal events. When the camera tracks the behavior trajectory of abnormal events, the multiple cameras are divided into camera groups with multiple viewing angles according to the topological relationship of the multiple cameras; when a camera recognizes an abnormal event within the field of view, the multiple cameras in the group automatically adjust the viewing angle in linkage to track and monitor the maximum viewing angle of the abnormal event; when the abnormal event moves and disappears from the maximum viewing angle range of a camera in a group, the cameras between groups automatically adjust the viewing angle in linkage to track the abnormal event according to the moving direction of the abnormal event.

[0013] Furthermore, it also includes a streaming server and a web client; the intelligent recognition server is also used to push the image data of the tracking process to the streaming server, and the streaming server pushes the real-time image data to the web client.

[0014] A second aspect of the present invention provides an intelligent video recognition linkage method, based on the intelligent video recognition linkage system, comprising the following steps:

[0015] According to the topological relationship of the multiple cameras, the multiple cameras are divided into camera groups with multiple viewing angle ranges;

[0016] When a camera identifies an abnormal event within its field of view, multiple cameras in the group automatically adjust the viewing angle to track and monitor the maximum viewing angle of the abnormal event.

[0017] When an abnormal event moves and disappears from the maximum viewing angle of a camera in a group, the cameras between groups are linked to automatically adjust the viewing angle and track the abnormal event according to the moving direction of the abnormal event.

[0018] Furthermore, the camera groups are specifically divided as follows: the parts with overlapping visual ranges of all cameras are grouped as one group, and the parts without overlapping visual ranges are grouped as a single group.

[0019] Furthermore, the identification of abnormal events is achieved through a preset deep neural network model.

[0020] Furthermore, the training data of the deep neural network model is obtained as follows:

[0021] (1) Generate a 3D model based on the three-dimensional motion data of the object through motion simulation technology;

[0022] (2) Generate two-dimensional image data based on the 3D model;

[0023] (3) Annotate the two-dimensional image data to obtain the training data.

[0024] Furthermore, multiple cameras in the group are linked to automatically adjust the viewing angle to track and monitor the maximum viewing angle of abnormal events. The specific method is: multiple cameras in the group are linked to automatically adjust the viewing angle, so that the abnormal event is at the center of the field of view of one or several cameras within the drivable adjustment range of the camera, thereby achieving the maximum tracking viewing angle of the abnormal event.

[0025] Furthermore, when an abnormal event moves and disappears from the maximum viewing angle of a camera in a group, the abnormal event is tracked. The specific method is as follows:

[0026] When an abnormal event disappears from the maximum field of view of a camera in a certain group, the angle and coordinates at which the abnormal event leaves the camera are given. According to the topological relationship of the camera positions, the camera group closest to the angle and coordinates is triggered. The camera group closest to the angle and coordinates automatically adjusts the viewing angle to the moving direction of the abnormal event and locks the maximum possible moving range of the abnormal event.

[0027] The beneficial effects of the present invention are as follows:

[0028] 1) The intelligent video recognition linkage method provided by the present invention realizes continuous tracking after an abnormal event occurs through communication between cameras and sharing of intelligent recognition result information, and video security linkage has no blind spots.

[0029] 2) The intelligent video recognition linkage method provided by the present invention is based on the data generation technology of motion simulation technology, which generates a large amount of two-dimensional image data through three-dimensional data, thereby achieving performance improvement of the video recognition model.

[0030] 3) The intelligent video recognition linkage system provided by the present invention solves the problem of video security linkage in the distribution network, and continuously tracks abnormal events after they occur, thus achieving video security linkage without blind spots. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 It is a structural block diagram of the intelligent video recognition linkage system in an embodiment of the present invention.

[0033] Figure 2 The principle state diagram of the intelligent video recognition linkage method in the embodiment of the present invention is as follows:

[0034] Figure 3 Schematic diagram of data simulation in an embodiment of the present invention.

[0035] Figure 4It is a top plan view of the distribution station room and the camera angle in an embodiment of the present invention.

[0036] Figure 5 This is a top plan view of the distribution station room and the camera angle when an abnormal event occurs in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0038] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention.

[0039] The embodiment of the present invention provides an intelligent video recognition linkage system of multiple cameras based on physical topology relationship, and an intelligent video recognition linkage method based on the intelligent video recognition linkage system. After an abnormal event occurs, continuous tracking can be achieved to achieve video security linkage without blind spots.

[0040] like Figure 1 As shown, the first aspect of an embodiment of the present invention provides an intelligent video recognition linkage system, including multiple cameras, an internal local area network, a video decoding server, an intelligent recognition server, a finite state machine decision server based on recognition results, a streaming server and a web client, etc.

[0041] Specifically, in this embodiment, multiple cameras are responsible for collecting station room video data, the internal LAN is responsible for transmitting the video stream data collected by the camera, and the video decoding server is responsible for decoding the video stream. The intelligent recognition server is responsible for detecting and identifying the parsed image sequence, and passing the detection and identification results to the finite state machine decision server; at the same time, the intelligent recognition server pushes the image data of the tracking process to the stream server, and the user views the real-time image data through the web client. The recognition results of the intelligent recognition server include two conclusions: an abnormal event has occurred and no abnormal event has occurred. Based on the recognition results, the finite state machine decision server tracks the behavior trajectory of abnormal events through the intelligent video recognition linkage strategy.

[0042] Specifically, in the embodiment of the present invention, the intelligent video recognition linkage strategy divides the status of the video data acquired by the camera into two states: no abnormal event occurs and abnormal event occurs; when no abnormal event occurs within the field of view of all cameras, each camera maintains intelligent recognition within an independent field of view; when an abnormal event occurs within the field of view of some cameras (assuming that the field of view of each camera is Θ, and the cameras are divided into multiple related groups based on the topological relationship of multiple cameras, which are further divided into intra-group relationships and inter-group relationships), an intra-group linkage mechanism is formed based on the intra-group relationship. When the abnormal event moves beyond the maximum field of view of the intra-group relationship, an inter-group linkage mechanism is formed based on the inter-group relationship to continuously track the abnormal event.

[0043] like Figure 2 As shown, the intelligent video recognition linkage method of the present invention mainly includes two parts: one under no abnormal event and one under abnormal event. The specific process is as follows:

[0044] S1. Divide the multiple cameras into camera groups with multiple viewing angles based on the topological relationship of the multiple cameras.

[0045] Specifically, Figure 4 As shown, all cameras in the distribution station room are divided into 7 groups, the part with overlapping visual range is one group (the group within the dotted line range), and the part without overlapping visual range is a single group.

[0046] S2. When an abnormal event occurs within the field of view of a camera, the camera acquires video data of the area, identifies the occurrence of the abnormal event through an intelligent video recognition algorithm, and feeds back the trajectory information and direction trend to the corresponding camera in the group and the surrounding camera group. The cameras in the group track the abnormal event.

[0047] Specifically, assuming that the field of view of each camera is Θ, when an abnormal event occurs within the field of view of a camera in a group, the group linkage mechanism will be triggered, and the camera linkage will automatically adjust the viewing angle to ensure that the abnormal event is at the center of one or several cameras within the drivable adjustment range of the camera, thereby achieving the maximum tracking angle of the abnormal event and performing all-round detection, tracking, and early warning of the abnormal event.

[0048] In an embodiment of the present invention, the intelligent recognition server recognizes abnormal events by using a preset deep neural network model, and the training data of the deep neural network model is based on the data generation technology of motion simulation technology, which mixes real data and motion simulation data.

[0049] like Figure 3 As shown, the specific method of obtaining training data is as follows:

[0050] (1) Generate a 3D model based on the three-dimensional motion data of the object through motion simulation technology.

[0051] (2) Generate a large amount of two-dimensional image data based on the 3D model.

[0052] (3) Annotate the two-dimensional image data automatically or / and manually, and provide the annotated two-dimensional image data to the deep neural network model for training and learning, so as to improve the accuracy of the deep neural network model.

[0053] After training, the deep neural network model can identify different objects, including flame rats, etc., with the time indicator reduced by more than 20% and the accuracy increased by more than 40%, laying the necessary data and method foundation for the perception analysis of power workers.

[0054] S3. When an abnormal event moves and disappears from the maximum viewing angle of a camera in a group, the inter-group camera linkage mechanism is mobilized to form inter-group camera linkage behavior to continuously track the abnormal event.

[0055] Specifically, assuming that the field of view of each camera group is φ, when an abnormal event disappears from the maximum field of view of a camera in a group, the inter-group linkage mechanism will be triggered. The inter-group linkage mechanism is based on the results of intelligent video recognition, and gives the angle and coordinates of the abnormal event leaving the camera. According to the topological relationship of the camera position, the camera group closest to the angle is triggered. The camera group automatically adjusts the viewing angle to the moving direction of the abnormal event, thereby ensuring that the maximum possible moving range of the abnormal event is locked, and all-round detection, tracking, and early warning of suspicious people or objects are carried out. Figure 5 As shown, when the movement trend of the abnormal event is as shown in the blue trend in the figure, Group 6 and Group 4 in the figure will be mobilized to monitor, track and warn the abnormal event.

[0056] An embodiment of the present invention also provides a method for linking intelligent video recognition of cameras within and between groups. Based on the above-mentioned camera grouping, the input is the topological position relationship of the station room cameras, the initial angle and the position coordinate sequence of abnormal events recognized by intelligent video, and the output result is the camera that needs to be adjusted and the adjustment angle.

[0057] The specific algorithm steps are as follows:

[0058] Step 1: Initialize and obtain the angles and topological relationships of each camera, as well as the abnormal event location coordinate sequence B(t) of intelligent video recognition.

[0059] Step 2: If an abnormal event occurs within the field of view of camera C, camera C tracks the movement of the abnormal event. The abnormal event moves pixel vector X on the screen, and camera C automatically adjusts angle a to correspond to the moving pixel vector X on the screen.

[0060] X=B(t)-B(t-1), where B(t) represents the coordinate corresponding to time t.

[0061] Step 3: When the abnormal event is about to leave the field of view of the camera group where camera C is located, find the camera H that is closest to the vector X direction from the adjacent camera group M of camera C, and adjust the movement of camera H by -X.

[0062] H = argmin{cosɑ*lP(H)-P(C)|}, H∈M, cosɑ>0, (M is derived from the topological relationship);

[0063] P(C) represents the coordinates of the C camera, P(H) represents the coordinates of the H camera, and ɑ is the angle between P(C) and P(H) with the abnormal event (e.g., a person) as the vertex;

[0064] cosɑ=X*(P(H)-P(C)) / |X|*|P(H)-P(C)|.

[0065] Step 4: When an abnormal event appears within the field of view of camera H, camera H tracks the movement of the abnormal event. The abnormal event moves pixel vector X on the screen, and camera H automatically adjusts angle a to correspond to the moving pixel vector X on the screen.

[0066] Step 5: The abnormal event leaves and the range of all cameras ends.

[0067] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. An intelligent video recognition linkage method, implemented based on an intelligent video recognition linkage system, characterized in that: The steps include: According to the topological relationship of the multiple cameras, the multiple cameras are divided into camera groups with multiple viewing angle ranges; When a camera identifies an abnormal event within its field of view, multiple cameras in the group automatically adjust the viewing angle to track and monitor the maximum viewing angle of the abnormal event. When an abnormal event moves and disappears from the maximum viewing angle of a camera in a group, the cameras between groups are linked to automatically adjust the viewing angle and track the abnormal event according to the moving direction of the abnormal event; The camera groups are divided as follows: the part with overlapping visual ranges of all cameras is a group, and the part without overlapping visual ranges is a separate group; The identification of the abnormal event is achieved through a preset deep neural network model; The training data of the deep neural network model is obtained as follows: (1) Generate a 3D model based on the three-dimensional motion data of the object through motion simulation technology; (2) Generate two-dimensional image data based on the 3D model; (3) labeling the two-dimensional image data to obtain the training data; The multiple cameras in the group are linked to automatically adjust the viewing angle to track and monitor the maximum viewing angle of abnormal events. The specific method is: multiple cameras in the group are linked to automatically adjust the viewing angle to make the abnormal event in the center of the field of view of one or several cameras within the adjustable range of the camera, so as to achieve the maximum tracking angle of the abnormal event; When an abnormal event moves and disappears from the maximum viewing angle of a camera in a group, the abnormal event is tracked as follows: When an abnormal event disappears from the maximum field of view of a camera in a group, the angle and coordinates at which the abnormal event leaves the camera are given, and the camera group closest to the angle and coordinates is triggered according to the topological relationship of the camera positions. The camera group closest to the angle and coordinates automatically adjusts the viewing angle to the moving direction of the abnormal event, and locks the maximum possible moving range of the abnormal event. An intelligent video recognition linkage system, including a camera, an internal LAN, a video decoding server, an intelligent recognition server, and a finite state machine decision server based on recognition results; The camera is used to collect video data from the station room; The internal LAN is used to transmit video stream data collected by the camera; The video decoding server is used to decode the video stream; The intelligent recognition server is equipped with a deep neural network model, which is used to detect and recognize the decoded image sequence, and pass the detection and recognition results to the finite state machine decision server. The recognition results include two conclusions: abnormal events have occurred or no abnormal events have occurred. The finite state machine decision server based on the recognition result is used to control the camera to track the behavior trajectory of abnormal events. When the camera tracks the behavior trajectory of abnormal events, the multiple cameras are divided into camera groups with multiple viewing angles according to the topological relationship of the multiple cameras; when a camera recognizes an abnormal event within the field of view, the multiple cameras in the group automatically adjust the viewing angle in linkage to track and monitor the maximum viewing angle of the abnormal event; when the abnormal event moves and disappears from the maximum viewing angle range of a camera in a group, the cameras between groups automatically adjust the viewing angle in linkage to track the abnormal event according to the moving direction of the abnormal event.

2. The intelligent video recognition linkage method according to claim 1, characterized in that: The system also includes a streaming server and a web client; the intelligent recognition server is also used to push the image data of the tracking process to the streaming server, and the streaming server pushes the real-time image data to the web client.

Images