Secure video surveillance system and video data processing method

Abstract

FIELD: video surveillance systems.SUBSTANCE: invention can be used in territorially distributed automated security systems. Initial data is pre-formed and specified: a list of target objects, reference images of target objects, the target object image match coefficient, the background image change coefficient, the initial background image update time period, and video camera parameters in the video monitoring tool. Video data is received from a digital video camera and a thermal imager and the video is split into a sequence of frames. The first frame is recorded into the storage device and sent to the database server as a background image, which is updated according to the specified time period. From the second frame, computer vision algorithms are used to search for target objects, for which each new frame is sequentially compared with the saved background image until the end of the video surveillance period.EFFECT: reducing the volume of video data traffic transmitted over communication channels while maintaining the image quality of specified objects and increasing the security of the transmitted information.5 cl, 6 dwg

Description

[0001] Field of technology to which the invention relates

[0002] The invention relates to video surveillance systems and can be used in territorially distributed automated security systems.

[0003] Technology Level

[0004] The proliferation of CCTV cameras in both the public and commercial sectors places increased demands on bandwidth for video data transmission, especially in real time, and creates a significant load on the data network. However, security in such systems is typically limited to the use of technical devices to ensure their safety (for example, a vandal-resistant camera housing) and the installation of one-time single-factor authentication when connecting to the camera over the network.

[0005] a) Description of analogues

[0006] The following are known: "Video surveillance system with distributed nodes and a method for controlling such a system" under patent RU 2518194 C2 dated 18.12.2009, IPC H04N 7 / 18 (2006 / 01), H04L 29 / 08 (2006 / 01), published. 10.06.2014, Bulletin No. 16, "Adaptive security video surveillance system" under patent No. RU 2570146 C1 dated 16.07.2014, IPC G08B 13 / 196 (2006.01), published. 10.12.2015, Bulletin No. No. 34, “System for the Protection of Open Areas” No. RU 2692962 C1 dated September 21, 2018, IPC G08B 13 / 18 (2006 / 01), published June 28, 2019, Bulletin No. 19.

[0007] A characteristic feature of the listed video surveillance systems is that the technical means acting as sources of video information (video cameras and thermal imagers) and the technical means processing video images are geographically distributed and connected by communication channels.

[0008] The disadvantages of video surveillance systems include the lack of video information processing aimed at reducing the volume of information transmitted via communication channels, and the lack of information protection tools both on system elements and in communication channels.

[0009] A method and device for video monitoring are known according to patent RU 2633223 C2 from 2015.04.29, G08B 21 / 22 (2006.01), G08B 25 / 00 (2006.01), published. 2017.10.11, Bulletin No. 5, which consists in the fact that the monitoring equipment is turned on and continuously uploads video to the server until a legitimate user arrives and the system reads the identifier from the wearable wireless device.

[0010] The disadvantages of the method and device for video monitoring are the inapplicability of this method for geographically distributed video monitoring systems, as well as the lack of processing of the video stream in order to reduce the volume of video data traffic transmitted over communication channels.

[0011] A method for multi-channel remote video surveillance at construction and industrial sites and a mobile video recording system for its implementation are known under patent RU 2748517 C1 dated 2020.04.06, H04N 7 / 173 (2011.01), published. 2021.05.26, Bulletin No. 15, which consists in increasing the speed of detection of potentially dangerous situations through the use of additional personal video recording subsystems on individual protective equipment, increasing the reliability of recognition of dangerous events, reducing the time for receiving and transmitting data and eliminating the influence of the human factor.

[0012] The disadvantage of the method of multi-channel remote video surveillance at construction and industrial sites and the mobile video recording system is that in order to reduce the volume of video data traffic transmitted over communication channels, only image encoding methods are used, which reduce the amount of transmitted information due to the deterioration of image quality.

[0013] The closest in technical essence and functions to the declared system (prototype) is the distributed architecture of the forest video monitoring system according to patent RU 2554102 C2 dated 09.25.2013, IPC H04N 7 / 18 (2006 / 01), G08B 13 / 196 (2006 / 01), published 06.27.2015, Bulletin No. 18, containing a plurality of video monitoring points, each of which contains a video camera on a high-rise structure; a subsystem for controlling video monitoring and primary processing of video data, containing one or more video servers, wherein each video server is assigned one or more video monitoring points from the said plurality for controlling the video cameras of these one or more video monitoring points, wherein each server is connected to the video cameras of the video monitoring points assigned to it via a communication channel for receiving video data filmed by these video cameras; a subsystem for storing and secondary processing of data, containing one or more object servers;and one or more computer terminals, wherein said video servers, object servers and computer terminals are connected to a communication network, wherein each video server is configured to: perform an analysis of the captured video data to identify signs of a fire, upon detection of signs of a fire, form a potential hazard data object related to the identified possible fire, and send the potential hazard data object to the video data storage and secondary processing subsystem;wherein each object server is configured to: store and modify potentially dangerous data objects, and send potentially dangerous data objects to computer terminals, wherein a client application is installed on each terminal computer to ensure the operator's interaction with the video monitoring and primary video data processing subsystem and the storage and secondary data processing subsystem, wherein the client application is configured to receive potentially dangerous data objects from the storage and secondary data processing subsystem and to present potentially dangerous data objects to the operator.

[0014] The closest in technical essence and functions to the claimed method (prototype) is the method for detecting fire, implemented in the forest video monitoring system, according to patent RU 2554102 C2 dated 09.25.2013, IPC H04N 7 / 18 (2006 / 01), G08B 13 / 196 (2006 / 01), published 06.27.2015, Bull. No. 18, which consists in that, by means of a video server from a plurality of video servers: receiving video data from at least one video camera, performing an analysis of the video data to identify signs of a fire, upon identifying signs of a fire, generating a potential danger data object related to the identified possible fire, with a link to the video data in which signs of a fire were identified, and sending the potential danger data object to at least one of the object servers;by means of an object server: receiving a potential hazard data object, comparing the received potential hazard data object with previously stored potential hazard data objects, based on the results of the comparison, performing at least one of the following: storing the received potential hazard data object, modifying at least one of the previously stored potential hazard data objects, modifying the received potential hazard data object, and sending one or more potential hazard data objects to at least one computer terminal; and by means of the computer terminal: receiving potential hazard data objects and presenting the potential hazard data objects to the operator.

[0015] The disadvantage of the system and method (prototype) is that the video data is transmitted via communication channels to the database server in full, while there are no means of protecting information, the video server processes video data not in real time, but recorded in the database, from the video server to the object server both the generated data object and the video data itself are transmitted.

[0016] Technical problem

[0017] The technical problem that the invention is aimed at solving is the significant volume of video information transmitted over a data transmission network, caused by the increase in the number of network video surveillance equipment, the redundancy of non-targeted information in the transmitted video traffic, the remoteness of video data processing equipment from video cameras, and the low level of security of video surveillance systems.

[0018] The technical problem is caused by the lack of preliminary processing of the video stream before transmitting it over the data network, and insufficient technical means to ensure the protection of video information.

[0019] The technical problem is solved by pre-processing the video stream before transmitting it over a secure data network. This is accomplished using a microcomputer integrated with the video camera. This microcomputer integrates information security measures, including protection against network interference, unauthorized access, and guaranteed data erasure. The microcomputer, in real time, detects the appearance of pre-defined target images from the entire video stream, identifies multiple images of detected objects against the background, and transmits dynamically changing graphical information about the detected sets of target objects and a static background image via various data network routes, with adaptively changing frequency.

[0020] Disclosure of the invention (its essence)

[0021] a) The technical result that the invention is aimed at achieving

[0022] The technical result of the invention is a reduction in the volume of video data traffic transmitted over communication channels while maintaining the quality of the image of specified objects and increasing the security of the transmitted information.

[0023] 1. The technical result in a secure video surveillance system is achieved by the fact that in the known system, video monitoring means, each of which is connected to one or more video servers via communication channels; a video surveillance control subsystem containing at least one video server, wherein each server is connected to the video monitoring means assigned to it and other elements of the system via communication channels; a storage subsystem containing at least one database server, wherein each server is connected to the remaining elements of the system via a communication channel; one or more automated operator workstations with installed software allowing interaction with video monitoring means and elements of the control and storage subsystems, in addition, all elements of the video surveillance system are connected to a data transmission network, which consists of a plurality of heterogeneous communication channels, active and passive network equipment,software and interaction protocols; each video monitoring device is made in a single design, equipped with a rotating mechanism, the video surveillance device includes a digital video camera and a thermal imager, a microcomputer with a computer vision function and a time counter, a storage device in which designated reference images of target objects are stored and a background image is recorded, security measures that include means of protection against influence from the data transmission network, unauthorized access and means of guaranteed erasure of information, while the video output of the digital video camera and thermal imager is connected to the video input of the microcomputer, the storage device is connected directly to the microcomputer, the microcomputer itself is configured to generate control signals for all elements of the video monitoring device,for which purpose the first and second outputs of the microcomputer are connected to the inputs of a digital video camera and a thermal imager respectively, the microcomputer has a network interface for connection to a data transmission network for the exchange of information with the remaining elements of the system; each video server is configured to superimpose video data on detected objects, received from video monitoring equipment, onto a background image received from the database server, and to transmit the resulting image to the operator's automated workstation, in addition it has security measures, including means of protection against influence from the data transmission network, unauthorized access and means of guaranteed erasure of information, each video server is connected to the data transmission network through a firewall; each database server is configured to record the background images received from each video monitoring equipment,and also store all graphic images of target objects and a list of which video monitoring tools which graphic images of target objects are assigned to, in addition to having in its composition security means, including means of protection against the influence of the data transmission network, unauthorized access and means of guaranteed erasure of information, each database server is connected to the data transmission network through a firewall; the automated operator workstation is designed with the ability to display processed video data, enter new reference graphic images of target objects and assign them to specific video monitoring tools, and also manage the operation of video monitoring tools and video servers, while the automatic mode of operation of the system is priority, each automated operator workstation is connected to the data transmission network.

[0024] In an embodiment of the invention, each video monitoring means may additionally be equipped with an additional video camera with a higher resolution light-sensitive matrix.

[0025] In a variant of the invention, one or more collective display monitors may additionally be placed, configured to connect to a data transmission network and display processed video data with a level of detail higher than that on the operator's automated means.

[0026] 2. The technical result of the method is achieved in that in the known method, implemented in a video surveillance system containing a plurality of video monitoring means; a plurality of video servers, wherein each video server serves one or more video monitoring means; a plurality of collective display monitors and / or automated operator workstations; in addition, the initial data are preliminarily formed and specified: a list of target objects, reference images of target objects, a target object image match coefficient, a background image change coefficient, an initial background image refresh time period, video camera parameters, in the video monitoring means, reference images of target objects are recorded in a storage device, video data is received from a digital video camera and a thermal imager by means of a microcomputer coupled with a video camera and the video is divided into a sequence of frames,The first frame is recorded in the storage device and sent to the database server as a background image, which is updated in accordance with a specified time period, a time counter is started, from the second frame, using computer vision algorithms pre-installed in the microcomputer, a search for target objects is carried out, for which each new frame is sequentially compared with the saved background image until the end of the video surveillance time, if there are no changes, then the images continue to be compared frame by frame, if there are changes in the compared images, then the nature of the changes is determined, if the detected changes cannot be grouped into an object, then a decision is made that the changes relate to the background image and the coefficient of changes in the background image is determined, if it is greater than or equal to the set value, then the refresh period of the background image is changed,a new background image is recorded in the storage device of the video monitoring device and sent to the database server, if the ratio of changes in the background image is less, then the images continue to be compared frame by frame, while the background image is not sent, and the time counter is reset to zero, the procedure of comparing incoming frames with the background image continues until the end of the video surveillance time, if the detected changes can be grouped into an object, then the image of the detected object is compared with the scalable reference images of the target objects stored in the storage device of the video surveillance device, if the coefficient of coincidence of the image of the detected object is greater than or equal to the set value, then an identifier is assigned to the object, the position of the object in the frame is determined and images of only the selected object with the identifier and the position of the starting point in the frame are sent to the video server frame by frame,If the match ratio of the image of the detected object is less than the set value, then a notification is sent to the operator's automated workstation about the appearance of a non-target object; if a command is received from the operator to transmit an image of the detected non-target object, then the image of the detected non-target object is sent to the operator's automated workstation; if not, then video surveillance is continued, checking whether the time has come to update the background image; if yes, then a new background image is written to the storage device of the video surveillance device and sent to the database server; if not, then continue comparing incoming images; the database server receives the background image from the video monitoring device and writes it to the database; when a new background image is received, it is written to the database instead of the previous one; upon request from the video server, the background image is sent to the video server,receive images of new target objects from the operator, record the images of target objects in the database, upon receipt of a command from the operator, record which target object images need to be assigned to which video monitoring device and transmit them to the video monitoring device; in the video server, receive a video image of the selected object with an identifier, coordinates and position of the object in the frame from the video monitoring device, send a request to the database server for the provision of a background image corresponding to the video monitoring device, receive a background image from the database server, superimpose the video data of the detected objects on the background image and transmit the resulting video image to the automated operator workstation or collective display monitor; in the automated operator workstation, receive the resulting image from the video server if it is necessary to enter a reference image of a new target object,then they upload it to the database server, send a command to the database server to which video monitoring device which images of target objects need to be assigned, send a command to the database server to transfer the assigned images of target objects to the video monitoring devices, receive a notification from the video surveillance devices about the appearance of a non-target object, if the operator needs to view the image of the detected non-target object, then they send a command to the video surveillance device to provide an image of the detected non-target object.

[0027] In a variant of the invention, if the match coefficient of the image of the target object is less than the set value, a video camera with higher image clarity and detail is additionally connected and the procedure for comparing the new image of the target object with the standard is repeated.

[0028] b) Cause-and-effect relationship between features and technical result

[0029] By introducing a set of distinctive features into a known object, the declared technical solutions make it possible to reduce the volume of video data traffic transmitted over communication channels while maintaining the image quality of specified objects and to increase the security of the transmitted information.

[0030] Evidence of compliance of the claimed invention with the conditions of patentability

[0031] A conducted analysis of the prior art revealed that there are no existing analogues characterized by sets of features identical to all the features of the claimed method. Consequently, the claimed invention meets the patentability requirement of "novelty."

[0032] The listed new set of essential features is sufficient for the implementation of a secure video surveillance system that ensures a reduction in the volume of video data traffic transmitted over communication channels while maintaining the image quality of specified objects.

[0033] A search of prior art in this and related fields of technology to identify features that match the distinctive features of the claimed invention from prior art revealed that these features are not clearly evident from the prior art. The applicant's prior art findings do not reveal any prior art evidence of the impact of the essential features of the claimed invention on achieving the stated technical result. Consequently, the claimed invention meets the patentability requirement of "inventive step."

[0034] The “industrial applicability” of the method is determined by the presence of an element base, on the basis of which the elements of the system can be made to achieve the result specified in the invention.

[0035] Brief description of drawings

[0036] The claimed method is illustrated by drawings:

[0037] Figure 1 - generalized structural diagram of a secure video surveillance system;

[0038] Figure 2 - a generalized block diagram of the implementation of the claimed method;

[0039] Figure 3 - example of target object images;

[0040] Figure 4 - an example of detection and selection of target objects in a video image;

[0041] Figure 5 - an example of assigning identifiers to detected target objects;

[0042] Figure 6 - example of a table of distribution of target objects by video monitoring means.

[0043] Implementation of the invention

[0044] The secure video surveillance system and the method for processing and transmitting video data over a data transmission network consists of:

[0045] a plurality of video monitoring devices 1 (Fig. 1), each of which consists of a digital video camera 1.1 (Fig. 1), a thermal imager 1.2 (Fig. 1), a storage device 1.3 (Fig. 1), a microcomputer 1.4 (Fig. 1) with a computer vision function and a time counter,

[0046] video surveillance control subsystem 2 (Fig. 1), containing at least one video server 2.1 (Fig. 1),

[0047] storage subsystem 3 (Fig. 1), containing at least one database server 3.1 (Fig. 1),

[0048] one or more automated operator stations 4 (Fig. 1),

[0049] collective use monitor 5 (Fig. 1),

[0050] data transmission network 6 (Fig. 1),

[0051] Information security tools 7 (Fig. 1), including protection against data network attacks, unauthorized access, and guaranteed data erasure. Protection against data network attacks is typically achieved through the use of firewalls, the main functions of which include:

[0052] 1. Restricting external user access to servers protected by a firewall;

[0053] 2. restricting access of internal users to external resources;

[0054] 3. filtering of incoming and outgoing traffic (by protocol, by address, by port);

[0055] 4. Security policy violation alarm;

[0056] 5. Security policy audit.

[0057] [Laponina O.R. L24 Fundamentals of Network Security. Part 1. Firewalls: Tutorial / O.R. Laponina - M .: National Open University "INTUIT", 2014. - 378 p., ill., table. - (Series "Fundamentals of Information Technology")]. The gradation and requirements for firewalls are defined by the Order of the FSTEC of Russia dated February 9, 2016 No. 9 "Requirements for firewalls".

[0058] Guaranteed data erasure tools are designed to securely delete data from storage media. For example, if a video monitoring device is compromised, images of target objects, authentication identifiers, and key information for encryption protocols stored on the storage device must be erased. Otherwise, an attacker could use this information to disrupt the entire video surveillance system.

[0059] Each video monitoring device is made in a single design, wherein the video output of the digital video camera (1.1, Fig. 1) and the thermal imager (1.2, Fig. 1) is connected to the video input of the microcomputer (1.4, Fig. 1), the storage device (1.3, Fig. 1) is connected directly to the microcomputer, the microcomputer itself is designed with the possibility of generating control signals for all elements of the video monitoring device, for which the first and second outputs of the microcomputer are connected to the inputs of the digital video camera and the thermal imager, respectively, the microcomputer has a network interface for connection to the data transmission network for exchanging information with the remaining elements of the system.

[0060] Each video monitoring device is equipped with a rotating mechanism.

[0061] Each video monitoring device includes hardware and software protection tools, including protection tools against influence from the data transmission network, protection tools against unauthorized access, as well as tools for guaranteed erasure of information.

[0062] Each video monitoring device must comply with the requirements of GOST R 51558 - 2014 "Security television devices and systems. Classification. General technical requirements. Test methods."

[0063] Digital video camera (1.1, Fig. 1) is a video camera, structurally and functionally combined with a device for analog-to-digital conversion of a video signal. In this case, a video camera is understood to be a device intended for television analysis of the transmitted scene using optoelectronic conversion and transmission of a television signal. [GOST R 51558 - 2014 "Security television means and systems. Classification. General technical requirements. Test methods"]. A video surveillance camera, for example, Hikvision DS-2CE16D1T-AVFIR3 [Electronic resource. URL: hikvisionpro.ru / catalog / element / ds-2ce16d1t-avfir3.html. Date of access 03 / 15 / 2025].

[0064] In an embodiment of the invention, each video monitoring device can be equipped with an additional video camera with a higher resolution of the light-sensitive matrix, for example, the Hikvision DS-2CD72205G0 / E IP Video Camera [Electronic resource. URL: hikvision24.ru / videokamery / ip-videokamera-hikvision-ds-2cd72205g0-e-70-200-mm / . Accessed on 03 / 15 / 2025], designed to obtain a highly detailed image of the object detected by the main video camera.

[0065] Thermal imager (thermal imaging camera) (1.2, Fig. 1) is an optoelectronic device designed for contactless (remote) observation, measurement and recording of temperature fields (spatio-temporal distribution of surface temperature of objects) located in the field of view of the device. [GOST R 54852-2024 "Buildings and structures. Methods for determining the performance of a thermal insulation shell based on a thermal imaging survey and in-kind measurements"]. The Mastermann BTK8-6.150 V thermal imaging video camera can be used as a thermal imager [Electronic resource. URL: hikvision24.ru / videokamery / mastermann / teplovizionnaya-videokamera-mastermann-btk8-6-150v / . Accessed 03 / 15 / 2025]

[0066] Microcomputer (Micro EVM) (1.4, Fig. 1) - a computer belonging to the class of computing machines, the central part of which is built on one or more microprocessors and developed based on the requirement to minimize physical volume. [GOST USSR 15971-90 "Buildings and structures. Methods for determining the indicators of a thermal protective shell based on a thermal imaging survey and in-kind measurements"]. A Dev Board Mini [Electronic resource. URL: coral.ai / products / dev-board-mini. Date of access 03 / 15 / 2025] or a GEEKOM A8 mini personal computer [Electronic resource. URL: geekom.ru / geekom-a8 / . Date of access 03 / 15 / 2025] can be used as a microcomputer.

[0067] The storage device (1.3, Fig. 1) is an information carrier intended for recording and storing data. [GOST R 70942-2023 "Storage devices of digital computers. Terms and definitions"]. The storage device can be built-in, for example, PCIe M.2 SSDs [Electronic resource. URL: transcend-info.com / product / intemal-ssd / mte255s. Date of access 03 / 15 / 2025] or alienable such as a USB drive or SD card, for example, Transcend microSDXC / SDHC 300S [Electronic resource. URL: transcend-info.com / product / memory-card / usd300s. Date of access 03 / 15 / 2025]. The storage device of the video monitoring device stores the designated reference images of the target objects and records the background image.

[0068] Hardware and software security tools - a general term that refers to technical (hardware), software and combined (hardware and software) means of information protection, as well as technical means involved in the exchange of video information. Information security tool - a technical, software, software and hardware tool, substance and (or) material intended or used to protect information. [GOST R 50922-2006 "Information Protection. Basic Terms and Definitions"]. With respect to video surveillance systems, information security tools must ensure reliable protection of information from the security threats described in [GOST R 51558-2014 "Security Television Tools and Systems. Classification. General Technical Requirements. Test Methods"].

[0069] Data network security tools are hardware and software security tools designed to ensure the security of information during its transmission over a data network. Firewalls can be used as such security tools, for example, ZYXEL USG FLEX 500 [Electronic resource. URL: knsneva.ru / product / mezhsetevoi-ekran-zyxel-zywall-usg-flex-500-usgflex500-eu0102f / . Accessed on March 15, 2025] or UserGate E1000, E3000 [Electronic resource. URL: usergate.com / ru / products / usergate-e. Accessed on March 15, 2025]

[0070] Means of protection against unauthorized access are software, hardware, or software and hardware means designed to prevent or significantly hinder unauthorized access to information. Protective covers, such as BEWARD B100хх-K220A [Electronic resource. URL: pro-spec.ru / catalog / kozhuhi / b10xx-k220a. Date of access 03 / 15 / 2025] and alarm systems, such as the Osnovo outdoor station tamper sensor [Electronic resource. URL: osnovo-shop.ru / product / osnovo-datcik-vskrytia-dla-ulicnyh-stancij / 69083?ysclid=m8c1pih8uh980970786], can serve as means of protection against unauthorized access. Accessed March 15, 2025]

[0071] Guaranteed data erasure tools are hardware, software, or software tools designed to permanently delete information from storage media such as hard magnetic disks (HDDs) and solid-state drives (SSDs), for example, the Impulse-7U hard drive disposal system [Electronic resource. URL: detsys.ru / catalog / utilizatory / impuls_5u / . Accessed 03 / 15 / 2025] or the SGU-2 Guaranteed Information Destruction System [Electronic resource. URL: cobra.ru / ru / prod / sgu. Accessed 03 / 15 / 2025]

[0072] Video server (2.1, Fig. 1) is a device within a digital security television system, designed to convert an analog video signal from the output of video cameras into digital format for the purpose of processing, transmitting it over a computer network, and / or recording it on a digital storage medium. [GOST R 51558-2014 "Security television means and systems. Classification. General technical requirements. Test methods"].

[0073] The video server in the proposed system is designed with the ability to overlay video data about detected objects, obtained from video monitoring equipment, on a background image received from the database server, and the ability to transmit the resulting image to the operator's automated workstation; in addition, it has security tools, including tools for protection against influence from the data transmission network, unauthorized access, and tools for guaranteed erasure of information; each video server is connected to the data transmission network through a firewall.

[0074] The video server in the proposed system can provide the resulting video stream by combining video data from the video monitoring tool and a background image from a database corresponding to the selected video monitoring tool, for example, using the Filmora video editor software [Electronic resource. URL: filmora.wondershare.com.ru / . Accessed 03 / 15 / 2025]. The Vizzara-IP-int video server can be used as a video server, for example [Electronic resource. URL: vizzara.ru / products / videoservers. Accessed 03 / 15 / 2025].

[0075] A database server (1.3, Fig. 1) is a specialized hardware and software device that has a large memory capacity for storing database information (DB), and computing power for the operation of a database management system (DBMS) and access control system. The database server must ensure the reliable storage of large amounts of information for a specified period.

[0076] The database server is capable of recording background images received from each video monitoring device, as well as storing all target object graphical images and a list of assigned target object graphical images to each video monitoring device. Additionally, it incorporates security measures, including protection against network interference, unauthorized access, and guaranteed data erasure. Each database server is connected to the network via a firewall.

[0077] For example, the ATLAS.SE-series network data storage can be used as a database server [Electronic resource. URL: atlasdv.ru / solutions-and-products / atlas-se. Accessed on March 15, 2025]. The Postgres PRO Standard software solution can be used as a DBMS [Electronic resource. URL: architect-design.ru / postgres-pro / subd-postgres-pro-standard. Accessed on March 15, 2025].

[0078] The automated operator's workstation (4, Fig. 1) is a personal computer with pre-installed software for controlling video monitoring equipment and working with a database. The URM-32-2M can be used as an automated operator's workstation [Electronic resource. URL: grumant.ru / production / catalog / 00053327. Accessed 03 / 15 / 2025].

[0079] The automated operator workstation is designed with the ability to display processed video data, enter new reference graphic images of target objects and assign them to specific video monitoring tools, as well as control the operation of video monitoring tools and video servers, with the automatic mode of operation of the system being the priority, each automated operator workstation is connected to the data transmission network.

[0080] Additionally, to automate the activities of the video surveillance operator, a control panel for rotary digital video cameras can be installed, for example, a PTZ control panel for Ps-link PS-C3 digital cameras [Electronic resource. URL: ps-link.ru / catalog / videonablyudenie / dop_oborudovanie_dlya_videonablyudeniya / prochee / 1077407 / . Date of access 03 / 15 / 2025].

[0081] A shared monitor (shared information display facility) (5, Fig. 1) is an information display facility designed for simultaneous perception by a large number of people or for the simultaneous display of several sources. [GOST 52870-2007 "Shared information display facilities. Requirements for visual display of information and measurement methods"]. A large screen, video wall, or display capable of displaying a high-resolution image can be used as a shared monitor, for example, a video wall assembled from 55" Samsung VM55B-R displays [Electronic resource. URL: Samsung / com / ru / business / smart-signage / video-wall. Accessed 03 / 15 / 2025].

[0082] Data transmission network (6, Fig. 1)) is a secondary telecommunication network for servicing senders and receivers of data messages [GOST 17657-1979 “Data transmission. Terms and definitions”].

[0083] A data transmission network consists of many heterogeneous communication channels, active and passive network equipment, software and interaction protocols.

[0084] Active network equipment is equipment containing electronic circuits that are powered from the electrical network or other power sources and perform the functions of conversion, signal amplification and others [GOST R 51513-1999 “Electromagnetic compatibility of technical equipment. Equipment for distribution networks of television and radio broadcasting receiving systems. Electromagnetic interference standards, noise immunity requirements and test methods”]. Examples of active network equipment include switches, such as the D-Link DMS-1250-10SP switch [Electronic resource. URL: dlink.ru / ru / products / 1 / 2724.html. Accessed 03 / 23 / 2025], Eltex ESR-3200L routers [Electronic resource. URL: eltexcm.ra / catalog / servisnye-marshrutizatory / servisnye-marshrutizatory-esr / esr-32001.html. Accessed March 23, 2025], D-Link DAP-3666 wireless access points [Electronic resource. URL: dlink.ru / ru / products / 2 / 2715.html. Accessed March 23, 2025]

[0085] Network communication protocols - designed to organize the interaction of various equipment, the most common is the TCP / IP protocol stack.

[0086] In the interests of the video surveillance system, the data transmission network must be able to guarantee the transmission of the required volume of video data within the allowed time.

[0087] The operation of the system can be described using the video data processing method.

[0088] The proposed method can be implemented using the block diagram shown in Figure 2.

[0089] In the process of video surveillance, four main elements actively participate in the exchange of video data: video monitoring equipment (blocks 8 to 26), database server (blocks 27 to 34), video server (blocks 35 to 39), automated operator workstation (blocks 40 to 47).

[0090] In block 8 (Fig. 2), the initial data that will be used in the process of recognizing target objects is preliminarily generated and specified, namely:

[0091] List and reference images of target objects,

[0092] duration of video surveillance (T видео ),

[0093] target object image matching coefficient (K цб ),

[0094] background image change coefficient (K) ф ),

[0095] Initial background refresh time period (T ф ),

[0096] video camera parameters.

[0097] For example, if video surveillance is intended to protect an area from unauthorized entry, then reference images of target objects would require images of vehicles and people to compare detected objects to. An example of target object images is shown in Figure 3.

[0098] The target object image match coefficient is set to increase the reliability and prevent false alarms of the recognition system and is usually determined empirically, taking into account the video surveillance conditions (precipitation, illumination, etc.), the capabilities of the technical detection equipment (the resolution of the video camera's light-sensitive matrix and the distance of the object), as well as the importance of the video surveillance object (a false alarm is better than missing a possible object).

[0099] The background image change ratio represents the difference between two background images and can be expressed as a percentage or a pixel count. The background image change ratio is set to reduce the amount of information transmitted by uninformative changes to the background image (leaf movement in the wind, birds flying, etc.).

[0100] Background image or background refers to the setting, environment, or surroundings where an action, event, or phenomenon occurs. [Electronic resource. Efremova's Explanatory Dictionary. URL: efremova.info / word / fon.html Accessed 03 / 15 / 2025].

[0101] For periodic background image refresh, set the initial background image refresh period. During video monitoring, the background image refresh period may increase or decrease depending on the background image change percentage.

[0102] Video camera parameters are technical characteristics that affect image quality (color or black and white, matrix sensitivity and size, focal length, presence of backlighting and built-in image correction programs, etc. [Electronic resource. URL: video.tcb-spb.ru / sistemy-bezopasnosti-i-videonablyudeniya / 182-kharakteristiki-kamer-videonablyudeniya. Accessed 03 / 15 / 2025]. The above parameters affect the accuracy of object recognition.

[0103] In block 9 (Fig. 2), reference target images assigned to the selected video surveillance device are recorded in the storage device on the video monitoring device. During video monitoring, new target images can be assigned and recorded in the storage device on the operator's command.

[0104] In block 10 (Fig. 2), video data from a digital video camera and a thermal imager are received by means of a microcomputer coupled with a video camera and the video is divided into a sequence of frames (for example, a method for automatically dividing a video into edited frames is known according to patent RU 2682274 C1 dated 04.06.2018, G06T 1 / 00 ​​(2006 / 01), G11B 27 / 00 (2006 / 01), H04N 21 / 845 (2011 / 01), published 19.03.2019, Bulletin No. 8).

[0105] In block 11 (Fig. 2), the first frame, which will be used as a background image, is recorded into a storage device connected to the microcomputer.

[0106] In block 12 (Fig. 2), the first frame to be used as the background image is sent to the database server.

[0107] In block 13 (Fig. 2), the microcomputer starts counting the time that will be used for comparison with the background image refresh period.

[0108] In block 14 (Fig. 2), a search for target objects is performed by means of a microcomputer using preset computer vision algorithms, for which each new frame is sequentially compared with a stored background image, for example, using computer vision algorithms based on the definition of the mean square error (MSE), the structural similarity index (SSIM), histogram comparison, methods based on functions (SIFT, SURF, ORB), methods based on deep learning [Electronic resource. URL: opencv.org. Accessed 15.03.2025], [Electronic resource. URL: tensorflow.org. Accessed 15.03.2025] or programs written in high-level languages, for example, Python, with the OpenCV library and the Tensorflow framework connected.

[0109] Computer vision is the ability of a functional unit to receive, process, and interpret visual data. [GOST 33707 - 2016 (ISO / IEC 2382: 2015) "Information technology. Vocabulary"].

[0110] In block 15 (Fig. 2), check whether the video surveillance time has ended. If not, proceed to block 16; if yes, end the video surveillance process.

[0111] In block 16 (Fig. 2), the comparison images are checked for changes. If there are any, the process proceeds to block 17; if not, the process proceeds to block 14 and continues frame-by-frame comparison.

[0112] In block 17 (Fig. 2), the nature of the detected changes is determined. If the changes can be grouped into an area similar in size and shape to the scalable images of the target objects, then proceed to block 18. An example of this operation is shown in Figure 4. If the changes are in the nature of individually located points (unable to group), then a decision is made to change the background image and proceed to block 24. [An analysis of existing methods for detecting objects in video is described in the dissertation for the degree of Candidate of Technical Sciences by A.V. Pastushkov, "Method and Algorithms for Searching for an Object in a Video Stream," Tomsk, 2017].

[0113] If the changes cannot be grouped into an object, the changes are considered to be related to the background image, and the background image change factor is determined in block 24 (Fig. 2). Expressed as a percentage or units (pix), the factor indicates how much the incoming image from the video camera has changed from the background image stored in the memory device. If the background match factor is greater than or equal to the required value, proceed to block 25. If not, the time counter is reset and the system proceeds to block 13, where the time counter is reset and video surveillance continues.

[0114] In block 25 (Fig. 2), the value of the background update time period is increased by a specified value and they move on to block 12.

[0115] If the changes can be grouped into an object, then in block 18 (Fig. 2), the identified areas of changes, which are the image of the detected object, are compared with the scalable reference images of the target objects stored in the storage device. Methods for searching for objects in images and a comparison of existing object search algorithms are described in the article “Analysis of object search algorithms in images using various modifications of convolutional neural networks” published in the VSU Bulletin, series: System Analysis and Information Technology, 2019, No. 3, pp. 123-137. And also “I see, therefore I exist: a review of Deep Learning in Computer Vision (part 2)” [Electronic resource. URL: habr.com / ru / companies / mipt / articles / 458190 / . Accessed 03 / 15 / 2025]. If the coincidence coefficient of the image of the detected object (K об ) is greater than or equal to the set coefficient of coincidence of the image of the target object, then go to block 19, if not, then to block 21.

[0116] In block 19 (Fig. 2), an identifier is assigned to the matching image of the detected object, and the object's position in the frame is determined. An example of this operation is shown in Figure 4.2. The identifier is transmitted along with the image of the detected object and serves for marking when multiple objects are detected, for tracking the object between video monitoring devices, and as an identifier in the database.

[0117] Information about the position of an object in the frame reduces the processing time of video data on the server [detailed methods are described in the dissertation for the degree of Doctor of Technical Sciences by A.V. Khamukhin “Highly efficient algorithms for semantic processing of video images and control of technical vision instrument complexes” Moscow, 2016].

[0118] In block 20 (Fig. 2), images of only the selected object with the identifier and the position of the starting point in the frame are sent to the video server frame by frame. Software solutions are known that are capable of performing similar operations with video data, for example, Adobe Premiere Pro [Electronic resource. URL: adobe.com / products / premiere.html. Accessed on 15.03.2025] or Video Cutter JS [Electronic resource. URL: video-cutter-js.com. Accessed on 15.03.2025], [Electronic resource. URL: neerc.ifmo.ru / wiki / index.php?title = Location_of_objects_on_an_image. Accessed on 15.03.2025].

[0119] If the image matching coefficient of the detected object (Kob) is less than the set image matching coefficient of the target object, then a notification is sent to the operator’s automated workstation about the appearance of a non-target object (block 21 (Fig. 2)).

[0120] In block 22 (Fig. 2), if a command is received from the operator to transmit an image of a detected non-target object, then proceed to block 23; if not, then continue video surveillance and proceed to block 26.

[0121] In block 23 (Fig. 2), an image of the identified non-target object is sent to the operator’s workstation.

[0122] In block 26 (Fig. 2), they check whether the time to update the background image has come or not; if it has, then they go to block 11; if not, then they go to block 14 and continue the comparison.

[0123] The database server receives a background image from video surveillance equipment (block 27 (Fig. 2)).

[0124] In block 28 (Fig. 2), the received background image is recorded in the database for a specific video monitoring tool.

[0125] In block 29 (Fig. 2), if a new background image has arrived, then go to block 28 and rewrite the background image; if not, then go to block 30.

[0126] In block 30 (Fig. 2), upon request from the video server, a recorded background image is sent from a specific video monitoring tool to the video server (block 37).

[0127] In block 31 (Fig. 2), images of new target objects transmitted from the operator are received, for example, if it is necessary to expand the search and add trucks, or if it is necessary to refine the image of the target object, for example, to search only for blue cars.

[0128] In block 32 (Fig. 2), upon command from the operator, the received images of target objects are recorded in the database.

[0129] In block 33 (Fig. 2), upon command from the operator, which images of target objects need to be added or redistributed are recorded to which video monitoring device (block 9).

[0130] In block 34 (Fig. 2), upon command from the operator, new designated images of target objects are transmitted to the video monitoring equipment.

[0131] In block 35 (Fig. 2), the video server receives a video image of the selected object with an identifier, coordinates and position of the object in the frame from the video monitoring tool.

[0132] In block 36 (Fig. 2), a request is sent to the database server to provide a background image recorded from the same video monitoring device as the received image of the selected object.

[0133] In block 37 (Fig. 2), the requested background image is received from the database server.

[0134] In block 38 (Fig. 2), the final video image is obtained. For this, the video image of the detected object is superimposed on the background image based on the coordinates and position of the object in the frame. For this, the Adobe After Effects software tool can be used [Electronic resource. URL adobe.com / products / aftereffects.html. Accessed 03 / 15 / 2025].

[0135] In block 39 (Fig. 2), the final video image is transmitted to the operator’s workstation or collective display monitor.

[0136] In block 40 (Fig. 2) the operator’s workstation receives the final image from the video server.

[0137] In block 41 (Fig. 2), if the operator needs to enter a new reference image of the target object, then go to block 42; if not, then go to block 40.

[0138] In block 42 (Fig. 2), new reference images of target objects are loaded onto the database server.

[0139] In block 43 (Fig. 2), a command is transmitted to the database server to record which images of target objects need to be distributed to which video monitoring means; an example of distributing images of target objects to video monitoring means is shown in Figure 6.

[0140] In block 44 (Fig. 2), a command is transmitted to the database server (block 31) to transmit the designated images of target objects to the video monitoring equipment.

[0141] In block 45 (Fig. 2), a notification is received from video surveillance equipment about the appearance of a non-target object.

[0142] In block 46 (Fig. 2), if the operator needs to examine the image of the detected non-target object, then go to block 47; if not, then go to block 45.

[0143] In block 47 (Fig. 2), a command is transmitted to the video surveillance device (block 22), from which the notification was received, to provide an image of the detected non-target object.

[0144] Thus, due to the preliminary processing of the video stream in the video monitoring tool and the separate transmission of the background image and video data of the detected objects over the data transmission network, as well as the use of information security tools, a reduction in the volume of video data traffic transmitted over communication channels is ensured while maintaining the image quality of the specified objects and increasing the security of the transmitted information.

Claims

1. A secure video surveillance system comprising a plurality of video monitoring devices, each of which is connected to one or more video servers via communication channels; a video surveillance control subsystem comprising at least one video server, wherein each server is connected to the video monitoring devices assigned to it and other elements of the system via communication channels; a storage subsystem comprising at least one database server, wherein each server is connected to the remaining elements of the system via a communication channel; one or more automated operator workstations with installed software that allows interaction with the video monitoring devices and elements of the control and storage subsystems, characterized in that all elements of the video surveillance system are connected to a data transmission network, which consists of a plurality of heterogeneous communication channels, active and passive network equipment,software and interaction protocols; each video monitoring device is made in a single design, equipped with a rotating mechanism and consists of a digital video camera and a thermal imager, a microcomputer with a computer vision function and a time counter, a storage device in which the designated reference images of target objects are stored and the background image is recorded, security measures, including means of protection against influence from the data transmission network, unauthorized access and a means of guaranteed erasure of information, while the video output of the digital video camera and thermal imager is connected to the video input of the microcomputer, the storage device is connected directly to the microcomputer, the microcomputer itself is configured to generate control signals for all elements of the video monitoring device, for which purpose the first and second outputs of the microcomputer are connected to the inputs of the digital video camera and thermal imager, respectively,the microcomputer has a network interface for connection to the data transmission network for the exchange of information with the other elements of the system; each video server is designed with the ability to superimpose video data about detected objects, received from video monitoring equipment, on a background image received from the database server, and the ability to transmit the resulting image to the automated workstation of the operator, in addition, it has in its composition security means, including means of protection from the influence of the data transmission network, unauthorized access and means of guaranteed erasure of information, each video server is connected to the data transmission network through a firewall; each database server is designed with the ability to record the background images received from each video monitoring equipment, as well as to store all graphic images of target objects and a list,which video monitoring tool is assigned which graphic images of target objects, in addition, it must have in its composition security means, including means of protection against the influence of the data transmission network, unauthorized access and means of guaranteed erasure of information, each database server is connected to the data transmission network through a firewall; the automated operator workstation is designed with the ability to display processed video data, enter new reference graphic images of target objects and assign them to specific video monitoring tools, and also control the operation of video monitoring tools and video servers, while the automatic mode of operation of the system is priority, each automated operator workstation is connected to the data transmission network.

2. The system according to paragraph 1, in which each video monitoring device can be equipped with an additional video camera with a higher resolution light-sensitive matrix.

3. The system according to paragraph 1, in which one or more collective display monitors may additionally be placed, configured with the ability to connect to a data transmission network and display processed video data with a level of detail higher than that on the operator’s automated means.

4. A method for processing video data implemented in a video surveillance system containing a plurality of video monitoring means; a plurality of video servers, wherein each video server services one or more video monitoring means; a plurality of collective display monitors and / or automated operator workstations; characterized in that the following initial data are preliminarily generated and specified: a list of target objects, reference images of target objects, a target object image match coefficient, a background image change coefficient, an initial background image refresh time period, and video camera parameters, in the video monitoring means, reference images of target objects are recorded in a storage device, video data is received from a digital video camera and a thermal imager by means of a microcomputer coupled with the video camera, and the video is divided into a sequence of frames,The first frame is recorded in the storage device and sent to the database server as a background image, which is updated in accordance with a specified time period, a time counter is started, from the second frame, using computer vision algorithms pre-installed in the microcomputer, a search for target objects is carried out, for which each new frame is sequentially compared with the saved background image until the end of the video surveillance time, if there are no changes, then the images continue to be compared frame by frame, if there are changes in the compared images, then the nature of the changes is determined, if the detected changes cannot be grouped into an object, then a decision is made that the changes relate to the background image and the coefficient of changes in the background image is determined, if it is greater than or equal to the set value, then the refresh period of the background image is changed,a new background image is recorded in the storage device of the video monitoring device and sent to the database server, if the ratio of changes in the background image is less, then the images continue to be compared frame by frame, while the background image is not sent, and the time counter is reset to zero, the procedure of comparing incoming frames with the background image continues until the end of the video surveillance time, if the detected changes can be grouped into an object, then the image of the detected object is compared with the scalable reference images of the target objects stored in the storage device of the video surveillance device, if the coefficient of coincidence of the image of the detected object is greater than or equal to the set value, then an identifier is assigned to the object, the position of the object in the frame is determined and images of only the selected object with the identifier and the position of the starting point in the frame are sent to the video server frame by frame,If the match ratio of the image of the detected object is less than the set value, then a notification is sent to the operator's automated workstation about the appearance of a non-target object; if a command is received from the operator to transmit an image of the detected non-target object, then the image of the detected non-target object is sent to the operator's automated workstation; if not, then video surveillance is continued, they check whether the time has come to update the background image; if yes, then they write a new background image to the storage device of the video surveillance device and send it to the database server; if not, then they continue comparing incoming images; the database server receives the background image from the video monitoring device and writes it to the database; when a new background image is received, it is written to the database instead of the previous one; upon request from the video server, a background image is sent to the video server,receive images of new target objects from the operator, record the images of target objects in the database, upon receipt of a command from the operator, record which target object images should be assigned to which video monitoring device and transmit them to the video monitoring device; in the video server, receive a video image of the selected object with an identifier, coordinates and position of the object in the frame from the video monitoring device, send a request to the database server for the provision of a background image corresponding to the video monitoring device, receive a background image from the database server, superimpose the video data of the detected objects on the background image and transmit the resulting video image to the automated operator workstation or collective display monitor; in the automated operator workstation, receive the resulting image from the video server if it is necessary to enter a reference image of a new target object,then they upload it to the database server, send a command to the database server to which video monitoring device which images of target objects should be assigned, send a command to the database server to transfer the assigned images of target objects to the video monitoring devices, receive a notification from the video surveillance devices about the appearance of a non-target object, if the operator needs to view the image of the detected non-target object, then send a command to the video surveillance device to provide an image of the detected non-target object.

5. The method according to paragraph 4, in which, when recognizing an object, if the coefficient of coincidence of the image of the target object is less than the established value, a video camera with higher image clarity and detail is additionally connected and the procedure of comparing the new image of the target object with the standard is repeated.

Images