Intelligent video monitoring system based on multi-camera data fusion

Abstract

The invention relates to the technical field of security and protection, in particular to an intelligent video monitoring system based on multi-view camera data fusion, which comprises a plurality of multi-view cameras used for shooting a monitoring area; the rotation unit is used for adjusting the shooting angle, the shooting position and the shooting focus of each camera in the multiple cameras; the verification unit is used for determining and comparing picture-in-picture features in the video streams shot by the cameras in the single multi-view camera within the first preset time, and comparing the picture-in-picture features of the video streams shot by the fixed single-view camera before and after replacement within the second preset time; the alarm device is used for receiving the alarm signal and sending out a sound-light alarm; the system provided by the invention avoids misjudgment caused by shooting limitation of a single camera by utilizing a mode of arranging the devices, greatly improves the accuracy of judging whether a picture is modified or not, and effectively improves the safety and accuracy of the intelligent video monitoring system based on multi-view camera data fusion.

Description

Technical Field

[0001] The present invention relates to the field of security technology, and in particular to an intelligent video monitoring system based on multi-camera data fusion. Background Art

[0002] The intelligent video surveillance system based on multi-camera data fusion is an intelligent system that uses multiple cameras to collect video data from different perspectives and uses advanced data fusion technology to comprehensively process these data to achieve more accurate and comprehensive monitoring functions. Compared with the traditional monocular camera monitoring system, it can overcome the limitations of single-viewing angles, provide richer scene information and more accurate target analysis results. Each camera independently collects video streams, which contain scene information from different perspectives, such as the shape, color, and motion trajectory of objects. Data from different perspectives complement each other and are widely used in the security industry. However, many thieves use various technological means to interfere with cameras, so an intelligent video surveillance system is urgently needed to provide better protection.

[0003] China Patent Authorization Announcement No.: CN118840787B discloses a safety warning system and method for an overhead passenger device based on target tracking. The invention discloses a safety warning system and method for an overhead passenger device based on target tracking, including: inputting the current video frame into the YOLOv8 target detection model to obtain a first detection result; obtaining the initial state of the first target frame corresponding to the target object; inputting the frame number difference v, the time interval between each frame and the initial state into the state transition model to predict the predicted position of the corresponding first target frame, and obtaining the second detection result of the i+vth video frame; finding the corresponding second target frame that best matches the first target frame, and using the corresponding second target frame that best matches as the updated target frame; sounding and light alarming according to the category of the first target frame or the updated target frame; repeating the above process until the category of the updated target frame is a non-violation behavior, and ending the cycle; the invention can effectively monitor and warn the riding behavior of the overhead passenger device.

[0004] Chinese Patent Authorization Announcement No.: CN114500871 B discloses a multi-channel video analysis method, device and medium. This invention discloses a multi-channel video analysis method, device and medium. Obtain the first real-time video data corresponding to the multiple lenses of the multi-eye camera respectively; perform picture-in-picture synthesis processing on the multiple first real-time video data, and use the synthesized picture-in-picture video data as the second real-time video data; transmit the second real-time video data to the preset intelligent analysis module to mark the target image in the second real-time video data through the preset intelligent analysis module; encode the marked second real-time video data to obtain compressed video data; receive the second real-time video data request sent by the user, and push the compressed video data to the user. Through the above method, the purpose of analyzing multiple videos at the same time by low computing power devices can be achieved.

[0005] However, the above method has the following problems: when there is a problem with the surveillance video or it is tampered with, the comparison and prediction of the video data fails to provide a good early warning, and simply relying on images to monitor the device cannot effectively protect the surveillance camera from being invaded. Summary of the invention

[0006] To this end, the present invention provides an intelligent video surveillance system based on multi-camera data fusion, which is used to overcome the problems in the prior art that when the surveillance video has problems or is tampered with, the comparison and prediction of video data cannot provide a good early warning, and simply relying on images to monitor the device cannot effectively protect the surveillance camera from intrusion.

[0007] To achieve the above object, the present invention provides an intelligent video monitoring system based on multi-camera data fusion, comprising:

[0008] A plurality of multi-view cameras for photographing the monitoring area;

[0009] A rotation unit, which is connected to the multi-camera, and is used to adjust the shooting angle, shooting position and shooting focus of each camera in the multi-camera, and to extract a rotating monocular camera, adjust the rotating monocular camera to be consistent with the shooting position of the other fixed monocular cameras in the same multi-camera, and select a single fixed monocular camera to replace the rotating monocular camera;

[0010] a verification unit, which is connected to the multi-camera and the rotation unit, respectively, and is used to determine and compare the picture-in-picture features in the video stream captured by each camera in the multi-camera within a first preset time, and determine whether to issue an alarm signal according to the comparison result; and to compare the video stream captured by the rotating monocular camera with the video stream captured by each fixed monocular camera, and determine whether to issue an alarm signal according to the comparison result; and to compare the picture-in-picture features of the video stream captured by the fixed monocular camera before and after the replacement within a second preset time, and determine whether to issue an alarm signal according to the comparison result;

[0011] An alarm device, which is connected to the multi-eye camera and the verification unit respectively, and is used to receive the alarm signal, and in the state of receiving the alarm signal, replace the shooting position of each of the multi-eye cameras shooting the same monitoring area, and issue an audible and visual alarm;

[0012] Among them, the shooting position is the corresponding position area in the monitoring area photographed by the multi-eye camera.

[0013] Furthermore, the verification unit determines a picture-in-picture feature in a video stream captured by each camera in the multi-camera, wherein:

[0014] If the picture taken by a monocular camera is completely contained in the picture taken by another monocular camera, then the feature of the target object existing in the picture taken by the monocular camera is the picture-in-picture feature;

[0015] If the picture taken by the monocular camera partially overlaps with the picture taken by the other monocular camera, the feature of the target object existing in the overlapping part of the picture is the picture-in-picture feature;

[0016] The picture-in-picture features are the visual features and motion features of the same target object that appear simultaneously in two monocular cameras; the visual features include the shape features, color features, and texture features of the target object; the motion features include the movement direction and movement speed of the target object.

[0017] Furthermore, the verification unit compares the picture-in-picture features in the video streams captured by each camera in the multi-camera, wherein:

[0018] If the target object does not move within the first preset time, the verification unit compares the visual features of the target object in the video streams captured by each camera;

[0019] If the target moves within the first preset time, the verification unit compares the motion features of the target in the video streams captured by each camera;

[0020] Among them, the first preset time is positively correlated with the number of monocular cameras included in a single multi-camera.

[0021] Furthermore, the verification unit determines whether to issue the alarm signal according to the comparison result while comparing the visual features of the target object in the video streams captured by each camera, wherein:

[0022] If the visual features are inconsistent, the verification unit sends out the alarm signal.

[0023] Furthermore, the verification unit calculates the movement rate of the target object in the video stream shot by each camera while comparing the movement features of the target object in the video stream shot by each camera, wherein:

[0024] If the movement rates are inconsistent, the verification unit sends out the alarm signal;

[0025] If the movement rates are consistent, the movement rate is compared with a preset rate, and when the movement rate is greater than or equal to the preset rate, the verification unit sends the alarm signal;

[0026] The preset speed is positively correlated with the height of the target object.

[0027] Further, the rotation unit extracts any single monocular camera from the multi-camera, sets the monocular camera as the rotating monocular camera, and adjusts the shooting position of the rotating monocular camera to be consistent with the shooting positions of the remaining fixed monocular cameras in the same multi-camera in a periodic time period.

[0028] The periodic time period is positively correlated with the number of monocular cameras included in a single multi-camera.

[0029] Furthermore, the verification unit compares the video stream captured by the rotating monocular camera with the video stream captured by each of the fixed monocular cameras in the same periodic time period, and determines whether to issue an alarm signal according to the comparison result, wherein:

[0030] If the picture-in-picture features of the target object in the video stream captured by the rotating monocular camera are inconsistent with the picture-in-picture features of the same target object in the video stream captured by the fixed monocular camera, the verification unit sends out the alarm signal.

[0031] Furthermore, the rotation unit selects a single fixed monocular camera and replaces the rotating monocular camera, wherein:

[0032] The rotation unit replaces the rotating monocular camera with the fixed monocular camera while adjusting the shooting position of the rotating monocular camera to be consistent with the fixed monocular camera.

[0033] Furthermore, the verification unit compares the picture-in-picture features of the video streams captured by the fixed monocular camera before and after the replacement within the second preset time, and determines whether to issue an alarm signal according to the comparison result, wherein:

[0034] If the picture-in-picture features of the video streams captured by the fixed monocular camera before and after the replacement within the second preset time are inconsistent, the verification unit sends the alarm signal;

[0035] The second preset time is positively correlated with the number of monocular cameras included in a single multi-camera.

[0036] Furthermore, upon receiving the alarm signal, the alarm device swaps the shooting positions of the multi-eye cameras shooting the same monitored area, emits an audible and visual alarm, and simultaneously shuts down the multi-eye cameras with inconsistent picture-in-picture features.

[0037] Compared with the prior art, the beneficial effect of the present invention lies in that the system of the present invention can verify the video stream captured by the camera from multiple dimensions by comparing the features of the same target object in the pictures captured by different cameras in the multi-camera. Different cameras have different shooting angles, focal lengths, etc., and the acquired target feature information is rich and diverse. In the monitoring scenario, one camera may capture the full front view of the target object, and another may capture local details on the side. By comprehensively comparing the shape, color, texture and other features of the target object under these different perspectives, it is possible to more comprehensively and accurately judge whether the picture conforms to the actual situation, avoid misjudgment caused by the shooting limitations of a single camera, and greatly improve the accuracy of judging whether the picture has been modified. At the same time, the target object features captured by each camera can complement and verify each other. If a certain camera presents a slight deviation in the color of the target object due to light problems, other cameras can provide more accurate color references under different light conditions. Through comparison and correction, it is possible to more reliably identify whether there are abnormal changes in the target object features in the picture, thereby accurately judging the authenticity of the picture, and effectively improving the security and accuracy of the intelligent video surveillance system based on multi-camera data fusion.

[0038] Furthermore, by comparing the motion characteristics of moving targets in video streams captured by different cameras, the accuracy of target motion analysis can be improved. Under normal circumstances, the motion characteristics of moving targets have a certain regularity. By comparing the motion characteristics under different cameras, abnormal behaviors and events that do not conform to the normal pattern can be discovered in time, and sudden acceleration, turning, and stopping of the target can be detected, thereby realizing abnormal detection and early warning. In the field of intelligent security, this comparison helps to quickly discover suspicious behaviors, improve security prevention capabilities, and further enhance the security and accuracy of intelligent video surveillance systems based on multi-camera data fusion.

[0039] Furthermore, the present invention provides a rotating monocular camera, which periodically shoots the same video content with different fixed monocular cameras. When the fixed camera is mainly used to continuously focus on the static and dynamic conditions of a specific area, it may not be sensitive enough to capture some abnormal conditions with subtle changes or intermittent occurrences. The periodic participation of the rotating camera in comparison can bring observation perspectives at different time points and different dynamic conditions. When the rotating monocular camera is periodically compared with the fixed monocular camera, if the fixed camera fails and causes an abnormality in the picture, the problem can be discovered more quickly by comparing the difference between the two pictures. At the same time, the periodic comparison of the rotating monocular camera can be used as a supplement and verification mechanism to ensure that even if the fixed monocular camera has temporary data loss or errors, it can be repaired or restored by comparing the data with the rotating monocular camera, thereby ensuring the integrity of the data of the entire monitoring system, and further improving the security and accuracy of the intelligent video surveillance system based on multi-camera data fusion.

[0040] Furthermore, the present invention can effectively eliminate the monitoring blind spots by alternately replacing the rotating monocular camera and the fixed monocular camera, and swapping the shooting areas of different multi-cameras in the same monitoring area when receiving an alarm signal. Due to the limitations of the installation position and angle, each camera may have a certain visual blind spot, that is, an area that cannot be effectively monitored. By swapping the positions of different cameras, the area originally in the blind spot can be effectively monitored, and the area that was previously monitored can be observed from different angles from the new camera perspective, thereby optimizing the monitoring perspective and coverage, so that the entire monitoring area can be covered more comprehensively, reducing monitoring blind spots, and enriching the monitoring angle. Different cameras have different observation perspectives for the same monitoring area. After swapping the camera, this change in perspective helps to examine the situation in the monitoring area with a new perspective. For some abnormal behaviors or events, it may be difficult to detect from the original camera perspective, but it may become obvious after changing the perspective, further improving the security and accuracy of the intelligent video monitoring system based on multi-camera data fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a structural schematic diagram of an intelligent video monitoring system based on multi-camera data fusion of the present invention;

[0042] Figure 2 A schematic diagram of a picture-in-picture feature of a full-screen embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a picture-in-picture feature with partially overlapping pictures according to an embodiment of the present invention;

[0044] Figure 4 is a flow chart of determining the motion characteristics of a target object according to an embodiment of the present invention;

[0045] Among them, 1, the first monocular camera image; 2, the second monocular camera image; 3, the third monocular camera image; 4, the fourth monocular camera image; 5, the target object. DETAILED DESCRIPTION

[0046] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0048] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0049] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] See also Figure 1 As shown, it is a structural schematic diagram of an intelligent video monitoring system based on multi-camera data fusion of the present invention, an intelligent video monitoring system based on multi-camera data fusion, comprising:

[0051] A plurality of multi-view cameras for photographing the monitoring area;

[0052] A rotation unit, which is connected to the multi-camera, is used to adjust the shooting angle, shooting position and shooting focus of each camera in the multi-camera, and to extract a rotating monocular camera, adjust the rotating monocular camera to be consistent with the shooting position of the other fixed monocular cameras in the same multi-camera, and select a single fixed monocular camera to replace the rotating monocular camera;

[0053] a verification unit, which is connected to the multi-camera and the rotation unit, respectively, and is used to determine and compare the picture-in-picture features in the video stream captured by each camera in a single multi-camera within a first preset time, and determine whether to issue an alarm signal according to the comparison result; and to compare the video stream captured by the rotating monocular camera with the video stream captured by each fixed monocular camera, and determine whether to issue an alarm signal according to the comparison result; and to compare the picture-in-picture features of the video stream captured by the fixed monocular camera before and after the replacement within a second preset time, and determine whether to issue an alarm signal according to the comparison result;

[0054] An alarm device is connected to the multi-camera and the verification unit respectively, and is used to receive an alarm signal, and when receiving the alarm signal, it changes the shooting positions of the multi-cameras shooting the same monitoring area and sends out an audible and visual alarm;

[0055] Among them, the shooting position is the corresponding position area in the monitoring area photographed by the multi-eye camera.

[0056] Specifically, the verification unit determines the picture-in-picture features in the video stream captured by each camera in a single multi-camera, wherein:

[0057] If the picture taken by a monocular camera is completely contained in the picture taken by another monocular camera, the feature of the target object existing in the picture taken by the monocular camera is a picture-in-picture feature;

[0058] If the image captured by a monocular camera partially overlaps with the image captured by another monocular camera, the feature of the target object in the overlapping part of the image is the picture-in-picture feature;

[0059] The picture-in-picture features are the visual features and motion features of the same target object that appear in two monocular cameras at the same time; the visual features include the shape features, color features, and texture features of the target object; the motion features include the movement direction and movement rate of the target object.

[0060] Please cooperate Figure 2 See also Figure 3As shown, they are respectively a schematic diagram of a picture-in-picture feature of a fully-enclosed picture according to an embodiment of the present invention and a schematic diagram of a picture-in-picture feature of a partially-overlapped picture according to an embodiment of the present invention;

[0061] In implementation, the second monocular camera image 2 captured by the second monocular camera is included in the first monocular camera image 1 captured by the first monocular camera, and the target object 5 is captured in both the first monocular camera image 1 and the second monocular camera image 2, then the visual features and motion features of the target object 5 are picture-in-picture features;

[0062] The third monocular camera image 3 taken by the third monocular camera and the fourth monocular camera image 4 taken by the fourth monocular camera partially overlap, and the target object 5 is photographed in the overlapping part, so the visual features and motion features of the target object 5 are picture-in-picture features.

[0063] Specifically, the system of the present invention can verify the video stream captured by the camera from multiple dimensions by comparing the features of the same target object in the pictures captured by different cameras in the multi-camera. Different cameras have different shooting angles, focal lengths, etc., and the acquired target feature information is rich and diverse. In the monitoring scenario, one camera may capture the full front view of the target object, and another may capture local details on the side. By comprehensively comparing the shape, color, texture and other features of the target object under these different perspectives, it is possible to more comprehensively and accurately judge whether the picture conforms to the actual situation, avoid misjudgment caused by the shooting limitations of a single camera, and greatly improve the accuracy of judging whether the picture has been modified. At the same time, the target object features captured by each camera can complement and verify each other. If a certain camera presents a slight deviation in the color of the target object due to light problems, other cameras can provide more accurate color references under different light conditions. Through comparison and correction, it can more reliably identify whether there are abnormal changes in the target object features in the picture, thereby accurately judging the authenticity of the picture, effectively improving the security and accuracy of the intelligent video surveillance system based on multi-camera data fusion.

[0064] Specifically, the verification unit compares the picture-in-picture features in the video streams captured by each camera in a single multi-camera, where:

[0065] If the target object does not move within the first preset time, the verification unit compares the visual features of the target object in the video streams captured by each camera;

[0066] If the target object moves within the first preset time, the verification unit compares the motion characteristics of the target object in the video streams captured by each camera;

[0067] In implementation, the visual feature is the most obvious distinguishing feature of a stationary object in a video stream. If the target object does not move within a first preset time, the target object is determined to be a stationary object. The verification unit compares the visual features of the target object in the video stream captured by each camera.

[0068] The motion feature is the most obvious distinguishing feature of a moving object in a video stream. If the target object moves within a first preset time, the target object is determined to be a moving object. The verification unit compares the motion features of the target object in the video stream captured by each camera.

[0069] Among them, the first preset time is positively correlated with the number of monocular cameras included in a single multi-camera.

[0070] It can be understood that the more monocular cameras a single multi-eye camera includes, the more video streams are captured per unit time, the longer the time for capturing the video stream obtained by the verification unit, and the more accurate the result of judging the target object.

[0071] Optionally, the number of monocular cameras included in a single multi-camera is 2, and the first preset time is 10 seconds;

[0072] A single multi-camera contains 3 monocular cameras, and the first preset time is 15 seconds;

[0073] A single multi-camera contains 6 monocular cameras, and the first preset time is 30 seconds.

[0074] Specifically, the verification unit determines whether to issue an alarm signal based on the comparison result while comparing the visual features of the target object in the video stream captured by each camera.

[0075] If the visual features are inconsistent, the verification unit sends out an alarm signal.

[0076] In implementation, the visual features of the target object include shape features, color features and texture features of the target object; if the target object in the two compared video streams appears yellow in one video stream and red in the other video stream, the verification unit sends an alarm signal;

[0077] If the top shape of the target object in the two compared video streams in one video stream contains a flower, and the flower does not appear on the top of the target object in the other video stream, the verification unit sends an alarm signal;

[0078] It is understandable that the multi-eye camera is set at a position higher than the target object, so that the shape features of the top of the target object will not be blocked by other objects;

[0079] If the target object in the two compared video streams has a grid texture in one video stream and a stripe texture in the other video stream, the verification unit sends out an alarm signal.

[0080] See also Figure 4 As shown in FIG. 1 , it is a flow chart for determining the motion characteristics of a target object according to an embodiment of the present invention. The verification unit calculates the motion rate of the target object in the video stream shot by each camera while comparing the motion characteristics of the target object in the video stream shot by each camera.

[0081] If the movement rates are inconsistent, the verification unit sends out an alarm signal;

[0082] If the movement rates are consistent, the movement rate is compared with the preset rate, and when the movement rate is greater than or equal to the preset rate, the verification unit sends an alarm signal;

[0083] In implementation, the motion characteristics of the target object include the motion direction and motion speed of the target object; different purpose cameras analyze the motion direction of the target object by establishing direction coordinates. If the motion directions of the target object in each video stream are inconsistent, the verification unit sends an alarm signal.

[0084] In the implementation, the camera calculates the motion rate of the target moving in different video streams, and if the motion rates are inconsistent, the verification unit sends an alarm signal;

[0085] If the movement rates are consistent, the movement rate is compared with the preset rate, and the preset rate is set to 2 meters per second; if the movement rate is 4 meters per second, which is greater than the preset rate of 2 meters per second, the verification unit sends an alarm signal.

[0086] The preset rate is positively correlated with the height of the target.

[0087] It is understandable that the target object monitored in security monitoring is a person. The taller the person is, the longer the stride is during normal walking, the faster the walking speed is, and the greater the preset speed is.

[0088] Optionally, the height of the target is 1.6 meters per second and the preset rate is 1.3 meters per second;

[0089] The target height is 1.7 meters per second, and the preset speed is 1.5 meters per second;

[0090] The target height is 1.8 meters per second and the preset rate is 1.7 meters per second.

[0091] Specifically, by comparing the motion characteristics of moving targets in video streams captured by different cameras, the accuracy of target motion analysis can be improved. Under normal circumstances, the motion characteristics of moving targets have a certain regularity. By comparing the motion characteristics under different cameras, abnormal behaviors and events that do not conform to the normal pattern can be discovered in time, and sudden acceleration, turning, and stopping of the target can be detected, thereby realizing abnormal detection and early warning. In the field of intelligent security, this comparison helps to quickly discover suspicious behaviors, improve security prevention capabilities, and further enhance the security and accuracy of intelligent video surveillance systems based on multi-camera data fusion.

[0092] Specifically, the rotation unit extracts any monocular camera from a single multi-camera, sets the monocular camera as the rotating monocular camera, and adjusts the shooting position of the rotating monocular camera to be consistent with the shooting positions of the other fixed monocular cameras in the same multi-camera in a periodic time period.

[0093] In implementation, the rotation unit extracts any monocular camera as the rotating monocular camera, and sets the shooting image range of the rotating monocular camera to be consistent with the shooting image range of the fixed monocular camera.

[0094] The periodic time period is positively correlated with the number of monocular cameras contained in a single multi-camera system.

[0095] It is understandable that the more monocular cameras a single multi-camera contains, the longer the time it takes to perform a matching comparison between the rotating monocular camera and all the fixed monocular cameras. While ensuring the accuracy of each comparison, the comparison time should not be less than 10 seconds, so the cycle time period will increase.

[0096] A single multi-camera contains two monocular cameras, and the cycle time period is 10 seconds;

[0097] A single multi-camera contains three monocular cameras, and the cycle time period is 12 seconds;

[0098] A single multi-camera contains 6 monocular cameras, and the cycle time period is 15 seconds.

[0099] Specifically, the verification unit compares the video streams captured by the rotating monocular camera with the video streams captured by the fixed monocular cameras in the same period, and determines whether to issue an alarm signal according to the comparison result, wherein:

[0100] If the picture-in-picture features of the target object in the video stream shot by the rotating monocular camera are inconsistent with the picture-in-picture features of the same target object in the video stream shot by the fixed monocular camera, the verification unit sends an alarm signal.

[0101] In implementation, frame difference analysis is performed when switching the shooting range of the monocular camera. For the video stream shot by the rotating monocular camera, the difference between adjacent frames is calculated. At the moment of camera rotation, the difference between adjacent frames usually increases significantly, which manifests as sudden changes in picture content, color, brightness, etc. This difference can be quantified by calculating the difference in pixel values, the structural similarity index (SSIM) of the image, and other indicators.

[0102] Specifically, the present invention sets up a rotating monocular camera, which periodically shoots the same video content of different fixed monocular cameras. When the fixed camera is mainly used to continuously pay attention to the static and dynamic conditions of a specific area, it may not be sensitive enough to capture some abnormal conditions with subtle changes or intermittent occurrences. The periodic participation of the rotating camera in comparison can bring observation perspectives at different time points and different dynamic conditions. When the rotating monocular camera is periodically compared with the fixed monocular camera, if the fixed camera fails and causes an abnormality in the picture, the problem can be discovered more quickly by comparing the difference between the two pictures. At the same time, the periodic comparison of the rotating monocular camera can be used as a supplement and verification mechanism to ensure that even if the fixed monocular camera has temporary data loss or errors, it can be repaired or restored by comparing with the data of the rotating monocular camera, thereby ensuring the integrity of the data of the entire monitoring system, and further improving the security and accuracy of the intelligent video surveillance system based on multi-camera data fusion.

[0103] Specifically, the rotation unit selects a single fixed monocular camera and a rotating monocular camera to replace, where:

[0104] The rotation unit replaces the rotating monocular camera with the fixed monocular camera while adjusting the shooting position of the rotating monocular camera to be consistent with the fixed monocular camera.

[0105] In implementation, a fixed monocular camera is randomly selected, the shooting position of the rotating monocular camera is adjusted to be consistent with the fixed monocular camera, and then the fixed monocular camera is set as the rotating monocular camera, and the rotating monocular camera is set as the fixed monocular camera.

[0106] Specifically, the present invention can effectively eliminate the monitoring blind spot by alternately replacing the rotating monocular camera and the fixed monocular camera, and swapping the shooting areas of different multi-cameras in the same monitoring area when receiving an alarm signal. Due to the limitations of the installation position and angle, each camera may have a certain visual blind spot, that is, an area that cannot be effectively monitored. By swapping the positions of different cameras, the area originally in the blind spot can be effectively monitored, and the area that was previously monitored can be observed from different angles from the new camera perspective, thereby optimizing the monitoring perspective and coverage, so that the entire monitoring area can be covered more comprehensively, reducing monitoring blind spots, and enriching the monitoring angle. Different cameras have different observation perspectives for the same monitoring area. After swapping the camera, this change in perspective helps to examine the situation in the monitoring area with a new perspective. For some abnormal behaviors or events, it may be difficult to detect from the original camera perspective, but it may become obvious after changing the perspective, further improving the security and accuracy of the intelligent video monitoring system based on multi-camera data fusion.

[0107] Specifically, the verification unit compares the picture-in-picture features of the video streams captured by the fixed monocular camera before and after the replacement within the second preset time, and determines whether to issue an alarm signal according to the comparison result, wherein:

[0108] If the picture-in-picture features of the video streams captured by the fixed monocular camera before and after the replacement are inconsistent within the second preset time, the verification unit sends an alarm signal;

[0109] In implementation, the second preset time is set to 10 seconds, and the video stream recorded by the fixed monocular camera before replacement within 10 seconds before the replacement moment is compared with the video stream recorded by the fixed monocular camera after replacement within 10 seconds after the replacement moment. If the picture-in-picture features in the two video streams are inconsistent, the verification unit sends an alarm signal.

[0110] The second preset time is positively correlated with the number of monocular cameras included in a single multi-camera.

[0111] It is understandable that, the more monocular cameras a single multi-eye camera includes, the longer the comparison time is, and the second preset time is increased while ensuring the accuracy of the comparison of the video streams before and after the replacement.

[0112] Optionally, the number of monocular cameras included in a single multi-camera is 2, and the second preset time is 3 seconds;

[0113] A single multi-camera contains 3 monocular cameras, and the second preset time is 5 seconds;

[0114] A single multi-camera contains 6 monocular cameras, and the second preset time is 10 seconds.

[0115] Specifically, when the alarm device receives an alarm signal, it switches the shooting positions of the multi-eye cameras shooting the same monitoring area, issues an audible and visual alarm, and simultaneously shuts down the multi-eye cameras with inconsistent picture-in-picture features.

[0116] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent video surveillance system based on multi-camera data fusion, characterized in that: include: A plurality of multi-view cameras for photographing the monitoring area; A rotation unit, which is connected to the multi-camera, and is used to adjust the shooting angle, shooting position and shooting focus of each camera in the multi-camera, and to extract a rotating monocular camera, adjust the rotating monocular camera to be consistent with the shooting position of the other fixed monocular cameras in the same multi-camera, and select a single fixed monocular camera to replace the rotating monocular camera; a verification unit, which is connected to the multi-camera and the rotation unit, respectively, and is used to determine and compare the picture-in-picture features in the video stream captured by each camera in the multi-camera within a first preset time, and determine whether to issue an alarm signal according to the comparison result; and to compare the video stream captured by the rotating monocular camera with the video stream captured by each fixed monocular camera, and determine whether to issue an alarm signal according to the comparison result; and to compare the picture-in-picture features of the video stream captured by the fixed monocular camera before and after the replacement within a second preset time, and determine whether to issue an alarm signal according to the comparison result; An alarm device, which is connected to the multi-eye camera and the verification unit respectively, and is used to receive the alarm signal, and in the state of receiving the alarm signal, replace the shooting position of each of the multi-eye cameras shooting the same monitoring area, and issue an audible and visual alarm; Among them, the shooting position is the corresponding position area in the monitoring area photographed by the multi-eye camera.

2. The intelligent video surveillance system based on multi-camera data fusion according to claim 1 is characterized in that: The verification unit determines a picture-in-picture feature in a video stream captured by each camera in the multi-camera, wherein: If the picture taken by a monocular camera is completely contained in the picture taken by another monocular camera, then the feature of the target object existing in the picture taken by the monocular camera is the picture-in-picture feature; If the picture taken by the monocular camera partially overlaps with the picture taken by the other monocular camera, the feature of the target object existing in the overlapping part of the picture is the picture-in-picture feature; The picture-in-picture features are the visual features and motion features of the same target object that appear simultaneously in two monocular cameras; the visual features include the shape features, color features, and texture features of the target object; the motion features include the movement direction and movement speed of the target object.

3. The intelligent video monitoring system based on multi-camera data fusion according to claim 2 is characterized in that: The verification unit compares the picture-in-picture features in the video streams captured by each camera in the multi-camera, wherein: If the target object does not move within the first preset time, the verification unit compares the visual features of the target object in the video streams captured by each camera; If the target moves within the first preset time, the verification unit compares the motion features of the target in the video streams captured by each camera; Among them, the first preset time is positively correlated with the number of monocular cameras included in a single multi-camera.

4. The intelligent video monitoring system based on multi-camera data fusion according to claim 3 is characterized in that: The verification unit determines whether to issue the alarm signal according to the comparison result while comparing the visual features of the target object in the video streams captured by each camera, wherein: If the visual features are inconsistent, the verification unit sends out the alarm signal.

5. The intelligent video monitoring system based on multi-camera data fusion according to claim 4 is characterized in that: The verification unit calculates the movement rate of the target object in the video stream shot by each camera while comparing the movement features of the target object in the video stream shot by each camera, wherein: If the movement rates are inconsistent, the verification unit sends out the alarm signal; If the movement rates are consistent, the movement rate is compared with a preset rate, and when the movement rate is greater than or equal to the preset rate, the verification unit sends the alarm signal; The preset speed is positively correlated with the height of the target object.

6. The intelligent video monitoring system based on multi-camera data fusion according to claim 5 is characterized in that: The rotation unit extracts any single monocular camera from the multi-camera, sets the monocular camera as the rotation monocular camera, and adjusts the shooting position of the rotation monocular camera to be consistent with the shooting positions of the remaining fixed monocular cameras in the same multi-camera in a periodic time period. The periodic time period is positively correlated with the number of monocular cameras included in a single multi-camera.

7. The intelligent video monitoring system based on multi-camera data fusion according to claim 6 is characterized in that: The verification unit compares the video stream captured by the rotating monocular camera with the video stream captured by each fixed monocular camera in the same periodic time period, and determines whether to issue an alarm signal according to the comparison result, wherein: If the picture-in-picture features of the target object in the video stream captured by the rotating monocular camera are inconsistent with the picture-in-picture features of the same target object in the video stream captured by the fixed monocular camera, the verification unit sends out the alarm signal.

8. The intelligent video monitoring system based on multi-camera data fusion according to claim 7 is characterized in that: The rotation unit selects a single fixed monocular camera and replaces it with the rotating monocular camera, wherein: The rotation unit replaces the rotating monocular camera with the fixed monocular camera while adjusting the shooting position of the rotating monocular camera to be consistent with the fixed monocular camera.

9. The intelligent video monitoring system based on multi-camera data fusion according to claim 8 is characterized in that: The verification unit compares the picture-in-picture features of the video streams captured by the fixed monocular camera before and after the replacement within the second preset time, and determines whether to issue an alarm signal according to the comparison result, wherein: If the picture-in-picture features of the video streams captured by the fixed monocular camera before and after the replacement within the second preset time are inconsistent, the verification unit sends the alarm signal; The second preset time is positively correlated with the number of monocular cameras included in a single multi-camera.

10. The intelligent video monitoring system based on multi-camera data fusion according to claim 9 is characterized in that: When receiving the alarm signal, the alarm device switches the shooting positions of the multi-eye cameras shooting the same monitored area, emits an audible and visual alarm, and simultaneously turns off the multi-eye cameras with inconsistent picture-in-picture features.

Images