A multi-site integrated video surveillance and personnel behavior intelligent analysis system

Abstract

This invention discloses a multi-site integrated video surveillance and intelligent personnel behavior analysis system. This invention relates to the field of scene monitoring technology and solves the problem that relying on manual calibration or simple geometric alignment makes it difficult to accurately identify common areas between cameras. Addressing the complex layout of different sites, the system dynamically adjusts the camera focal length and selects the optimal frame combination to maximize the extraction of effective information from intersecting areas. Even when there are deviations in the camera's monitoring range, it can still ensure that there are no blind spots in key areas through common area identification. The image stitching processing end uses a pixel ratio-based weight calibration mechanism to further improve the visual consistency of the stitched images, providing a high-quality global visual foundation for subsequent personnel behavior analysis.

Description

Technical Field

[0001] This invention relates to the field of scene monitoring technology, specifically to a multi-site integrated video monitoring and personnel behavior intelligent analysis system. Background Technology

[0002] In various scenarios such as industrial plants, commercial parks, and transportation hubs, security control and efficient management rely on real-time monitoring of the entire area and precise analysis of personnel behavior. However, existing multi-site monitoring systems generally suffer from the following technical pain points:

[0003] Monitoring cameras in multiple sites often come from different manufacturers, are deployed in scattered locations, and have different parameters (such as focal length, viewing angle, and resolution), making it difficult to identify the overlapping areas of the monitoring areas of different cameras. Traditional image stitching methods often rely on manual calibration or simple geometric alignment, which makes it difficult to accurately identify common areas between cameras. This often results in problems such as ghosting, discontinuity, or abrupt transitions in the stitched images. In particular, when the pixel features of the overlapping areas are significantly different due to inconsistent focal lengths, the image fusion effect is poor, which can easily create blind spots and affect the integrity of the overall monitoring.

[0004] Furthermore, the complex architectural structures of multiple sites (such as multi-story buildings and overlapping areas) make it difficult for traditional monitoring systems to dynamically optimize probe parameters and accurately calibrate overlapping areas by incorporating architectural spatial characteristics (such as BIM models). This further complicates the integration of monitoring footage and the analysis of personnel behavior. Therefore, there is an urgent need for an integrated system capable of accurately merging monitoring footage from multiple sites and intelligently analyzing personnel behavior to improve the intelligence and precision of security management at multiple sites. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multi-site integrated video surveillance and personnel behavior intelligent analysis system, which solves the problem of difficulty in accurately identifying common areas between probes due to reliance on manual calibration or simple geometric alignment.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multi-site integrated video surveillance and personnel behavior intelligent analysis system, comprising:

[0007] The monitoring integration center performs comprehensive analysis and processing on the monitoring areas associated with different monitoring cameras, identifies common areas associated with different monitoring areas, and merges multiple sets of monitoring images to confirm the splicing area of ​​multiple monitoring areas. Its main execution terminals include:

[0008] The parameter debugging and processing end confirms the location of the monitoring probes within the BIM multi-site building model, identifies the overlapping range of different monitoring probes based on their monitoring ranges, adjusts the focal length of different monitoring probes with overlapping ranges, confirms the frames associated with several different focal lengths, and identifies the overlapping areas from several sets of associated frames.

[0009] Two sets of monitoring probes with overlapping ranges are denoted as an intersecting probe group. The focal length of the two monitoring probes in the intersecting probe group is adjusted to confirm the monitoring images acquired under different focal length values. The monitoring images associated with the two monitoring probes are then combined in pairs to confirm several monitoring image groups.

[0010] Confirm the intersection area of ​​two monitoring screens within the monitoring screen group: identify the different pixel values ​​associated with different pixels in the monitoring screen, and confirm the change characteristics between adjacent pixels based on the confirmed pixel values. Confirm several change characteristics associated with different pixels in the two monitoring screens, and identify whether there is a common change characteristic area in the two monitoring screens. The change characteristics between pixels in different screens within the change characteristic area are consistent. If it exists, record the change characteristic area; if it does not exist, do not record it.

[0011] Confirm the area proportion of the changing characteristic area, and record the confirmed area proportion as the image characteristic of the current monitoring screen group;

[0012] The proportion of the area associated with different monitoring screen groups is confirmed, and the maximum value is selected from the different area proportions. The monitoring screen group associated with the maximum value is recorded as the standard screen group of the cross-probe group, and the changing feature area within the standard screen group is recorded as the cross area.

[0013] The common area calibration end restores the original focal length of monitoring probes with overlapping ranges, records whether there are overlapping areas in the current image, and performs area calibration to identify whether there are common areas within the calibrated areas:

[0014] The monitoring screen corresponding to a single monitoring probe within a standard screen group is recorded as a standard screen, and overlapping areas are marked within the standard screen;

[0015] Record the monitoring image generated by the corresponding monitoring probe under the current focal length monitoring state as the actual image, and confirm whether there is an intersection area in the actual image based on the intersection area marked in the standard image. If there is, record it as the built-in area; if not, stop the processing process.

[0016] The actual images associated with the two monitoring probes in the cross probe group are confirmed, and the associated built-in areas are confirmed simultaneously. The confirmed cross area is used as a reference to identify whether there is a common area between the two built-in areas. If there is, the confirmed common area is directly transmitted to the image splicing processing terminal. If there is no common area, no processing is performed.

[0017] The image stitching processing unit, based on the common areas associated with different monitoring areas, and according to the location of these common areas and the pixel characteristics within different monitoring screens, recalibrates the pixels and completes the image stitching process based on the calibrated pixel values, confirming the stitching area:

[0018] Based on the identified common areas, in the subsequent monitoring process, the different pixel values ​​associated with pixels at the same location are identified, and a weighting factor is determined based on the ratio of these different pixel values. Then, based on the weighting factor, the merged pixel values ​​of the corresponding pixels in the merged area are determined. The pixel values ​​of pixels at the same location are labeled as X1 and X2, with a ratio of (X1:X2). Confirm the merged pixel value after pixel merging processing in the corresponding area;

[0019] Then, the merged pixel values ​​of the same location points are confirmed in turn to complete the image stitching process for the corresponding different monitoring areas, and the stitching area is confirmed.

[0020] The personnel analysis center executes different processing processes in different processing areas to identify any anomalies in the path behavior of corresponding personnel and promptly displays signals. Its main execution terminals include:

[0021] The facial recognition terminal performs facial recognition on relevant personnel appearing in the monitoring area, compares and verifies the recognized faces with the database, and identifies whether the corresponding faces belong to strangers.

[0022] The trajectory analysis module confirms the trajectories of unfamiliar individuals appearing in the spliced ​​area. It identifies the entry and exit points of these individuals within the spliced ​​area, determines the baseline direction, and then confirms the associated trajectories based on this baseline direction to determine if the unfamiliar individuals' behavior is abnormal.

[0023] Confirm the entry point of the stranger within the splicing area, then confirm the disappearance point of the corresponding stranger. Using the entry point as the starting point and the disappearance point as the ending point, confirm the direction of movement of the corresponding stranger and use it as the reference direction.

[0024] Next, the movement routes of unfamiliar personnel in the splicing area are confirmed. The movement routes are divided into several movement points. The movement characteristics of the corresponding movement segments are confirmed from the adjacent movement points. The movement characteristics are the radial distance associated with the movement from the beginning of the previous movement point to the end of the next movement point. The angle between the corresponding movement characteristics and the reference direction is confirmed. If the angle is ≥120°, the corresponding movement segment is recorded as an abnormal segment; otherwise, no marking is performed.

[0025] Once the abnormal segment is confirmed, the proportion of the abnormal segment in the overall route is identified. If the overall proportion is ≥40%, such unfamiliar personnel are recorded as abnormal personnel, and an abnormal signal is generated simultaneously for warning processing, which is available for relevant management personnel to view and facilitate timely response and handling measures. Otherwise, the personnel in the model are marked but no abnormal signal warning is generated.

[0026] This invention provides a multi-site integrated video surveillance and intelligent personnel behavior analysis system. Compared with existing technologies, it has the following advantages:

[0027] This invention uses a BIM building model to locate the position of monitoring probes, and combines focal length adjustment and pixel change features to identify overlapping areas, accurately calibrating the common areas of the monitoring images from different probes. This solves the problem of image fragmentation caused by differences in viewing angle and focal length in traditional multi-probe monitoring. The linkage between parameter adjustment and common area calibration ensures pixel-level calibration of overlapping areas, making the spliced ​​images transition naturally in overlapping areas and avoiding abrupt phenomena such as "ghosting" and "faults". This achieves global integration of monitoring images from multiple sites.

[0028] To address the complex layouts of different sites, the system dynamically adjusts the probe focal length and selects the optimal frame combination to maximize the extraction of effective information in overlapping areas. Even when there are deviations in the probe's monitoring range, it can still ensure that there are no blind spots in key areas through common area identification. The image stitching processing end uses a weight calibration mechanism based on pixel ratios to further improve the visual consistency of the stitched images, providing a high-quality global visual foundation for subsequent personnel behavior analysis.

[0029] By establishing a baseline direction by locating the entry and exit points of strangers, and combining this with quantitative analysis of the radial distance and the angle between the directions, abnormal behaviors such as "wandering aimlessly" and "significant route reversals" can be accurately identified. By using a threshold of abnormal segment percentage (≥40%), high-risk strangers are marked as abnormal individuals and alerts are triggered. This solves the problem of relying on manual monitoring and difficulty in timely detection of abnormal behaviors in traditional surveillance, achieving an upgrade from "passive monitoring" to "proactive early warning," and providing management personnel with a basis for decision-making in quickly handling safety hazards. Attached Figure Description

[0030] Figure 1This is a schematic diagram of the principle framework of the present invention;

[0031] Figure 2 This is a schematic diagram illustrating the existence of the common region in this invention;

[0032] Figure 3 This is a schematic diagram showing the absence of a common region in this invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] First Embodiment

[0035] Please see Figure 1 This application provides a multi-site integrated video surveillance and personnel behavior intelligent analysis system, including a monitoring integration center and a personnel analysis center, wherein the input nodes of the monitoring integration center and the personnel analysis center are electrically connected;

[0036] The monitoring integration center includes a parameter debugging and processing terminal, a common area calibration terminal, and a video splicing and processing terminal. The parameter debugging and processing terminal, the common area calibration terminal, and the video splicing and processing terminal are electrically connected from the output node to the input node. The personnel analysis center includes a face recognition terminal and a movement trajectory analysis terminal. The face recognition terminal and the movement trajectory analysis terminal are electrically connected from the output node to the input node.

[0037] The monitoring integration center performs comprehensive analysis and processing on the monitoring areas associated with different monitoring probes in multiple stations, identifies common areas associated with different monitoring areas, and merges multiple sets of monitoring images based on the identified common areas to confirm the fused image of multiple monitoring areas.

[0038] Methods for image blending include:

[0039] The parameter debugging and processing end confirms the location of the monitoring probes within the BIM multi-site building model, identifies the overlapping range of different monitoring probes based on their monitoring ranges, adjusts the focal length of different monitoring probes with overlapping ranges, confirms the frames associated with several different focal lengths, and identifies the overlapping areas from several sets of associated frames.

[0040] Two sets of monitoring probes with overlapping ranges are denoted as an intersecting probe group. The focal length of the two monitoring probes in the intersecting probe group is adjusted to confirm the monitoring images acquired under different focal length values. The monitoring images associated with the two monitoring probes are then combined in pairs to confirm several monitoring image groups (each monitoring image group contains two images, each from two different monitoring probes within the intersecting probe group):

[0041] Confirm the intersection area of ​​two monitoring screens within the monitoring screen group: Identify the different pixel values ​​associated with different pixels in the monitoring screen, and confirm the change characteristics between adjacent pixels based on the confirmed pixel values ​​(the change characteristics are confirmed from top to bottom and from left to right, that is: if the pixel value of pixel A is 10, the pixel value of pixel B to the right is 20, and the pixel value of pixel C below is 30, then the change characteristic between A and B is 10, and the change characteristic between A and C is 20, which is the pixel value that A needs to change relative to B. If the pixel value of pixel D to the right of B is 10, then the change characteristic between B and D is -10). Confirm several change characteristics associated with different pixels in the two monitoring screens, and identify whether there is a common change characteristic area in the two monitoring screens. The change characteristics between pixels in different screens within the change characteristic area are consistent. If it exists, record the change characteristic area; if it does not exist, do not record it.

[0042] Confirm the area proportion of the change feature area and record the confirmed area proportion as the screen feature of the current monitoring screen group. Specifically, the area proportion is the area proportion of the change feature area in the corresponding monitoring screen. There are two sets of area proportions. By averaging the two sets of area proportions, the associated area proportion can be obtained.

[0043] The proportion of the area associated with different monitoring screen groups is confirmed, and the maximum value is selected from the different area proportions. The monitoring screen group associated with the maximum value is recorded as the standard screen group of the cross-probe group, and the changing feature area within the standard screen group is recorded as the cross area.

[0044] Specifically, the standard image set here is designed to ensure a better image standard when combining and stitching images later, making it easier to integrate and stitch the images together.

[0045] The common area calibration end restores the original focal length of monitoring probes with overlapping ranges, records whether there are overlapping areas in the current image, and performs area calibration. From the calibrated areas, it identifies whether there are common areas. If they exist, it performs the stitching process between subsequent images to confirm the stitching area; if they do not exist, it does not perform any processing.

[0046] The monitoring screen corresponding to a single monitoring probe within a standard screen group is recorded as a standard screen, and overlapping areas are marked within the standard screen;

[0047] Record the monitoring image generated by the corresponding monitoring probe under the current focal length monitoring state as the actual image, and confirm whether there is an intersection area in the actual image based on the intersection area marked in the standard image. If there is, record it as the built-in area; if not, stop the processing process.

[0048] The actual images associated with both monitoring probes in the cross-probe group are confirmed, and the associated built-in areas are simultaneously confirmed. Using the confirmed cross-area as a benchmark, it is determined whether there is a common area between the two built-in areas. If so (e.g.) Figure 2 As shown, in Figure 2 In the image, if there are two dashed arcs representing the edges of different built-in areas, then there must be a common area between the two built-in areas. If this common area exists, it is directly transmitted to the image stitching processing unit. If it does not exist (e.g....), then... Figure 3 If the result is negative, no action will be taken.

[0049] The image stitching processing end, based on the common areas associated with different monitoring areas, and according to the location of the common areas and the pixel characteristics within different monitoring screens, recalibrates the pixels and completes the image stitching process of different monitoring screens based on the calibrated pixel values ​​to obtain the stitched area. The specific processing procedure is as follows:

[0050] Based on the identified common areas, in the subsequent monitoring process, the different pixel values ​​associated with pixels at the same location are identified, and a weighting factor is determined based on the ratio of these different pixel values. Then, based on the weighting factor, the merged pixel value of the corresponding region is determined. Specifically, if pixels at the same location are A or B, with pixel value A being 20 and pixel value B being 10, then A:B = 2:1, where A accounts for two-thirds and B accounts for one-third. The merged pixel value = 20 × two-thirds + 10 × one-third. The pixel values ​​of the pixels at the same location are labeled X1 and X2, with a ratio of (X1:X2). Confirm the merged pixel value after pixel merging processing in the corresponding area;

[0051] Then, the merged pixel values ​​of the same location points are confirmed in turn to complete the image stitching process for the corresponding different monitoring areas, and the stitching area is confirmed.

[0052] Second Embodiment

[0053] In the specific implementation process, compared with the above embodiments, this embodiment mainly focuses on the path analysis process of the corresponding personnel, and identifies whether there are any related anomalies in the movement trajectory associated with the corresponding personnel. Its main execution end is the personnel analysis center.

[0054] The personnel analysis center executes different processing processes in different processing areas to identify whether there are any anomalies in the path behavior of the corresponding personnel and displays the signals in a timely manner.

[0055] Its main execution components include a facial recognition unit and a movement trajectory analysis unit. The facial recognition unit performs facial recognition on relevant personnel appearing in the monitored area, compares the recognized faces with a database to verify whether the corresponding face belongs to an unfamiliar person. The specific method of recognition is as follows:

[0056] The system uses cameras to detect people in the monitored area in real time, automatically selecting and extracting facial regions (excluding background, body and other interference);

[0057] The captured facial images are preprocessed (such as deblurring and lighting correction), and then unique facial features (such as facial contours and eye corner distance) are extracted using algorithms to generate a string of digital feature codes (similar to a "facial ID card").

[0058] The extracted feature codes are compared one by one with the face feature codes stored in the database, and the similarity is calculated (e.g., more than 90% is a high match);

[0059] If the similarity exceeds a set threshold (e.g., 85%), it is determined to be a "known person"; if all comparison results are below the threshold, it is determined to be a "stranger face" and a corresponding alert is triggered (e.g., alarm, anomaly marker).

[0060] Since facial recognition comparison is a common technology, it will not be elaborated on here. The facial recognition terminal simply performs facial recognition to distinguish between strangers and people in the same venue.

[0061] The trajectory analysis module confirms the trajectories of unfamiliar individuals appearing in the spliced ​​area. It identifies the entry and exit points of these individuals within the spliced ​​area, determines the baseline direction, and then confirms the associated trajectories based on this baseline direction to determine if their behavior is abnormal. The specific identification method is as follows:

[0062] Confirm the entry point of the stranger within the splicing area, then confirm the disappearance point of the corresponding stranger. Using the entry point as the starting point and the disappearance point as the ending point, confirm the direction of movement of the corresponding stranger and use it as the reference direction.

[0063] Next, the movement routes of unfamiliar personnel in the splicing area are confirmed. The movement routes are divided into several movement points. The movement characteristics of the corresponding movement segments are confirmed from the adjacent movement points. The movement characteristics are the radial distance associated with the movement from the beginning of the previous movement point to the end of the next movement point. The angle between the corresponding movement characteristics and the reference direction is confirmed. If the angle is ≥120°, the corresponding movement segment is recorded as an abnormal segment; otherwise, no marking is performed.

[0064] Once the abnormal segment is confirmed, the proportion of the abnormal segment in the overall route is identified. If the overall proportion is ≥40%, such unfamiliar personnel are recorded as abnormal personnel, and an abnormal signal is generated simultaneously for warning processing, which is available for relevant management personnel to view and facilitate timely response and handling measures. Otherwise, the personnel in the model are marked but no abnormal signal warning is generated.

[0065] Specifically, when relevant personnel are walking and their routes change drastically, indicating that the person in front of them is walking aimlessly and is also an unfamiliar person, it indicates a significant personnel issue. In this case, the relevant personnel analysis process can be carried out based on the specific route analysis to achieve a better warning and handling effect.

[0066] Some of the data in the above formulas are numerical calculations with dimensions removed, and the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0067] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A multi-site integrated video surveillance and people behavior intelligent analysis system, characterized in that, The application comprises the following steps: Firstly, the monitoring range of different monitoring probes is confirmed through parameter debugging, and the cross range is confirmed, and then the focal length adjustment process is performed to confirm whether there is a cross region between frames associated with different focal lengths, and the specific method is as follows: Two monitoring probes with cross range are recorded as a cross probe group, and the focal length of the two monitoring probes in the cross probe group is adjusted, the monitoring pictures obtained under different focal length values are confirmed, and the monitoring pictures associated with the two monitoring probes are combined two by two to confirm a plurality of monitoring picture groups; Different pixel values associated with different pixel points in the monitoring pictures are identified, and the change characteristics between adjacent pixel points are confirmed based on the confirmed pixel values, a plurality of change characteristics associated with different pixel points in the two monitoring pictures are confirmed, and whether there is a same change characteristic region in the two monitoring pictures is identified, the change characteristics between different pixel points in the change characteristic region are consistent, if there is, the change characteristic region is recorded, and if not, no record is made; The area ratio of the change characteristic region is confirmed, and the confirmed area ratio is recorded as the picture characteristic of the current monitoring picture group; The area ratios associated with different monitoring picture groups are confirmed, and the maximum value is selected from the different area ratios, and the monitoring picture group associated with the maximum value is recorded as the standard picture group of the cross probe group, and the change characteristic region in the standard picture group is recorded as the cross region; Secondly, the pictures of the cross region are recorded through the common region calibration end, and whether there is a common region in the recorded cross region is identified, and the specific method is as follows: The monitoring picture corresponding to a single monitoring probe in the standard picture group is recorded as a standard picture, and the cross region in the standard picture is calibrated; The monitoring picture generated by the corresponding monitoring probe under the current focal length monitoring state is recorded as an actual picture, and whether there is a cross region in the actual picture is confirmed according to the calibrated cross region in the standard picture, if there is, it is recorded as an embedded region, if not, the processing process is stopped; The actual pictures associated with the two monitoring probes in the cross probe group are confirmed, the embedded regions associated with the two monitoring probes are confirmed synchronously, and whether there is a common region between the two embedded regions is identified based on the confirmed cross region, if there is, the confirmed common region is directly transmitted to the picture splicing processing end, if not, no processing is performed; Finally, the monitoring pictures with common regions are spliced and processed through the picture splicing processing end, the pixel calibration is performed again according to the pixel characteristics in different monitoring pictures, and the splicing processing process of the monitoring region is completed according to the pixel values after the calibration processing; The personnel analysis center executes different processing processes in different processing regions, identifies whether the path behavior of the corresponding personnel has related abnormalities, and displays signals in time. The specific method for confirming the splicing region by the picture splicing processing end is as follows:

2. The multi-site integrated video surveillance and people behavior intelligent analysis system of claim 1, wherein, ​ According to the identified common region, in the subsequent monitoring process, the different pixel values associated with the same position pixel points are identified, and according to the ratio relationship of the different pixel values, the weight factor is identified, and then according to the weight factor, the merged pixel value of the corresponding region pixel point after merging is identified. The pixel values of the same position pixel points are marked as X1 and X2, and the ratio relationship is (X1:X2). Adopting: , the merged pixel value of the corresponding region pixel point after merging processing is identified; And the same position point of the merging pixel value is confirmed in turn, the picture splicing process corresponding to different monitoring areas is completed, and the splicing area is confirmed.

3. The multi-site integrated video surveillance and people behavioral intelligent analysis system of claim 1, wherein, The personnel analysis center comprises: The face recognition end performs face recognition on the relevant personnel appearing in the monitoring area, compares and verifies the recognized face with the database, and identifies whether the corresponding face is a stranger.

4. The multi-site integrated video surveillance and people behavioral intelligent analysis system of claim 3, wherein, The personnel analysis center further comprises: The behavior trajectory analysis end confirms the trajectory of the stranger appearing in the splicing area, confirms the entry point and the disappearance point of the corresponding stranger in the splicing area, confirms the reference direction, and then confirms the behavior characteristics associated in the marching process according to the reference direction, and identifies whether the behavior of the stranger is abnormal.

5. The multi-site integrated video surveillance and people behavioral intelligent analysis system of claim 4, wherein, The specific way of the behavior trajectory analysis end for identifying the abnormal behavior of the stranger is: Confirm the entry point of the stranger in the splicing area, and then confirm the disappearance point of the corresponding stranger, take the entry point as the starting point and the disappearance point as the end point, confirm the marching direction of the corresponding stranger, and take it as the reference direction; Then, the marching route of the stranger in the splicing area is confirmed, the marching route is divided into a plurality of marching points, the marching characteristics of the corresponding marching section are confirmed from the adjacent marching points, the marching characteristics are the marching vectors associated from the previous marching point in the adjacent marching points to the next group of marching points, the included angle between the corresponding marching characteristics and the reference direction is confirmed, if the included angle is greater than or equal to 120°, the corresponding marching section is recorded as an abnormal section, otherwise, no marking is performed; After the abnormal section is confirmed, the overall proportion of the abnormal section in the entire marching route is identified, if the overall proportion is greater than or equal to 40%, the stranger is recorded as an abnormal person, and an abnormal signal is generated simultaneously for warning processing, for the relevant management personnel to check.

6. The multi-site integrated video surveillance and people behavioral intelligent analysis system of claim 5, wherein, If the overall proportion is less than 40%, the model personnel is marked but no abnormal signal warning is generated.

Images