Multi-camera monitoring video splicing method and device, electronic equipment and medium

Abstract

This application discloses a video stitching method, apparatus, electronic device, and storage medium for multi-camera surveillance, aiming to solve the problems of poor regional adaptability, inaccurate feature matching, lack of multi-dimensional synchronization, single parameter tuning mechanism, and insufficient balance between data storage and security in the prior art, and to achieve high-precision, seamless, and adaptive panoramic stitching of large-scale multi-camera surveillance videos.

Description

Technical Field

[0001] This application relates to the field of video data processing technology, specifically to a video stitching method, device, electronic device, and storage medium for multi-camera surveillance. Background Technology

[0002] In the field of video surveillance, with the rapid development of smart cities, smart security, and smart transportation, the application scenarios of multi-camera monitoring systems are becoming increasingly widespread. From urban roads and commercial complexes to industrial parks and residential communities, seamless wide-area monitoring is required through the collaboration of multiple cameras. Seamlessly stitching video streams from multiple cameras into a single panoramic video stream is the core technology for achieving continuous monitoring of wide-area scenes. This technology not only needs to solve fundamental problems such as time synchronization, geometric distortion, and perspective changes between video streams, but also needs to adapt to new challenges brought about by different installation environments, dynamic scene changes, and complex lighting conditions.

[0003] In existing technologies, multi-camera video stitching is mainly achieved through single-dimensional image correction, feature point detection and matching, and image fusion techniques. Its processing logic is often globally unified, failing to consider differences in the installation environment of different cameras and regional scene characteristics. For example, some technologies only perform uniform time synchronization and geometric correction on the video streams of all cameras, ignoring differences in camera installation height, device spacing, and scene type in different areas. This leads to geometric misalignment of corrected video frames when stitching across regions. Feature point extraction often uses single-dimensional extraction networks, failing to consider both basic edge features and deep semantic features. In dynamic scenes (such as areas with heavy traffic or pedestrian flow) and complex lighting scenes (such as backlighting or nighttime lighting), incomplete feature point extraction and matching errors are prone to occur, resulting in defects such as ghosting, breaks, and obvious seams in the stitched video.

[0004] Furthermore, existing video synchronization technologies mostly achieve synchronization only in the temporal dimension, lacking multi-dimensional synchronization of spatial coordinates and frame rate. This makes it impossible for video frames from different cameras to be accurately aligned in the spatiotemporal dimensions. The parameters of stitching algorithms are mostly preset fixed values. Although some technologies have achieved dynamic adjustment of stitching parameters, this only adjusts the camera's operating parameters in a single dimension, failing to combine this with video preprocessing parameters to form a closed-loop adjustment across the entire chain. This makes it unable to cope with the degradation of stitching quality caused by environmental changes and equipment status fluctuations. Simultaneously, regarding the storage and security of stitched data, existing technologies mostly adopt a globally unified storage method, failing to manage data by monitoring area. This results in low data retrieval efficiency, and the encryption methods are relatively simple, unable to achieve layered encryption for different monitoring areas, making it difficult to meet the high data security requirements of the security monitoring field. Summary of the Invention

[0005] This application provides a video stitching method, electronic device, apparatus, and storage medium for multi-camera surveillance, aiming to solve the problems of poor regional adaptability, inaccurate feature matching, lack of multi-dimensional synchronization, single parameter tuning mechanism, and insufficient balance between data storage and security in the prior art, and to achieve high-precision, seamless, and adaptive panoramic stitching of large-scale multi-camera surveillance videos.

[0006] In a first aspect, embodiments of this application provide a video stitching method for multi-camera surveillance, including: Adaptive preprocessing is performed on the raw surveillance video streams collected by each camera. At the same time, the real-time operating parameters and installation environment parameters of each camera are obtained. Based on the installation environment parameters, the multiple cameras are divided into regions. Then, combined with the real-time operating parameters, the surveillance video streams after the regions are divided are categorized and calibrated and synchronized in multiple dimensions to obtain the categorization calibration and synchronization information. Based on the partition calibration synchronization information, dynamic distortion correction and spatial geometric registration are performed on each monitoring video stream to obtain a registered video frame; wherein, the registered video frame is a monitoring scene image in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration. Based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, multi-scale feature point extraction and hierarchical matching are performed on the registered video frames to obtain accurate feature point pairs between adjacent and cross-regional video frames. Based on a preset adaptive stitching algorithm, cross-regional geometric fusion and scene-based seamless stitching are performed on the precise feature point pairs to obtain a stitched video frame sequence. Then, multi-dimensional image enhancement and adaptive optimization are performed on the stitched video frame sequence to obtain a high-fidelity panoramic video stream. The high-fidelity panoramic video stream is input into the trained multi-dimensional quality assessment model to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, and obtain the comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

[0007] Optionally, in some embodiments of this application, the adaptive preprocessing of the raw surveillance video streams collected by each camera, the acquisition of real-time operating parameters and installation environment parameters of each camera, the division of multiple cameras into regions based on the installation environment parameters, and the partitioning and multi-dimensional synchronization of each surveillance video stream after region division in combination with the real-time operating parameters, to obtain partitioning and synchronization information, includes: The original surveillance video streams captured by each camera are sequentially subjected to adaptive noise reduction, dynamic contrast enhancement, and frame rate unification to complete the adaptive preprocessing of the video stream and obtain the preprocessed surveillance video stream. Data is extracted from the control module and environmental perception module of each camera to obtain the real-time operating parameters and installation environment parameters of each camera. The real-time operating parameters include at least the camera's real-time exposure value, focus parameters, dynamic frame rate, and real-time viewing angle. The installation environment parameters include at least the camera's installation height, area scene type, surrounding lighting conditions, and device spacing. Based on the regional scene type and device spacing in the installation environment parameters, the multiple cameras are divided into several camera monitoring sub-regions, and each sub-region corresponds to a group of cameras for collaborative monitoring. The real-time operating parameters of each camera are input into the preset partition calibration model. The intrinsic dynamic matrix and extrinsic spatial matrix of each monitoring sub-region are generated respectively to complete the partition calibration of each monitoring video stream and obtain the partition calibration information. The partitioning and calibration information of each monitoring sub-area and the time and frame rate information of each camera are input into the preset multi-dimensional synchronization algorithm to synchronize the time, space and frame rate of each monitoring video stream, and adjust the timestamp, spatial coordinates and frame sequence of each monitoring video stream to obtain multi-dimensional synchronization information. The partition calibration information and the multi-dimensional synchronization information are integrated according to the monitoring sub-regions to obtain the partition calibration synchronization information.

[0008] Optionally, in some embodiments of this application, the step of performing dynamic distortion correction and spatial geometric registration on each monitoring video stream based on the partition calibration synchronization information to obtain registered video frames includes: Obtain the partition calibration synchronization information integrated by monitoring sub-regions, and extract the intrinsic dynamic matrix, extrinsic spatial matrix, and multi-dimensional synchronization parameters; Based on the intrinsic dynamic matrix and the real-time exposure value and ambient lighting conditions in the real-time working parameters, dynamic radial and tangential distortion corrections are performed on each monitoring video stream to obtain a distortion-free video frame set; wherein, the intrinsic dynamic matrix includes at least dynamic focal length parameters, real-time optical center parameters, and environmental adaptation distortion coefficients; Based on the external parameter space matrix and multi-dimensional synchronization parameters, and using the preset global three-dimensional coordinate system as a reference, the distortion-free video frame set is subjected to spatial coordinate mapping and geometric registration, and the video frames of each monitoring sub-area are unified into the global three-dimensional coordinate system to obtain the initial registered video frame set. Scene feature calibration points are collected in the global three-dimensional coordinate system. The positions of the feature calibration points in the initial registration video frame set are detected and deviations are analyzed in real time. Based on the deviation analysis results, the extrinsic parameter space matrix is ​​finely adjusted, and the initial registration video frame set is corrected for secondary registration to obtain the registered video frames.

[0009] Optionally, in some embodiments of this application, the hierarchical feature extraction network includes at least a shallow feature extraction sub-network, a mid-level feature extraction sub-network, and a deep feature extraction sub-network; the shallow feature extraction sub-network is used to extract the basic features of edges and corners of video frames, the mid-level feature extraction sub-network is used to extract the texture and contour features of video frames, and the deep feature extraction sub-network is used to extract the target contours and scene semantic features of video frames; the scene-based feature matching strategy includes at least a static scene matching strategy, a dynamic scene matching strategy, and a complex lighting scene matching strategy.

[0010] Optionally, in some embodiments of this application, after the step of performing cross-regional geometric fusion and scene-based seamless stitching on the precise feature point pairs based on a preset adaptive stitching algorithm to obtain a stitched video frame sequence, and then performing multi-dimensional image enhancement and adaptive optimization on the stitched video frame sequence to obtain a high-fidelity panoramic video stream, the method includes: The registered video frames and precise feature point pairs of each monitoring sub-region are associated with the corresponding intrinsic dynamic matrix and extrinsic spatial matrix to obtain cross-region data association results; wherein, the cross-region data association results are used to perform collaborative correction on the monitoring video streams of each sub-region during cross-region stitching; A pre-set intelligent tagging program records the correspondence between the registered video frames, precise feature point pairs, and the stitched video frame sequence, generating a cross-regional stitching relationship mapping map. The cross-regional stitching relationship mapping map is used to find the original camera video stream corresponding to each frame image when tracing back the panoramic video stream. Create a regional data storage folder, and create corresponding data tables in the folder according to the monitored sub-regions. Store the cross-regional data association results and cross-regional splicing relationship mapping map in the corresponding data tables according to the sub-regions. Each data table is converted into a fixed-length character sequence to obtain a character sequence set. The character sequence set is then added to the index of the corresponding data table to construct a regional data table containing a hash index. Obtain the unique device identifier of each camera, match the corresponding basic encrypted character in the preset encrypted character library based on the device identifier, and randomly generate a unique encrypted string in combination with the monitoring sub-area code; The data storage folders and data tables in each region are encrypted using the proprietary encryption string in a layered manner.

[0011] Secondly, embodiments of this application provide a video stitching device for multi-camera surveillance, comprising: The processing module is used to perform adaptive preprocessing on the raw surveillance video streams collected by each camera, and at the same time obtain the real-time working parameters and installation environment parameters of each camera. Based on the installation environment parameters, the multiple cameras are divided into regions, and then combined with the real-time working parameters, the monitoring video streams after the regions are categorized and calibrated and synchronized in multiple dimensions to obtain the categorization calibration and synchronization information. The registration module is used to perform dynamic distortion correction and spatial geometric registration on each monitoring video stream based on the partition calibration synchronization information to obtain registered video frames; wherein, the registered video frames are monitoring scene images in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration processing. The matching module is used to perform multi-scale feature point extraction and hierarchical matching on the registered video frames based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, so as to obtain accurate feature point pairs between adjacent and cross-regional video frames. The stitching module is used to perform cross-regional geometric fusion and scene-based seamless stitching on the precise feature point pairs based on a preset adaptive stitching algorithm to obtain a stitched video frame sequence. Then, the stitched video frame sequence is subjected to multi-dimensional image enhancement and adaptive optimization to obtain a high-fidelity panoramic video stream. The module is used to input the high-fidelity panoramic video stream into the trained multi-dimensional quality assessment model, and to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time to obtain a comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

[0012] Accordingly, this application also provides an electronic device, including a memory, a processor, and a processor program stored in the memory and executable on the processor, wherein the processor executes the program as described in any of the methods above.

[0013] This application also provides a storage medium storing a processor program that, when executed by a processor, implements any of the methods described above.

[0014] This application provides a video stitching method, apparatus, electronic device, and storage medium for multi-camera surveillance. By dividing the multi-camera video stream into regions based on installation environment parameters, it achieves partition calibration and multi-dimensional synchronization of the surveillance video stream, solving the problem of poor regional adaptability in existing technologies with globally unified processing. This makes calibration and synchronization more closely match the scene characteristics of each sub-region. Through dynamic distortion correction and spatial geometric registration based on a global three-dimensional coordinate system, combined with secondary correction of scene feature calibration points, it achieves precise alignment of video frames in spatiotemporal and geometric dimensions, eliminating the effects of viewing angle deviation and distortion from different cameras. Furthermore, it uses a hierarchical feature extraction network to extract multi-scale feature points, combined with a scene-based feature matching strategy to adapt to different scene types, significantly improving performance. This method improves the accuracy of feature point matching in complex dynamic scenes and lighting conditions, avoiding ghosting and breaks in the stitching. Through cross-regional geometric fusion and seamless scene-based stitching using an adaptive stitching algorithm, combined with multi-dimensional image enhancement optimization, a high-fidelity panoramic video stream is obtained, improving the visual quality of the stitched video. Real-time monitoring using a multi-dimensional quality assessment model establishes a dual-closed-loop adjustment mechanism for camera parameters and preprocessing parameters, enabling dynamic parameter adjustment across the entire chain. This allows the stitching system to adapt to environmental changes and equipment status fluctuations, continuously ensuring stitching quality. Simultaneously, through regional data storage, hash index construction, and layered encryption processing, the method improves the efficiency of stitched data retrieval while meeting the high data security requirements of the security monitoring field. The method and system of this application are applicable to various large-scale multi-camera monitoring scenarios such as urban traffic, commercial complexes, and industrial parks, achieving seamless, high-precision, and adaptive panoramic video stitching, enhancing the wide-area monitoring capabilities and intelligence level of the monitoring system. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the video stitching method for multi-camera monitoring provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the video splicing device for multi-camera monitoring provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0019] In this application, the term "exemplary" is used to mean "serving as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0020] This application provides a video stitching method, apparatus, server, and computer-readable storage medium for multi-camera surveillance, which will be described in detail below.

[0021] like Figure 1 As shown, Figure 1This is a schematic diagram of a video splicing system for multi-camera monitoring provided in an embodiment of this application. The system may include multiple terminals 100 and a server 200, which are connected via a network. The server 200 integrates a video splicing device for multi-camera monitoring, such as... Figure 1 In the server, terminal 100 can access server 200.

[0022] In this embodiment, the server 200 is mainly used to acquire a first pet facial image of the pet to be detected; perform facial feature key point detection on the first pet facial image to obtain pet facial feature key points; perform pet identity recognition based on pet facial feature key points to obtain recognition results; if the recognition result is recognition failure, perform image preprocessing on the first pet facial image to obtain a processed target pet facial image; and identify the pet's identity information based on the target pet facial image.

[0023] In this embodiment, the server 200 can be a standalone server, a server network, or a server cluster. For example, the server 200 described in this embodiment includes, but is not limited to, a computer, a network terminal, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing. In this embodiment, communication between the server and the terminal can be achieved through any communication method, including but not limited to mobile communication based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP / IP Protocol Suite (TCP / IP) and User Datagram Protocol (UDP).

[0024] It is understood that the terminal 100 used in this application embodiment can be a device that includes both receiving and transmitting hardware, i.e., a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a terminal may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal 100 may be a smart feeder.

[0025] Next, we will introduce the video stitching method for multi-camera monitoring provided in the embodiments of this application.

[0026] In the embodiments of the video stitching method for multi-camera monitoring in this application, the video stitching device for multi-camera monitoring is used as the execution subject. For simplicity and ease of description, this execution subject will be omitted in subsequent method embodiments. The video stitching device for multi-camera monitoring is applied to a server. The method includes: acquiring a first pet facial image of a pet to be detected; performing facial feature key point detection on the first pet facial image to obtain pet facial feature key points; performing pet identity recognition based on the pet facial feature key points to obtain a recognition result; if the recognition result is a recognition failure, performing image preprocessing on the first pet facial image to obtain a processed target pet facial image; and recognizing the pet's identity information based on the target pet facial image.

[0027] This application provides a video stitching method for multi-camera surveillance, which can be executed by an electronic device or a server. This application example illustrates the method using an electronic device. The electronic device includes a touchscreen display and a processor. The touchscreen display presents a graphical user interface (GUI) and receives user commands applied to the GUI. When the user operates the GUI via the touchscreen, the GUI can control local content on the electronic device or control content on the server side in response to the received commands.

[0028] The video stitching solution for multi-camera monitoring provided in this application achieves partitioned calibration and multi-dimensional synchronization of the monitoring video stream by dividing the multi-camera area based on installation environment parameters. This solves the problem of poor regional adaptability in existing technologies with globally unified processing, making calibration and synchronization more closely match the scene characteristics of each sub-region. Through dynamic distortion correction and spatial geometric registration based on a global three-dimensional coordinate system, combined with secondary correction of scene feature calibration points, precise alignment of video frames in spatiotemporal and geometric dimensions is achieved, eliminating the effects of viewpoint deviation and distortion from different cameras. Furthermore, multi-scale feature point extraction is achieved through a hierarchical feature extraction network, combined with a scene-based feature matching strategy to adapt to different scene types, significantly improving the performance of complex dynamic scenes. The accuracy of feature point matching in scene and lighting conditions avoids ghosting and breaks in the stitching. Through cross-regional geometric fusion and seamless scene-based stitching using an adaptive stitching algorithm, combined with multi-dimensional image enhancement optimization, a high-fidelity panoramic video stream is obtained, improving the visual quality of the stitched video. Real-time monitoring using a multi-dimensional quality assessment model constructs a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters, achieving dynamic parameter adjustment across the entire chain. This allows the stitching system to adapt to environmental changes and equipment status fluctuations, continuously ensuring stitching quality. Simultaneously, through regional data storage, hash index construction, and layered encryption processing, the efficiency of stitched data retrieval is improved while meeting the high data security requirements of the security monitoring field. The method and system of this application are applicable to various large-scale multi-camera monitoring scenarios such as urban traffic, commercial complexes, and industrial parks, enabling seamless, high-precision, and adaptive panoramic video stitching, improving the wide-area monitoring capabilities and intelligence level of the monitoring system.

[0029] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0030] A video stitching method for multi-camera surveillance includes: adaptive preprocessing of the original surveillance video streams collected by each camera, while obtaining the real-time operating parameters and installation environment parameters of each camera, dividing the multiple cameras into regions based on the installation environment parameters, and then combining the real-time operating parameters to perform partition calibration and multi-dimensional synchronization on each surveillance video stream after partitioning, so as to obtain partition calibration and synchronization information. Based on the partition calibration synchronization information, dynamic distortion correction and spatial geometric registration are performed on each monitoring video stream to obtain registered video frames. The registered video frames are monitoring scene images in the same coordinate system obtained after dynamic distortion correction and spatial geometric registration of each monitoring video stream. Based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, multi-scale feature point extraction and hierarchical matching are performed on the registered video frames to obtain precise feature point pairs between adjacent and cross-regional video frames. Based on a preset adaptive stitching algorithm, cross-regional geometric fusion and field fusion are performed on the precise feature point pairs. Seamless stitching of the scene yields a stitched video frame sequence. This sequence is then subjected to multi-dimensional image enhancement and adaptive optimization to obtain a high-fidelity panoramic video stream. The high-fidelity panoramic video stream is input into a trained multi-dimensional quality assessment model to monitor and comprehensively evaluate its stitching and visual quality in real time, resulting in a comprehensive quality assessment result. Based on this result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time operating parameters of each camera, video preprocessing parameters, and the core stitching parameters in the adaptive stitching algorithm.

[0031] Please see Figure 1 , Figure 1 This application provides a flowchart illustrating a video stitching method for multi-camera monitoring. The specific flow of this multi-camera monitoring video stitching method is as follows: Step 101: Perform adaptive preprocessing on the raw monitoring video streams collected by each camera, and at the same time obtain the real-time working parameters and installation environment parameters of each camera. Divide the multiple cameras into regions based on the installation environment parameters, and then combine the real-time working parameters to perform partition calibration and multi-dimensional synchronization on each monitoring video stream after partitioning, so as to obtain partition calibration and synchronization information.

[0032] This step is the basic preprocessing and calibration synchronization stage for video stitching. Its core lies in breaking away from the existing globally unified processing model and performing regional processing based on the camera's installation environment. Simultaneously, it achieves multi-dimensional synchronization of time, space, and frame rate, providing accurate foundational data for subsequent correction and stitching. The specific implementation steps of this step are as follows: S1011. Adaptive Preprocessing of Video Streams. The raw surveillance video streams captured by 50 cameras undergo adaptive denoising, dynamic contrast enhancement, and frame rate unification processing sequentially. Adaptive denoising employs a Gaussian mixture model-based adaptive denoising algorithm, increasing denoising intensity in dynamic target-dense areas such as pedestrian streets and intersections, and decreasing denoising intensity in static areas such as underground parking garages to avoid over-denoising leading to feature point loss. Dynamic contrast enhancement uses an adaptive histogram equalization algorithm, enhancing contrast for backlit main road cameras and underground parking garage cameras with nighttime lighting, while maintaining the original contrast for indoor cameras in commercial pedestrian streets. Frame rate unification unifies the video stream frame rate of all cameras to 25fps. Frames from cameras with frame rates higher than 25fps are extracted, and frames from cameras with frame rates lower than 25fps are interpolated to obtain the preprocessed surveillance video stream, ensuring frame rate consistency across all video streams.

[0033] S1012. Parameter Extraction. Real-time operating parameters are extracted through the control modules of each camera. These parameters include the camera's real-time exposure value, focus parameters, dynamic frame rate, and real-time viewing angle. Each camera's control module has a built-in data acquisition interface that collects and uploads parameters in real time at a frequency of 100ms. Installation environment parameters are extracted through the environmental perception modules (light sensor, position sensor) on each camera. These parameters include the camera's installation height (3m-10m), the type of the area scene (pedestrian street, main road, intersection, underground parking garage), surrounding lighting conditions (strong light, backlight, weak light, supplemental lighting), and the distance between devices (2m-50m). The environmental perception modules collect environmental data at a frequency of 500ms and perform averaging to ensure parameter stability.

[0034] S1013. Regional Division. Based on the regional scene type and device spacing in the installation environment parameters, the K-means clustering algorithm is used to divide the multiple cameras into regional areas. Cameras with the same scene type and a device spacing ≤10m are divided into a monitoring sub-area. Finally, four camera monitoring sub-areas are divided: pedestrian street sub-area (12 cameras), main road sub-area (15 cameras), intersection sub-area (18 cameras), and underground parking garage sub-area (5 cameras). Each sub-area is a group of cameras for collaborative monitoring. Subsequent calibration, synchronization, and correction are all processed according to the sub-area, making the processing strategy more in line with the scene characteristics of each area.

[0035] S1014. Zone Calibration. A pre-defined zone calibration model is constructed. This model is a dynamic calibration model based on camera calibration theory, and calibration is performed separately for each monitored sub-region. The real-time operating parameters of each camera are input into the zone calibration model. Based on the scene characteristics of each monitored sub-region, the model generates an intrinsic dynamic matrix and an extrinsic spatial matrix for each sub-region: the intrinsic dynamic matrix is ​​a dynamic matrix that changes with the real-time operating parameters of the camera, including at least dynamic focal length parameters, real-time optical center parameters, and environmental adaptation distortion coefficients, capable of adapting to real-time changes in camera exposure, focus, and other parameters; the extrinsic spatial matrix is ​​a spatial matrix based on the camera installation position in each sub-region, reflecting the camera's position and orientation in the local coordinate system of the sub-region. After completing the zone calibration of each monitoring video stream, zone calibration information divided by sub-region is obtained, with each sub-region corresponding to an independent set of calibration parameters.

[0036] S1015, Multi-dimensional Synchronization. A pre-defined multi-dimensional synchronization algorithm is constructed, which integrates the Network Time Protocol (NTP), a three-dimensional spatial coordinate mapping algorithm, and a frame rate synchronization algorithm to achieve three-dimensional synchronization of time, space, and frame rate. The partition calibration information of each monitoring sub-region and the time and frame rate information of each camera are input into the multi-dimensional synchronization algorithm: Time synchronization unifies the timestamps of all cameras to the reference clock of the cloud platform through the NTP protocol, and the time synchronization error is controlled within 10ms; Spatial synchronization is based on the extrinsic spatial matrix of each sub-region, mapping the spatial coordinates of the video frames of each camera to the local coordinate system of the sub-region, and then unifying them to the global temporary coordinate system through coordinate transformation to achieve precise spatial alignment; Frame rate synchronization is based on the frame rate unification result of step S1011, adjusting the frame sequence of each video stream to ensure that each camera has a corresponding video frame at the same timestamp. After completing the multi-dimensional synchronization, the multi-dimensional synchronization information of each sub-region is obtained, including the synchronized timestamp, spatial coordinates, and frame sequence parameters.

[0037] S1016. Information Integration. The partition calibration information and multi-dimensional synchronization information of each monitoring sub-region are integrated by sub-region. The intrinsic dynamic matrix and extrinsic spatial matrix of each sub-region are bound with the synchronized time, space, and frame rate parameters to generate partition calibration synchronization information for each sub-region. The partition calibration synchronization information of each sub-region is an independent data packet, which contains all the basic calibration and synchronization parameters required for video stitching in that region. Subsequent steps will be processed based on this data packet.

[0038] Step 102: Based on the partition calibration synchronization information, perform dynamic distortion correction and spatial geometric registration on each monitoring video stream to obtain the registered video frame; wherein, the registered video frame is a monitoring scene image in the same coordinate system obtained after each monitoring video stream has undergone dynamic distortion correction and spatial geometric registration processing.

[0039] This step is the correction and registration stage for video stitching. Its core lies in achieving dynamic distortion correction and spatial geometric registration based on a global 3D coordinate system. Combined with secondary correction using scene feature calibration points, it eliminates camera distortion and spatial viewpoint deviations, unifying all video frames to the same global coordinate system, laying the foundation for subsequent feature point matching and stitching. The specific implementation steps of this step are as follows: S1021. Parameter Extraction. Extract the intrinsic dynamic matrix, extrinsic spatial matrix, and multi-dimensional synchronization parameters (timestamp, spatial coordinates, frame rate) from the partition calibration synchronization information data packets of each monitoring sub-region. Establish a parameter index table by sub-region for easy subsequent retrieval.

[0040] S1022. Dynamic Distortion Correction. Based on the extracted intrinsic dynamic matrix and real-time exposure values ​​and ambient lighting conditions from the real-time operating parameters, dynamic radial and tangential distortion correction is performed on each monitoring video stream. Radial distortion correction employs a dynamic correction algorithm based on polynomial fitting. The correction coefficients are adjusted according to the environment-adaptive distortion coefficients in the intrinsic dynamic matrix, adjusting for different lighting conditions. For example, the radial distortion correction intensity is increased for cameras on main roads under strong light, while the correction intensity is appropriately reduced for cameras in underground parking garages under weak light. Tangential distortion correction employs a dynamic correction algorithm based on the camera imaging model. The correction matrix is ​​dynamically adjusted by combining the camera's real-time exposure values ​​and focus parameters to avoid correction deviations caused by changes in camera parameters. After distortion correction is completed for the video streams of each monitoring sub-region, a set of distortion-free video frames for each sub-region is obtained. The video frames in this set have eliminated the distortion effects caused by the physical characteristics of the lens.

[0041] S1023. Initial Spatial Geometric Registration. A pre-defined global 3D coordinate system is constructed, with the geographical center of the city's core business district as the origin, east as the X-axis, north as the Y-axis, and the vertical ground as the Z-axis. Based on the extracted extrinsic spatial matrix and multi-dimensional synchronization parameters, and using this global 3D coordinate system as a reference, spatial coordinate mapping and geometric registration are performed on the distortion-free video frame sets of each sub-region. First, the spatial coordinates of the video frames in the local coordinate systems of each sub-region are transformed to the global 3D coordinate system. Then, based on the spatial synchronization parameters in the multi-dimensional synchronization parameters, the geometry of the video frames is stretched, translated, and rotated to ensure that the geometry of each video frame is consistent with the geometry of the real-world scene. After completing the initial spatial geometric registration, the video frames of each monitored sub-region are unified into the global 3D coordinate system, resulting in the initially registered video frame set.

[0042] S1024. Secondary Registration and Correction. In the real scene corresponding to the global 3D coordinate system, scene feature calibration points are collected. Feature calibration points are fixed and easily identifiable physical points in the scene, such as the base of a street lamp post in a pedestrian street, the corner of a traffic sign on a main road, the traffic light pillar at an intersection, and the intersection of parking lines in an underground parking garage. At least 20 scene feature calibration points are collected for each monitoring sub-area, and the precise coordinates of each calibration point in the global 3D coordinate system are recorded. The positions of the feature calibration points in the initial registration video frame set are detected and analyzed in real time. The pixel coordinates of the calibration points in the video frame are located by image recognition algorithm, and compared with the real coordinates in the global 3D coordinate system to calculate the deviation value between the pixel coordinates and the real coordinates. If the deviation value exceeds a preset threshold (5 pixels), the extrinsic parameter space matrix is ​​fine-tuned according to the deviation analysis results, and the fine-tuned extrinsic parameter space matrix is ​​reapplied to the initial registration video frame set for secondary registration and correction; if the deviation value does not exceed the preset threshold, the initial registration video frame set remains unchanged. After completing the secondary registration and correction, the registered video frame is obtained. This video frame is a monitoring scene image unified to the global three-dimensional coordinate system. The video frames of each camera are precisely aligned in terms of geometry and spatial position, without any viewpoint deviation or distortion.

[0043] Step 103: Based on the preset hierarchical feature extraction network and scene-based feature matching strategy, perform multi-scale feature point extraction and hierarchical matching on the registered video frames to obtain accurate feature point pairs between adjacent and cross-regional video frames.

[0044] This step is the feature point extraction and matching stage for video stitching. Its core lies in overcoming the limitations of existing single-dimensional feature extraction technologies. It achieves multi-scale feature point extraction through a hierarchical feature extraction network and combines this with a scene-based feature matching strategy to adapt to different scene types. This not only enables feature point matching between adjacent camera video frames but also between video frames across different regions, improving the accuracy and comprehensiveness of feature point matching. The specific implementation steps of this step are as follows: S1031. Construct a hierarchical feature extraction network. The hierarchical feature extraction network is a multi-scale feature extraction network based on a convolutional neural network (CNN), including a shallow feature extraction sub-network, a mid-level feature extraction sub-network, and a deep feature extraction sub-network. These three sub-networks are independent of each other, but their outputs are fused together to achieve multi-scale feature point extraction from basic to deep layers. Shallow feature extraction sub-network: Based on a lightweight CNN architecture, it uses 3×3 convolutional kernels with a stride of 1. It is mainly used to extract basic features such as edges and corners of video frames. This sub-network has a fast extraction speed and can quickly locate basic feature points in video frames, providing a basis for subsequent matching. Mid-level feature extraction sub-network: Based on the first 8 layers of the VGG16 network, it uses a hybrid convolution kernel of 3×3 and 5×5, mainly used to extract mid-level features such as texture and contour of video frames. This sub-network can extract more recognizable texture and contour features on the basis of basic features, and improve the uniqueness of feature points. Deep Feature Extraction Subnetwork: Based on the first 16 layers of the ResNet50 network, a residual connection structure is introduced. It is mainly used to extract deep features such as target contours and scene semantics in video frames. This subnetwork can identify dynamic targets (such as vehicles and pedestrians) and scene semantics (such as roads and buildings) in video frames, and achieve target-level feature point extraction in dynamic scenes.

[0045] The extraction results from the three sub-networks will be fused to generate a multi-scale feature point set, which includes feature points in three dimensions: basic, intermediate, and deep, taking into account both the quantity and recognizability of feature points.

[0046] S1032. Formulate a scenario-based feature matching strategy. The scenario-based feature matching strategy is an adaptive matching strategy based on different scenario types, including a static scenario matching strategy, a dynamic scenario matching strategy, and a complex lighting scenario matching strategy. Different matching strategies are adapted for the four monitoring sub-areas in this embodiment: Static scene matching strategy: Applicable to the underground parking garage sub-area, which is a static scene with few dynamic targets. A fast matching algorithm based on FLANN is adopted, combined with the Euclidean distance of feature points for matching. The matching threshold is set to 0.6 to ensure the speed and accuracy of matching. Dynamic scene matching strategy: Applicable to pedestrian streets, main roads, and intersections, which are dynamic scenes with dense traffic and pedestrian flow. A robust matching algorithm based on RANSAC is adopted. First, preliminary matching is performed through feature descriptors, and then erroneous matching points are filtered out through the RANSAC algorithm. The erroneous matching point filtering rate is ≥95%, ensuring the matching accuracy in dynamic scenes. Matching strategy for complex lighting and shadow scenes: Applicable to the main road sub-area with backlight and the underground parking garage sub-area with nighttime supplemental lighting. These areas have complex lighting and shadow conditions and feature points are easily occluded. The matching algorithm based on feature point gray-level invariant moments is adopted to extract the gray-level invariant moment features of feature points, maintain the recognizability of feature points when the lighting and shadow change, and improve the matching success rate in complex lighting and shadow scenes.

[0047] S1033. Multi-scale Feature Point Extraction. The registered video frames of each monitored sub-region are input into a hierarchical feature extraction network. Feature points are extracted through three sub-networks: shallow, medium, and deep. The extraction results from these three sub-networks are then fused to generate a multi-scale feature point set for each video frame. Each video frame's feature point set contains multi-dimensional feature points, including edges, corners, textures, contours, target contours, and scene semantics, with ≥500 feature points per frame to ensure comprehensive coverage. Simultaneously, feature descriptions are generated for the feature point sets. SIFT feature descriptors are used to generate a 128-dimensional feature description vector for each feature point, which is then used for subsequent feature point matching.

[0048] S1034. Hierarchical Feature Point Matching. Based on the scene type of each monitored sub-region, the corresponding scene-based feature matching strategy is invoked to perform hierarchical matching on the multi-scale feature point set. The matching process is divided into adjacent camera video frame matching and cross-region camera video frame matching. Adjacent camera video frame matching: Feature point matching is performed on video frames of adjacent cameras in the same monitoring sub-area. Preliminary matching is performed based on the similarity of feature description vectors, and then erroneous matching points are filtered out through the corresponding scene-based matching strategy to obtain feature point pairs between adjacent video frames. Cross-regional camera video frame matching: Feature point matching is performed on video frames of cameras in the boundary areas of adjacent monitoring sub-regions, such as cameras at the boundary between main road sub-regions and intersection sub-regions, or cameras at the boundary between pedestrian street sub-regions and main road sub-regions. A cross-scene fusion matching algorithm is used, combined with the scene-based matching strategy of the two sub-regions, to obtain feature point pairs between cross-regional video frames.

[0049] The accuracy of the matched feature point pairs is verified by calculating the matching error. If the matching error is ≤3 pixels, the feature point pair is considered accurate. This process ultimately yields accurate feature point pairs between adjacent and cross-regional video frames within each monitoring sub-region, providing a precise matching benchmark for subsequent video stitching.

[0050] Step 104: Based on the preset adaptive stitching algorithm, perform cross-regional geometric fusion and scene-based seamless stitching on the precise feature point pairs to obtain the stitched video frame sequence. Then, perform multi-dimensional image enhancement and adaptive optimization on the stitched video frame sequence to obtain a high-fidelity panoramic video stream.

[0051] This step is the fusion and optimization stage of video stitching. Its core lies in achieving cross-regional geometric fusion and seamless scene-based stitching, breaking through the limitations of existing single-region stitching technologies. Simultaneously, through multi-dimensional image enhancement and adaptive optimization, the visual quality of the stitched video is improved, resulting in a high-fidelity panoramic video stream. The specific implementation steps of this step are as follows: S1041. Construct an adaptive stitching algorithm. This adaptive stitching algorithm integrates a homography matrix calculation algorithm, a cross-regional geometric fusion algorithm, and a scene-based seamless fusion algorithm. It can adaptively adjust stitching parameters based on the scene characteristics and precise feature point pairs of different monitoring sub-regions. Homography matrix calculation algorithm: Based on accurate feature point pairs, the least squares method is used to calculate the homography matrix between each video frame, reflecting the geometric transformation relationship between video frames and providing a basis for geometric alignment; Cross-regional geometric fusion algorithm: For the boundary areas of different monitoring sub-regions, a multi-matrix fusion geometric alignment algorithm is adopted to unify the homography matrix of each sub-region to the global three-dimensional coordinate system, thereby realizing cross-regional geometric fusion and avoiding geometric misalignment in the boundary areas; Scene-based seamless fusion algorithms include graph cut fusion algorithm, multi-band hybrid fusion algorithm and Poisson fusion algorithm. The fusion algorithm is adaptively selected for different scene types. For example, the graph cut fusion algorithm is used for static scenes, the multi-band hybrid fusion algorithm is used for dynamic scenes, and the Poisson fusion algorithm is used for scenes with complex lighting and shadows.

[0052] S1042. Cross-regional geometric fusion. Precise feature point pairs from each monitoring sub-region are input into the adaptive stitching algorithm. First, a homography matrix calculation algorithm is used to calculate the corresponding homography matrix for each precise feature point pair, obtaining the geometric transformation relationship between video frames. Then, a cross-regional geometric fusion algorithm is used to transform the homography matrices of each monitoring sub-region to a global three-dimensional coordinate system. Cross-regional geometric alignment and fusion are performed on the video frames of each sub-region. Geometric smoothing is applied to the video frames in the boundary areas to eliminate geometric stitching deviations between sub-regions, achieving global geometric alignment of all video frames from all monitoring sub-regions.

[0053] S1043. Scene-based Seamless Stitching. Based on the video frames after cross-regional geometric fusion, the scene-based seamless fusion algorithm in the adaptive stitching algorithm is called according to the scene type of each monitoring sub-area to perform scene-based seamless stitching: For static video frames of the underground parking garage sub-area, a graph cut fusion algorithm is used to find the optimal stitching seam by calculating the energy function between video frames, avoiding obvious stitching seams; for dynamic video frames of pedestrian streets, main roads, and intersections, a multi-band hybrid fusion algorithm is used to perform multi-scale fusion of the stitching area to eliminate ghosting of dynamic targets; for complex lighting and shadow video frames with backlighting and supplementary lighting, a Poisson fusion algorithm is used to maintain the consistency of lighting and color in the stitching area. After completing scene-based seamless stitching, the video frames of all monitoring sub-areas are stitched into a whole to obtain a stitched video frame sequence. This frame sequence is a continuous panoramic video frame without obvious stitching seams or geometric misalignment.

[0054] S1044. Multi-dimensional Image Enhancement. This involves performing multi-dimensional image enhancement processing on the stitched video frame sequence, including four dimensions: color correction, brightness balancing, sharpening enhancement, and contrast optimization. Color correction: A color correction algorithm based on the gray world assumption is used to uniformly correct the colors of video frames in each monitoring sub-area, eliminate color deviations between different cameras, and ensure color consistency of panoramic video. Brightness balance: An adaptive brightness equalization algorithm is used to reduce the brightness in bright areas and increase the brightness in dim areas to ensure the brightness uniformity of the panoramic video and eliminate differences in brightness. Sharpening Enhancement: The Laplacian sharpening algorithm is used to sharpen and enhance the edges and contours of the panoramic video, improving the video's clarity; Contrast optimization: A contrast optimization algorithm based on gamma transform is adopted to adaptively adjust the gamma value according to the grayscale distribution of video frames, thereby improving the visual depth of the video.

[0055] S1045. Adaptive Optimization. An adaptive optimization model is constructed, based on a deep learning algorithm, to perform quality checks on the video frame sequence after multi-dimensional image enhancement. The detection metrics include seam visibility, color consistency, brightness uniformity, sharpness, and ghosting rate. If any detection metric fails to meet a preset threshold, the model adaptively adjusts the image enhancement parameters and re-enhances the image; if all metrics meet the preset threshold, the video frame sequence remains unchanged. After adaptive optimization, a high-fidelity panoramic video stream is obtained. This stream is a seamless panoramic surveillance video covering the city's core business district and surrounding roads, characterized by consistent color, uniform brightness, high sharpness, and no seams or ghosting, achieving the high visual quality required for security monitoring.

[0056] S1046. Cross-regional data management and encryption. To ensure the efficiency and security of retrieval of stitched data, after obtaining the high-fidelity panoramic video stream, the core data in the stitching process is managed across regions and encrypted in layers. The specific steps are as follows: Data association: The registered video frames and precise feature point pairs of each monitoring sub-region are associated with the corresponding intrinsic dynamic matrix and extrinsic spatial matrix to obtain cross-region data association results. These results are used to perform collaborative correction on the monitoring video streams of each sub-region during cross-region stitching to ensure the consistency of the stitching. Generate a mapping map: Through a preset intelligent labeling program, a unique label is added to each video frame and feature point pair, recording the correspondence between the registered video frames, precise feature point pairs and the stitched video frame sequence, generating a cross-regional stitching relationship mapping map. This map is used to quickly find the original camera video stream corresponding to each frame image when tracing back the panoramic video stream, improving the efficiency of tracing the source. Regional storage: Create a regional data storage folder in the cloud platform's storage server, and create corresponding data tables in the folder according to the four monitoring sub-regions. Store the cross-region data association results and cross-region splicing relationship mapping map in the corresponding data tables according to the sub-regions to realize regional data management. Build a hash index: Convert each data table into a fixed-length character sequence (character length is 256 bits) to obtain a character sequence set. Add the character sequence set to the index of the corresponding data table to build a regional data table containing a hash index. The hash index enables fast data retrieval with a retrieval response time of ≤100ms. Generate encrypted strings: Obtain the unique device identifier of each camera, and in the cloud platform's preset encrypted character library (containing 26 English letters and 10 numbers), match the corresponding basic encrypted characters based on the last 6 digits of the device identifier, and then combine them with the unique code of each monitoring sub-area (e.g., the pedestrian street sub-area code is A01), and generate a 128-bit exclusive encrypted string through a random number generation algorithm. Each monitoring sub-area corresponds to a set of exclusive encrypted strings. Layered encryption: Layered encryption is applied to the data storage folders and data tables of each region using dedicated encryption strings. The folders are encrypted using the AES-128 encryption algorithm, and the data tables are encrypted using the MD5 encryption algorithm. This layered encryption ensures the security of security monitoring data and prevents data leakage.

[0057] Step 105: Input the high-fidelity panoramic video stream into the trained multi-dimensional quality assessment model to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, and obtain the comprehensive quality assessment result. Based on the comprehensive quality assessment result, construct a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

[0058] This step is the quality assessment and parameter adjustment stage for video stitching. Its core lies in achieving multi-dimensional real-time monitoring of stitching and visual quality, constructing a full-link dual-closed-loop parameter adjustment mechanism, and overcoming the limitations of existing technologies that adjust only one parameter. This enables coordinated dynamic adjustment of camera operating parameters, video preprocessing parameters, and stitching algorithm parameters, allowing the stitching system to adapt to environmental changes and equipment status fluctuations, continuously ensuring stitching quality. The specific implementation steps of this step are as follows: S1051. Train the multi-dimensional quality assessment model. The multi-dimensional quality assessment model is a hybrid assessment model based on ResNet101 and Support Vector Machine (SVM). The training process of the model is as follows: Construct a training dataset: Collect multi-camera surveillance video stitching samples with different scenes and different stitching quality, with a sample size of ≥100,000. Each sample is labeled with a score for stitching quality indicators (stitching seam visibility, geometric misalignment, ghosting rate) and visual quality indicators (color consistency, brightness uniformity, sharpness, contrast). The score is based on a scale of 1-10, with higher scores indicating better quality. Feature extraction: Input the training dataset into the ResNet101 network to extract the deep features of the video samples and generate feature vectors; Model training: Input the deep feature vector into the SVM classifier, use the splicing quality and visual quality scores as labels to train the model, optimize the model parameters using gradient descent until the model's evaluation error is ≤5%; Model Validation: Construct a test dataset to validate the trained model. The validation accuracy is ≥95%, and the model training is complete.

[0059] The trained multi-dimensional quality assessment model can monitor and comprehensively evaluate the stitching quality and visual quality of panoramic video streams in real time, and output scores for each indicator and a comprehensive score.

[0060] S1052. Real-time Quality Monitoring and Evaluation. The obtained high-fidelity panoramic video stream is input frame by frame into the trained multi-dimensional quality evaluation model. The model monitors the video stream in real time at a frequency of 25fps, outputting individual scores for each frame for stitching quality indicators (stitching seam clarity, geometric misalignment, ghosting rate) and visual quality indicators (color consistency, brightness uniformity, sharpness, contrast). A weighted summation method is then used to calculate the comprehensive score, with a weight of 0.6 for stitching quality indicators and 0.4 for visual quality indicators. The maximum comprehensive score is 10 points. If the comprehensive score is ≥8 points, the stitching quality is considered excellent; if 6 points ≤ comprehensive score < 8 points, the stitching quality is considered good; if the comprehensive score < 6 points, the stitching quality is considered unqualified. All individual scores and the comprehensive score are integrated to obtain the comprehensive quality evaluation result, which is uploaded to the cloud platform's monitoring terminal every 1 second for maintenance personnel to view.

[0061] S1053. Construct a dual-closed-loop adjustment mechanism. Based on the comprehensive quality assessment results, a dual-closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed. This mechanism includes a closed loop for adjusting camera operating parameters and a closed loop for adjusting video preprocessing parameters. The two closed loops are independent of each other but work together to form a complete parameter adjustment system. Camera operating parameter adjustment closed loop: Taking the comprehensive score of stitching quality and visual quality as the control target, and the real-time operating parameters of the camera (real-time exposure value, focus parameters, real-time viewing angle) as the adjustment object, an adjustment closed loop is constructed through PID control algorithm. When the comprehensive score decreases, the operating parameters of the camera are automatically adjusted to improve the video acquisition quality. Video preprocessing parameter adjustment closed loop: Taking the comprehensive score of stitching quality and visual quality as the control target, and video preprocessing parameters (denoising intensity, contrast enhancement coefficient, frame rate adjustment parameters) as the adjustment objects, an adjustment closed loop is constructed through fuzzy control algorithm. When the comprehensive score decreases, the preprocessing parameters are automatically adjusted to improve the video preprocessing quality.

[0062] Both adjustment loops are linked to the adaptive stitching algorithm, and the parameter adjustment results will be fed back to the adaptive stitching algorithm in real time, realizing the coordinated adjustment of the three-dimensional parameters.

[0063] S1054. Dynamic Parameter Adjustment. Based on the comprehensive quality assessment results, the real-time operating parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm are dynamically adjusted through a dual closed-loop adjustment mechanism. If the overall score is ≥8 points (excellent), all parameters will remain unchanged and the system will continue to run with the current parameters; If the overall score is between 6 and 8 (good), then make slight adjustments to the parameters: fine-tune the camera's real-time exposure and focus parameters by ±5%; fine-tune the noise reduction intensity and contrast enhancement coefficient of the video preprocessing by ±10%; and fine-tune the fusion area width of the adaptive stitching algorithm by ±5 pixels. If the overall score is less than 6 points (unqualified), the parameters will be significantly adjusted: the real-time viewing angle and exposure value of the camera will be recalibrated by ±20%; the noise reduction, contrast enhancement and frame rate parameters of video preprocessing will be reset; the homography matrix and fusion parameters of the adaptive stitching algorithm will be recalculated until the overall score is ≥8 points.

[0064] The parameters are adjusted dynamically in real time, with an adjustment response time of ≤500ms, ensuring that the stitching system can quickly adapt to environmental changes (such as sudden changes in lighting and weather) and equipment status fluctuations (such as slight camera shifts and focus deviations), and continuously output high-fidelity panoramic video streams.

[0065] This application provides a video stitching method for multi-camera surveillance. It adaptively preprocesses the original surveillance video streams captured by each camera, simultaneously acquiring the real-time operating parameters and installation environment parameters of each camera. Based on the installation environment parameters, the multiple cameras are divided into regions. Then, combined with the real-time operating parameters, the video streams in each region are categorized and synchronized in multiple dimensions. After obtaining the categorization and synchronization information, dynamic distortion correction and spatial geometric registration are performed on each video stream based on this information to obtain registered video frames. The registered video frames are monitoring scene images in the same coordinate system obtained after dynamic distortion correction and spatial geometric registration of each video stream. Then, based on the preprocessing... A hierarchical feature extraction network and a scene-based feature matching strategy are established to perform multi-scale feature point extraction and hierarchical matching on the registered video frames, obtaining accurate feature point pairs between adjacent and cross-regional video frames. Then, based on a preset adaptive stitching algorithm, cross-regional geometric fusion and scene-based seamless stitching are performed on the accurate feature point pairs to obtain a stitched video frame sequence. The stitched video frame sequence is then subjected to multi-dimensional image enhancement and adaptive optimization to obtain a high-fidelity panoramic video stream. Finally, the high-fidelity panoramic video stream is input into a trained multi-dimensional quality evaluation model to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, obtaining a comprehensive quality evaluation result. Based on the quality assessment results, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm. In the multi-camera monitoring video stitching scheme provided in this application, by dividing the multi-camera into regions based on installation environment parameters, the partitioning and multi-dimensional synchronization of the monitoring video stream are achieved, solving the problem of poor regional adaptability in the global unified processing of existing technologies, making the calibration and synchronization more consistent with the scene characteristics of each sub-region; through dynamic distortion correction and spatial geometric registration based on the global three-dimensional coordinate system, combined with secondary correction of scene feature calibration points, precise alignment of video frames in the spatiotemporal and geometric dimensions is achieved, eliminating the The effects of different camera viewing angle deviations and distortions are mitigated; multi-scale feature point extraction is achieved through a hierarchical feature extraction network, combined with a scene-based feature matching strategy to adapt to different scene types, significantly improving the accuracy of feature point matching in complex dynamic scenes and lighting conditions, avoiding ghosting and breaks in the stitching; through cross-regional geometric fusion and seamless scene-based stitching using an adaptive stitching algorithm, combined with multi-dimensional image enhancement optimization, a high-fidelity panoramic video stream is obtained, improving the visual quality of the stitched video; through real-time monitoring of a multi-dimensional quality assessment model, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed, realizing dynamic parameter adjustment across the entire chain, allowing the stitching system to adapt to environmental changes and equipment status fluctuations, continuously ensuring stitching quality;Meanwhile, by using regional data storage, hash index construction, and layered encryption processing, the efficiency of retrieval of stitched data is improved while meeting the high data security requirements of the security monitoring field. The method and system of this application are applicable to various large-scale multi-camera monitoring scenarios such as urban traffic, commercial complexes, and industrial parks, and can achieve seamless, high-precision, and adaptive panoramic video stitching, enhancing the wide-area monitoring capabilities and intelligence level of the monitoring system.

[0066] To facilitate better implementation of the video splicing method for multi-camera monitoring in this application embodiment, this application embodiment also provides a video splicing device for multi-camera monitoring, wherein the meanings of the terms are the same as those in the video splicing system for multi-camera monitoring described above, and specific implementation details can be found in the description in the system embodiment.

[0067] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a video stitching device for multi-camera monitoring provided in an embodiment of this application. Specifically, the video stitching device for multi-camera monitoring may include a processing module 201, a registration module 202, a matching module 203, a stitching module 204, and a construction module 205, as follows: The processing module 201 is used to perform adaptive preprocessing on the raw monitoring video streams collected by each camera, and at the same time obtain the real-time working parameters and installation environment parameters of each camera. Based on the installation environment parameters, the multiple cameras are divided into regions, and then the monitoring video streams after the regions are divided are calibrated and synchronized in multiple dimensions in combination with the real-time working parameters to obtain the calibrated synchronization information. The registration module 202 is used to perform dynamic distortion correction and spatial geometric registration on each monitoring video stream based on the partition calibration synchronization information to obtain a registered video frame; wherein, the registered video frame is a monitoring scene image in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration processing. The matching module 203 is used to perform multi-scale feature point extraction and hierarchical matching on the registered video frames based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, so as to obtain accurate feature point pairs between adjacent and cross-regional video frames. The stitching module 204 is used to perform cross-regional geometric fusion and scene-based seamless stitching on the precise feature point pairs based on a preset adaptive stitching algorithm to obtain a stitched video frame sequence. Then, the stitched video frame sequence is subjected to multi-dimensional image enhancement and adaptive optimization to obtain a high-fidelity panoramic video stream. The construction module 205 is used to input the high-fidelity panoramic video stream into the trained multi-dimensional quality assessment model, to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, and to obtain a comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

[0068] This application provides a video stitching device for multi-camera monitoring. The processing module 201 performs adaptive preprocessing on the original monitoring video streams collected by each camera, and simultaneously acquires the real-time operating parameters and installation environment parameters of each camera. Based on the installation environment parameters, the multiple cameras are divided into regions. Then, combined with the real-time operating parameters, the video streams after region division are categorized and synchronized in multiple dimensions. After obtaining the categorization and synchronization information, the registration module 202 performs dynamic distortion correction and spatial geometric registration on each monitoring video stream based on the categorization and synchronization information to obtain registered video frames. The registered video frames are monitoring scene images in the same coordinate system obtained after dynamic distortion correction and spatial geometric registration of each monitoring video stream. Then, the matching module 203, based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, performs multi-scale feature point extraction and hierarchical matching on the registered video frames to obtain accurate feature point pairs between adjacent and cross-regional video frames. Next, the stitching module 204, based on a preset adaptive stitching algorithm, performs cross-regional geometric fusion and scene-based seamless stitching on the accurate feature point pairs to obtain a stitched video frame sequence. Then, the stitched video frame sequence undergoes multi-dimensional image enhancement and adaptive optimization to obtain a high-fidelity panoramic video stream. Finally, the construction module 205 inputs the high-fidelity panoramic video stream into a trained multi-dimensional quality evaluation model to perform real-time monitoring and comprehensive evaluation of the stitching quality and visual quality of the high-fidelity panoramic video stream. A comprehensive quality assessment result is obtained. Based on this result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm. In the video stitching scheme for multi-camera monitoring provided in this application, the regional division of the monitoring video stream based on installation environment parameters achieves partition calibration and multi-dimensional synchronization, solving the problem of poor regional adaptability in the existing technology's globally unified processing, making calibration and synchronization more consistent with the scene characteristics of each sub-region. Through dynamic distortion correction and spatial geometric registration based on the global three-dimensional coordinate system, combined with secondary correction of scene feature calibration points, the video frames are synchronized in both spatiotemporal and geometric dimensions. Precise alignment eliminates the effects of perspective deviation and distortion from different cameras; multi-scale feature point extraction is achieved through a hierarchical feature extraction network, combined with a scene-based feature matching strategy to adapt to different scene types, significantly improving the accuracy of feature point matching in complex dynamic scenes and lighting scenes, avoiding stitching ghosting and breaks; cross-regional geometric fusion and scene-based seamless stitching through adaptive stitching algorithms, combined with multi-dimensional image enhancement optimization, yields a high-fidelity panoramic video stream, improving the visual quality of the stitched video; real-time monitoring through a multi-dimensional quality assessment model constructs a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters, realizing dynamic parameter adjustment across the entire chain, allowing the stitching system to adapt to environmental changes and equipment status fluctuations, continuously ensuring stitching quality;Meanwhile, by using regional data storage, hash index construction, and layered encryption processing, the efficiency of retrieval of stitched data is improved while meeting the high data security requirements of the security monitoring field. The method and system of this application are applicable to various large-scale multi-camera monitoring scenarios such as urban traffic, commercial complexes, and industrial parks, and can achieve seamless, high-precision, and adaptive panoramic video stitching, enhancing the wide-area monitoring capabilities and intelligence level of the monitoring system.

[0069] Furthermore, embodiments of this application also provide an electronic device, such as... Figure 3 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically: The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more processor-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 3 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: Processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 302, and by calling data stored in memory 302, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, processor 301 may include one or more processing cores; preferably, processor 301 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles video stitching for wireless multi-camera monitoring. It is understood that the modem processor may not be integrated into processor 301.

[0070] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and video stitching methods for multi-camera monitoring by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0071] The electronic device also includes a power supply 303 that supplies power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0072] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0073] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in the embodiments of this application, the processing 301 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 302 according to the following instructions, and the processing 301 runs the applications stored in the memory 302 to realize various functions, as follows: Adaptive preprocessing is performed on the raw surveillance video streams collected by each camera. At the same time, the real-time operating parameters and installation environment parameters of each camera are obtained. Based on the installation environment parameters, the multiple cameras are divided into regions. Then, combined with the real-time operating parameters, the surveillance video streams after the regions are divided are categorized and calibrated and synchronized in multiple dimensions to obtain the categorization calibration and synchronization information. Based on the partition calibration synchronization information, dynamic distortion correction and spatial geometric registration are performed on each monitoring video stream to obtain a registered video frame; wherein, the registered video frame is a monitoring scene image in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration. Based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, multi-scale feature point extraction and hierarchical matching are performed on the registered video frames to obtain accurate feature point pairs between adjacent and cross-regional video frames. Based on a preset adaptive stitching algorithm, cross-regional geometric fusion and scene-based seamless stitching are performed on the precise feature point pairs to obtain a stitched video frame sequence. Then, multi-dimensional image enhancement and adaptive optimization are performed on the stitched video frame sequence to obtain a high-fidelity panoramic video stream. The high-fidelity panoramic video stream is input into the trained multi-dimensional quality assessment model to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, and obtain the comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

[0074] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0075] This application embodiment achieves partitioned calibration and multi-dimensional synchronization of surveillance video streams by dividing the video stream into regions based on installation environment parameters using multiple cameras. This solves the problem of poor regional adaptability in existing technologies with globally unified processing, making calibration and synchronization more closely match the scene characteristics of each sub-region. Through dynamic distortion correction and spatial geometric registration based on a global three-dimensional coordinate system, combined with secondary correction of scene feature calibration points, precise alignment of video frames in spatiotemporal and geometric dimensions is achieved, eliminating the effects of viewing angle deviation and distortion from different cameras. Furthermore, multi-scale feature point extraction is achieved through a hierarchical feature extraction network, combined with a scene-based feature matching strategy to adapt to different scene types, significantly improving performance in complex dynamic scenes and lighting conditions. The accuracy of feature point matching avoids ghosting and breaks in the stitching. Through cross-regional geometric fusion and seamless scene-based stitching using an adaptive stitching algorithm, combined with multi-dimensional image enhancement optimization, a high-fidelity panoramic video stream is obtained, improving the visual quality of the stitched video. Real-time monitoring using a multi-dimensional quality assessment model constructs a dual-closed-loop adjustment mechanism for camera parameters and preprocessing parameters, achieving dynamic parameter adjustment across the entire chain. This allows the stitching system to adapt to environmental changes and equipment status fluctuations, continuously ensuring stitching quality. Simultaneously, through regional data storage, hash index construction, and layered encryption processing, the efficiency of stitched data retrieval is improved while meeting the high data security requirements of the security monitoring field. The method and system of this application are applicable to various large-scale multi-camera monitoring scenarios such as urban traffic, commercial complexes, and industrial parks, enabling seamless, high-precision, and adaptive panoramic video stitching, enhancing the wide-area monitoring capabilities and intelligence level of the monitoring system.

[0076] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a processor-readable storage medium and loaded and executed by a processor.

[0077] Therefore, embodiments of this application provide a storage medium storing multiple instructions that can be loaded by a processor to execute steps in any of the video stitching methods for multi-camera monitoring provided in embodiments of this application. For example, the instructions can execute the following steps: Adaptive preprocessing is performed on the raw surveillance video streams collected by each camera. At the same time, the real-time operating parameters and installation environment parameters of each camera are obtained. Based on the installation environment parameters, the multiple cameras are divided into regions. Then, combined with the real-time operating parameters, the surveillance video streams after the regions are divided are categorized and calibrated and synchronized in multiple dimensions to obtain the categorization calibration and synchronization information. Based on the partition calibration synchronization information, dynamic distortion correction and spatial geometric registration are performed on each monitoring video stream to obtain a registered video frame; wherein, the registered video frame is a monitoring scene image in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration. Based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, multi-scale feature point extraction and hierarchical matching are performed on the registered video frames to obtain accurate feature point pairs between adjacent and cross-regional video frames. Based on a preset adaptive stitching algorithm, cross-regional geometric fusion and scene-based seamless stitching are performed on the precise feature point pairs to obtain a stitched video frame sequence. Then, multi-dimensional image enhancement and adaptive optimization are performed on the stitched video frame sequence to obtain a high-fidelity panoramic video stream. The high-fidelity panoramic video stream is input into the trained multi-dimensional quality assessment model to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, and obtain the comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

[0078] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0079] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0080] Since the instructions stored in the storage medium can execute the steps in any of the video stitching methods for multi-camera monitoring provided in the embodiments of this application, the beneficial effects that any of the video stitching methods for multi-camera monitoring provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0081] The foregoing has provided a detailed description of a video stitching method, apparatus, electronic device, and storage medium for multi-camera monitoring provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A video stitching method for multi-camera surveillance, characterized in that, The method includes: Adaptive preprocessing is performed on the raw surveillance video streams collected by each camera. At the same time, the real-time operating parameters and installation environment parameters of each camera are obtained. Based on the installation environment parameters, the multiple cameras are divided into regions. Then, combined with the real-time operating parameters, the surveillance video streams after the regions are divided are categorized and calibrated and synchronized in multiple dimensions to obtain the categorization calibration and synchronization information. Based on the partition calibration synchronization information, dynamic distortion correction and spatial geometric registration are performed on each monitoring video stream to obtain a registered video frame; wherein, the registered video frame is a monitoring scene image in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration. Based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, multi-scale feature point extraction and hierarchical matching are performed on the registered video frames to obtain accurate feature point pairs between adjacent and cross-regional video frames. Based on a preset adaptive stitching algorithm, cross-regional geometric fusion and scene-based seamless stitching are performed on the precise feature point pairs to obtain a stitched video frame sequence. Then, multi-dimensional image enhancement and adaptive optimization are performed on the stitched video frame sequence to obtain a high-fidelity panoramic video stream. The high-fidelity panoramic video stream is input into the trained multi-dimensional quality assessment model to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time, and obtain the comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

2. The video stitching method for multi-camera monitoring according to claim 1, characterized in that, The process involves adaptive preprocessing of the raw surveillance video streams captured by each camera, simultaneously acquiring the real-time operating parameters and installation environment parameters of each camera, dividing the multiple cameras into regions based on the installation environment parameters, and then combining the real-time operating parameters to perform partition calibration and multi-dimensional synchronization on each monitored video stream after region division, obtaining partition calibration and synchronization information, including: The original surveillance video streams captured by each camera are sequentially subjected to adaptive noise reduction, dynamic contrast enhancement, and frame rate unification to complete the adaptive preprocessing of the video stream and obtain the preprocessed surveillance video stream. Data is extracted from the control module and environmental perception module of each camera to obtain the real-time operating parameters and installation environment parameters of each camera. The real-time operating parameters include at least the camera's real-time exposure value, focus parameters, dynamic frame rate, and real-time viewing angle. The installation environment parameters include at least the camera's installation height, area scene type, surrounding lighting conditions, and device spacing. Based on the regional scene type and device spacing in the installation environment parameters, the multiple cameras are divided into several camera monitoring sub-regions, and each sub-region corresponds to a group of cameras for collaborative monitoring. The real-time operating parameters of each camera are input into the preset partition calibration model. The intrinsic dynamic matrix and extrinsic spatial matrix of each monitoring sub-region are generated respectively to complete the partition calibration of each monitoring video stream and obtain the partition calibration information. The partitioning and calibration information of each monitoring sub-area and the time and frame rate information of each camera are input into the preset multi-dimensional synchronization algorithm to synchronize the time, space and frame rate of each monitoring video stream, and adjust the timestamp, spatial coordinates and frame sequence of each monitoring video stream to obtain multi-dimensional synchronization information. The partition calibration information and the multi-dimensional synchronization information are integrated according to the monitoring sub-regions to obtain the partition calibration synchronization information.

3. The video stitching method for multi-camera monitoring according to claim 2, characterized in that, The step of performing dynamic distortion correction and spatial geometric registration on each monitored video stream based on the partition calibration synchronization information to obtain registered video frames includes: Obtain the partition calibration synchronization information integrated by monitoring sub-regions, and extract the intrinsic dynamic matrix, extrinsic spatial matrix, and multi-dimensional synchronization parameters; Based on the intrinsic dynamic matrix and the real-time exposure value and ambient lighting conditions in the real-time working parameters, dynamic radial and tangential distortion corrections are performed on each monitoring video stream to obtain a distortion-free video frame set; wherein, the intrinsic dynamic matrix includes at least dynamic focal length parameters, real-time optical center parameters, and environmental adaptation distortion coefficients; Based on the external parameter space matrix and multi-dimensional synchronization parameters, and using the preset global three-dimensional coordinate system as a reference, the distortion-free video frame set is subjected to spatial coordinate mapping and geometric registration, and the video frames of each monitoring sub-area are unified into the global three-dimensional coordinate system to obtain the initial registered video frame set. Scene feature calibration points are collected in the global three-dimensional coordinate system. The positions of the feature calibration points in the initial registration video frame set are detected and deviations are analyzed in real time. Based on the deviation analysis results, the extrinsic parameter space matrix is ​​finely adjusted, and the initial registration video frame set is corrected for secondary registration to obtain the registered video frames.

4. The video stitching method for multi-camera monitoring according to claim 3, characterized in that, The hierarchical feature extraction network includes at least a shallow feature extraction sub-network, a medium feature extraction sub-network, and a deep feature extraction sub-network; the shallow feature extraction sub-network is used to extract the basic features of the edge and corner points of the video frame, the medium feature extraction sub-network is used to extract the texture and contour features of the video frame, and the deep feature extraction sub-network is used to extract the target contour and scene semantic features of the video frame. The scenario-based feature matching strategy includes at least a static scenario matching strategy, a dynamic scenario matching strategy, and a complex lighting and shadow scenario matching strategy.

5. The video stitching method for multi-camera monitoring according to claim 4, characterized in that, The step of performing cross-regional geometric fusion and scene-based seamless stitching on the precise feature point pairs based on the preset adaptive stitching algorithm to obtain a stitched video frame sequence, and then performing multi-dimensional image enhancement and adaptive optimization on the stitched video frame sequence to obtain a high-fidelity panoramic video stream includes: The registered video frames and precise feature point pairs of each monitoring sub-region are associated with the corresponding intrinsic dynamic matrix and extrinsic spatial matrix to obtain cross-region data association results; wherein, the cross-region data association results are used to perform collaborative correction on the monitoring video streams of each sub-region during cross-region stitching; A pre-set intelligent tagging program records the correspondence between the registered video frames, precise feature point pairs, and the stitched video frame sequence, generating a cross-regional stitching relationship mapping map. The cross-regional stitching relationship mapping map is used to find the original camera video stream corresponding to each frame image when tracing back the panoramic video stream. Create a regional data storage folder, and create corresponding data tables in the folder according to the monitored sub-regions. Store the cross-regional data association results and cross-regional splicing relationship mapping map in the corresponding data tables according to the sub-regions. Each data table is converted into a fixed-length character sequence to obtain a character sequence set. The character sequence set is then added to the index of the corresponding data table to construct a regional data table containing a hash index. Obtain the unique device identifier of each camera, match the corresponding basic encrypted character in the preset encrypted character library based on the device identifier, and randomly generate a unique encrypted string in combination with the monitoring sub-area code; The data storage folders and data tables in each region are encrypted using the proprietary encryption string in a layered manner.

6. A video stitching system for multi-camera surveillance, characterized in that, include: The processing module is used to perform adaptive preprocessing on the raw surveillance video streams collected by each camera, and at the same time obtain the real-time working parameters and installation environment parameters of each camera. Based on the installation environment parameters, the multiple cameras are divided into regions, and then combined with the real-time working parameters, the monitoring video streams after the regions are categorized and calibrated and synchronized in multiple dimensions to obtain the categorization calibration and synchronization information. The registration module is used to perform dynamic distortion correction and spatial geometric registration on each monitoring video stream based on the partition calibration synchronization information to obtain registered video frames; wherein, the registered video frames are monitoring scene images in the same coordinate system obtained by each monitoring video stream after dynamic distortion correction and spatial geometric registration processing. The matching module is used to perform multi-scale feature point extraction and hierarchical matching on the registered video frames based on a preset hierarchical feature extraction network and a scene-based feature matching strategy, so as to obtain accurate feature point pairs between adjacent and cross-regional video frames. The stitching module is used to perform cross-regional geometric fusion and scene-based seamless stitching on the precise feature point pairs based on a preset adaptive stitching algorithm to obtain a stitched video frame sequence. Then, the stitched video frame sequence is subjected to multi-dimensional image enhancement and adaptive optimization to obtain a high-fidelity panoramic video stream. The module is used to input the high-fidelity panoramic video stream into the trained multi-dimensional quality assessment model, and to monitor and comprehensively evaluate the stitching quality and visual quality of the high-fidelity panoramic video stream in real time to obtain a comprehensive quality assessment result. Based on the comprehensive quality assessment result, a dual closed-loop adjustment mechanism for camera parameters and preprocessing parameters is constructed to dynamically adjust the real-time working parameters of each camera, video preprocessing parameters, and core stitching parameters in the adaptive stitching algorithm.

7. An electronic device, characterized in that, include: A memory, a processor, and a processor program stored in the memory and executable on the processor, wherein the processor executes the program as a step of the video stitching method for multi-camera surveillance as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The computer processing program stores a video stitching method for multi-camera monitoring as described in any one of claims 1 to 6 that can be loaded by a processor and executed.

Images