A video monitoring multi-camera screen splicing method and system

Abstract

The application provides a video monitoring multi-camera screen splicing method and system, relates to the technical field of video splicing, and obtains a first video stream collected by multiple cameras deployed in an open urban area; hardware encoding and decoding processing is performed on the video stream to generate a second video stream; distortion correction is performed on each frame of video picture in the second video stream through a pre-constructed distortion correction model to obtain a third video stream; a space coordinate system is established to align and register each frame of video picture in the third video stream to generate a fourth video stream; a semantic alignment model is used to identify the respective target motion paths of personnel and vehicles in the fourth video stream, and the central positions of two target motion paths and the field of view ranges of multiple cameras in the fourth video stream are adjusted to obtain a fifth video stream, so that a panoramic monitoring picture is further generated, and the geometric consistency and target trajectory continuity of multi-camera video splicing in a complex urban scene are improved.

Description

Technical Field

[0001] This application relates to the field of video stitching technology, and in particular to a method and system for stitching together images from multiple video surveillance cameras. Background Technology

[0002] In intelligent security and traffic management in urban open areas, there is an urgent need to efficiently fuse video footage from multiple cameras to create a panoramic monitoring view with wide coverage and continuous perspective. Such scenarios require systems capable of automatically handling image distortion, spatial positional deviations, and the continuity of target motion across cameras from different viewpoints, thereby providing a high-quality, seamlessly stitched panoramic video stream for subsequent intelligent analysis.

[0003] In existing technologies, a typical approach employs an image stitching method based on feature point matching. This method first extracts stable corner points or edge features from each video frame and then uses geometric transformations to project multiple images onto a unified plane for fusion. However, this approach is susceptible to occlusion, dynamic target interference, and lens distortion in complex urban scenes, leading to feature matching failures or misalignments, making it difficult to guarantee semantic consistency at the stitching boundaries. Furthermore, it does not consider the correlation of target motion trajectories between different cameras, resulting in breaks or ghosting when people or vehicles cross the field of view, failing to meet the spatiotemporal continuity requirements of high-precision panoramic monitoring. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for stitching together images from multiple video surveillance cameras, in order to solve the problems of feature matching misalignment and target trajectory breakage caused by occlusion, dynamic interference and distortion in the prior art.

[0005] To address the aforementioned technical problems, in a first aspect, this application provides a method for stitching together images from multiple video surveillance cameras, comprising:

[0006] Acquire the first video stream captured by multiple cameras deployed in open urban areas;

[0007] The first video stream is subjected to hardware encoding and decoding processing to generate a second video stream;

[0008] The third video stream is obtained by correcting the distortion of each frame in the second video stream using a pre-built distortion correction model.

[0009] A spatial coordinate system is established, and the video frames in the third video stream are aligned and registered based on the spatial coordinate system to generate a fourth video stream;

[0010] The semantic alignment model is used to identify the target motion paths of people and vehicles in the fourth video stream, and the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras are adjusted to obtain the fifth video stream;

[0011] Identify overlapping and non-overlapping areas between each frame of the fifth video stream to generate a panoramic monitoring image.

[0012] Optionally, distortion correction is performed on each frame of the second video stream using a pre-built distortion correction model to obtain a third video stream, including:

[0013] Obtain the lens parameters and actual deployment data of the multiple cameras, and mark fixed marker points in each frame of the second video stream;

[0014] Based on the location distribution of the fixed marker points, the region division module of the distortion correction model divides the key and non-key regions in each frame of video into areas affected by lens distortion.

[0015] Based on the lens parameters, the distortion patterns of key and non-key areas in each frame of video are analyzed by the distortion correction model's pattern analysis module. Combined with the actual deployment data, the first compensation coefficient for different key areas is determined, and the second compensation coefficient for non-key areas is determined based on the first compensation coefficient of adjacent key areas.

[0016] Based on the actual spatial location of the fixed marker point, the positional offset of the pixels covered by the deformed sub-region in the key area relative to the corresponding fixed marker point is calculated by the calculation module of the distortion correction model.

[0017] Based on the first compensation coefficient, the second compensation coefficient, and the position offset, the distortion correction module of the distortion correction model corrects the distortion of each frame of the second video stream to form the third video stream.

[0018] Optionally, a spatial coordinate system is established, and the video frames in the third video stream are aligned and registered based on the spatial coordinate system to generate a fourth video stream, including:

[0019] Obtain the basic geographic data of the open area of ​​the city;

[0020] Based on actual deployment data, fixed marker points, and geographic base data, a spatial coordinate system is constructed, and the initial positioning information of the pixels of each video frame in the spatial coordinate system is determined.

[0021] Calculate the coordinate difference of the same fixed marker point in the corresponding video frames of adjacent cameras to form positioning deviation data;

[0022] Based on the positioning deviation data and the lens parameters of the multiple cameras, the pixel spatial positions of fixed marker points and surrounding areas in each frame of the third video stream are adjusted to form a fourth video stream.

[0023] Optionally, identifying the target motion paths of people and vehicles in the fourth video stream using a semantic alignment model includes:

[0024] The semantic alignment model's recognition module identifies the image regions corresponding to people and vehicles in each frame of video footage and determines the initial position information of people and vehicles relative to fixed marker points in each frame of video footage.

[0025] Based on the acquisition time sequence of adjacent frames, the corresponding image regions of the same person and the same vehicle in adjacent frames are associated and integrated through the association module of the semantic alignment model to form a person image sequence and a vehicle image sequence.

[0026] The center positions of the same person in each frame of the video image sequence and the center positions of the same vehicle in each frame of the video image sequence are connected to form the initial movement paths of the person and the vehicle, respectively.

[0027] Based on the variation of the center position of the same target in the spatial coordinate system, the spatial position of the fixed marker point, and the geographical features of the urban open area, abnormal position points in the initial movement paths of the personnel and vehicles are identified and eliminated, and target movement paths of the personnel and vehicles are generated.

[0028] Optionally, by adjusting the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras, a fifth video stream is obtained, including:

[0029] The center positions of the two target movement paths are compared with the relative positions of the corresponding fixed marker points in the spatial coordinate system to determine the direction and degree of positional offset of the two target movement paths relative to the fixed marker points.

[0030] According to the position offset direction and position offset degree, the center positions of the two target motion paths in the fourth video stream are adjusted to obtain the intermediate video stream;

[0031] Based on the actual deployment data and lens parameters of the multiple cameras, the concentrated distribution area and the covered blank area of ​​the two target motion paths in the intermediate video stream are identified to determine the adjustment direction and adjustment range of the field of view of each camera.

[0032] Based on the adjustment direction and adjustment range, combined with the actual deployment data and lens parameters, the field of view of each frame of the intermediate video stream is adjusted to form the fifth video stream.

[0033] Optionally, based on the actual deployment data and lens parameters of the multiple cameras, the concentrated distribution area and the coverage blank area of ​​the two target motion paths in the intermediate video stream are identified to determine the adjustment direction and adjustment range of the field of view of each camera, including:

[0034] Based on the center positions of the two target motion paths in the intermediate video stream, the actual deployment data of the multiple cameras, and the lens parameters, the initial field of view of each camera is determined in the spatial coordinate system.

[0035] The distribution ratio of each center position of the two target motion paths within each initial field of view is statistically analyzed, and the sub-ranges with a distribution ratio higher than a preset ratio within each initial field of view are defined as the concentrated distribution area of ​​the target motion path. The sub-area covered by the target center position is defined as the coverage blank area. The target center position is the center position that is not covered by the initial field of view of any camera.

[0036] The initial field of view of each camera is spatially compared with the corresponding concentrated distribution area and the coverage blank area to determine the adjustment direction of the field of view of each camera.

[0037] Based on the actual deployment data of the multiple cameras, the spatial straight-line distance from the boundary of the initial field of view of each camera to the corresponding side boundary of the corresponding concentrated distribution area or the covered blank area is calculated, and the spatial straight-line distance of each camera is converted into the adjustment range of the field of view.

[0038] Optionally, identifying overlapping and non-overlapping regions between frames in the fifth video stream to generate a panoramic monitoring image includes:

[0039] Based on the fixed marker points in each video frame corresponding to adjacent cameras in the fifth video stream, the overlapping and non-overlapping areas between each video frame are identified.

[0040] Extract the first pixel information of each video frame corresponding to different cameras from each overlapping area, and determine the pixel priority of each camera by combining the actual deployment data.

[0041] According to the pixel priority of each camera, the second pixel information and the third pixel information are superimposed to form the fourth pixel information of the overlapping area. The second pixel information is the pixel information of each video frame corresponding to the camera with a pixel priority higher than or equal to the preset priority, and the third pixel information is the pixel information of each video frame corresponding to the camera with a pixel priority lower than the preset priority.

[0042] According to the spatial location of fixed marker points, the fourth pixel information in each frame of the fifth video stream is stitched together with the pixel information of non-overlapping areas to form a panoramic monitoring image.

[0043] Secondly, this application provides a video surveillance system for stitching together footage from multiple cameras, including:

[0044] The acquisition module is used to acquire the first video stream collected by multiple cameras deployed in open urban areas;

[0045] The processing module is used to perform hardware encoding and decoding processing on the first video stream to generate a second video stream;

[0046] The correction module is used to correct the distortion of each frame of the second video stream using a pre-built distortion correction model to obtain the third video stream;

[0047] The alignment module is used to establish a spatial coordinate system and perform alignment and registration processing on each frame of the third video stream based on the spatial coordinate system to generate a fourth video stream.

[0048] The adjustment module is used to identify the target motion paths of people and vehicles in the fourth video stream through a semantic alignment model, and adjust the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras to obtain the fifth video stream;

[0049] The identification module is used to identify overlapping and non-overlapping areas between each frame of the fifth video stream in order to generate a panoramic monitoring image.

[0050] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the steps of the video surveillance multi-camera splicing method described in the first aspect above.

[0051] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements the steps of a video surveillance method for stitching together images from multiple cameras as described in the first aspect above.

[0052] The video surveillance method for stitching together images from multiple cameras provided in this application has the following advantages: By sequentially executing steps such as video stream acquisition, hardware encoding and decoding, distortion correction based on a pre-built model, image registration in a spatial coordinate system, semantically driven target motion path alignment and field of view optimization, and overlapping area identification and panoramic synthesis, end-to-end generation of a highly consistent panoramic surveillance image from multiple heterogeneous video sources is achieved. This not only eliminates image distortion caused by lens physical characteristics but also ensures the geometric alignment accuracy of multi-view images through a unified spatial reference. Furthermore, it introduces a semantic-level target trajectory understanding mechanism to dynamically coordinate the cross-camera field of view layout, thereby improving the structural coherence and target behavior continuity of the stitching result in complex open scenes, and ultimately outputting a panoramic surveillance image with complete coverage, natural boundaries, and clear semantics.

[0053] Furthermore, by combining camera lens parameters and actual deployment information, fixed marker points are marked in the video frames, and distortion-sensitive and non-sensitive areas are divided accordingly. Then, the deformation patterns of different areas are analyzed, compensation coefficients and pixel offsets are calculated, and finally, fine-grained area-by-area distortion correction is achieved. This overcomes the problem of local image distortion caused by uneven distribution of nonlinear distortion in urban scenes due to traditional global or coarse-grained correction methods. The corrected video is closer to the real spatial geometric relationship, providing a high-fidelity visual foundation for subsequent spatial alignment and semantic association. It also alleviates feature misalignment and splicing breaks caused by distortion interference, and enhances the robustness and splicing quality of the system in complex environments. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a method for stitching together images from multiple video surveillance cameras, provided in an embodiment of this application;

[0056] Figure 2 A schematic diagram illustrating a specific implementation of a method for stitching together images from multiple video surveillance cameras, provided in an embodiment of this application;

[0057] Figure 3 This is a schematic diagram of a video surveillance system for splicing images from multiple cameras, provided in an embodiment of this application. Detailed Implementation

[0058] To address the problems of existing feature point matching-based stitching methods in complex urban scenes, such as unstable matching due to occlusion, dynamic interference, and lens distortion, and the lack of modeling for the continuity of target motion across cameras, making it difficult to generate semantically consistent and spatiotemporally coherent panoramic images, this application provides a stitching method for multiple video surveillance camera images. The core idea of ​​this method is to establish a mapping relationship between pixels and the real world by introducing fixed marker points and spatial references, and to coordinate the adjustment of the camera field of view and target trajectory before stitching, thereby avoiding the misalignment risk caused by unreliable underlying features at the source and ensuring the overall consistency of the panoramic image in both geometric structure and motion semantics.

[0059] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] The core of this application is to provide a method for stitching together images from multiple video surveillance cameras, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0061] Step 101: Obtain the first video stream collected by multiple cameras deployed in the open area of ​​the city.

[0062] In this step, the urban open area refers to the area without closed management as defined by the urban public space planning. This area includes urban roads, urban squares, public green spaces, etc., and is used to carry the daily passage and activities of people and vehicles. It is the physical space range for the deployment of multiple cameras and video collection.

[0063] The first video stream refers to the sequence of raw video data directly acquired by multiple cameras. This video stream includes continuous video frame images, frame acquisition timestamps, camera device identifiers, and other information.

[0064] In this embodiment, after determining the deployment locations of multiple cameras based on the geographical layout plan of the urban open area, the video acquisition module of each camera is activated. The camera's image sensor converts the light signal within the monitoring range into an electrical signal and continuously captures frame image data containing the activities of people and vehicles. Then, a collection timestamp and camera device identifier are added to each frame of data. The original frame image data collected by all cameras are aggregated and integrated through the data transmission link to finally obtain the first video stream covering different monitoring points in the urban open area.

[0065] Step 102: Perform hardware encoding and decoding processing on the first video stream to generate the second video stream.

[0066] In this step, the second video stream refers to the video data sequence obtained after hardware encoding and decoding of the first video stream. The video stream includes compressed frame image data and standardized video format information.

[0067] In this embodiment, the original image data in the first video stream is read frame by frame by hardware encoding and decoding. The original image data is compressed to remove duplicate and redundant pixel information in each frame. Then, the processed video data is converted into a unified standard video format, and the data validity is verified frame by frame to avoid data loss or damage. Finally, all processed frame images are integrated in the order of acquisition time to generate the second video stream.

[0068] Step 103: Perform distortion correction on each frame of the second video stream using a pre-built distortion correction model to obtain the third video stream.

[0069] In this step, the third video stream refers to the video data sequence after distortion correction, where the image has no lens distortion deviation and the pixel position matches the real spatial distribution.

[0070] It should be noted that the distortion correction model may include a region division module, a pattern analysis module, a calculation module, and a correction module. These modules are functionally limited descriptions, and this embodiment does not limit the specific structural design of each module.

[0071] Therefore, the structure of the above distortion correction model is exemplary. The embodiments of this application do not specifically limit the design of the structure or the design of the specific parameters within the structure, and can be set accordingly according to the actual situation.

[0072] In the embodiments of this application, such as Figure 2 As shown, step 103 specifically includes the following steps:

[0073] Step 301: Obtain the lens parameters and actual deployment data of the multiple cameras, and mark the fixed marker points in each frame of the second video stream.

[0074] In this step, lens parameters refer to relevant data describing the physical properties of multiple camera lenses. These parameters include lens focal length, lens shooting angle, etc., and are used to analyze the image distortion characteristics produced by the lens.

[0075] Actual deployment data refers to the actual installation location, height, and angle of multiple cameras in open urban areas, which is used to analyze image distortion in conjunction with spatial scene analysis.

[0076] Fixed markers refer to the pixels corresponding to objects whose positions remain unchanged in each frame of the second video stream. Their positions do not change with the movement of people or vehicles, and they serve as positional references for distortion correction.

[0077] In this embodiment, the lens parameters of multiple cameras in the open urban area are first retrieved, and the actual deployment data of multiple cameras in the open urban area are collected. Then, the video frames in the second video stream are traversed frame by frame, and the pixels corresponding to objects with fixed positions in the frames are selected and marked as fixed marker points.

[0078] Step 302: Based on the location distribution of the fixed marker points, the region division module of the distortion correction model is used to divide the key and non-key regions in each frame of video image affected by lens distortion.

[0079] In this step, the region segmentation module refers to the functional module in the distortion correction model used to segment the video image into regions, distinguishing regions with different degrees of distortion based on the location distribution of fixed marker points.

[0080] Key areas refer to the areas in each frame of video that are more significantly affected by lens distortion and have more obvious pixel deformation; non-key areas refer to the areas in each frame of video that are less affected by lens distortion and have less pixel deformation.

[0081] In this embodiment, the location distribution information of fixed marker points in each frame of the second video stream is first extracted. Then, this information is input into the region division module of the distortion correction model. The region division module then divides the areas with obvious distortion effects in each frame of the video into key areas and the areas with weak distortion effects into non-key areas based on the distribution of the fixed marker points.

[0082] Step 303: Based on the lens parameters, analyze the deformation patterns of key and non-key areas in each frame of video using the distortion correction model's pattern analysis module. Combined with the actual deployment data, determine the first compensation coefficient for different key areas, and determine the second compensation coefficient for non-key areas based on the first compensation coefficient of adjacent key areas.

[0083] In this step, the distortion law refers to the trend and degree of positional shift of video image pixels caused by the lens, and the distortion law varies in different regions.

[0084] The first compensation coefficient refers to the distortion compensation value set for the key area, which is used to accurately correct pixel deformation in the key area.

[0085] The second compensation coefficient refers to the distortion compensation value set for non-critical areas, adapting to slight deformations in these areas. This embodiment does not limit the value of each compensation coefficient; it can be set according to the actual situation.

[0086] In this embodiment, the lens parameters are first input into the distortion correction model's pattern analysis module. The pattern analysis module analyzes the deformation patterns of key and non-key areas in each frame of video, and matches the degree of deformation of each key area with actual deployment data. A corresponding first compensation coefficient is determined for different key areas. Then, the first compensation coefficients of adjacent key areas are extracted. The second compensation coefficient is obtained by calculating the arithmetic mean of the first compensation coefficients of two adjacent key areas. Through the above calculation method, the second compensation coefficient can be adapted to the slight deformation characteristics of non-key areas between two adjacent key areas, ensuring that the distortion correction of non-key areas is consistent and unified with the correction effect of adjacent key areas, and avoiding correction discontinuities.

[0087] Step 304: Based on the actual spatial position of the fixed marker point, calculate the position offset of the pixels covered by the deformable sub-region in the key area relative to the corresponding fixed marker point through the calculation module of the distortion correction model.

[0088] In this step, the deformable sub-region refers to a series of small, continuous regions with the same deformation characteristics that are subdivided within the key region.

[0089] Position offset refers to the sum of the horizontal and vertical positional deviations of the pixels covered by the deformable sub-region in the key area compared to the corresponding fixed marker point.

[0090] In this embodiment, the actual spatial position of each fixed marker point is first determined, and the position information is input into the calculation module of the distortion correction model. Then, the calculation module extracts the deformed sub-regions in the key area, compares the current position of the pixel in the deformed sub-region with the actual spatial position of the corresponding fixed marker point, calculates the horizontal offset value and vertical offset value of each pixel, and integrates the two types of offset values ​​to obtain the position offset.

[0091] Step 305: According to the first compensation coefficient, the second compensation coefficient, and the position offset, the distortion correction module of the distortion correction model is used to correct the distortion of each frame of the second video stream to form the third video stream.

[0092] In this embodiment, the first compensation coefficient, the second compensation coefficient, and the position offset are first input into the correction module of the distortion correction model. The correction module adjusts the pixel position of the key area according to the first compensation coefficient and adjusts the pixel position of the non-key area according to the second compensation coefficient. Combined with the position offset, the offset pixels are corrected to their true corresponding positions. After the distortion correction of all frames is completed, the corrected video frames are integrated in the order of acquisition time to form a third video stream.

[0093] The embodiments of this application can eliminate the image distortion problem caused by the camera lens, make the pixel position of the video image conform to the real spatial distribution, and solve the problem of deviation in subsequent spatial alignment and splicing caused by the distortion of the original image.

[0094] Step 104: Establish a spatial coordinate system, and perform alignment and registration processing on each frame of the third video stream based on the spatial coordinate system to generate a fourth video stream.

[0095] In this step, the spatial coordinate system refers to a spatial coordinate system established based on the geographic basic data of the urban open area, with a fixed geographic point as the coordinate origin. This spatial coordinate system includes horizontal coordinates and vertical coordinates.

[0096] The fourth video stream refers to the video data sequence after spatial alignment and registration processing, where the pixel spatial positions of each video frame are uniform and the positions of the marker points of adjacent camera frames are consistent.

[0097] In this embodiment of the application, step 104 specifically includes the following steps:

[0098] Step 401: Obtain the basic geographic data of the open area of ​​the city.

[0099] In this step, the geographic baseline data refers to geographic information such as the planar geographic outline, spatial orientation benchmark, and ground elevation reference of the urban open area.

[0100] In this embodiment of the application, the planar outline, orientation benchmark and elevation reference data of the urban open area are first extracted from the geographic mapping information corresponding to the urban open area, and the above data are sorted to obtain the geographic basic data of the urban open area.

[0101] Step 402: Based on the actual deployment data, fixed marker points, and geographic base data, construct a spatial coordinate system and determine the initial positioning information of the pixels of each video frame in the spatial coordinate system.

[0102] In this step, the initial positioning information refers to the preliminary coordinate values ​​of the pixels in each frame of the third video stream in the spatial coordinate system, which is the initial quantization result of the pixel spatial position.

[0103] In this embodiment, based on the geographic data of the open urban area, combined with the actual deployment data of multiple cameras and the location of fixed markers, a coordinate origin is selected and the coordinate axes are defined to construct a spatial coordinate system. Then, each pixel of each frame of the third video stream is mapped and matched with the spatial coordinate system to determine the coordinate value corresponding to each pixel, thus forming initial positioning information.

[0104] Step 403: Calculate the coordinate difference of the same fixed marker point in the corresponding video frames of adjacent cameras to form positioning deviation data.

[0105] In this step, adjacent cameras refer to multiple cameras deployed in adjacent locations within an open urban area, whose monitoring ranges overlap.

[0106] The same fixed marker point refers to the marker point corresponding to the same real fixed object captured in video footage from different cameras.

[0107] The coordinate difference value refers to the difference in coordinate values ​​of the same fixed marker point in the spatial coordinate system corresponding to the video frames of adjacent cameras.

[0108] Positioning deviation data refers to the quantified set of deviations formed by integrating the coordinate differences of all the same fixed marker points.

[0109] In this embodiment of the application, cameras with adjacent deployment locations within the open urban area are first selected, the same fixed marker point corresponding to the video images of the adjacent cameras is extracted, and then the spatial coordinate values ​​of the fixed marker point in the two video images are read respectively. The difference between the horizontal coordinate and the difference between the vertical coordinate are calculated, and all differences are integrated to form positioning deviation data.

[0110] Step 404: Based on the positioning deviation data and the lens parameters of the multiple cameras, adjust the pixel spatial positions of the fixed marker points and surrounding areas in each frame of the third video stream to form a fourth video stream.

[0111] In this step, the surrounding area refers to a continuous square pixel area extending outward from the fixed marker point with a preset pixel range. This preset pixel range is determined based on the lens parameters of multiple cameras and covers all pixels around the fixed marker point that are directly affected by the positioning deviation.

[0112] In this embodiment, the coordinate adjustment value of the fixed marker point is first determined based on the positioning deviation data, and the range of the surrounding area of ​​the fixed marker point is determined in combination with the lens parameters. Then, according to the coordinate adjustment value, the spatial coordinate position of all pixels in the fixed marker point and its surrounding area is adjusted synchronously. After the position adjustment of all frames is completed, the processed images are integrated in the order of acquisition time to form the fourth video stream.

[0113] This application embodiment realizes spatial alignment and registration of video images from multiple cameras, eliminates spatial positional deviations between images, and provides a unified spatial positional basis for subsequent target motion path recognition.

[0114] Step 105: Identify the target motion paths of people and vehicles in the fourth video stream using a semantic alignment model, and adjust the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras to obtain the fifth video stream.

[0115] In this step, the target motion path of each person and vehicle refers to the trajectory of the continuous movement of the person and vehicle in the open urban area after the abnormal position points are identified and removed through the semantic alignment model. It corresponds to the two subsequent target motion paths. The target motion path of the person corresponds to the line connecting the continuous positions of the same person in each frame, and the target motion path of the vehicle corresponds to the line connecting the continuous positions of the same vehicle in each frame.

[0116] The field of view refers to the spatial area that each camera can monitor. It is determined by the actual deployment data of the cameras and the lens parameters, and covers the spatial boundaries of the images that the cameras can capture.

[0117] The fifth video stream refers to the video data sequence obtained after adjusting the center position of the target's motion path and the field of view of multiple cameras. The path position is accurate and the camera field of view is adapted to the path distribution.

[0118] The explanation of the semantic alignment model and its specific structural design can be found in relevant technologies, and will not be elaborated here.

[0119] In this embodiment of the application, the first step is to "identify the target motion paths of people and vehicles in the fourth video stream using a semantic alignment model". This step specifically includes the following steps:

[0120] Step 501: Identify the image regions corresponding to people and vehicles in each frame of video using the recognition module of the semantic alignment model, and determine the initial position information of people and vehicles relative to fixed marker points in each frame of video.

[0121] In this step, the recognition module refers to the functional module in the semantic alignment model used to identify the corresponding image regions of people and vehicles in the scene and determine the initial position of the target.

[0122] An image region refers to a continuous pixel area in each frame of a video that can completely contain a person or vehicle, and can distinguish between different people and different vehicles.

[0123] Initial position information refers to the spatial position values ​​of personnel and vehicles relative to fixed marker points in each frame of video footage, used to reflect the initial orientation of the target in the frame.

[0124] In this embodiment, the semantic alignment model recognition module is first activated, and each frame of the video in the fourth video stream is traversed frame by frame. Through image feature comparison, the image regions corresponding to people and vehicles in each frame are identified, and the image regions of different people and different vehicles are distinguished. Then, with the spatial position of the fixed marker point as a reference, the horizontal and vertical position differences of the center of the image region of people and vehicles relative to the fixed marker point are calculated, thereby determining the initial position information of people and vehicles in each frame.

[0125] Step 502: Based on the acquisition time sequence of adjacent frames, the corresponding image regions of the same person and the same vehicle in adjacent frames are associated and integrated through the association module of the semantic alignment model to form a person image sequence and a vehicle image sequence.

[0126] In this step, the association module refers to the functional module in the semantic alignment model used to associate the same target image region in adjacent frames and integrate the continuous image information of the target.

[0127] A personnel image sequence refers to a continuous set of images formed by sequentially associating and integrating the corresponding image regions of the same person in adjacent frames according to the acquisition time.

[0128] A vehicle image sequence refers to a continuous set of images formed by sequentially associating and integrating corresponding image regions of the same vehicle in adjacent frames according to the acquisition time.

[0129] In this embodiment, the acquisition timestamps of each frame in the fourth video stream are first extracted, and the correspondence between adjacent frames is determined by sorting them according to the timestamps. Then, the feature similarity of the image regions of people and vehicles in adjacent frames is compared through the association module of the semantic alignment model. Next, the image regions whose feature similarity meets the preset standard are determined to be the regions corresponding to the same person and the same vehicle. All adjacent frame image regions of the same person and the same vehicle are associated and integrated in chronological order to form a personnel image sequence and a vehicle image sequence.

[0130] Step 503: Connect the center positions of the same person in each frame of the personnel image sequence and the center positions of the same vehicle in each frame of the vehicle image sequence to form the preliminary motion paths of the personnel and vehicles respectively.

[0131] In this step, the center position refers to the pixel position corresponding to the geometric center of the corresponding image area of ​​the same person or vehicle in each frame of the image sequence, which is also the corresponding coordinate point of the image area in the spatial coordinate system.

[0132] The preliminary motion path refers to the continuous trajectory formed by connecting the center positions of the same target in each frame before abnormal locations are removed. It may include abnormal locations caused by image interference.

[0133] In this embodiment, the geometric center of the image region of the same person in each frame of the video image sequence is extracted one by one, and the coordinates of the center position in the spatial coordinate system are determined. Then, according to the acquisition time sequence, all the center positions of the same person are connected in sequence to form the preliminary movement path of the person. Using the same method, the center positions of each frame of the same vehicle in the vehicle image sequence are extracted and connected in sequence to form the preliminary movement path of the vehicle.

[0134] Step 504: Based on the variation range of the center position of the same target in the spatial coordinate system, the spatial position of the fixed marker point, and the geographical features of the urban open area, identify and remove abnormal position points in the preliminary movement paths of the personnel and vehicles, and generate the target movement paths of the personnel and vehicles respectively.

[0135] In this step, "same target" refers to the same person in a sequence of people images or the same vehicle in a sequence of vehicle images.

[0136] The magnitude of change refers to the sum of the differences in the horizontal and vertical coordinates of the center position of the same target in two consecutive video frames, reflecting the degree of positional change of the target in consecutive frames.

[0137] Abnormal location points refer to central locations in the initial movement path where the change in the adjacent central location exceeds a reasonable range and does not match the spatial location of fixed marker points or the geographical features of open urban areas. These are often caused by image interference or misidentification.

[0138] The target motion path refers to the continuous and reasonable motion trajectory of the same target in the spatial coordinate system after removing abnormal location points, which can truly reflect the actual movement of people and vehicles.

[0139] In this embodiment, a reasonable range for the variation of the center position of the same target in consecutive frames is first set. Then, the preliminary movement paths of personnel and vehicles are traversed one by one to calculate the variation of the center position of the same target in consecutive frames in the spatial coordinate system. If the variation of a certain center position exceeds the reasonable range, and combined with the spatial position of the fixed marker point and the geographical features of the urban open area, it is determined that the center position does not match the surrounding environment. Then, the center position is marked as an abnormal position point and removed. After removing all abnormal position points, the remaining center positions are reconnected in the order of acquisition time to generate the target movement paths of personnel and vehicles respectively.

[0140] Next, the process is executed: "Adjust the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras to obtain the fifth video stream." This step specifically includes the following steps:

[0141] Step 511: Compare the center positions of the two target movement paths with the relative positions of the corresponding fixed marker points in the spatial coordinate system to determine the direction and degree of positional offset of the two target movement paths relative to the fixed marker points.

[0142] In this step, the corresponding fixed marker point refers to the fixed marker point that is closest to the two target movement paths and can be used as a position reference. Each path segment corresponds to one or more matching fixed marker points.

[0143] The position offset direction refers to the offset direction of the center position of the two target movement paths relative to the corresponding fixed marker point in the spatial coordinate system.

[0144] The degree of positional offset refers to the difference in coordinates between the center positions of two target movement paths and their corresponding fixed marker points, quantifying the distance of the offset.

[0145] In this embodiment, firstly, in the spatial coordinate system, the fixed marker points corresponding to each segment of the two target motion paths are determined, the coordinate values ​​of the path center position and the corresponding fixed marker point are extracted, and the difference between the horizontal coordinate and the vertical coordinate between the path center position and the corresponding fixed marker point is calculated; then, the position offset direction is determined according to the sign of the difference, the position offset degree is determined according to the absolute value of the difference, and the offset information of all path segments is integrated to determine the overall offset direction and offset degree of the two target motion paths relative to the fixed marker points.

[0146] Step 512: Adjust the center positions of the two target motion paths in the fourth video stream according to the position offset direction and position offset degree to obtain the intermediate video stream.

[0147] In this step, the intermediate video stream refers to the video data sequence obtained after adjusting the center positions of the two target motion paths in the fourth video stream. The center positions of the motion paths of people and vehicles in the picture have been corrected to reasonable positions, and the camera field of view has not been adjusted.

[0148] In this embodiment, the adjustment values ​​of the center positions of the two target motion paths are first calculated according to the position offset direction and the degree of position offset; then, according to the adjustment values ​​and offset direction, the center position of each path is adjusted in the spatial coordinate system to a reasonable position relative to the corresponding fixed marker point to ensure that the path position matches the real spatial scene; after the adjustment is completed, all frames in the fourth video stream are integrated in the order of acquisition time to obtain the intermediate video stream.

[0149] Step 513: Based on the actual deployment data and lens parameters of the multiple cameras, identify the concentrated distribution area and the blank coverage area of ​​the two target motion paths in the intermediate video stream, so as to determine the adjustment direction and adjustment range of the field of view of each camera.

[0150] In this step, the concentrated distribution area refers to the central location of the two target movement paths, which is a relatively densely distributed sub-region within the initial field of view of the camera.

[0151] The covered blank area refers to a continuous sub-region in the area where the center of the two target movement paths is located, which is not covered by the initial field of view of any camera.

[0152] The adjustment direction refers to the direction in which the field of view of each camera needs to be adjusted to adapt to concentrated distribution areas and cover blank areas.

[0153] The adjustment range refers to the amount of adjustment required for the field of view of each camera, quantitatively reflecting the degree of field of view adjustment.

[0154] In this embodiment of the application, step 513 specifically includes the following steps:

[0155] Step 521: Based on the center positions of the two target motion paths in the intermediate video stream, the actual deployment data of the multiple cameras, and the lens parameters, the initial field of view of each camera is determined in the spatial coordinate system.

[0156] In this step, the initial field of view refers to the monitoring space range marked in the spatial coordinate system for each camera before field of view adjustment, based on actual deployment data and lens parameters. It reflects the initial monitoring spatial boundary of the camera and is determined by the camera's installation position, installation angle, and lens parameters.

[0157] In this embodiment, the distribution information of the center positions of the two target motion paths in the intermediate video stream in the spatial coordinate system is first extracted. Combined with the actual deployment data and lens parameters of multiple cameras, the initial spatial boundary that each camera can cover is delineated in the spatial coordinate system to clarify the horizontal and vertical boundary coordinates of the initial field of view of each camera.

[0158] Step 522: Calculate the distribution ratio of each center position of the two target motion paths within each initial field of view, and define the sub-ranges with a distribution ratio higher than a preset ratio within each initial field of view as the concentrated distribution area of ​​the target motion path, and define the sub-area covered by the target center position as the coverage blank area. The target center position is the center position that is not covered by the initial field of view of any camera.

[0159] In this step, the distribution ratio refers to the ratio of the number of target motion path center positions in a certain sub-range within the initial field of view of each camera to the total number of all center positions of the two target motion paths within that initial field of view. It is used to quantitatively reflect the density of path center positions in that sub-range.

[0160] The preset ratio refers to a pre-set standard for determining the distribution proportion of concentrated distribution areas. Sub-ranges exceeding this standard are considered concentrated distribution areas.

[0161] The target center position refers to the center position in the two target movement paths that is not covered by the initial field of view of any camera.

[0162] The blank area refers to the continuous sub-region where the center of all targets is located. This area is not covered by any camera's initial field of view and cannot capture path-related images.

[0163] In this embodiment of the application, the initial field of view of each camera is first analyzed one by one, and the number of all center positions of the two target motion paths within the initial field of view is counted. Then, the initial field of view is divided into multiple sub-ranges, and the number of center positions in each sub-range is counted, and the distribution ratio of each sub-range is calculated.

[0164] Then, sub-ranges with a distribution ratio higher than the preset ratio are designated as concentrated distribution areas of the target movement path; at the same time, the target center positions that are not covered by the initial field of view of any camera are selected, and the continuous spatial sub-regions where these target center positions are located are designated as coverage blank areas.

[0165] Step 523: Spatially compare the initial field of view of each camera with the corresponding concentrated distribution area and the coverage blank area to determine the adjustment direction of the field of view of each camera.

[0166] In this step, the corresponding concentrated distribution area refers to the concentrated distribution area that is closest to or partially overlaps with the initial field of view of a certain camera.

[0167] The corresponding blank coverage area refers to the blank coverage area adjacent to the initial field of view of a certain camera.

[0168] The direction of adjustment is determined based on the initial field of view and the spatial relationship between the two areas, ensuring that the adjusted camera field of view can cover the concentrated distribution area or the adjacent blank area.

[0169] In this embodiment, the initial field of view of each camera is first spatially compared and matched with the corresponding concentrated distribution area and the coverage blank area. Specifically: if part or all of the sub-area of ​​the concentrated distribution area is located outside the initial field of view of the camera, the adjustment direction of the camera's field of view is determined to be the spatial orientation toward the concentrated distribution area; if the coverage blank area is directly adjacent to the initial field of view of the camera, the adjustment direction of the camera's field of view is determined to be the spatial orientation toward the coverage blank area; if both situations exist simultaneously, the direction toward the concentrated distribution area is given priority as the main adjustment direction.

[0170] Step 524: Based on the actual deployment data of the multiple cameras, calculate the spatial straight-line distance from the boundary of the initial field of view of each camera to the corresponding side boundary of the corresponding concentrated distribution area or the covered blank area, and convert the spatial straight-line distance of each camera into the adjustment range of the field of view.

[0171] In this step, the corresponding side boundary refers to the side boundary of a concentrated distribution area or a blank coverage area that is closest to the initial field of view boundary of a certain camera and faces that initial field of view.

[0172] The spatial straight-line distance refers to the straight-line distance from the initial field of view boundary of the camera to the corresponding side boundary of the concentrated distribution area or the covered blank area, which quantitatively reflects the distance between the two.

[0173] In this embodiment, firstly, based on the actual deployment data of multiple cameras, the boundary coordinates of the initial field of view of each camera, as well as the coordinates of the corresponding side boundaries of the concentrated distribution area and the covered blank area, are determined; then, the spatial straight-line distance from the boundary of the initial field of view to the corresponding side boundary is obtained through coordinate calculation; combined with the actual deployment position and installation angle of the camera, this spatial straight-line distance is converted into a quantitative value that is adapted to the adjustment of the camera's field of view. This quantitative value is the adjustment range of the field of view of each camera, so as to ensure that the adjusted camera's field of view can cover the corresponding area.

[0174] Step 514: Based on the adjustment direction and adjustment range, and combined with the actual deployment data and lens parameters, adjust the field of view of each frame of the intermediate video stream to form the fifth video stream.

[0175] In this embodiment, the adjustment direction and adjustment range of each camera are first extracted. Combined with the actual deployment data and lens parameters of multiple cameras, the field of view of the video screen corresponding to each camera is adjusted. Specifically, according to the adjustment direction, the field of view of the camera is adjusted to face the concentrated distribution area or cover the blank area. According to the adjustment range, the adjustment size of the field of view is controlled to ensure that the field of view of each camera can cover the corresponding concentrated distribution area after adjustment, and fill the blank area as much as possible. After the field of view of all frames of video screen in the intermediate video stream is adjusted frame by frame, all processed screens are integrated according to the acquisition time order to form the fifth video stream.

[0176] This application embodiment ensures the accuracy of the target motion path and adapts the camera's field of view to the path distribution, providing accurate and suitable basic data for the identification of overlapping areas and the generation of panoramic monitoring images in the subsequent fifth video stream, thus ensuring the smoothness of the subsequent stitching process and the accuracy of the stitching results.

[0177] Step 106: Identify overlapping and non-overlapping areas between each frame of the fifth video stream to generate a panoramic monitoring image.

[0178] In this step, the overlapping area refers to the pixel area in the video images of two or more adjacent cameras in the fifth video stream where the monitoring range overlaps and contains the same real-world spatial scene.

[0179] Non-overlapping areas refer to the pixel areas in the fifth video stream that are unique to each camera's video feed and do not overlap with the monitoring range of other cameras.

[0180] A panoramic surveillance view refers to a complete and continuous surveillance view covering the entire open area of ​​a city, formed by overlaying and stitching the video images from all cameras in the fifth video stream according to overlapping and non-overlapping areas.

[0181] In this embodiment of the application, step 106 specifically includes the following steps:

[0182] Step 601: Based on the fixed marker points in each video frame corresponding to adjacent cameras in the fifth video stream, identify the overlapping and non-overlapping areas between each video frame.

[0183] In this step, the video frames corresponding to adjacent cameras refer to the video frames captured by cameras located adjacent to each other and with related monitoring ranges within the open area of ​​the city at the same acquisition time.

[0184] In this embodiment, firstly, video frames from adjacent cameras at the same acquisition time in the fifth video stream are extracted, and fixed marker points in the frames are located; then, the pixel positions corresponding to the same fixed marker point in different frames are matched, and continuous pixel areas with overlapping positions are marked as overlapping areas, while pixel areas in each frame that do not overlap with other frames are marked as non-overlapping areas.

[0185] Step 602: Extract the first pixel information of each video frame corresponding to different cameras from each overlapping area, and determine the pixel priority of each camera by combining the actual deployment data.

[0186] In this step, the first pixel information refers to the pixel color, pixel brightness, and pixel sharpness information extracted from the overlapping area in the video frames of different cameras.

[0187] Pixel priority refers to the priority level set for pixel information of different cameras in the overlapping area based on the actual deployment data of the cameras, which is used to determine the pixel selection criteria when superimposing.

[0188] In this embodiment, each overlapping region of the fifth video stream is traversed one by one to extract the pixel color, pixel brightness, and pixel clarity of the video images corresponding to different cameras in that region, thus obtaining the first pixel information. Then, combined with deployment data such as the actual installation height, installation angle, and shooting distance of the cameras, a corresponding pixel priority is set for each camera, with cameras in better deployment positions corresponding to higher pixel priorities.

[0189] Step 603: According to the pixel priority of each camera, the second pixel information and the third pixel information are superimposed to form the fourth pixel information of the overlapping area. The second pixel information is the pixel information of each video frame corresponding to the camera with a pixel priority higher than or equal to the preset priority, and the third pixel information is the pixel information of each video frame corresponding to the camera with a pixel priority lower than the preset priority.

[0190] In this step, preset priority refers to a pre-defined level standard used to distinguish the priority of pixel information usage.

[0191] The second pixel information refers to the first pixel information of a camera that meets the priority use criteria and whose pixel priority is higher than or equal to the preset priority.

[0192] The third pixel information refers to the first pixel information of cameras that do not meet the priority use standard and whose pixel priority is lower than the preset priority.

[0193] The fourth pixel information refers to the final pixel information of the overlapping area obtained after superimposing the second and third pixel information.

[0194] In this embodiment, the pixel priority of each camera is first divided according to the preset priority. The second pixel information with a pixel priority higher than or equal to the preset priority and the third pixel information with a pixel priority lower than the preset priority are selected. The second pixel information is used as the basis for superposition. The third pixel information is fused and superimposed with the second pixel information. The core features of the second pixel information are retained and the auxiliary features of the third pixel information are fused to form the fourth pixel information of the overlapping area.

[0195] Step 604: According to the spatial position of the fixed marker point, the fourth pixel information in each frame of the fifth video stream is stitched together with the pixel information of the non-overlapping area to form a panoramic monitoring image.

[0196] In this embodiment, the position of the fixed marker point in the spatial coordinate system is first used as the positioning reference to determine the splicing orientation and connection order of each video frame. The fourth pixel information after processing the overlapping area is used as the image connection part, and the pixel information of the non-overlapping area in each camera image is used as the independent image part. All pixel information is sequentially combined and connected according to the spatial orientation to complete the splicing processing of all frames and form a panoramic monitoring image covering the entire open area of ​​the city.

[0197] This application's embodiments solve the problems of overlapping areas being disordered and discontinuous when stitching together multiple camera images, achieving seamless and continuous panoramic video surveillance of open urban areas.

[0198] Figure 3 This is a schematic diagram illustrating a specific implementation of a video surveillance system for stitching together footage from multiple cameras, as provided in this application. (Refer to...) Figure 3 The system may include:

[0199] The acquisition module 31 is used to acquire the first video stream collected by multiple cameras deployed in the open area of ​​the city;

[0200] Processing module 32 is used to perform hardware encoding and decoding processing on the first video stream to generate a second video stream;

[0201] The correction module 33 is used to perform distortion correction on each frame of the second video stream using a pre-built distortion correction model to obtain the third video stream;

[0202] Alignment module 34 is used to establish a spatial coordinate system and perform alignment and registration processing on each frame of video in the third video stream based on the spatial coordinate system to generate a fourth video stream;

[0203] The adjustment module 35 is used to identify the target motion paths of people and vehicles in the fourth video stream through a semantic alignment model, and adjust the center position of the two target motion paths in the fourth video stream and the field of view of the multiple cameras to obtain the fifth video stream;

[0204] The identification module 36 is used to identify overlapping and non-overlapping areas between each frame of the fifth video stream to generate a panoramic monitoring image.

[0205] This application provides a video surveillance multi-camera image stitching system to implement the aforementioned video surveillance multi-camera image stitching method. Therefore, the specific implementation of the video surveillance multi-camera image stitching system can be found in the embodiment section of the video surveillance multi-camera image stitching method above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0206] This application also provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the steps of the video surveillance multi-camera image stitching method described above.

[0207] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, implements the steps of the video surveillance multi-camera splicing method described above.

[0208] In one exemplary embodiment, the computer storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0209] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the video surveillance multi-camera image stitching method.

[0210] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0211] The above provides a detailed description of a method and system for stitching together images from multiple video surveillance cameras, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for stitching together images from multiple video surveillance cameras, characterized in that, include: Acquire the first video stream captured by multiple cameras deployed in open urban areas; The first video stream is subjected to hardware encoding and decoding processing to generate a second video stream; The third video stream is obtained by correcting the distortion of each frame in the second video stream using a pre-built distortion correction model. A spatial coordinate system is established, and the video frames in the third video stream are aligned and registered based on the spatial coordinate system to generate a fourth video stream; The semantic alignment model is used to identify the target motion paths of people and vehicles in the fourth video stream, and the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras are adjusted to obtain the fifth video stream; Identify overlapping and non-overlapping areas between each frame of the fifth video stream to generate a panoramic monitoring image; Adjusting the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras yields a fifth video stream, including: The center positions of the two target movement paths are compared with the relative positions of the corresponding fixed marker points in the spatial coordinate system to determine the direction and degree of positional offset of the two target movement paths relative to the fixed marker points. According to the position offset direction and position offset degree, the center positions of the two target motion paths in the fourth video stream are adjusted to obtain the intermediate video stream; Based on the actual deployment data and lens parameters of the multiple cameras, the concentrated distribution area and the covered blank area of ​​the two target motion paths in the intermediate video stream are identified to determine the adjustment direction and adjustment range of the field of view of each camera. Based on the adjustment direction and adjustment range, and combined with the actual deployment data and lens parameters, the field of view of each frame of the intermediate video stream is adjusted to form the fifth video stream; Based on the actual deployment data and lens parameters of the multiple cameras, the concentrated distribution areas and coverage gaps of the two target motion paths in the intermediate video stream are identified to determine the adjustment direction and adjustment range of the field of view of each camera, including: Based on the center positions of the two target motion paths in the intermediate video stream, the actual deployment data of the multiple cameras, and the lens parameters, the initial field of view of each camera is determined in the spatial coordinate system. The distribution ratio of each center position of the two target motion paths within each initial field of view is statistically analyzed, and the sub-ranges with a distribution ratio higher than a preset ratio within each initial field of view are defined as the concentrated distribution area of ​​the target motion path. The sub-area covered by the target center position is defined as the coverage blank area. The target center position is the center position that is not covered by the initial field of view of any camera. The initial field of view of each camera is spatially compared with the corresponding concentrated distribution area and the coverage blank area to determine the adjustment direction of the field of view of each camera. Based on the actual deployment data of the multiple cameras, the spatial straight-line distance from the boundary of the initial field of view of each camera to the corresponding side boundary of the corresponding concentrated distribution area or the covered blank area is calculated, and the spatial straight-line distance of each camera is converted into the adjustment range of the field of view.

2. The method according to claim 1, characterized in that, The distortion correction of each frame in the second video stream is performed using a pre-built distortion correction model to obtain the third video stream, which includes: Obtain the lens parameters and actual deployment data of the multiple cameras, and mark fixed marker points in each frame of the second video stream; Based on the location distribution of the fixed marker points, the region division module of the distortion correction model divides the key and non-key regions in each frame of video into areas affected by lens distortion. Based on the lens parameters, the distortion patterns of key and non-key areas in each frame of video are analyzed by the distortion correction model's pattern analysis module. Combined with the actual deployment data, the first compensation coefficient for different key areas is determined, and the second compensation coefficient for non-key areas is determined based on the first compensation coefficient of adjacent key areas. Based on the actual spatial location of the fixed marker point, the positional offset of the pixels covered by the deformed sub-region in the key area relative to the corresponding fixed marker point is calculated by the calculation module of the distortion correction model. Based on the first compensation coefficient, the second compensation coefficient, and the position offset, the distortion correction module of the distortion correction model corrects the distortion of each frame of the second video stream to form the third video stream.

3. The method according to claim 1, characterized in that, Establish a spatial coordinate system, and perform alignment and registration processing on each frame of the third video stream based on the spatial coordinate system to generate a fourth video stream, including: Obtain the basic geographic data of the open area of ​​the city; Based on actual deployment data, fixed marker points, and geographic base data, a spatial coordinate system is constructed, and the initial positioning information of the pixels of each video frame in the spatial coordinate system is determined. Calculate the coordinate difference of the same fixed marker point in the corresponding video frames of adjacent cameras to form positioning deviation data; Based on the positioning deviation data and the lens parameters of the multiple cameras, the pixel spatial positions of fixed marker points and surrounding areas in each frame of the third video stream are adjusted to form a fourth video stream.

4. The method according to claim 1, characterized in that, Identifying the target motion paths of people and vehicles in the fourth video stream using a semantic alignment model includes: The semantic alignment model's recognition module identifies the image regions corresponding to people and vehicles in each frame of video footage and determines the initial position information of people and vehicles relative to fixed marker points in each frame of video footage. Based on the acquisition time sequence of adjacent frames, the corresponding image regions of the same person and the same vehicle in adjacent frames are associated and integrated through the association module of the semantic alignment model to form a person image sequence and a vehicle image sequence. The center positions of the same person in each frame of the video image sequence and the center positions of the same vehicle in each frame of the video image sequence are connected to form the initial movement paths of the person and the vehicle, respectively. Based on the variation of the center position of the same target in the spatial coordinate system, the spatial position of the fixed marker point, and the geographical features of the urban open area, abnormal position points in the initial movement paths of the personnel and vehicles are identified and eliminated, and target movement paths of the personnel and vehicles are generated.

5. The method according to claim 1, characterized in that, Identifying overlapping and non-overlapping regions between frames in the fifth video stream to generate a panoramic monitoring view includes: Based on the fixed marker points in each video frame corresponding to adjacent cameras in the fifth video stream, the overlapping and non-overlapping areas between each video frame are identified. Extract the first pixel information of each video frame corresponding to different cameras from each overlapping area, and determine the pixel priority of each camera by combining the actual deployment data. According to the pixel priority of each camera, the second pixel information and the third pixel information are superimposed to form the fourth pixel information of the overlapping area. The second pixel information is the pixel information of each video frame corresponding to the camera with a pixel priority higher than or equal to the preset priority, and the third pixel information is the pixel information of each video frame corresponding to the camera with a pixel priority lower than the preset priority. According to the spatial location of fixed marker points, the fourth pixel information in each frame of the fifth video stream is stitched together with the pixel information of non-overlapping areas to form a panoramic monitoring image.

6. A video surveillance system for stitching together images from multiple cameras, used in the video surveillance method for stitching together images from multiple cameras as described in any one of claims 1 to 5, characterized in that, include: The acquisition module is used to acquire the first video stream collected by multiple cameras deployed in open urban areas; The processing module is used to perform hardware encoding and decoding processing on the first video stream to generate a second video stream; The correction module is used to correct the distortion of each frame of the second video stream using a pre-built distortion correction model to obtain the third video stream; The alignment module is used to establish a spatial coordinate system and perform alignment and registration processing on each frame of the third video stream based on the spatial coordinate system to generate a fourth video stream. The adjustment module is used to identify the target motion paths of people and vehicles in the fourth video stream through a semantic alignment model, and adjust the center positions of the two target motion paths in the fourth video stream and the field of view of the multiple cameras to obtain the fifth video stream; The identification module is used to identify overlapping and non-overlapping areas between each frame of the fifth video stream in order to generate a panoramic monitoring image.

7. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a method for stitching together images from multiple video surveillance cameras as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a method for stitching together images from multiple video surveillance cameras as described in any one of claims 1 to 5.

Images