Panoramic stitching method, device and electronic equipment for airport surveillance video

Abstract

The present application provides a method, device and electronic equipment for panoramic stitching of airport surveillance videos, which relates to the field of data processing. A first surveillance video and a second surveillance video for a target airport are obtained, and the target airport corresponds to multiple airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos among the multiple airport surveillance videos; an interference pattern diagram is determined based on the first surveillance video and the second surveillance video; a static background extraction operation and a dynamic target removal operation are performed on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched; a first light wave field corresponding to the first camera used to shoot the first surveillance video and a second light wave field corresponding to the second camera used to shoot the second surveillance video are determined; based on the first area to be stitched and the second area to be stitched, a panoramic surveillance diagram is determined in combination with the first light wave field and the second light wave field. Implementation of this technical solution facilitates panoramic stitching of airport surveillance videos.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a method, device and electronic equipment for panoramic stitching of airport surveillance videos. Background Art

[0002] As important hubs for public transportation, airport operations involve complex scenarios such as aircraft taxiing, apron scheduling, ground handling, and passenger movements. To ensure safe and efficient operations, modern airports widely deploy multi-camera surveillance systems to cover critical areas such as runways, taxiways, and aprons. However, due to the large spatial scope and frequent dynamic objects in airport scenes, panoramic stitching of videos captured by multiple cameras presents challenges.

[0003] Currently, related panoramic stitching technologies rely on traditional methods such as feature point matching or image registration. However, in airport surveillance scenarios, due to the interference of dynamic objects such as taxiing aircraft, ground vehicles, and personnel movement, the stitching results are prone to dynamic interference, resulting in ghosting and stitching errors. Distortion caused by the differences in the perspectives of multiple cameras can also lead to noticeable seams in the stitched area and unnatural image fusion. Therefore, these traditional methods of feature point matching or image registration are not conducive to panoramic stitching of airport surveillance videos.

[0004] Therefore, there is an urgent need for a panoramic stitching method, device and electronic equipment for airport surveillance videos. Summary of the Invention

[0005] The present application provides a method, device and electronic equipment for panoramic stitching of airport surveillance videos, which facilitates panoramic stitching of airport surveillance videos.

[0006] In a first aspect of the present application, a panoramic stitching method for airport surveillance videos is provided, the method comprising: obtaining a first surveillance video and a second surveillance video for a target airport, the target airport corresponding to a plurality of airport surveillance videos, the first surveillance video and the second surveillance video being any two surveillance videos of the plurality of airport surveillance videos; determining an interference pattern diagram based on the first surveillance video and the second surveillance video; performing a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched; determining a first optical wave field corresponding to a first camera used to shoot the first surveillance video, and a second optical wave field corresponding to a second camera used to shoot the second surveillance video; and determining a panoramic surveillance diagram based on the first area to be stitched and the second area to be stitched in combination with the first optical wave field and the second optical wave field.

[0007] By employing the above technical solution, the interference pattern is introduced to convert the characteristic information in the first and second surveillance videos into a high-frequency phase interference feature map, enabling precise identification of static background and dynamic targets in each surveillance video. The dynamic target removal operation effectively filters out interference from dynamic targets such as aircraft and ground vehicles in the stitching process, ensuring clarity and accuracy in the stitched area and eliminating ghosting or blurring. By establishing a separate optical wavefield model for each camera, distortion caused by camera angle, field of view, and perspective distortion is addressed. The optical wavefield contains rich phase information, enabling natural alignment and fusion between multi-view videos, eliminating misalignment and distortion at seams and ensuring a smooth transition in the stitching result. The static background extraction operation based on the interference pattern map utilizes the high-frequency phase consistency characteristics of the static background to stably extract static background information in scenes with complex lighting conditions or frequent dynamic target motion. This approach avoids the failure of traditional background extraction methods based on inter-frame differencing due to lighting changes or dynamic interference, thereby improving the robustness of the stitching algorithm. By combining the first light wave field and the second light wave field, the high-frequency details captured in multiple videos are accurately integrated into the panoramic image, making the stitched image clearer in terms of space and detail. At the same time, the light wave field technology can dynamically compensate for the color differences and brightness changes between the areas covered by different cameras to ensure the color consistency of the panoramic image. The combination of the interference pattern image and the light wave field reduces the computational complexity of multi-step feature matching and complex registration operations in traditional stitching technology, enabling panoramic stitching to achieve near real-time updates in a high-frame-rate video streaming environment. This method utilizes the frequency domain characteristics of the interference pattern image and the light wave field, and can adapt to background feature extraction and stitching under different lighting conditions. Regardless of daytime, nighttime or weather conditions, the stitched panorama can maintain stable quality and consistency. Therefore, it is convenient to perform panoramic stitching of airport surveillance videos.

[0008] Optionally, determining the interference pattern diagram based on the first surveillance video and the second surveillance video specifically includes: performing Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; calculating the phase change between the first high-frequency texture and the second high-frequency texture to generate the interference pattern diagram.

[0009] By employing the above technical solution, Fourier transform converts surveillance video frames from the time domain into the frequency domain, highlighting high-frequency texture features within the image. These features primarily correspond to detailed information within the video. By acquiring high-frequency textures from both the first and second surveillance videos, key features can be more accurately extracted, avoiding aliasing between background textures and dynamic objects, and improving feature comparison accuracy. Images captured by different cameras exhibit certain parallax, and dynamic objects appear as discontinuous phase changes within video frames. By calculating the phase changes between high-frequency textures, dynamic objects can be more accurately detected and distinguished from static backgrounds. Static backgrounds exhibit high phase consistency across multiple frames, while dynamic objects experience phase fluctuations due to their constantly changing positions. Leveraging this characteristic, interference from dynamic objects can be more accurately removed. Interference pattern mapping, through phase change calculation, generates an image that fuses the high-frequency phase features of both videos. This image directly reflects key texture regions and phase consistency information within the video frames, serving as the fundamental data for panoramic stitching. Compared to traditional edge detection or feature point matching, interference pattern mapping combines frequency domain characteristics with phase information, making it more suitable for panoramic stitching of dynamic and complex scenes. Due to the differences in viewing angles of multiple cameras, there will be certain distortion and misalignment in the video frames. Through Fourier transform and phase correction, the high-frequency textures in the two videos can be accurately aligned.

[0010] Optionally, performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be stitched and a second area to be stitched specifically includes: inputting the interference pattern image into a light wave propagation model, and obtaining a first result through static background extraction; using a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; and determining the first area to be stitched and the second area to be stitched based on the first result and the second result using the light wave propagation model.

[0011] By employing the above technical solution and processing the interference pattern using a light wave propagation model, the characteristics of light wave propagation can be leveraged to highlight key texture information within the static background while simultaneously suppressing interference from dynamic objects. In a scene, the light wave propagation characteristics of the static background are highly stable, enabling clear separation of static objects such as runways and buildings. The impact of dynamic interference on light wave propagation is mitigated, resulting in more accurate static background extraction. A high-pass filter processes dynamic objects within the interference pattern, eliminating the interference of high-frequency, discontinuous dynamic objects on panoramic stitching. Dynamic objects appear as discontinuous distributions within high-frequency features, and the high-pass filter accurately identifies and removes these high-frequency interferences. After eliminating dynamic objects, only background features remain in the stitching area, avoiding ghosting or misalignment of dynamic objects during stitching. The dual operations of static background extraction and dynamic object removal are combined using the light wave propagation model to precisely locate the first and second stitching areas. Static background extraction provides high-quality background texture information, while the high-pass filter effectively eliminates dynamic interference. The combination of these two ensures strong texture consistency and minimal dynamic interference within the stitching area. The stitching area determination process reduces the presence of low-frequency background and high-frequency interference, ensuring stitching quality. Static background extraction ensures robust stitching, maintaining image stability even with frequent dynamic object changes. Dynamic object removal reduces visual clutter during real-time monitoring, resulting in clearer and more reliable panoramas. This streamlines the stitching process in multi-camera surveillance scenarios, improving processing efficiency and automation. This system is suitable for real-time panoramic stitching tasks, meeting the time-sensitive requirements of airport surveillance.

[0012] Optionally, determining the first light wave field corresponding to the first camera used to shoot the first surveillance video and the second light wave field corresponding to the second camera used to shoot the second surveillance video specifically includes: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera based on the interference pattern diagram; and using non-uniform refraction to correct the perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera to generate the first light wave field and the second light wave field.

[0013] By employing the above technical solution, an accurate light wave propagation model is established for each camera's capture area by constructing the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera. This facilitates a detailed description of the viewing angle and optical characteristics of each camera. Light wave field modeling enables each camera's video frame to accurately reflect the light propagation process based on physical laws, avoiding stitching errors caused by optical distortion. Non-uniform refraction correction precisely corrects the camera's light wave field, correcting for distortion caused by optical lenses and ensuring that each element in the image is more accurately aligned with the real world. Non-uniform refraction correction effectively eliminates localized distortion caused by lens distortion between different cameras, enhancing the geometric consistency of the stitched image. After correction, the image geometry more closely resembles the actual scene, avoiding the misalignment caused by distortion in traditional stitching methods. Correcting for perspective distortion eliminates image distortion caused by different shooting angles. After perspective distortion correction, the light wave field of each camera more closely matches the actual viewing angle, ensuring that multiple surveillance videos can be stitched together naturally. By optimizing the optical wavefield, we reduce stitching errors caused by differences in camera angles, perspective distortion, and non-uniform refraction in traditional stitching, ensuring more natural and detailed panoramic images. This method can process video streams captured by multiple cameras, ensuring optical and geometric correction for each video even with varying viewpoints and optical distortions. This results in more precise stitching, improving stitching quality in multi-camera scenarios, adapting to various viewing angle configurations, such as wide-angle and telephoto, and enhancing the adaptability and stability of stitching.

[0014] Optionally, determining a panoramic monitoring map based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field, specifically includes: obtaining an overlapping area between the first area to be stitched and the second area to be stitched; smoothing the overlapping area using a Gaussian beam model to obtain a first stitching result; determining a key alignment feature based on the overlapping area; aligning the first area to be stitched and the second area to be stitched using an interference phase change based on the key alignment feature to obtain a second stitching result; and generating the panoramic monitoring map based on the first stitching result and the second stitching result.

[0015] By adopting the above technical solution, the overlapping area is smoothed using a Gaussian beam model, which can reduce the obviousness of the seam or transition area between the two areas to be stitched, making the stitching result smoother and more natural. The application of the Gaussian beam model helps to smooth the transition in the overlapping area, avoiding abrupt seams or distortion during the stitching process, so that the panoramic image presents a seamless stitching effect. The stitching error caused by changes in lighting, texture or perspective is reduced, and the overall visual coherence is enhanced. By analyzing the overlapping area and determining the key alignment features, the stitching alignment process is made more precise, ensuring that the most important geometric structures in the image are accurately aligned after stitching. The identification of key alignment features helps to accurately align the two areas to be stitched, avoiding texture misalignment or geometric distortion caused by inaccurate alignment. This step can extract important features for complex scenes and dynamic environments, ensuring the consistency of key information when stitching images. Interferometric phase change alignment can accurately calibrate the relative positions of the areas to be stitched based on the interference pattern, thereby enhancing the geometric and optical consistency of the stitched image. Interferometric phase shift alignment provides a physical alignment method capable of handling perspective differences, optical distortion, and perspective changes in complex scenes. The panoramic monitoring image generated by combining the first stitching result with the second stitching result after a smooth transition ensures the naturalness and consistency of the final stitched image. The application of key alignment features and interferometric phase shifting provides accurate alignment results in dynamic environments, preventing dynamic objects from affecting the stitching effect. The use of Gaussian beam smoothing reduces the impact of fast-moving objects in dynamic scenes on the stitching result, enhancing the visual consistency of the panorama.

[0016] Optionally, the method further includes: determining a target dynamic area based on the second light wave field; predicting a motion trajectory of the dynamic target by analyzing energy changes of the target dynamic area in the second light wave field; determining an occluded area in the second area to be stitched; performing interpolation and reconstruction on the occluded area based on the motion trajectory of the dynamic target to fill in texture information lost due to occlusion of the dynamic target; for areas with blurred texture in the second stitching result, using low-frequency light wave components of the texture information to supplement missing texture details to obtain a third stitching result; and generating the panoramic monitoring map based on the first stitching result and the third stitching result.

[0017] By employing this technical solution, by analyzing the target's dynamic regions and predicting its trajectory, it is possible to proactively identify and track moving targets, avoiding occlusion or distortion during the stitching process. Predicting and tracking dynamic targets ensures target consistency in the stitched images, preventing errors or dislocations caused by dynamic targets. Pre-identifying the dynamic target's motion path allows for real-time adjustments during the stitching process, avoiding spatial misalignment caused by target motion. Interpolation reconstruction of occluded areas effectively compensates for texture loss caused by occlusion, ensuring the integrity of all stitched images. By filling in texture information through interpolation reconstruction, the stitched result maintains visual integrity, avoiding blank or distorted areas. For texture blurring during the stitching process, low-frequency light wave components are used to fill in lost detail, improving image detail and accuracy. This low-frequency light wave component filling accurately restores details lost during the stitching process, ensuring more realistic texture detail and enhancing visual quality. By filling in missing details and avoiding blurry areas during the stitching process, the final panoramic image is made clearer and more refined. This method enables the stitching result to better reflect the continuity of the real scene, avoiding abrupt or discontinuous parts in the image caused by occlusion or dynamic targets. It ensures that the final panoramic image has clear and delicate details in each area, and the overall effect is more natural. Dynamic target prediction and occluded area interpolation reconstruction can effectively cope with the challenges in complex dynamic environments such as airports, improve the adaptability of the stitching process to dynamic targets, occlusions and other factors, enable the stitched image to cope with changes in the actual scene, and avoid the dynamic environment from having too much impact on the stitching accuracy.

[0018] Optionally, the method further includes: obtaining a target video frame in a third surveillance video, where the third surveillance video is any one of the multiple airport surveillance videos except the first surveillance video and the second surveillance video; dynamically adjusting the panoramic skeleton of the third optical wave field corresponding to the third surveillance video according to the target video frame; and optimizing the consistency of the panoramic skeleton using an autocorrelation calibration mechanism of the interference phase to eliminate the drift problem caused by splicing.

[0019] By employing the above technical solution, the panoramic skeleton of the third optical wave field is dynamically adjusted based on the target video frame, allowing optimization based on changes in actual video content, thereby improving the geometric accuracy of the stitching process. Dynamic adjustment of the panoramic skeleton helps adapt to changes in the surveillance video environment, ensuring precise adjustment of the perspective and alignment of each camera during the stitching process. This adjustment effectively addresses stitching deviations caused by differences in camera position and perspective, enhancing the geometric consistency and accuracy of the stitched image. An interferometric phase autocorrelation calibration mechanism is used to optimize the panoramic skeleton, ensuring that the stitching process is free of drift caused by perspective deviation or image misalignment. This mechanism ensures image consistency when stitching multiple perspectives, avoiding misalignment, distortion, or unnatural transitions in the stitched image, thereby improving overall quality. The introduction of the autocorrelation calibration mechanism enhances the stability and robustness of the stitching process, especially in dynamic scenes or when coordinating multiple cameras. This enhances the stitching system's adaptability in dynamic environments, ensuring consistent stitching across multiple camera perspectives under all environmental conditions. In dynamic environments like airports, this optimization mechanism can reliably handle subtle changes in large-scale surveillance scenes, ensuring high-quality final panoramic images. By precisely adjusting the panoramic skeleton and calibration mechanism, visual artifacts introduced by the stitching algorithm are reduced, optimizing the quality of the final panoramic surveillance image. This enhances the coherence and consistency of the panoramic image, ensuring natural transitions and clear image content.

[0020] In a second aspect of the present application, a panoramic stitching device for airport surveillance videos is provided, wherein the panoramic stitching device comprises an acquisition module and a processing module, wherein the acquisition module is used to acquire a first surveillance video and a second surveillance video for a target airport, wherein the target airport corresponds to multiple airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos among the multiple airport surveillance videos; the processing module is used to determine an interference pattern diagram based on the first surveillance video and the second surveillance video; the processing module is further used to perform a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched; the processing module is further used to determine a first optical wave field corresponding to a first camera used to shoot the first surveillance video, and a second optical wave field corresponding to a second camera used to shoot the second surveillance video; the processing module is further used to determine a panoramic surveillance diagram based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field.

[0021] In a third aspect of the present application, an electronic device is provided, which includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs the method described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, the method described above is executed.

[0023] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages:

[0024] By introducing interference patterns, the feature information in the first and second surveillance videos is converted into high-frequency phase interference feature patterns, enabling precise identification of static background and dynamic targets in each video. The dynamic target removal operation effectively filters out interference from dynamic targets such as aircraft and ground vehicles, ensuring clarity and accuracy in the stitching process and eliminating ghosting or blurring. By establishing a separate optical wavefield model for each camera, distortion issues caused by camera angle, field of view, and perspective distortion are addressed. The optical wavefield contains rich phase information, enabling natural alignment and fusion between multi-view videos, eliminating misalignment and distortion at seams and ensuring a smooth transition in the stitching result. The static background extraction operation based on the interference pattern utilizes the high-frequency phase consistency characteristics of the static background to stably extract static background information in scenes with complex lighting conditions or frequent dynamic target motion. This approach avoids the failure of traditional background extraction methods based on inter-frame differencing due to lighting changes or dynamic interference, thereby improving the robustness of the stitching algorithm. By combining the first light wave field and the second light wave field, the high-frequency details captured in multiple videos are accurately integrated into the panoramic image, making the stitched image clearer in terms of space and detail. At the same time, the light wave field technology can dynamically compensate for the color differences and brightness changes between the areas covered by different cameras to ensure the color consistency of the panoramic image. The combination of the interference pattern image and the light wave field reduces the computational complexity of multi-step feature matching and complex registration operations in traditional stitching technology, enabling panoramic stitching to achieve near real-time updates in a high-frame-rate video streaming environment. This method utilizes the frequency domain characteristics of the interference pattern image and the light wave field, and can adapt to background feature extraction and stitching under different lighting conditions. Regardless of daytime, nighttime or weather conditions, the stitched panorama can maintain stable quality and consistency. Therefore, it is convenient to perform panoramic stitching of airport surveillance videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1A flowchart of a panoramic stitching method for airport surveillance videos provided in an embodiment of the present application;

[0026] Figure 2 Another schematic diagram of a flow chart of a panoramic stitching method for airport surveillance videos provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of a module of a panoramic stitching device for airport surveillance videos provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0029] Explanation of the reference numerals: 31, acquisition module; 32, processing module; 41, processor; 42, communication bus; 43, user interface; 44, network interface; 45, memory. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0031] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0032] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0033] As important public transportation hubs, airports involve complex scenarios such as aircraft taxiing, apron scheduling, ground handling, and passenger movement. To ensure safe and efficient operations, modern airports widely deploy multi-camera surveillance systems to cover critical areas such as runways, taxiways, and aprons. However, due to the vast space and frequent dynamic objects in airport scenes, panoramic stitching of videos captured by multiple cameras presents numerous challenges.

[0034] Existing panoramic stitching technologies rely on traditional methods such as feature point matching or image registration. However, in airport surveillance scenarios, dynamic objects such as taxiing aircraft, ground handling vehicles, and moving personnel significantly interfere with the stitching results, causing ghosting and stitching errors. Furthermore, differences in viewing angles between multiple cameras can cause distortion in the stitched area, resulting in noticeable seams and unnatural image fusion. Therefore, traditional feature point matching and image registration methods are less effective for panoramic stitching of airport surveillance videos.

[0035] In order to solve the above technical problems, this application provides a panoramic stitching method for airport surveillance video, referring to Figure 1 , Figure 1 This is a flow chart of a panoramic stitching method for airport surveillance videos provided in an embodiment of the present application. The panoramic stitching method is applied to a server and includes steps S110 to S150, which are as follows:

[0036] S110: Acquire a first surveillance video and a second surveillance video for a target airport. The target airport corresponds to multiple airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos from the multiple airport surveillance videos.

[0037] Specifically, the server needs to select surveillance videos for the target airport from the surveillance system. The target airport refers to the airport of interest, which may be a specific airport. The server is a separate computer system or a server cluster used to manage the airport's surveillance system. The target airport has multiple surveillance video sources. For example, the airport may deploy multiple cameras to cover different areas, such as runways, taxiways, boarding gates, parking lots, etc. Each camera will produce a surveillance video. Among these multiple surveillance videos, the server selects two specific videos, called the first surveillance video and the second surveillance video. These two videos come from different cameras and may be surveillance videos covering different areas of the airport. The two videos selected from the multiple surveillance videos do not need to be fixed, but can be any two surveillance videos. In other words, the first video and the second video can be any two surveillance videos, which may be two cameras from different locations or different perspectives.

[0038] For example, consider an airport with multiple surveillance cameras deployed to cover different areas. These cameras are installed at locations such as the runway, apron, and boarding gates. The first surveillance video is captured by a camera on the runway. This video might cover aircraft taxiing, taking off, and landing on the runway. The second surveillance video is captured by another camera on the apron, showing scenes such as aircraft parking and ground crew operations.

[0039] S120: Determine an interference pattern diagram based on the first monitoring video and the second monitoring video.

[0040] Specifically, an interference pattern map is an image based on the interaction between different frames in a video. By analyzing the image details in two videos, the generated interference pattern map can reveal the relationship between the different videos, especially the changes in dynamic objects. The generation of the interference pattern map is related to frequency domain image processing. Using techniques such as Fourier transform, it can analyze the subtle changes between the two video frames and find the phase differences between them. These differences manifest as the interference effects caused by dynamic objects in the image. This map can reveal detailed differences between different videos, especially in dynamic environments, such as moving objects and scene changes.

[0041] In one possible implementation, an interference pattern diagram is determined based on the first surveillance video and the second surveillance video, specifically including: performing Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; calculating the phase change between the first high-frequency texture and the second high-frequency texture to generate an interference pattern diagram.

[0042] Specifically, the Fourier transform converts signals from the time domain to the frequency domain. In the frequency domain, different parts of an image are represented by different frequency components. For video, the Fourier transform can help extract high-frequency and low-frequency information from the image. High-frequency components represent image details, such as edges and textures, while low-frequency components represent smooth areas, such as the background. Performing a frame-by-frame Fourier transform means the server performs a Fourier transform on each frame of the video, rather than processing the entire image. This enables detailed analysis of each moment in the video and captures changes in dynamic objects. After the Fourier transform, the frequency domain of the video frame contains high-frequency textures. These high-frequency textures contain image details, particularly changes associated with dynamic objects. The first and second high-frequency textures represent the high-frequency components of the first and second surveillance videos, respectively. Because the surveillance videos are from different viewpoints, the high-frequency components in the two videos may differ. In particular, when there are dynamic objects, changes in these dynamic elements are reflected in the high-frequency textures. Phase shift refers to the phase difference between the frequency components of the two images in the frequency domain. By calculating the phase shift between the high-frequency textures of the two videos, subtle changes in the images, particularly the motion of dynamic objects, can be revealed. When stitching together multiple surveillance videos, dynamic objects can cause variations between video frames. Calculating these phase shifts helps precisely align dynamic elements in different videos. An interference pattern is generated by calculating the phase shifts in the high-frequency textures of two videos. It reveals the differences between the two videos, particularly those caused by dynamic objects. The interference pattern reveals information such as the relative displacement between the two video sources and the target's trajectory, which is crucial for accurately aligning different viewpoints and dynamic objects during the stitching process.

[0043] For example, consider two surveillance videos: one from an airport runway camera and the other from an apron camera. These videos capture dynamic scenes in different areas, such as aircraft taxiing on the runway and ground crew operations on the apron. The first surveillance video contains the dynamic scene of the aircraft taxiing. The aircraft's movement on the runway causes changes in detail in the video, particularly in the area surrounding the aircraft. The second surveillance video captures ground crew operating the aircraft on the apron, also showing dynamic changes, such as personnel movement or equipment use. The server first performs a Fourier transform on each frame of the runway and apron camera videos, converting the images into the frequency domain. This yields high-frequency texture, representing the details in the image. By comparing the high-frequency textures of the runway and apron videos, the server calculates the phase changes in the details in the two videos. For example, when an aircraft taxis on the runway, it changes the high-frequency texture in the video, resulting in a phase difference. Similarly, when ground crew operate on the apron, changes in the video are also produced. By analyzing the phase changes in the two videos, the server generates an interference pattern. This figure reveals the dynamic differences between the two videos, especially in the high-frequency regions of the image. These differences are related to the motion trajectories of dynamic objects such as the taxiing aircraft and ground crew operations.

[0044] S130 , performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced.

[0045] Specifically, an interference pattern map has been generated. It shows the difference between dynamic targets and background caused by phase changes between two surveillance video frames. This map reveals the distinction between dynamic targets and static background. Static background extraction involves identifying unchanging portions of the video from the interference pattern map and distinguishing them from dynamic targets. Background portions are objects or areas that remain stationary in the video, while dynamic targets are portions of the video that show noticeable movement or change. This operation can utilize image processing techniques, such as background modeling and frame subtraction, to extract stable, unchanging background areas in the video. For example, runway markings and buildings are static background, while people and vehicles at the airport are dynamic targets. Dynamic target removal involves removing all changes caused by dynamic targets from the interference pattern map, retaining only the background information. By removing these dynamic targets, they are prevented from interfering with the stitching result. Dynamic targets can cause varying degrees of distortion, ghosting, or stitching errors in the two video sources. Therefore, by removing these dynamic targets, the server can obtain a more accurate stitching area. Dynamic target removal can be accomplished using a high-pass filter or background subtraction methods. A high-pass filter helps remove low-frequency content while retaining high-frequency content. The "joined region" refers to a portion of the image extracted from the interference pattern. This portion of the image represents the static background area after dynamic objects have been removed, making it suitable for stitching. The first and second "joined regions" are extracted regions from two surveillance videos, respectively. These regions will be combined into a seamless panorama in the subsequent stitching step.

[0046] In one possible implementation, a static background extraction operation and a dynamic target removal operation are performed on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced, specifically including: inputting the interference pattern image into a light wave propagation model, and obtaining a first result through static background extraction; using a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; and determining the first area to be spliced ​​and the second area to be spliced ​​using the light wave propagation model based on the first result and the second result.

[0047] Specifically, the light wave propagation model is a mathematical model that describes how light waves, or image information, propagate. In this context, the light wave propagation model is used to extract static background components from the interference pattern. It helps identify static background areas and distinguish dynamic targets. During static background extraction, the server uses the light wave propagation model to identify time-invariant components of the interference pattern. These components include areas such as airport grounds, buildings, and road markings that do not change over time or with object movement. The first result of static background extraction is the extracted background components from the interference pattern. This helps isolate stable, stitchable areas from complex surveillance images. The second result is the image after dynamic targets are removed, retaining the background information without dynamic targets such as people and vehicles. Based on these first and second results, the server further uses the light wave propagation model to determine image regions suitable for stitching. These regions are the first and second regions to be stitched. They are derived from two surveillance videos, each with the interference of dynamic targets removed, ready for stitching.

[0048] S140: Determine a first light wave field corresponding to a first camera used for shooting a first surveillance video, and a second light wave field corresponding to a second camera used for shooting a second surveillance video.

[0049] Specifically, a light wave field is a mathematical model that describes the propagation of light waves in an image or video. In the embodiments of this application, the light wave field is used to represent the relationship between a camera's perspective and the scene. Each camera's perspective produces a different light wave field when capturing a scene. These light wave fields affect the video's appearance, such as light refraction, perspective, and distortion. For the first surveillance video, the server determines the light wave field of the camera that captured the video. This determination is based on factors such as the camera's position, angle, focal length, and shooting environment. These factors determine the perspective, field of view, and possible distortion of the captured image. Similarly, the camera capturing the second surveillance video will also have a corresponding light wave field. The second camera may be located at a different position and angle, so its light wave field will differ from that of the first camera. The server determines the second camera's light wave field based on factors such as its position and angle.

[0050] In one possible implementation, determining a first light wave field corresponding to a first camera used to shoot a first surveillance video and a second light wave field corresponding to a second camera used to shoot a second surveillance video specifically includes: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera based on an interference pattern diagram; and using non-uniform refraction to correct perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera to generate the first light wave field and the second light wave field.

[0051] Specifically, the server analyzes the interference pattern to determine how to construct the initial light wave fields for the two cameras. These initial light wave fields are preliminary models of each camera's perspective, representing the light wave propagation characteristics of the scene captured by the camera. In real-world scenarios, different cameras will produce different light wave propagation effects when capturing the same scene. In particular, different camera perspectives, shooting distances, and focal lengths can lead to certain image distortions, particularly perspective distortion, where objects near and far differ in size. Non-uniform refraction correction adjusts for these non-uniform refraction effects, ensuring a more accurate correspondence between the light wave fields generated by the camera perspectives and the actual scene. This correction reduces parallax differences between cameras, resulting in a more natural stitching effect. After this correction, the first and second light wave fields are generated. These two wave fields accurately represent the scene depicted by the two cameras, taking into account factors such as perspective, refraction, and perspective. These two wave fields are used to guide the subsequent panoramic stitching process, ensuring better alignment between the two images and reducing distortion or unnatural seams during stitching.

[0052] S150: Determine a panoramic monitoring map based on the first area to be spliced ​​and the second area to be spliced, in combination with the first light wave field and the second light wave field.

[0053] Specifically, during the airport surveillance video stitching process, the first and second regions to be stitched refer to the image regions obtained through the aforementioned background extraction and dynamic object removal operations. These regions are the actual portions of the video that need to be stitched together, potentially due to overlapping or similar content captured by different cameras. The first and second optical wave fields refer to the light wave propagation characteristics of the scene captured by each camera. These optical wave fields describe the visual characteristics of the scene, including factors such as camera angle, distance, and lighting. These optical wave fields play a crucial role in image alignment and fusion during the stitching process, ensuring a natural transition between stitched regions with no noticeable seams. Combining the first and second regions to be stitched, along with their respective optical wave fields, means that the server utilizes the overlap of the two regions and the characteristics of the two optical wave fields to stitch the images. Specifically, the stitching algorithm smoothes, aligns, and fuses the images based on the overlap between the two regions, ensuring a more natural and coherent panoramic image. Ultimately, based on this information, the server generates a panoramic surveillance image of the entire surveillance area—a large-scale image that combines content captured by multiple cameras. The panoramic monitoring map provides a complete airport monitoring view, covering multiple areas, making it easier for managers to monitor every corner of the airport from a unified perspective.

[0054] In one possible implementation, a panoramic monitoring map is determined based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field. The method specifically includes: obtaining an overlapping area between the first area to be stitched and the second area to be stitched; smoothing the overlapping area using a Gaussian beam model to obtain a first stitching result; determining a key alignment feature based on the overlapping area; aligning the first area to be stitched and the second area to be stitched using an interference phase change based on the key alignment feature to obtain a second stitching result; and generating a panoramic monitoring map based on the first stitching result and the second stitching result.

[0055] Specifically, when stitching surveillance videos, the two video regions to be stitched may partially overlap. This overlap is where the image content captured by the two cameras intersects. This overlapping area is a key focus for the stitching algorithm, as it must ensure a seamless visual connection. The Gaussian beam model is a mathematical model that simulates the propagation of light waves and exhibits smooth and gentle transitions. When stitching images, the Gaussian beam model is used to process the overlapping areas, smoothing the transitions and reducing the likelihood of noticeable seams or abrupt transitions. This step produces the initial stitching result—the preliminarily processed stitched image—with a relatively natural transition effect. In the overlapping area, there are features that may aid in image alignment, such as texture, corners, or edges. These features are called key alignment features. By analyzing the image content in the overlapping area, the algorithm can determine which features are most important for stitching and use these features for subsequent fine-tuning. Interferometric phase shift technology can be used to analyze the phase differences between the two images and align key features of the two images by adjusting the image's position and rotation. Here, the algorithm leverages key alignment features in the overlapping area to perform phase correction and position adjustment, ensuring precise stitching of the two areas. Finally, through interferometric phase shift alignment, a second stitching result is obtained: this finely aligned stitched image. Finally, the server combines the first and second stitching results to generate the final panoramic surveillance image. This panorama covers a larger area of ​​the airport, ensuring a natural, seamless transition between images.

[0056] For example, consider an airport surveillance system with two cameras, one on the east and one on the west side of a runway. East camera 1 captures the eastern half of the runway, while west camera 2 captures the western half. The images captured by these two cameras overlap partially, such as at the junction of the two runway sections. Assume that the east and west images overlap in the middle of the runway, which is a critical area for stitching. During the stitching process, the Gaussian beam model is first used to smooth the overlapping area. For example, the overlapping area in the middle of the runway may have a seam due to different lighting conditions or camera angles. Using the Gaussian beam model, the overlapping area is smoothed, reducing the abruptness during stitching and producing the initial stitching result. The overlapping area may contain some landmark features, such as the runway edge or ground markings, which can serve as key alignment features. By analyzing these features, the algorithm determines the corresponding relationship between these points in the two images. Based on the key alignment features in the overlapping area, the server uses an interferometric phase change algorithm to finely align the two areas. For example, runway edges or ground markings can be precisely aligned, ensuring no noticeable misalignment between the two video streams when stitched together. After interferometric phase adjustment, the resulting second stitching results are even more accurate and seamless. Finally, combining the first and second stitching results creates a large panoramic surveillance image encompassing the entire airport runway, ensuring seamless connectivity and enabling monitoring personnel to comprehensively review the airport's conditions.

[0057] In one possible implementation, a target dynamic region is determined based on a second optical wave field; the motion trajectory of the dynamic target is predicted by analyzing energy changes in the target dynamic region in the second optical wave field; an occluded region in the second area to be stitched is determined; the occluded region is interpolated and reconstructed based on the motion trajectory of the dynamic target to fill in texture information lost due to occlusion of the dynamic target; for regions with blurred textures in the second stitching result, low-frequency optical wave components are used to supplement missing texture details based on the texture information to obtain a third stitching result; and a panoramic monitoring image is generated based on the first and third stitching results.

[0058] Specifically, by observing energy changes in dynamic regions of an object, the target's movement direction and speed can be inferred. Dynamic objects produce energy fluctuations or variations in the video, so analyzing these changes can help predict the target's trajectory. This helps understand where the dynamic object will appear, enabling subsequent processing to repair occluded areas. Occluded areas are areas of the image that are blocked or covered by the presence of a dynamic object, resulting in a loss of image detail. For example, a moving vehicle may obstruct portions of the ground or other objects, obscuring parts of the image. By analyzing the image, the server can identify these occluded areas and identify them as key areas for inpainting. Interpolation reconstruction is a commonly used image inpainting technique. In this step, the server infers the content of the occluded area based on the previously predicted trajectory of the dynamic object. For example, if a vehicle obscures the background, the server can use interpolation techniques to infer the background image previously obscured by the vehicle and fill in the occluded area with this image information, restoring the lost texture. Texture blurring can occur due to a variety of reasons, such as camera angle, focal length, or uneven lighting. For these areas, the server analyzes existing texture information, such as color and lighting characteristics, and uses a low-frequency light wave component completion method to repair the blurred areas. Low-frequency light wave components primarily reflect the large-scale structure and shape of the image, while the restoration of texture information can make the image details more complete, ultimately resulting in the third stitching result, that is, the repaired and optimized stitching image. In this step, the server combines the first and third stitching results obtained previously to generate the final panoramic monitoring image. This panoramic image integrates images from different cameras, ensuring the integrity of the entire monitoring area. Even under the influence of dynamic targets, the image remains clear and seamless.

[0059] In one possible implementation, refer to Figure 2 , Figure 2 Another flow diagram of a panoramic stitching method for airport surveillance videos provided in an embodiment of the present application includes steps S210 to S230, which are as follows: S210: Acquire a target video frame from a third surveillance video, where the third surveillance video is any surveillance video from multiple airport surveillance videos other than the first and second surveillance videos; S220: Dynamically adjust the panoramic skeleton of the third optical wave field corresponding to the third surveillance video based on the target video frame; S230: Optimize the consistency of the panoramic skeleton using an autocorrelation calibration mechanism of the interferometric phase to eliminate drift caused by stitching.

[0060] Specifically, the third surveillance video refers to a video selected from multiple airport surveillance videos, excluding the first and second surveillance videos. This video may be from the perspective of other cameras within the airport. The goal of this step is to extract key target video frames from this third video—important image frames relevant to the stitching process. Assume that the airport has multiple cameras. The first surveillance video captured by camera 1 and the second surveillance video captured by camera 2 cover both sides of the runway, while the third surveillance video captured by camera 3 may monitor the apron or taxiway. By extracting the target frames from the third video, the server can use this information to perform panoramic stitching. The third optical wave field is the light wave propagation model corresponding to the third surveillance video. This optical wave field represents the perspective and imaging characteristics of camera 3. The panoramic skeleton is a virtual framework or structure that represents the basic layout of the entire surveillance video stitching process. During the stitching process, the server needs to dynamically adjust the optical wave field based on the image content and perspective of the third video to ensure that the images are properly aligned when the panorama is synthesized. If the viewing angle of camera 3 is different from that of camera 1 or camera 2, or the position of camera 3 changes, the server needs to dynamically adjust its optical wave field according to the image information of the third surveillance video to maintain the spatial consistency of the panorama.

[0061] Interferometric phase refers to the phase difference between different images or light waves. In image stitching, interferometric phase is used to assess image alignment. The autocorrelation calibration mechanism is an algorithm that optimizes stitching consistency by comparing phase changes between different video frames, reducing visual errors such as drift during stitching, which can result in image misalignment or distortion. This mechanism enables the server to dynamically adjust the stitching results by comparing phase changes across different viewpoints and viewing distances, ensuring precise alignment of content from different video sources and avoiding unnatural gaps or offsets when stitching. Suppose, during the stitching process, the images from camera 1 and camera 2 are precisely aligned, but the image from camera 3 is misaligned due to a viewpoint difference or movement. The server uses the autocorrelation calibration mechanism based on interferometric phase to detect this misalignment and optimize the panoramic skeleton, eliminating stitching drift caused by the misaligned viewpoint. Drift refers to image misalignment or misalignment when stitching images from multiple camera perspectives. This problem is caused by errors resulting from different camera viewpoints, focal lengths, or camera motion. By applying an interferometric phase autocorrelation calibration mechanism, the server can accurately correct these errors, eliminating drift during image stitching and making the final stitching result more natural and seamless. If the camera in the third surveillance video is slightly offset from the first and second surveillance videos due to vibration or slight position shift, the server automatically corrects these errors through the interferometric phase autocorrelation calibration mechanism, ensuring that the third video's image can be correctly stitched together with the first two videos.

[0062] This application also provides a panoramic stitching device for airport surveillance video, referring to Figure 3 , Figure 3 A schematic diagram of a module for a panoramic stitching device for airport surveillance videos provided in an embodiment of the present application. The panoramic stitching device is a server, comprising an acquisition module 31 and a processing module 32. The acquisition module 31 acquires a first surveillance video and a second surveillance video for a target airport, where the target airport corresponds to multiple airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos from the multiple airport surveillance videos. The processing module 32 determines an interference pattern diagram based on the first surveillance video and the second surveillance video. The processing module 32 performs a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched. The processing module 32 determines a first optical wave field corresponding to a first camera used to shoot the first surveillance video and a second optical wave field corresponding to a second camera used to shoot the second surveillance video. The processing module 32 determines a panoramic surveillance map based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field.

[0063] In one possible implementation, the processing module 32 determines an interference pattern diagram based on the first surveillance video and the second surveillance video, specifically including: the processing module 32 performs Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; the processing module 32 calculates the phase change between the first high-frequency texture and the second high-frequency texture to generate an interference pattern diagram.

[0064] In one possible implementation, the processing module 32 performs a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced, specifically including: the processing module 32 inputs the interference pattern image into the light wave propagation model, and obtains a first result through static background extraction; the processing module 32 uses a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; the processing module 32 uses the light wave propagation model to determine the first area to be spliced ​​and the second area to be spliced ​​based on the first result and the second result.

[0065] In one possible implementation, the processing module 32 determines a first light wave field corresponding to a first camera used to shoot a first surveillance video, and a second light wave field corresponding to a second camera used to shoot a second surveillance video. Specifically, the processing module 32 constructs an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera based on the interference pattern diagram; the processing module 32 uses non-uniform refraction to correct the perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera to generate the first light wave field and the second light wave field.

[0066] In one possible embodiment, the processing module 32 determines a panoramic monitoring map based on the first area to be stitched and the second area to be stitched, in combination with the first light wave field and the second light wave field, specifically including: the acquisition module 31 acquires the overlapping area between the first area to be stitched and the second area to be stitched; the processing module 32 uses a Gaussian beam model to smooth the overlapping area to obtain a first stitching result; the processing module 32 determines the key alignment feature based on the overlapping area; the processing module 32 uses interference phase change to align the first area to be stitched and the second area to be stitched based on the key alignment feature to obtain a second stitching result; the processing module 32 generates a panoramic monitoring map based on the first stitching result and the second stitching result.

[0067] In one possible implementation, the processing module 32 determines the target dynamic area based on the second light wave field; the processing module 32 predicts the motion trajectory of the dynamic target by analyzing the energy changes of the target dynamic area in the second light wave field; the processing module 32 determines the occluded area in the second area to be stitched; the processing module 32 interpolates and reconstructs the occluded area based on the motion trajectory of the dynamic target to fill in the texture information lost due to the occlusion of the dynamic target; the processing module 32 uses low-frequency light wave components to supplement the missing texture details in the area with blurred texture in the second stitching result through texture information to obtain a third stitching result; the processing module 32 generates a panoramic monitoring map based on the first stitching result and the third stitching result.

[0068] In one possible implementation, the acquisition module 31 acquires a target video frame in a third surveillance video, where the third surveillance video is any one of multiple airport surveillance videos except the first surveillance video and the second surveillance video; the processing module 32 dynamically adjusts the panoramic skeleton of the third optical wave field corresponding to the third surveillance video according to the target video frame; the processing module 32 uses an autocorrelation calibration mechanism of the interference phase to optimize the consistency of the panoramic skeleton to eliminate the drift problem caused by splicing.

[0069] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0070] This application also provides an electronic device, referring to Figure 4 , Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include: at least one processor 41, at least one network interface 44, a user interface 43, a memory 45, and at least one communication bus 42.

[0071] The communication bus 42 is used to realize the connection and communication between these components.

[0072] The user interface 43 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 43 may also include a standard wired interface and a wireless interface.

[0073] The network interface 44 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0074] The processor 41 may include one or more processing cores. Using various interfaces and circuits, the processor 41 connects to various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 45, as well as accesses data stored in the memory 45, to perform various server functions and process data. Optionally, the processor 41 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 41 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may also be implemented as a separate chip, rather than integrated into the processor 41.

[0075] Among them, the memory 45 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 45 includes a non-transitory computer-readable storage medium. The memory 45 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 45 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 45 may also be optionally at least one storage device located away from the aforementioned processor 41. As Figure 4 As shown, the memory 45 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a panoramic stitching method of airport surveillance videos.

[0076] exist Figure 4 In the electronic device shown, the user interface 43 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 41 can be used to call an application program for a panoramic stitching method of airport surveillance video stored in the memory 45. When executed by one or more processors, the electronic device executes one or more methods as in the above-mentioned embodiments.

[0077] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0078] The present application also provides a computer-readable storage medium storing instructions, which, when executed by one or more processors, enable an electronic device to execute one or more of the methods described in the above embodiments.

[0079] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0081] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0082] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0083] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory includes various media that can store program code, such as USB flash drives, mobile hard drives, magnetic disks, or optical disks.

[0084] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A panoramic stitching method for airport surveillance video, characterized in that: The method comprises: Obtain a first surveillance video and a second surveillance video for a target airport, where the target airport corresponds to multiple airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos from the multiple airport surveillance videos; determining an interference pattern diagram based on the first surveillance video and the second surveillance video; Performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced; Determining a first light wave field corresponding to a first camera used to shoot the first surveillance video, and a second light wave field corresponding to a second camera used to shoot the second surveillance video; determining the first light wave field corresponding to the first camera used to shoot the first surveillance video, and the second light wave field corresponding to the second camera used to shoot the second surveillance video, specifically comprising: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera based on the interference pattern diagram; and using non-uniform refraction to correct perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera to generate the first light wave field and the second light wave field; Determining a panoramic monitoring map based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field; determining the panoramic monitoring map based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field, specifically includes: obtaining an overlapping area between the first area to be stitched and the second area to be stitched; smoothing the overlapping area using a Gaussian beam model to obtain a first stitching result; and determining a key alignment feature based on the overlapping area; According to the key alignment feature, the first area to be stitched and the second area to be stitched are aligned using interference phase change to obtain a second stitching result; and the panoramic monitoring image is generated according to the first stitching result and the second stitching result.

2. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The determining of the interference pattern diagram according to the first monitoring video and the second monitoring video specifically includes: Performing Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; The phase change between the first high-frequency texture and the second high-frequency texture is calculated to generate the interference pattern diagram.

3. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The step of performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be stitched and a second area to be stitched specifically includes: Inputting the interference pattern into a light wave propagation model and obtaining a first result through static background extraction; Using a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; The first area to be spliced ​​and the second area to be spliced ​​are determined using the light wave propagation model according to the first result and the second result.

4. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The method further comprises: determining a target dynamic area according to the second optical wave field; Predicting the motion trajectory of the dynamic target by analyzing the energy change of the dynamic region of the target in the second light wave field; Determining an occluded area in the second area to be stitched; According to the motion trajectory of the dynamic target, the occluded area is interpolated and reconstructed to fill in the texture information lost due to the occlusion of the dynamic target; For the area with blurred texture in the second stitching result, the missing texture details are supplemented by using the low-frequency light wave component according to the texture information to obtain a third stitching result; The panoramic monitoring image is generated according to the first stitching result and the third stitching result.

5. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The method further comprises: Acquire a target video frame in a third surveillance video, where the third surveillance video is any one of the plurality of airport surveillance videos except the first surveillance video and the second surveillance video; Dynamically adjusting the panoramic skeleton of the third optical wave field corresponding to the third monitoring video according to the target video frame; The panorama skeleton is optimized for consistency by using an autocorrelation calibration mechanism of the interferometric phase to eliminate the drift problem caused by stitching.

6. A panoramic stitching device for airport surveillance video, characterized in that: The panoramic stitching device comprises an acquisition module (31) and a processing module (32), wherein: The acquisition module (31) is used to acquire a first surveillance video and a second surveillance video for a target airport, wherein the target airport corresponds to a plurality of airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos among the plurality of airport surveillance videos; The processing module (32) is used to determine an interference pattern diagram based on the first monitoring video and the second monitoring video; The processing module (32) is further configured to perform a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced; The processing module (32) is further configured to determine a first light wave field corresponding to a first camera used to shoot the first surveillance video, and a second light wave field corresponding to a second camera used to shoot the second surveillance video; the determining of the first light wave field corresponding to the first camera used to shoot the first surveillance video, and the second light wave field corresponding to the second camera used to shoot the second surveillance video, specifically comprising: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to the interference pattern diagram; and using non-uniform refraction to correct perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera to generate the first light wave field and the second light wave field; The processing module (32) is further configured to determine a panoramic monitoring map based on the first area to be spliced ​​and the second area to be spliced, in combination with the first optical wave field and the second optical wave field; the determining of the panoramic monitoring map based on the first area to be spliced ​​and the second area to be spliced, in combination with the first optical wave field and the second optical wave field, specifically comprises: obtaining an overlapping area between the first area to be spliced ​​and the second area to be spliced; smoothing the overlapping area using a Gaussian beam model to obtain a first splicing result; determining a key alignment feature based on the overlapping area; aligning the first area to be spliced ​​and the second area to be spliced ​​using an interference phase change based on the key alignment feature to obtain a second splicing result; and generating the panoramic monitoring map based on the first splicing result and the second splicing result.

7. An electronic device, characterized in that: The electronic device comprises a processor (41), a memory (45), a user interface (43) and a network interface (44), wherein the memory (45) is used to store instructions, the user interface (43) and the network interface (44) are both used to communicate with other devices, and the processor (41) is used to execute the instructions stored in the memory (45) so that the electronic device executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 5 is performed.

Images