Panoramic stitching method and device for airport monitoring video, and electronic equipment

Abstract

The invention provides a panoramic stitching method and device for an airport monitoring video and electronic equipment, and relates to the field of data processing. A first monitoring video and a second monitoring video for the target airport are obtained, the target airport corresponds to a plurality of airport monitoring videos, and the first monitoring video and the second monitoring video are any two monitoring videos in the plurality of airport monitoring videos; determining an interference mode pattern according to the first monitoring video and the second monitoring video; performing static background extraction operation and dynamic target removal operation on the interference mode pattern to obtain a first to-be-spliced region and a second to-be-spliced region; determining a first light wave field corresponding to a first camera used for shooting the first monitoring video and a second light wave field corresponding to a second camera used for shooting the second monitoring video; and based on the first to-be-spliced area and the second to-be-spliced area, combining the first light wave field and the second light wave field, and determining a panoramic monitoring image. By implementing the technical scheme, panoramic stitching of airport monitoring videos is facilitated.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a method, device and electronic equipment for panoramic stitching of airport surveillance videos. Background Art

[0002] As an important hub for public transportation, airports involve complex scenarios such as aircraft taxiing, apron scheduling, ground handling operations, and passenger activities. To ensure safe and efficient operations, modern airports widely deploy multi-camera monitoring systems to cover key areas such as runways, taxiways, and aprons. However, due to the large spatial range and frequent dynamic targets of airport scenes, videos shot by multiple cameras face challenges in the panoramic stitching process.

[0003] At present, the relevant panoramic stitching technology relies on the traditional method of feature point matching or image registration. However, in the airport monitoring scene, due to the interference of dynamic targets such as taxiing aircraft, ground handling vehicles and personnel movement, the stitching results are prone to dynamic interference leading to ghosting and stitching errors, as well as distortion caused by differences in multi-camera perspectives, resulting in obvious seams in the stitching area and unnatural image fusion. Therefore, the traditional method of feature point matching or image registration is not conducive to panoramic stitching of airport monitoring videos.

[0004] Therefore, there is an urgent need for a panoramic stitching method, device and electronic equipment for airport surveillance videos. Summary of the invention

[0005] The present application provides a method, device and electronic equipment for panoramic stitching of airport surveillance videos, which facilitates panoramic stitching of airport surveillance videos.

[0006] In a first aspect of the present application, a method for panoramic stitching of airport surveillance videos is provided, the method comprising: acquiring a first surveillance video and a second surveillance video for a target airport, the target airport corresponding to a plurality of airport surveillance videos, the first surveillance video and the second surveillance video being any two surveillance videos of the plurality of airport surveillance videos; determining an interference pattern diagram according to the first surveillance video and the second surveillance video; performing a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched; determining a first optical wave field corresponding to a first camera used to shoot the first surveillance video, and a second optical wave field corresponding to a second camera used to shoot the second surveillance video; determining a panoramic surveillance diagram based on the first area to be stitched and the second area to be stitched in combination with the first optical wave field and the second optical wave field.

[0007] By adopting the above technical solution, through the introduction of the interference pattern map, the feature information in the first surveillance video and the second surveillance video is converted into a high-frequency feature map of phase interference, which can accurately identify the static background and dynamic targets in each surveillance video. The dynamic target removal operation can effectively filter the interference of dynamic targets such as aircraft and ground vehicles on the splicing process, ensure the clarity and accuracy of the area to be spliced, and eliminate ghosting or blurring problems. By establishing an independent light wave field model for each camera, the distortion problem caused by the camera shooting angle, field of view and perspective deformation is solved. The light wave field contains rich phase information, which can realize the natural alignment and fusion between multi-view videos, eliminate the dislocation and deformation at the seams, and ensure a smooth transition of the splicing results. The static background extraction operation based on the interference pattern map uses the high-frequency phase consistency characteristics of the static background to stably extract static background information in scenes with complex lighting conditions or frequent movement of dynamic targets. This method avoids the problem that the traditional background extraction method based on inter-frame difference fails due to lighting changes or dynamic interference, and improves the robustness of the splicing algorithm. By combining the first light wave field and the second light wave field, the high-frequency details captured in multiple videos are accurately integrated into the panoramic image, making the stitched image clearer in terms of space and detail. At the same time, the light wave field technology can dynamically compensate for the color differences and brightness changes between the areas covered by different cameras to ensure the color consistency of the panoramic image. The combination of the interference pattern map and the light wave field reduces the computational complexity of multi-step feature matching and complex registration operations in traditional stitching technology, enabling panoramic stitching to achieve near real-time updates in a high-frame-rate video streaming environment. This method utilizes the frequency domain characteristics of the interference pattern map and the light wave field, and can adapt to background feature extraction and stitching under different lighting conditions. Regardless of daytime, nighttime or weather changes, the stitched panorama can maintain stable quality and consistency. Therefore, it is convenient to perform panoramic stitching of airport surveillance videos.

[0008] Optionally, determining the interference pattern diagram based on the first surveillance video and the second surveillance video specifically includes: performing Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; calculating the phase change between the first high-frequency texture and the second high-frequency texture to generate the interference pattern diagram.

[0009] By adopting the above technical solution, Fourier transform converts the surveillance video frame in the time domain to the frequency domain, highlighting the high-frequency texture features in the image, which mainly correspond to the detail information in the video. By obtaining the high-frequency texture of the first and second surveillance videos, key features can be extracted more accurately, the aliasing of background texture and dynamic targets can be avoided, and the accuracy of feature comparison can be improved. There is a certain parallax in the shooting images of different cameras, and dynamic targets appear as discontinuous phase changes in video frames. By calculating the phase changes between high-frequency textures, dynamic targets and static backgrounds can be keenly detected and distinguished. The static background has high phase consistency in multiple frames, while the dynamic target causes phase fluctuations due to the constant change of position. Using this feature, the interference of dynamic targets can be more accurately removed. The interference pattern image generates an image that combines the high-frequency phase features of the two videos by calculating the phase change. This image directly reflects the key texture area and phase consistency information in the video frame, and is the basic data for panoramic stitching. Compared with traditional edge detection or feature point matching, the interference pattern image combines frequency domain characteristics and phase information, and is more suitable for panoramic stitching of dynamic and complex scenes. Due to the differences in viewing angles of multiple cameras, there will be certain distortions and misalignments in the video frames. Through Fourier transform and phase correction, the high-frequency textures in the two videos can be accurately aligned.

[0010] Optionally, performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be stitched and a second area to be stitched specifically includes: inputting the interference pattern image into a light wave propagation model, and obtaining a first result through static background extraction; using a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; and determining the first area to be stitched and the second area to be stitched based on the first result and the second result using the light wave propagation model.

[0011] By adopting the above technical solution and processing the interference pattern map using the light wave propagation model, the key texture information in the static background can be highlighted by using the light wave propagation characteristics, while suppressing the interference of dynamic targets. In the scene, the light wave propagation characteristics of the static background have high stability, and can clearly separate static targets such as runways and buildings. The influence of dynamic interference on light wave propagation is weakened, making the static background extraction more accurate. The high-pass filter processes the dynamic targets in the interference pattern map, and can eliminate the interference of high-frequency and discontinuous dynamic targets on the panoramic stitching. The dynamic targets are shown as discontinuous distributions in the high-frequency features, and the high-pass filter can accurately identify and remove these high-frequency interferences. After eliminating the dynamic targets, only the background features are retained in the stitching area, avoiding the ghosting or dislocation problems of dynamic targets in the stitching. The dual operations of static background extraction and dynamic target removal are integrated through the light wave propagation model to accurately locate the first area to be stitched and the second area to be stitched. The static background extraction provides high-quality background texture information, and the high-pass filter effectively eliminates dynamic interference. The combination of the two ensures strong texture consistency and less dynamic interference in the stitching area. The stitching area determination process reduces the mixing of low-frequency background and high-frequency interference, ensuring the quality of the stitching results. Static background extraction ensures the robustness of the stitching results, and maintains the stability of the stitching image even if the dynamic target changes frequently. Dynamic target removal reduces visual interference in real-time monitoring, making the panorama clearer and more reliable. It simplifies the stitching process in multi-camera monitoring scenarios and improves processing efficiency and automation. It is suitable for real-time panoramic stitching tasks to meet the high timeliness requirements of airport monitoring.

[0012] Optionally, determining the first light wave field corresponding to the first camera used to shoot the first surveillance video, and the second light wave field corresponding to the second camera used to shoot the second surveillance video, specifically includes: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to the interference pattern diagram; and using non-uniform refraction to correct perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera, to generate the first light wave field and the second light wave field.

[0013] By adopting the above technical solution, an accurate light wave propagation model is established for the shooting area of ​​each camera by constructing the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera. This helps to describe the viewing angle and optical characteristics of each camera in detail. Light wave field modeling enables the video frames shot by each camera to accurately reflect the light propagation process based on physical laws, avoiding stitching errors caused by optical distortion. Through non-uniform refraction correction, the light wave field of the camera is accurately corrected, the distortion caused by the optical lens is corrected, and each element in the image can be more accurately aligned with the real world. Non-uniform refraction correction effectively eliminates the local distortion caused by lens distortion of different cameras and enhances the geometric consistency of the stitched image. After correction, the geometric shape of the image is closer to the actual scene, avoiding the dislocation phenomenon caused by distortion in the traditional stitching method. By correcting the perspective distortion, the image distortion caused by different shooting perspectives is eliminated. After the perspective distortion is corrected, the light wave field of each camera is more consistent with the actual perspective relationship, ensuring that multiple surveillance videos can be naturally stitched together. By optimizing the light wave field, the stitching errors caused by the difference in shooting angles, perspective distortion and non-uniform refraction between different cameras in traditional stitching are reduced, ensuring that the panoramic image is more natural and detailed. This method can process video streams shot by multiple cameras, and can still ensure the optical and geometric correction of each video even in different viewing angles and different optical distortions, thereby achieving more accurate stitching, improving the stitching quality in multi-camera scenes, adapting to a variety of viewing angle configurations such as wide-angle and telephoto, and enhancing the adaptability and stability of stitching.

[0014] Optionally, determining the panoramic monitoring map based on the first area to be stitched and the second area to be stitched in combination with the first optical wave field and the second optical wave field specifically includes: obtaining an overlapping area between the first area to be stitched and the second area to be stitched; smoothing the overlapping area using a Gaussian beam model to obtain a first stitching result; determining a key alignment feature based on the overlapping area; aligning the first area to be stitched and the second area to be stitched based on the key alignment feature using interference phase change to obtain a second stitching result; generating the panoramic monitoring map based on the first stitching result and the second stitching result.

[0015] By adopting the above technical solution, the overlapping area is smoothed by the Gaussian beam model, which can reduce the obviousness of the seam or transition zone between the two areas to be stitched, making the stitching result smoother and more natural. The application of the Gaussian beam model helps to smoothly transition in the overlapping area, avoids abrupt seams or distortion in the stitching process, and makes the panoramic image present a seamless stitching effect. The stitching error caused by changes in illumination, texture or perspective is reduced, and the overall visual coherence is enhanced. By analyzing the overlapping area and determining the key alignment features, the alignment process of stitching is made more accurate, ensuring that the most important geometric structures in the image are accurately aligned after stitching. The identification of key alignment features helps to accurately align the two areas to be stitched, avoiding texture dislocation or geometric distortion caused by inaccurate alignment. This step can extract important features for complex scenes and dynamic environments, and ensure the consistency of key information when stitching images. Interferometric phase change alignment can accurately calibrate the relative position of the areas to be stitched based on the interference pattern diagram, thereby enhancing the geometric and optical consistency of the stitched image. Interferometric phase change alignment provides a physical alignment method that can handle viewpoint differences, optical distortion, and perspective changes in complex scenes. The panoramic monitoring image generated by combining the first stitching result and the second stitching result after smooth transition ensures the naturalness and consistency of the final stitched image. The application of key alignment features and interferometric phase change can provide accurate alignment results in dynamic environments and prevent dynamic targets from affecting the stitching effect. The use of Gaussian beam smoothing reduces the impact of fast-moving objects in dynamic scenes on the stitching results and enhances the visual consistency of the panorama.

[0016] Optionally, the method further includes: determining a target dynamic area according to the second optical wave field; predicting a motion trajectory of the dynamic target by analyzing energy changes of the target dynamic area in the second optical wave field; determining an occluded area in the second area to be stitched; interpolating and reconstructing the occluded area according to the motion trajectory of the dynamic target to fill in texture information lost due to occlusion of the dynamic target; for an area with blurred texture in the second stitching result, using a low-frequency optical wave component according to the texture information to complete the missing texture details to obtain a third stitching result; generating the panoramic monitoring image according to the first stitching result and the third stitching result.

[0017] By adopting the above technical solution, by analyzing the dynamic area of ​​the target and predicting the motion trajectory of the dynamic target, the moving target can be identified and tracked in advance to avoid the occlusion or distortion during the stitching process. The prediction and tracking of dynamic targets can ensure the consistency of the target in the image stitching and avoid the splicing error or break caused by the appearance of dynamic targets. The motion path of the dynamic target is identified in advance, so that the stitching process can be adjusted in real time to avoid the spatial dislocation caused by the motion of the target in the stitching. The occluded area interpolation reconstruction technology can effectively fill the texture loss caused by the occlusion of the dynamic target and ensure the integrity of the stitching image in all areas. By filling the texture information through the interpolation reconstruction technology, the stitching result can maintain visual integrity and avoid blank or distorted areas during stitching. For the texture fuzzy area generated in the stitching process, the lost detail information can be restored by using low-frequency light wave components for completion, thereby improving the fineness and accuracy of the image. The low-frequency light wave component completion technology can accurately restore the details that are blurred or lost during the stitching process, ensuring that the texture details of the image are more realistic and increasing the visual quality. By completing the missing details and avoiding blurry parts in the stitching process, the final panoramic image is clearer and more refined. This method enables the stitching results to better reflect the continuity of the real scene and avoids abrupt or incoherent parts in the image caused by occlusion or dynamic targets. It ensures that the final panoramic image has clear and delicate details in each area, and the overall effect is more natural. Dynamic target prediction and occluded area interpolation reconstruction can effectively cope with the challenges in complex dynamic environments such as airports, improve the adaptability of the stitching process to dynamic targets, occlusions and other factors, enable the stitching image to cope with changes in the actual scene, and avoid the dynamic environment from having too much impact on the stitching accuracy.

[0018] Optionally, the method also includes: obtaining a target video frame in a third surveillance video, wherein the third surveillance video is any one of the multiple airport surveillance videos except the first surveillance video and the second surveillance video; dynamically adjusting the panoramic skeleton of the third optical wave field corresponding to the third surveillance video according to the target video frame; and optimizing the consistency of the panoramic skeleton using an autocorrelation calibration mechanism of the interference phase to eliminate the drift problem caused by stitching.

[0019] By adopting the above technical solution, by dynamically adjusting the panoramic skeleton of the third light wave field according to the target video frame, it can be optimized according to the changes in the actual video content, thereby improving the geometric accuracy in the stitching process. Dynamic adjustment of the panoramic skeleton helps to adapt to changes in the environment in the surveillance video and ensure that the viewing angle and alignment relationship of each camera are accurately adjusted during the stitching process. This adjustment can effectively deal with the stitching deviation caused by the difference in the position and viewing angle of different cameras, and enhance the geometric consistency and accuracy of the stitched image. The panoramic skeleton is optimized by using the autocorrelation calibration mechanism of the interference phase to ensure that there is no drift problem caused by viewing angle deviation or image misalignment during the stitching of the panorama. This mechanism can ensure the consistency of the image when processing multiple viewing angles, avoid misalignment, distortion or unnatural transition in the stitched image, and improve the overall quality. Through the introduction of the autocorrelation calibration mechanism, the stability and robustness of the stitching process are enhanced, especially when facing dynamic scenes or coordination problems between multiple cameras. The adaptability of the stitching system in a dynamic environment is enhanced, so that the stitching between multiple camera viewing angles can be consistent under any environmental conditions. For dynamic environments such as airports, this optimization mechanism can stably handle subtle changes in large-scale surveillance scenes, ensuring high-quality output of the final panoramic image. By precisely adjusting the panoramic skeleton and calibration mechanism, visual defects caused by the stitching algorithm are reduced, and the quality of the final panoramic surveillance image is optimized. The coherence and consistency of the panoramic surveillance image are enhanced, ensuring that the transition of the image is natural and the image content is clear and correct.

[0020] In a second aspect of the present application, a panoramic stitching device for airport surveillance videos is provided, the panoramic stitching device comprising an acquisition module and a processing module, wherein the acquisition module is used to acquire a first surveillance video and a second surveillance video for a target airport, the target airport corresponds to a plurality of airport surveillance videos, the first surveillance video and the second surveillance video are any two surveillance videos of the plurality of airport surveillance videos; the processing module is used to determine an interference pattern diagram according to the first surveillance video and the second surveillance video; the processing module is further used to perform a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched; the processing module is further used to determine a first optical wave field corresponding to a first camera used to shoot the first surveillance video, and a second optical wave field corresponding to a second camera used to shoot the second surveillance video; the processing module is further used to determine a panoramic surveillance diagram based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field.

[0021] In the third aspect of the present application, an electronic device is provided, which includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs the method described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, the method described above is executed.

[0023] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: By introducing the interference pattern map, the feature information in the first surveillance video and the second surveillance video is converted into a high-frequency feature map of phase interference, which can accurately identify the static background and dynamic targets in each surveillance video. The dynamic target removal operation can effectively filter the interference of dynamic targets such as aircraft and ground vehicles on the splicing process, ensure the clarity and accuracy of the area to be spliced, and eliminate ghosting or blurring problems. By establishing an independent light wave field model for each camera, the distortion problem caused by the camera shooting angle, field of view and perspective deformation is solved. The light wave field contains rich phase information, which can realize the natural alignment and fusion between multi-view videos, eliminate the dislocation and deformation at the seams, and ensure a smooth transition of the splicing results. The static background extraction operation based on the interference pattern map uses the high-frequency phase consistency characteristics of the static background to stably extract static background information in complex lighting conditions or scenes with frequent movement of dynamic targets. This method avoids the problem that the traditional background extraction method based on inter-frame difference fails due to lighting changes or dynamic interference, and improves the robustness of the splicing algorithm. By combining the first light wave field and the second light wave field, the high-frequency details captured in multiple videos are accurately integrated into the panoramic image, making the stitched image clearer in terms of space and detail. At the same time, the light wave field technology can dynamically compensate for the color differences and brightness changes between the areas covered by different cameras to ensure the color consistency of the panoramic image. The combination of the interference pattern map and the light wave field reduces the computational complexity of multi-step feature matching and complex registration operations in traditional stitching technology, enabling panoramic stitching to achieve near real-time updates in a high-frame-rate video streaming environment. This method utilizes the frequency domain characteristics of the interference pattern map and the light wave field, and can adapt to background feature extraction and stitching under different lighting conditions. Regardless of daytime, nighttime or weather changes, the stitched panorama can maintain stable quality and consistency. Therefore, it is convenient to perform panoramic stitching of airport surveillance videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a flow chart of a panoramic stitching method of airport surveillance video provided in an embodiment of the present application; Figure 2 Another schematic diagram of a flow chart of a panoramic stitching method of an airport surveillance video provided in an embodiment of the present application; Figure 3 A schematic diagram of a module of a panoramic stitching device for airport surveillance video provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0025] Explanation of the reference numerals: 31, acquisition module; 32, processing module; 41, processor; 42, communication bus; 43, user interface; 44, network interface; 45, memory. DETAILED DESCRIPTION

[0026] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0027] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0028] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] As an important public transportation hub, airports involve complex scenes such as aircraft taxiing, apron scheduling, ground handling operations, and passenger activities. In order to ensure safe and efficient operations, modern airports widely deploy multi-camera monitoring systems to cover key areas such as runways, taxiways, and aprons. However, due to the vast space of airport scenes and frequent dynamic targets, videos shot by multiple cameras face many challenges when performing panoramic stitching.

[0030] Existing panoramic stitching technologies rely on traditional methods such as feature point matching or image registration. However, in airport surveillance scenarios, dynamic targets such as taxiing aircraft, ground handling vehicles, and personnel movement significantly interfere with the stitching results, resulting in ghosting and stitching errors. In addition, the difference in viewing angles between multiple cameras may also cause distortion in the stitching area, making the seams obvious and the image fusion unnatural. Therefore, traditional feature point matching and image registration methods are less effective in processing panoramic stitching of airport surveillance videos.

[0031] In order to solve the above technical problems, the present application provides a panoramic stitching method for airport surveillance video, referring to Figure 1 , Figure 1 The present invention provides a flow chart of a panoramic stitching method for an airport surveillance video provided in an embodiment of the present invention. The panoramic stitching method is applied to a server and includes steps S110 to S150, which are as follows: S110. Acquire a first surveillance video and a second surveillance video for a target airport. There are multiple airport surveillance videos corresponding to the target airport. The first surveillance video and the second surveillance video are any two surveillance videos among the multiple airport surveillance videos.

[0032] Specifically, the server needs to select surveillance videos for the target airport from the monitoring system. The target airport refers to the airport of interest, which may be a specific airport. Among them, the server is a separate computer system or a server cluster used to manage the airport's monitoring system. The target airport has multiple surveillance video sources. For example, the airport will deploy multiple cameras to cover different areas, such as runways, taxiways, boarding gates, parking lots, etc. Each camera will produce a surveillance video. Among these multiple surveillance videos, the server selects two specific videos, called the first surveillance video and the second surveillance video. These two videos come from different cameras and may be surveillance videos covering different areas of the airport. The two videos selected from the multiple surveillance videos do not need to be fixed, but are any two surveillance videos. In other words, the first video and the second video can be any two surveillance videos, which may be two cameras from different locations or different perspectives.

[0033] For example, suppose there is an airport where multiple surveillance cameras are deployed to cover different areas. The cameras are installed at locations such as the runway, apron, and boarding gate. The first surveillance video is a surveillance video taken by a camera on the airport runway. This video may cover the taxiing, takeoff, and landing of the aircraft on the runway. The second surveillance video is a surveillance video taken by another camera on the apron, showing scenes such as the parking of the aircraft on the apron and the operation of ground staff.

[0034] S120: Determine an interference pattern diagram according to the first monitoring video and the second monitoring video.

[0035] Specifically, the interference pattern map is an image based on the mutual influence between different frames in the video. By analyzing the image details in two videos, the interference pattern map generated can reveal the relationship between different videos, especially the changes of dynamic targets. The generation of the interference pattern map is related to the frequency domain processing of the image. It can analyze the subtle changes between the two video frames through techniques such as Fourier transform and find the phase difference between them. These differences appear as interference effects caused by dynamic targets in the image. The map can reveal the detailed differences between different videos, especially in dynamic environments, such as moving targets and scene changes.

[0036] In a possible implementation, an interference pattern diagram is determined based on the first surveillance video and the second surveillance video, specifically including: performing Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; calculating the phase change between the first high-frequency texture and the second high-frequency texture to generate an interference pattern diagram.

[0037] Specifically, Fourier transform can convert signals from the time domain to the frequency domain. In the frequency domain, different parts of an image are represented as different frequency components. For videos, Fourier transform can help extract high-frequency and low-frequency information in an image. The high-frequency part represents details in the image, such as edges, textures, etc., while the low-frequency part represents smooth areas of the image, such as the background. Performing Fourier transform frame by frame means that the server performs Fourier transform on each frame of the video instead of processing it as a whole. This enables detailed analysis of each moment in the video and captures the changes of dynamic targets. After Fourier transform, the frequency domain of the video frame contains high-frequency textures. These high-frequency textures contain detailed parts of the image, especially changes related to dynamic targets. The first high-frequency texture and the second high-frequency texture represent the high-frequency components of the first surveillance video and the second surveillance video, respectively. Since the surveillance videos are from different perspectives, the high-frequency components in the two videos may be different, especially when there are dynamic targets, the changes of these dynamic elements will be reflected in the high-frequency textures. Phase change refers to the phase difference between the frequency components of two images in the frequency domain. By calculating the phase change between the high-frequency textures of the two videos, subtle changes in the image, especially the movement of dynamic targets, can be revealed. When stitching together multiple surveillance videos, dynamic targets can cause changes between video frames, and calculating these phase changes can help accurately align dynamic elements in different videos. The interference pattern is the result of calculating the phase changes of high-frequency textures in two videos. It shows the differences between the two videos, especially the changes caused by dynamic targets. The interference pattern can reveal information such as the relative displacement between the two video sources and the movement trajectory of the target, which is very important for accurately aligning different perspectives and dynamic targets during the stitching process.

[0038] For example, suppose there are two surveillance videos: one from a runway camera at an airport and the other from an apron camera. The two videos capture dynamic scenes in different areas, such as an airplane taxiing on the runway and ground crew operating on the apron. The first surveillance video contains the dynamic scene of the airplane taxiing. The movement of the airplane on the runway causes changes in details in the video, especially in the area around the airplane. The second surveillance video captures the scene of ground crew operating the airplane on the apron, which also has dynamic changes, such as the movement of personnel or the use of equipment. The server first performs a Fourier transform on each frame of the runway camera video and the apron camera video to convert the image to the frequency domain. In this way, we can get the high-frequency texture, that is, the details in the image. By comparing the high-frequency texture of the runway video and the apron video, the server calculates the phase change of the details in the two videos. For example, when the airplane taxis on the runway, it changes the high-frequency texture of the video, resulting in a phase difference. Similarly, when the ground crew operates on the apron, changes will also occur in the video. By analyzing the phase changes of the two videos, the server generates an interference pattern map. This figure reveals the dynamic differences between the two videos, especially in the high-frequency areas of the image. These differences are related to the motion trajectories of dynamic objects such as the aircraft taxiing and ground crew operations.

[0039] S130 , performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced.

[0040] Specifically, an interference pattern map has been generated before, which shows the difference between dynamic targets and background caused by the phase change between two surveillance video frames. The map reveals the distinction between dynamic targets and static background. Static background extraction refers to identifying the parts of the video that do not change from the interference pattern map and distinguishing them from dynamic targets. The background part is the object or area that does not move in the video, while the dynamic target refers to the part that has obvious movement or change in the video. This operation can use image processing techniques such as background modeling and frame difference method to extract the stable and static background area in the video. For example, the markings and buildings on the runway are static backgrounds, while the people and vehicles on the airport are dynamic targets. Dynamic target removal refers to removing all changes caused by dynamic targets in the interference pattern map, so that only the background information is retained. By removing these dynamic targets, they can be prevented from interfering with the stitching result during the stitching process. Dynamic targets may cause different degrees of deformation, ghosting or stitching errors in the two video sources. Therefore, by removing these dynamic targets, the server can obtain a more accurate stitching area. The dynamic target removal operation can be completed using a high-pass filter or background subtraction method. The high-pass filter helps to remove the low-frequency part and retain the high-frequency part. The area to be stitched refers to a part of the image extracted from the interference pattern map. This part of the image is the static background area after the dynamic target has been removed, which is suitable for stitching operations. The first area to be stitched and the second area to be stitched are extracted areas from two surveillance videos, respectively. These areas will be merged into a seamless panorama in the subsequent stitching step.

[0041] In one possible implementation, a static background extraction operation and a dynamic target removal operation are performed on the interference pattern image to obtain a first area to be stitched and a second area to be stitched, specifically including: inputting the interference pattern image into a light wave propagation model, and obtaining a first result through static background extraction; using a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; and determining the first area to be stitched and the second area to be stitched using a light wave propagation model based on the first result and the second result.

[0042] Specifically, the light wave propagation model is a mathematical model that describes how light waves or image information propagate. In this context, the light wave propagation model is used to extract the static background part in the interference pattern map. It helps to identify which areas are static backgrounds and distinguish dynamic targets. In the static background extraction process, the server uses the light wave propagation model to identify the parts of the interference pattern map that do not change over time. These background parts include areas such as airport sites, buildings, and markings that do not change over time or with the movement of objects. The first result of static background extraction is the background part extracted from the interference pattern map. It helps to separate stable and spliced ​​areas from complex surveillance images. The second result is the image after removing dynamic targets, and what is retained is the background information after dynamic targets such as people and vehicles are removed. Based on the first result and the second result, the server further determines the image area suitable for splicing through the light wave propagation model. These areas are the first area to be spliced ​​and the second area to be spliced, which are from two surveillance videos respectively, and the interference of dynamic targets has been removed and are ready for splicing.

[0043] S140: Determine a first light wave field corresponding to a first camera used for shooting a first surveillance video, and a second light wave field corresponding to a second camera used for shooting a second surveillance video.

[0044] Specifically, the light wave field is a mathematical model that describes the propagation of light waves in an image or video. In an embodiment of the present application, the light wave field is used to represent the relationship between the camera's perspective and the scene. Each camera will produce different light wave fields when shooting scenes through its perspective, and these light wave fields will affect the performance of the video image, such as light refraction, perspective effect, and distortion. For the first surveillance video, the server will determine the light wave field of the camera based on which camera the video was shot by. The determination of the light wave field is based on factors such as the position, angle, focal length, and shooting environment of the camera. These factors determine the perspective effect, field of view, and possible distortion of the image captured by the camera. Similarly, the camera shooting the second surveillance video will also have a corresponding light wave field. The second camera may be located at a different position and the angle may also be different, so the light wave field of the camera will be different from the light wave field of the first camera. The server will determine its light wave field based on factors such as the position and angle of the second camera.

[0045] In a possible implementation, determining a first light wave field corresponding to a first camera used to shoot a first surveillance video and a second light wave field corresponding to a second camera used to shoot a second surveillance video specifically includes: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to an interference pattern diagram; and using non-uniform refraction to correct perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera to generate the first light wave field and the second light wave field.

[0046] Specifically, the server determines how to construct the initial light wave fields of the two cameras by analyzing the interference pattern diagram. These initial light wave fields are preliminary models of the perspective of each camera, representing the light wave propagation characteristics in the scene captured by the camera. In actual scenes, different cameras will produce different light wave propagation effects for capturing the same scene. In particular, when the camera has different perspectives, different shooting distances, and different focal lengths, the captured image will produce certain distortions, especially perspective distortion, that is, the size difference between far and near objects is significant. Non-uniform refraction correction is to adjust these non-uniform refraction effects to ensure that the correspondence between the light wave field generated by the camera perspective and the actual scene is more accurate. Through correction, the parallax difference between cameras can be reduced, making the stitching effect more natural. After completing the above correction, the first light wave field and the second light wave field are generated. These two light wave fields represent the accurate description of the scene by the two cameras, respectively, taking into account factors such as perspective, refraction, and perspective. These two light wave fields will be used to guide the subsequent panoramic stitching process to ensure that the two images can be better aligned and reduce distortion or unnatural seams during stitching.

[0047] S150: Determine a panoramic monitoring map based on the first to-be-joined area and the second to-be-joined area in combination with the first light wave field and the second light wave field.

[0048] Specifically, in the process of stitching the airport surveillance video, the first area to be stitched and the second area to be stitched refer to the image areas obtained by the operations such as background extraction and dynamic target removal mentioned above. These areas are the parts of the video that actually need to be stitched, which may be areas where the content of the pictures taken by different cameras overlaps or is close. The first light wave field and the second light wave field refer to the light wave propagation characteristics of the scene captured by each camera. These light wave fields describe the visual characteristics of the scene, including factors such as camera shooting angle, distance, and lighting. These light wave fields play an important role in image alignment and fusion during the stitching process, ensuring that the transition between the stitching areas is natural and without obvious seams. Combining the first area to be stitched, the second area to be stitched, and their respective light wave fields means that the server will use the overlapping parts of the two areas and the characteristics of the two light wave fields to stitch the images. Specifically, the stitching algorithm will smooth, align, and fuse the images based on the overlapping areas of the two areas to ensure that the final generated panoramic image is more natural and coherent visually. Finally, based on the above information, the server generates a panoramic surveillance map of the entire surveillance area, which is a large-scale image that combines the content captured by multiple cameras. The panoramic surveillance map provides a complete airport surveillance view that can cover multiple areas, allowing managers to monitor every corner of the airport from a unified perspective.

[0049] In a possible implementation, based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field, a panoramic monitoring map is determined, specifically including: obtaining an overlapping area between the first area to be stitched and the second area to be stitched; smoothing the overlapping area using a Gaussian beam model to obtain a first stitching result; determining a key alignment feature based on the overlapping area; aligning the first area to be stitched and the second area to be stitched based on the key alignment feature using an interference phase change to obtain a second stitching result; generating a panoramic monitoring map based on the first stitching result and the second stitching result.

[0050] Specifically, in the splicing of surveillance videos, the two video areas to be spliced ​​may overlap in part, and this overlapping area is the part where the image content captured by the two cameras intersects. This overlapping area is the area that the splicing algorithm needs to focus on, because it must ensure that the overlapping area is visually seamless when splicing. The Gaussian beam model is a mathematical model used to simulate the propagation of light waves, and has smooth and soft transition characteristics. When splicing images, the Gaussian beam model is used to process the overlapping area, which can smooth the transition and reduce the obvious seams or abrupt transitions that may appear during splicing. Through this step, the first splicing result, that is, the spliced ​​image after preliminary processing, is obtained, which has a more natural transition effect. In the overlapping area, there are some features that may help align the images, such as texture, corners or edges. These features are called key alignment features. By analyzing the image content of the overlapping area, the algorithm can determine which features are most important for splicing, and use these features for subsequent fine alignment. Interferometric phase change technology can be used to analyze the phase difference between two images, and align the key features of the two images by adjusting the position and rotation of the images. Here, the algorithm uses the key alignment features of the overlapping area to perform phase correction and position adjustment to ensure the precise stitching of the two areas. Finally, the second stitching result is obtained through interference phase change alignment, which is the stitching image after fine alignment. Finally, the server combines the first stitching result with the second stitching result to generate the final panoramic monitoring map. This panorama covers a larger area of ​​the airport and ensures that the transition between images is natural and seamless.

[0051] For example, suppose there is an airport monitoring system with two cameras, one on the east side and the other on the west side of the runway. Camera 1 on the east side captures the east half of the runway, and camera 2 on the west side captures the west half of the runway. In the images captured by the two cameras, the images on the east and west sides overlap, such as the junction of the two runway sections. Assume that the images on the east and west sides overlap in the middle of the runway, which is the key area for stitching. During the stitching process, the Gaussian beam model is first used to smooth the overlapping area. For example, the overlapping area in the middle of the runway may have seams due to different lighting and shooting angles. Through the Gaussian beam model, the overlapping area can be smoothly transitioned, reducing the abruptness during stitching, and forming the first stitching result. In the overlapping area, there may be some iconic features, such as the edge line of the runway, ground markings, etc., which can be used as key alignment features. By analyzing these features, the algorithm determines the corresponding relationship between these points in the two images. Based on the key alignment features of the overlapping area, the server finely aligns the two areas through the interferometric phase change algorithm. For example, the runway edge or ground markings can be precisely aligned to ensure that there is no obvious misalignment in the images of the two video streams when they are stitched together. After interferometric phase adjustment, the second stitching result is obtained, and the image is more accurate and seamless. Finally, the first stitching result and the second stitching result are combined to generate a large panoramic monitoring image that includes the image of the entire airport runway, ensuring seamless connection and facilitating monitoring personnel to fully view the airport situation.

[0052] In a possible implementation, a target dynamic area is determined based on a second optical wave field; a motion trajectory of the dynamic target is predicted by analyzing energy changes in the target dynamic area in the second optical wave field; an occluded area in the second area to be stitched is determined; based on the motion trajectory of the dynamic target, the occluded area is interpolated and reconstructed to fill in texture information lost due to occlusion of the dynamic target; for an area with blurred texture in the second stitching result, a low-frequency optical wave component is used to fill in the missing texture details through texture information to obtain a third stitching result; and a panoramic monitoring image is generated based on the first stitching result and the third stitching result.

[0053] Specifically, by observing the energy changes in the dynamic area of ​​the target, the target's moving direction and speed can be inferred. Dynamic targets will produce energy fluctuations or changes in the video, so analyzing these changes can help predict the target's motion trajectory. This helps to understand where the dynamic target will appear, so that the occluded area can be repaired in subsequent processing. Occluded areas refer to areas where the image is blocked or covered due to the presence of dynamic targets, resulting in the loss of image details in these areas. For example, a moving vehicle may block part of the ground or other targets, making part of the image invisible. By analyzing the image, the server can determine these blocked areas and use them as key areas to be repaired. Interpolation reconstruction is a commonly used image restoration technology. In this step, the server infers the content of the blocked area based on the previously predicted motion trajectory of the dynamic target. For example, if a vehicle blocks the background area, the server can use interpolation technology to infer the background image that was previously blocked by the vehicle, and fill these image information into the blocked area to restore the lost texture. Texture blurring areas may be caused by multiple reasons, such as different camera angles, focal length problems, or uneven lighting. For these areas, the server analyzes the existing texture information, such as color, lighting characteristics, etc., and uses the low-frequency light wave component completion method to repair the blurred areas. The low-frequency light wave component mainly reflects the large-scale structure and shape in the image, and the restoration of texture information can make the image details more complete, and finally obtain the third stitching result, that is, the stitching image after repair and optimization. In this step, the server combines the first stitching result and the third stitching result obtained above to generate the final panoramic monitoring map. This panoramic map integrates images from different cameras to ensure the integrity of the entire monitoring area, and the image remains clear and seamless under the influence of dynamic targets.

[0054] In one possible implementation, refer to Figure 2 , Figure 2 Another flowchart of a panoramic stitching method for airport surveillance videos provided in an embodiment of the present application. The method includes steps S210 to S230, which are as follows: S210, obtaining a target video frame in a third surveillance video, where the third surveillance video is any surveillance video other than the first surveillance video and the second surveillance video among multiple airport surveillance videos; S220, dynamically adjusting the panoramic skeleton of the third optical wave field corresponding to the third surveillance video according to the target video frame; S230, optimizing the consistency of the panoramic skeleton using an autocorrelation calibration mechanism of the interference phase to eliminate the drift problem caused by stitching.

[0055] Specifically, the third surveillance video refers to other videos selected from multiple airport surveillance videos except the first and second surveillance videos. This video may come from the perspective of other cameras in the airport. The goal of this step is to extract key target video frames from this third video, that is, important image frames related to stitching in the surveillance. Assuming that there are multiple cameras in the airport, the first surveillance video shot by camera 1 and the second surveillance video shot by camera 2 are responsible for covering both sides of the runway, while the third surveillance video shot by camera 3 may be responsible for monitoring the apron or taxiway. By extracting the target frame in the third video, the server can use the information of this frame for panoramic stitching. The third optical wave field refers to the optical wave propagation model corresponding to the third surveillance video, which represents the perspective and imaging characteristics of camera 3. The panoramic skeleton is a virtual framework or structure used to represent the basic layout of the entire surveillance video stitching. During the stitching process, the server needs to dynamically adjust the optical wave field according to the image content and perspective of the third video to ensure that these images can be correctly aligned when synthesizing the panorama. If the viewing angle of camera 3 is different from that of camera 1 or camera 2, or the position of camera 3 changes, the server needs to dynamically adjust its optical wave field according to the image information of the third monitoring video to maintain the spatial consistency of the panorama.

[0056] Among them, interferometric phase involves the phase difference between different images or light waves. In image stitching, interferometric phase is used to evaluate the alignment between images. The autocorrelation calibration mechanism is an algorithm that optimizes the consistency of stitching by comparing the phase changes between different video frames to reduce visual errors such as drift during stitching, such as image misalignment or distortion. This mechanism helps the server dynamically adjust the stitching results by comparing the phase changes at different viewing angles and viewing distances to ensure that the content of different video sources is accurately aligned and avoid unnatural gaps or offsets when stitching images. Assume that during the stitching process, the images between camera 1 and camera 2 are already accurately aligned, but the image of camera 3 is not accurately aligned due to the difference in viewing angle or movement. The server uses the autocorrelation calibration mechanism of interferometric phase to detect this inconsistency and optimize the panoramic skeleton to eliminate the stitching drift problem caused by inconsistent viewing angles. The drift problem refers to the situation where images may be misaligned or misaligned when stitching images from multiple camera perspectives. This problem is caused by errors caused by different viewing angles, different focal lengths or movements of the cameras. By applying the interferometric phase autocorrelation calibration mechanism, the server can accurately correct these errors and eliminate drift during image stitching, making the final stitching result more natural and seamless. If the camera in the third surveillance video is slightly offset from the first and second surveillance videos due to vibration or slight position shift, the server automatically corrects these errors through the interferometric phase autocorrelation calibration mechanism to ensure that the image of the third video can be correctly stitched together with the images of the first two videos.

[0057] The present application also provides a panoramic stitching device for airport surveillance video, referring to Figure 3 , Figure 3 A module schematic diagram of a panoramic stitching device for airport surveillance videos provided in an embodiment of the present application. The panoramic stitching device is a server, and the server includes an acquisition module 31 and a processing module 32, wherein the acquisition module 31 acquires a first surveillance video and a second surveillance video for a target airport, and the target airport corresponds to multiple airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos among the multiple airport surveillance videos; the processing module 32 determines an interference pattern diagram according to the first surveillance video and the second surveillance video; the processing module 32 performs a static background extraction operation and a dynamic target removal operation on the interference pattern diagram to obtain a first area to be stitched and a second area to be stitched; the processing module 32 determines a first optical wave field corresponding to a first camera used to shoot the first surveillance video, and a second optical wave field corresponding to a second camera used to shoot the second surveillance video; the processing module 32 determines a panoramic surveillance diagram based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field.

[0058] In a possible implementation, the processing module 32 determines the interference pattern diagram based on the first monitoring video and the second monitoring video, specifically including: the processing module 32 performs Fourier transform on the first monitoring video and the second monitoring video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; the processing module 32 calculates the phase change between the first high-frequency texture and the second high-frequency texture to generate an interference pattern diagram.

[0059] In one possible implementation, the processing module 32 performs a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced, specifically including: the processing module 32 inputs the interference pattern image into a light wave propagation model, and obtains a first result through static background extraction; the processing module 32 uses a high-pass filter to remove dynamic targets in the interference pattern image to obtain a second result; the processing module 32 uses a light wave propagation model to determine the first area to be spliced ​​and the second area to be spliced ​​based on the first result and the second result.

[0060] In a possible implementation, the processing module 32 determines a first light wave field corresponding to a first camera used to shoot a first surveillance video, and a second light wave field corresponding to a second camera used to shoot a second surveillance video, specifically including: the processing module 32 constructs an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to an interference pattern diagram; the processing module 32 uses non-uniform refraction to correct the perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera, to generate the first light wave field and the second light wave field.

[0061] In a possible implementation, the processing module 32 determines a panoramic monitoring map based on the first area to be stitched and the second area to be stitched, in combination with the first optical wave field and the second optical wave field, specifically including: the acquisition module 31 acquires the overlapping area between the first area to be stitched and the second area to be stitched; the processing module 32 uses a Gaussian beam model to smooth the overlapping area to obtain a first stitching result; the processing module 32 determines a key alignment feature based on the overlapping area; the processing module 32 uses interference phase change to align the first area to be stitched and the second area to be stitched based on the key alignment feature to obtain a second stitching result; the processing module 32 generates a panoramic monitoring map based on the first stitching result and the second stitching result.

[0062] In a possible implementation, the processing module 32 determines the target dynamic area according to the second optical wave field; the processing module 32 predicts the motion trajectory of the dynamic target by analyzing the energy change of the target dynamic area in the second optical wave field; the processing module 32 determines the occluded area in the second area to be stitched; the processing module 32 interpolates and reconstructs the occluded area according to the motion trajectory of the dynamic target, and fills in the texture information lost due to the occlusion of the dynamic target; the processing module 32 uses the low-frequency optical wave component to complement the missing texture details of the area with blurred texture in the second stitching result through the texture information, and obtains a third stitching result; the processing module 32 generates a panoramic monitoring map according to the first stitching result and the third stitching result.

[0063] In a possible implementation, the acquisition module 31 acquires a target video frame in a third surveillance video, where the third surveillance video is any one of a plurality of airport surveillance videos except the first surveillance video and the second surveillance video; the processing module 32 dynamically adjusts the panoramic skeleton of the third optical wave field corresponding to the third surveillance video according to the target video frame; the processing module 32 optimizes the consistency of the panoramic skeleton using an autocorrelation calibration mechanism of the interference phase to eliminate the drift problem caused by stitching.

[0064] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0065] The present application also provides an electronic device, referring to Figure 4 , Figure 4The electronic device may include: at least one processor 41 , at least one network interface 44 , a user interface 43 , a memory 45 , and at least one communication bus 42 .

[0066] The communication bus 42 is used to realize the connection and communication between these components.

[0067] The user interface 43 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 43 may also include a standard wired interface and a wireless interface.

[0068] The network interface 44 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0069] Among them, the processor 41 may include one or more processing cores. The processor 41 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 45, and calling data stored in the memory 45. Optionally, the processor 41 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 41 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 41, and it can be implemented separately through a chip.

[0070] Among them, the memory 45 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 45 includes a non-transitory computer-readable storage medium. The memory 45 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 45 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 45 may also be optionally at least one storage device located away from the aforementioned processor 41. As Figure 4 As shown, the memory 45 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a panoramic stitching method of airport surveillance videos.

[0071] exist Figure 4 In the electronic device shown, the user interface 43 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 41 can be used to call an application program for a panoramic stitching method of airport surveillance video stored in the memory 45. When executed by one or more processors, the electronic device executes one or more methods in the above-mentioned embodiments.

[0072] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0073] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors, the electronic device executes one or more of the methods described in the above embodiments.

[0074] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0079] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A panoramic stitching method for airport surveillance video, characterized in that: The method comprises: Acquire a first surveillance video and a second surveillance video for a target airport, the target airport corresponds to a plurality of airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos among the plurality of airport surveillance videos; Determine an interference pattern diagram according to the first monitoring video and the second monitoring video; Performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced; Determine a first optical wave field corresponding to a first camera used to shoot the first surveillance video, and a second optical wave field corresponding to a second camera used to shoot the second surveillance video; Based on the first to-be-joined area and the second to-be-joined area, a panoramic monitoring map is determined in combination with the first optical wave field and the second optical wave field.

2. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The determining of the interference pattern diagram according to the first monitoring video and the second monitoring video specifically includes: Performing Fourier transform on the first surveillance video and the second surveillance video frame by frame to obtain a first high-frequency texture and a second high-frequency texture; The phase change between the first high-frequency texture and the second high-frequency texture is calculated to generate the interference pattern diagram.

3. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The step of performing a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be stitched and a second area to be stitched specifically includes: Inputting the interference pattern diagram into a light wave propagation model, and obtaining a first result through static background extraction; Using a high-pass filter to remove dynamic targets in the interference pattern diagram to obtain a second result; The first area to be spliced ​​and the second area to be spliced ​​are determined using the light wave propagation model according to the first result and the second result.

4. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The determining of the first optical wave field corresponding to the first camera used for shooting the first surveillance video and the second optical wave field corresponding to the second camera used for shooting the second surveillance video specifically includes: constructing an initial light wave field corresponding to the first camera and an initial light wave field corresponding to the second camera according to the interference pattern diagram; The first light wave field and the second light wave field are generated by correcting the perspective distortion of the initial light wave field corresponding to the first camera and the initial light wave field corresponding to the second camera by using non-uniform refraction.

5. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The determining of the panoramic monitoring map based on the first to-be-joined area and the second to-be-joined area in combination with the first optical wave field and the second optical wave field specifically includes: Acquire an overlapping area between the first area to be stitched and the second area to be stitched; The overlapping area is smoothed using a Gaussian beam model to obtain a first stitching result; determining a key alignment feature based on the overlapping area; According to the key alignment feature, the first to-be-joined area and the second to-be-joined area are aligned by using interference phase change to obtain a second stitching result; The panoramic monitoring image is generated according to the first stitching result and the second stitching result.

6. The panoramic stitching method of airport surveillance video according to claim 5, characterized in that: The method further comprises: determining a target dynamic area according to the second optical wave field; Predicting the motion trajectory of the dynamic target by analyzing the energy change of the dynamic area of ​​the target in the second light wave field; Determine an occluded area in the second area to be stitched; According to the motion trajectory of the dynamic target, the occluded area is interpolated and reconstructed to fill in the texture information lost due to the occlusion of the dynamic target; For the area with blurred texture in the second stitching result, the missing texture details are supplemented by using the low-frequency light wave component according to the texture information to obtain a third stitching result; The panoramic monitoring map is generated according to the first stitching result and the third stitching result.

7. The panoramic stitching method of airport surveillance video according to claim 1, characterized in that: The method further comprises: Acquire a target video frame in a third surveillance video, where the third surveillance video is any surveillance video among the plurality of airport surveillance videos except the first surveillance video and the second surveillance video; According to the target video frame, dynamically adjusting the panoramic skeleton of the third optical wave field corresponding to the third monitoring video; The panorama skeleton is optimized for consistency by using an autocorrelation calibration mechanism of the interferometric phase, so as to eliminate the drift problem caused by stitching.

8. A panoramic stitching device for airport surveillance video, characterized in that: The panoramic stitching device comprises an acquisition module (31) and a processing module (32), wherein: The acquisition module (31) is used to acquire a first surveillance video and a second surveillance video for a target airport, the target airport corresponds to a plurality of airport surveillance videos, and the first surveillance video and the second surveillance video are any two surveillance videos among the plurality of airport surveillance videos; The processing module (32) is used to determine an interference pattern diagram based on the first monitoring video and the second monitoring video; The processing module (32) is further used to perform a static background extraction operation and a dynamic target removal operation on the interference pattern image to obtain a first area to be spliced ​​and a second area to be spliced; The processing module (32) is further used to determine a first optical wave field corresponding to a first camera used to shoot the first monitoring video, and a second optical wave field corresponding to a second camera used to shoot the second monitoring video; The processing module (32) is further used to determine a panoramic monitoring map based on the first area to be spliced ​​and the second area to be spliced, in combination with the first optical wave field and the second optical wave field.

9. An electronic device, characterized in that: The electronic device comprises a processor (41), a memory (45), a user interface (43) and a network interface (44), wherein the memory (45) is used to store instructions, the user interface (43) and the network interface (44) are both used to communicate with other devices, and the processor (41) is used to execute the instructions stored in the memory (45) so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images