Multi-camera image superimposition and fusion method and system in aerial work monitoring

Abstract

The application discloses a high-altitude operation monitoring multi-camera image superposition fusion method and system, relates to the field of image processing, and comprises the following steps: arranging multiple wide-angle cameras at key positions of a work site to construct a monitoring camera array; performing time synchronization and coordination on the monitoring camera array based on a network time protocol, performing trial sampling, and obtaining a monitoring camera trial sampling image set; performing independent analysis on double-path and multi-path image superposition region identification and fusion strategies, and constructing an image superposition region fusion strategy; performing real-time monitoring on the work site, superimposing and fusing monitoring images based on the image superposition region fusion strategy, and obtaining an initial superposition and fusion image; collecting environmental information, performing intensive analysis on the initial superposition and fusion image, and obtaining a target superposition and fusion image. The application solves the technical problems of low image superposition and fusion quality and insufficient monitoring accuracy of existing high-altitude operation monitoring, and achieves the technical effects of improving image superposition and fusion quality and monitoring accuracy.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method and system for overlaying and fusing multi-camera images in high-altitude operation monitoring. Background Technology

[0002] High-altitude operation monitoring is crucial for ensuring worker safety, improving work efficiency, and achieving traceability of the work process. Comprehensive and accurate acquisition and processing of image information is the core of high-altitude operation monitoring. Currently, the main method for solving the problem of image acquisition and processing in high-altitude operation monitoring is to deploy multiple cameras and present the monitored scene using simple image stitching or direct multi-screen display. However, current methods suffer from problems such as asynchronous shooting times of the cameras, a lack of targeted image overlay fusion strategies, and insufficient consideration of the impact of environmental factors on image quality. These issues lead to problems like time misalignment, poor overlay fusion effects, and unclear image information, making it difficult to meet the needs of precise high-altitude operation monitoring.

[0003] Currently, high-altitude operation monitoring suffers from technical problems such as low image overlay and fusion quality and insufficient monitoring accuracy. Summary of the Invention

[0004] This application provides a method and system for multi-camera image overlay and fusion in high-altitude operation monitoring. It employs a wide-angle camera array deployed at the work site. This array undergoes time synchronization and coordination before trial sampling to acquire a set of monitoring image samples. After image overlay region identification and analysis, a fusion strategy is constructed for real-time monitoring and acquisition. An initial overlay fused image is generated based on the fusion strategy. This initial overlay fused image, along with independently acquired environmental information data, is input into an image enhancement and analysis module. After processing, a target overlay fused image is output. These techniques solve the technical problems of low image overlay and fusion quality and insufficient monitoring accuracy in existing high-altitude operation monitoring, achieving the technical effect of improving both image overlay and fusion quality and monitoring accuracy.

[0005] This application provides a method for multi-camera image overlay and fusion in high-altitude operation monitoring, comprising: deploying multiple wide-angle cameras at key locations at the work site to construct a monitoring camera array; performing time synchronization coordination of the monitoring camera array based on the Network Time Protocol (NTP), and using the monitoring camera array to perform trial sampling to obtain a set of monitoring camera trial sample images; performing independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies based on the monitoring camera trial sample image set to construct an image overlay region fusion strategy; using the monitoring camera array to perform real-time monitoring of the work site, and overlaying and fusing the monitoring images based on the image overlay region fusion strategy to obtain an initial overlay fused image; collecting environmental information, and performing enhancement analysis on the initial overlay fused image based on the environmental information to obtain a target overlay fused image.

[0006] In a possible implementation, based on the set of sampled images from the surveillance cameras, independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies is performed to construct an image overlay region fusion strategy. The following processing is executed: The set of sampled images from the surveillance cameras is transformed using a feature point matching algorithm under a unified spatial coordinate system to obtain a transformed set of sampled images from the surveillance cameras; based on the camera calibration parameters and coordinate mapping relationships of each wide-angle camera in the surveillance camera array, a heatmap of the transformed set of sampled images from the surveillance cameras is identified to determine the set of dual-channel overlay regions and the set of densely overlay regions; based on the set of dual-channel overlay regions and the set of densely overlay regions, and combined with the transformed set of sampled images from the surveillance cameras, an independent analysis of the fusion strategy is performed to construct the image overlay region fusion strategy.

[0007] In a possible implementation, based on the dual-path overlay region set and the dense overlay region set, and combined with the set of sampled images from the conversion monitoring camera, an independent analysis of the fusion strategy is performed to construct the image overlay region fusion strategy, and the following processing is executed: Based on the dual-path overlay region set and the dense overlay region set, key optimization regions and stable regions are identified in the set of sampled images from the conversion monitoring camera, obtaining a set of dual-path overlay key optimization region images, a set of dual-path overlay stable region images, a set of dense overlay key optimization region images, and a set of dense overlay stable region images; the set of dual-path overlay key optimization region images and the set of dense overlay stable region images are traversed. The image overlay region fusion strategy is analyzed by analyzing the set of regional image groups, the set of densely overlaid key optimization region image groups, and the set of densely overlaid stable region image groups to obtain the set of fusion sub-strategies for dual-path overlay key optimization region image groups, the set of fusion sub-strategies for dual-path overlay stable region image groups, the set of fusion sub-strategies for densely overlaid key optimization region image groups, and the set of fusion sub-strategies for densely overlaid stable region image groups. The set of fusion sub-strategies for dual-path overlay key optimization region image groups, the set of fusion sub-strategies for dual-path overlay stable region image groups, the set of fusion sub-strategies for densely overlaid key optimization region image groups, and the set of fusion sub-strategies for densely overlaid stable region image groups are stored in separate regions to construct the image overlay region fusion strategy.

[0008] In a possible implementation, the following processing is performed: Extracting a first dual-path overlay key optimization region image group from the set of dual-path overlay key optimization region image groups, wherein the first dual-path overlay key optimization region image group is any one of the dual-path overlay key optimization region image groups in the set; using a target detection algorithm to identify key fusion objects in the fused image to obtain a first set of dual-path overlay key fusion objects; taking the image overlay fusion quality of the first set of dual-path overlay key fusion objects as a fusion constraint that it must meet a preset quality requirement, and combining the first dual-path overlay key optimization region image group, performing region fusion sub-strategy analysis to obtain a set of dual-path overlay key optimization region image fusion sub-strategies.

[0009] In a possible implementation, the image overlay fusion quality of the first dual-path overlay key fusion object set must meet a preset quality requirement as a fusion constraint. Combined with the first dual-path overlay key optimization region image group, a region fusion sub-strategy analysis is performed to obtain a set of dual-path overlay key optimization region image fusion sub-strategies. The following processes are then performed: a key optimization region image fusion processor is pre-constructed; the key optimization region image fusion processor is used to perform region fusion analysis on the first dual-path overlay key optimization region image group to obtain a first key optimization region overlay fusion image; based on the fusion constraint and the first key optimization region overlay fusion image, the parameters of the key optimization region image fusion processor are updated and optimized, and the updated key optimization region image fusion processor is used as a dual-path overlay key optimization region image fusion sub-strategy to obtain the set of dual-path overlay key optimization region image fusion sub-strategies.

[0010] In a possible implementation, the following processing is performed: Extracting a first dual-path superimposed stable region image group from the set of dual-path superimposed stable region image groups, wherein the first dual-path superimposed stable region image group is any one of the dual-path superimposed stable region image groups in the set; randomly extracting a first pixel from the first dual-path superimposed stable region image group, and randomly iterating the first pixel according to a preset fusion iteration radius to determine a second pixel; connecting the first pixel and the second pixel to construct a first superimposed fusion line segment, and constructing a neighborhood of the first superimposed fusion line segment with the center of the first superimposed fusion line segment as the center and the length of the first superimposed fusion line segment as the diameter; dividing the neighborhood of the first superimposed fusion line segment into a first... The left neighborhood of the first superimposed and fused line segment and the right neighborhood of the first superimposed and fused line segment are superimposed and fused. Based on the first dual-path superimposed stable region image group, the pixel difference analysis of the image group in the left neighborhood and the right neighborhood of the first superimposed and fused line segment is performed respectively. If the pixel difference of the image group in the left neighborhood of the first superimposed and fused line segment and the pixel difference of the image group in the right neighborhood of the first superimposed and fused line segment are both less than or equal to the preset image group pixel difference threshold, then the second pixel point is iterated until the image edge of the first dual-path superimposed stable region image group is reached, the superimposed path of the first dual-path superimposed stable region image group is obtained, and the superimposed path of the first dual-path superimposed stable region image group is used as the dual-path superimposed stable region fusion sub-strategy to obtain the dual-path superimposed stable region image fusion sub-strategy set.

[0011] In a possible implementation, the following processing is performed: when the pixel difference between the left and right neighboring image groups of the first overlay fusion line segment is not less than or equal to a preset image group pixel difference threshold, the first overlay fusion line segment is adjusted at an angle a preset number of times, and image group pixel difference analysis is performed after each angle adjustment; if the pixel difference between the left and right neighboring image groups of the first overlay fusion line segment is still not less than or equal to the preset image group pixel difference threshold after the preset number of angle adjustments, the first pixel is selected again.

[0012] In a possible implementation, environmental information is collected, and the initial overlay fused image is enhanced based on the environmental information to obtain a target overlay fused image. The following processing is performed: environmental information is collected according to preset environmental indicators; a pre-constructed image enhancement analyzer is obtained, and the image enhancement analyzer is used to enhance the environmental information and the initial overlay fused image to obtain the target overlay fused image.

[0013] In a possible implementation, the following process is performed: image quality scoring is performed on the target overlay and fused image, and the parameters of the image enhancement analyzer are updated and optimized based on the image quality scoring results.

[0014] This application also provides a multi-camera image overlay and fusion system for high-altitude operation monitoring, comprising: a monitoring camera array construction module, used to deploy multiple wide-angle cameras at key locations at the work site to construct a monitoring camera array; a trial sampling module, used to perform time synchronization coordination of the monitoring camera array based on the Network Time Protocol, and to perform trial sampling using the monitoring camera array to obtain a set of monitoring camera trial sample images; an image overlay region fusion strategy construction module, used to perform independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategy based on the set of monitoring camera trial sample images to construct an image overlay region fusion strategy; an overlay fusion module, used to perform real-time monitoring of the work site using the monitoring camera array, and to overlay and fuse the monitoring images based on the image overlay region fusion strategy to obtain an initial overlay fused image; and an enhancement analysis module, used to collect environmental information, and to perform enhancement analysis on the initial overlay fused image based on the environmental information to obtain a target overlay fused image.

[0015] The proposed method and system for multi-camera image overlay and fusion in high-altitude operation monitoring involves first deploying multiple wide-angle cameras at key locations at the work site to construct a monitoring camera array. Next, the monitoring camera array is synchronized and coordinated using a network time protocol. Trial sampling is then performed using the array to obtain a set of trial sampled images. Based on this set, independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies is conducted to construct an image overlay region fusion strategy. The monitoring camera array is then used to monitor the work site in real time, and the monitored images are overlaid and fused based on the image overlay region fusion strategy to obtain an initial overlay fused image. Finally, environmental information is collected, and enhancement analysis is performed on the initial overlay fused image based on this environmental information to obtain a target overlay fused image. This achieves the technical effect of improving image overlay fusion quality and monitoring accuracy. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a flowchart illustrating the multi-camera image overlay and fusion method for high-altitude operation monitoring provided in this embodiment of the application.

[0018] Figure 2 This is a schematic diagram of the structure of a multi-camera image overlay and fusion system for high-altitude operation monitoring provided in an embodiment of this application.

[0019] Figure labeling: 10 for surveillance camera array construction module, 20 for trial sampling module, 30 for image overlay region fusion strategy construction module, 40 for overlay fusion module, and 50 for enhanced analysis module. Detailed Implementation

[0020] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below.

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same or different subsets of all possible embodiments and can be combined with each other without conflict. The terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0023] This application provides a method for overlaying and fusing multi-camera images in high-altitude operation monitoring, such as... Figure 1 As shown, the method includes:

[0024] Step S100: Deploy multiple wide-angle cameras at key locations on the work site to construct a monitoring camera array.

[0025] Specifically, a wide-angle camera is a type of camera capable of capturing a wide range of scenes. Its lens has a short focal length and a wide angle of view, typically exceeding 90 degrees, covering a large monitoring area. In high-altitude work monitoring, it is used to capture images of a large work site, obtaining more comprehensive visual information. The process begins with an on-site survey of the work site. Based on the spatial layout, work area, and monitoring requirements, key locations are identified, such as the area around the work platform, material storage areas, and access points. Then, based on the wide-angle camera's parameters and the site conditions, the appropriate installation height and angle range are calculated. Next, the camera bracket is installed at the selected location, and the camera is fixed in place. The camera's direction and focal length are adjusted to ensure that the camera's field of view covers the entire work area with some overlap, and that the camera can capture clear and complete images of the work scene. Finally, preliminary image acquisition and testing are conducted to check the camera's installation effectiveness and image quality.

[0026] Step S200: Time synchronization and coordination of the monitoring camera array is performed based on the Network Time Protocol, and trial sampling is performed using the monitoring camera array to obtain a set of trial sampled images from the monitoring cameras.

[0027] Specifically, Network Time Protocol (NTP) is a network protocol used to synchronize computer clocks. It keeps the system time of various devices consistent by exchanging timestamp information between computers. In this application, the NTP protocol is used to synchronize the time of multiple cameras to ensure that the acquired images have a unified time reference, thereby facilitating subsequent frame-level alignment and image fusion processing.

[0028] Set up an NTP server and connect each camera to it via the network. Configure the NTP client on the camera, setting parameters such as the server address and synchronization period, and enable time synchronization. Monitor and adjust the synchronization accuracy to keep the time error between cameras within milliseconds. Then, start the camera array for trial sampling, setting the sampling frequency and duration to collect a certain number of image samples, which are then stored on the server or storage device to form a set of trial sampled images from the surveillance cameras.

[0029] Step S300: Based on the set of sampled images from the surveillance cameras, perform independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies, and construct an image overlay region fusion strategy.

[0030] Specifically, the sampled images from the surveillance cameras undergo preprocessing, such as grayscale conversion and noise filtering. Feature point extraction and matching are performed on both dual-channel and multi-channel image pairs to determine the range and location of overlapping areas between different camera images. For the identified overlapping areas, image fusion algorithms based on multi-scale transformation or deep learning are used to preprocess the images in the overlapping areas, such as illumination equalization and edge smoothing, to reduce differences and visual discontinuities between images. Illumination equalization can be achieved through algorithms such as histogram equalization or adaptive limited contrast histogram equalization (CLAHE). Simultaneously, different fusion strategies are formulated based on factors such as image content and texture features, such as weighted averaging and Gaussian pyramid fusion, to construct image overlapping area fusion strategies. For example, Gaussian pyramid fusion can be used for areas with clear textures and rich details, while weighted average fusion can be used for areas with simple textures and smooth color transitions. The constructed image overlapping area fusion strategies are stored as configuration files or data structures for use in image fusion during real-time monitoring.

[0031] In one possible implementation, based on the set of test-sampled images from the surveillance cameras, independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies is performed to construct an image overlay region fusion strategy. Step S300 further includes step S310, which uses a feature point matching algorithm to perform a perspective transformation in a unified spatial coordinate system on the set of test-sampled images from the surveillance cameras, obtaining a transformed set of test-sampled images from the surveillance cameras. Specifically, feature point matching algorithms, such as SIFT and ORB, are used to detect feature points in each image in the set of test-sampled images from the surveillance cameras, extracting key feature points in the images. Feature point matching is performed between images from different cameras to find corresponding feature point pairs. Using the matched feature point pairs, methods such as homography matrix estimation are used to calculate the perspective transformation parameters from the perspective of each camera to the unified spatial coordinate system. Based on the calculated perspective transformation parameters, geometric transformations (such as perspective transformation, affine transformation, etc.) are performed on the original test-sampled images to transform the images to the unified spatial coordinate system, generating a transformed set of test-sampled images from the surveillance cameras.

[0032] Step S320: Based on the camera calibration parameters and coordinate mapping relationships of each wide-angle camera in the surveillance camera array, perform superimposed heatmap identification on the set of sampled images from the converted surveillance cameras to determine the set of dual-path superimposed regions and the set of densely superimposed regions. Specifically, according to the camera calibration parameters (such as focal length, principal point coordinates, distortion coefficient, etc.) and coordinate mapping relationships of each wide-angle camera, convert the image pixel coordinates in the set of sampled images from the converted surveillance cameras into coordinates in the global coordinate system. Calculate the coverage area of ​​each image in the global coordinate system and determine the overlapping areas between different images to construct a superimposed heatmap. On the superimposed heatmap, count how many images cover each pixel in the overlapping area, i.e., the superimposed number. Based on the threshold of the superimposed number (e.g., a superimposed number of 2 indicates a dual-path superimposed region, and a superimposed number greater than or equal to 3 indicates a densely superimposed region), identify the dual-path superimposed regions and the densely superimposed regions, and collect them to form corresponding sets.

[0033] Step S330: Based on the dual-path overlay region set and the dense overlay region set, and combined with the sampled images from the conversion monitoring camera, an independent analysis of the fusion strategy is performed to construct the image overlay region fusion strategy. Specifically, for different regions in the dual-path overlay region set and the dense overlay region set, and combined with the image features in the sampled images from the conversion monitoring camera, an independent analysis of the fusion strategy is performed. For each region in the dual-path overlay region set, the features of its corresponding two images are analyzed, such as texture, color, and contrast. A fusion strategy based on multi-scale transformation, such as Gaussian pyramid fusion, can be adopted. By decomposing and fusing the images at different scales, a smooth transition and detail preservation of the image can be achieved. For each region in the dense overlay region set, since it involves the overlay of multiple images, a deep learning-based fusion strategy can be adopted. Models such as convolutional neural networks (CNN) are used to learn image features and automatically generate a more reasonable and natural fusion result. Through experiments and tests on the sampled images from the conversion monitoring camera, the parameters in the fusion strategy are continuously adjusted and optimized to achieve the best fusion effect. Finally, the fusion strategies for different overlay areas are integrated to form a complete image overlay area fusion strategy, which is used to guide image fusion processing in real-time monitoring.

[0034] This approach eliminates the impact of differences in perspectives between different cameras by performing a perspective transformation on the sampled images from the surveillance cameras in a unified spatial coordinate system. Based on camera calibration parameters and coordinate mapping relationships, it identifies the superimposed heatmap, which can clearly determine the dual-path superimposed area and the densely superimposed area. This provides a basis for adopting targeted fusion strategies for different superimposed areas, improving the targeting and effectiveness of the fusion processing.

[0035] In one possible implementation, based on the dual-path overlay region set and the dense overlay region set, and combined with the sampled images from the conversion monitoring camera, an independent analysis of the fusion strategy is performed to construct the image overlay region fusion strategy. Step S330 further includes step S331, which involves identifying key optimization regions and stable regions in the sampled images from the conversion monitoring camera based on the dual-path overlay region set and the dense overlay region set, to obtain a set of dual-path overlay key optimization region images, a set of dual-path overlay stable region images, a set of dense overlay key optimization region images, and a set of dense overlay stable region images. Specifically, key optimization regions refer to areas with frequently changing dynamic content, sensitive edge fusion, and critical tasks, such as areas where workers are active, material handling areas, boundaries between areas of different colors or textures, operating parts of high-altitude work equipment, and areas near safety protection facilities. These areas require detailed analysis and processing during image fusion to ensure the accuracy and clarity of the fusion results. Stable regions refer to static backgrounds, distant buildings, platform ground, and other areas in an image that have minimal changes, excluding key optimization regions. During image fusion, fast stitching methods can be applied to overlay images according to a preset overlay route to improve fusion efficiency.

[0036] For identifying key optimization regions, dynamic target detection can be performed on images in the sampled image set of the surveillance camera. For example, optical flow can be used to calculate the motion vectors of pixels in the image sequence to identify areas where dynamic content frequently changes. Edge detection algorithms (such as Canny edge detection, Sobel edge detection, etc.) can be used to identify edge-blending sensitive areas in the image. Semantic segmentation algorithms can be combined to determine key regions of the task based on the semantic information of the image content.

[0037] For the identification of stable regions, background modeling can be used to compare the pixels in the image with the background model, and regions that match the background features can be identified as stable regions.

[0038] Finally, the identified key optimization regions and stable regions are intersected with the dual-path overlay region set and the dense overlay region set, respectively, to obtain the dual-path overlay key optimization region image set, the dual-path overlay stable region image set, the dense overlay key optimization region image set, and the dense overlay stable region image set.

[0039] Step S332 involves traversing the sets of dual-path superimposed key optimization region images, dual-path superimposed stable region images, densely superimposed key optimization region images, and densely superimposed stable region images to perform fusion strategy analysis, thereby obtaining sets of dual-path superimposed key optimization region image fusion sub-strategies, dual-path superimposed stable region image fusion sub-strategies, densely superimposed key optimization region image fusion sub-strategies, and densely superimposed stable region image fusion sub-strategies. Specifically, for the dual-path superimposed key optimization region image set, corresponding image data is collected, a training dataset is constructed, and a convolutional neural network model is trained using a deep learning framework (such as TensorFlow, PyTorch, etc.) to enable the model to learn the features and fusion rules of the images in that region and generate corresponding fusion sub-strategies. For the dual-path superimposed stable region image set, based on the features and geometric relationships of the images, a preset superimposed route is designed, and a fast stitching algorithm (such as affine transformation stitching based on feature point matching) is used for image superimposition. The stitching parameters and methods are recorded to form fusion sub-strategies. For densely stacked key optimization region image sets, due to the involvement of multiple image stacks, the analysis scale is further refined by segmenting the images into smaller sub-regions. A deep learning fusion strategy is applied to each sub-region to generate a more refined fusion sub-strategy. For densely stacked stable region image sets, the stacking order and weight allocation scheme of the multiple images are determined, and a fast stitching method is used for image stacking. Pixel fusion calculations during the stacking process are optimized to obtain fusion sub-strategies. Finally, sets of fusion sub-strategies for dual-path stacked key optimization region images, dual-path stacked stable region images, densely stacked key optimization region images, and densely stacked stable region images are obtained respectively.

[0040] Step S333 involves storing the set of image fusion sub-strategies for dual-path overlay key optimization regions, dual-path overlay stable regions, dense overlay key optimization regions, and dense overlay stable regions in separate regions to construct the image overlay region fusion strategy. Specifically, a storage structure, such as a database table or folder directory, is created to store different types of fusion sub-strategy sets. Then, the set of image fusion sub-strategies for dual-path overlay key optimization regions, dual-path overlay stable regions, dense overlay key optimization regions, and dense overlay stable regions are stored in their respective storage structures. When storing each fusion sub-strategy, its region type, region location information (such as the coordinate range in the global coordinate system), image feature description (such as texture features, color features, etc.), and specific parameters of the fusion algorithm (such as the weight file path of the convolutional neural network model, the transformation matrix of the fast stitching algorithm, etc.) are recorded. Finally, an indexing mechanism is constructed to quickly retrieve the corresponding fusion sub-strategy based on image region information during real-time monitoring, thereby constructing a complete image overlay region fusion strategy to guide the real-time image fusion processing.

[0041] This implementation method constructs separate fusion sub-strategies for different regions (dual-path overlay regions, densely overlay regions) and different types (critical optimization regions, stable regions), and stores them in separate regions. This makes the overall image overlay region fusion strategy more adaptable and flexible. It can flexibly apply corresponding fusion sub-strategies based on the characteristics and changes of different regions in the actual monitoring scenario, better coping with the complex and ever-changing high-altitude operation monitoring environment. When the monitoring scene or camera layout changes, only the changed parts need to be re-analyzed and the corresponding fusion sub-strategies adjusted, without requiring large-scale modifications to the entire fusion strategy, reducing system maintenance costs and the difficulty of adapting to new environments.

[0042] In one possible implementation, step S332 further includes step S3321, extracting a first dual-path overlay key optimization region image group from the set of dual-path overlay key optimization region image groups, wherein the first dual-path overlay key optimization region image group is any one of the dual-path overlay key optimization region image groups in the set. Specifically, the storage structure and indexing method of the set of dual-path overlay key optimization region image groups are determined. Based on actual needs (such as starting analysis from the earliest image group, starting analysis from an image group at a specific location, randomization, etc.), an index or query condition is selected to extract the corresponding first dual-path overlay key optimization region image group from the set. For example, if the image group set is stored in a list arranged in chronological order, then the first image group can be extracted as the first dual-path overlay key optimization region image group through list indexing operations.

[0043] Step S3322: A target detection algorithm is used to identify key fusion objects in the fused image, obtaining a first set of key fusion objects for dual-path overlay. Specifically, a target detection algorithm, such as a deep learning-based target detection algorithm (YOLO, Faster R-CNN, etc.) or a traditional feature-based target detection algorithm (Haar features + Adaboost algorithm, etc.), is used to analyze the fused image in the extracted first dual-path overlay key optimization region image group to identify the key fusion objects. Key fusion objects can be objects of significant monitoring importance, such as workers, high-altitude work equipment, and safety protection facilities. By training a target detection model on training data, it can accurately locate and identify these key fusion objects and output their location information (such as bounding box coordinates) and category information. The detection results output by the algorithm are processed and organized to form the first set of key fusion objects for dual-path overlay.

[0044] Step S3323: Taking the image overlay fusion quality of the first dual-path overlay key fusion object set meeting preset quality requirements as a fusion constraint, and combining the first dual-path overlay key optimization region image group, a regional fusion sub-strategy analysis is performed to obtain a set of dual-path overlay key optimization region image fusion sub-strategies. Specifically, preset quality requirement indicators (such as image sharpness, target integrity, color consistency, edge smoothness, etc.) and corresponding image quality evaluation algorithms (such as PSNR, SSIM, etc.) are determined. Various possible fusion strategies are tested on the first dual-path overlay key optimization region image group, such as trying different weighted average coefficients and different Gaussian pyramid fusion scales. The image quality evaluation algorithm is used to quantitatively evaluate the image quality of the key fusion object region under each fusion strategy. Based on the evaluation results, optimization algorithms or decision algorithms are used to select fusion sub-strategies that meet the quality requirements, and the parameters and methods of these sub-strategies are recorded, ultimately forming a set of dual-path overlay key optimization region image fusion sub-strategies.

[0045] For example, using image sharpness and edge smoothness as quality requirements, the SSIM algorithm is used to evaluate image quality. By adjusting the weight parameters in the weighted average fusion strategy, the SSIM value of the key fusion object region after fusion reaches a preset threshold or higher, thereby determining the corresponding weight parameters as part of the fusion sub-strategy.

[0046] This implementation method uses the image overlay and fusion quality of key objects in the fused image meeting preset quality requirements as a fusion constraint to conduct sub-strategy analysis, enabling monitoring personnel to observe the details and status of important objects more clearly, thereby better carrying out safety monitoring and management of high-altitude operations.

[0047] In one possible implementation, the image overlay fusion quality of the first dual-path overlay key fusion object set must meet a preset quality requirement as a fusion constraint. Combined with the first dual-path overlay key optimization region image group, a region fusion sub-strategy analysis is performed to obtain a set of dual-path overlay key optimization region image fusion sub-strategies. Step S3323 further includes step S33231, pre-constructing a key optimization region image fusion processor. Specifically, the functions and objectives to be implemented by the key optimization region image fusion processor are clarified, such as improving the clarity of dynamic targets and enhancing edge details. Then, a suitable image fusion algorithm and technology are selected as the basis of the fusion processor. For example, a Gaussian pyramid fusion algorithm is selected to handle edge-fusion sensitive regions, and a convolutional neural network is selected to handle regions with frequently changing dynamic content. Based on the selected algorithm and technology, the architecture and processing flow of the fusion processor are designed, and corresponding program code or hardware circuits are written. Finally, preliminary parameter settings and initialization are performed. For example, when constructing a deep learning-based key optimization region image fusion processor, a convolutional neural network model structure can be constructed first, the parameter range and initial values ​​of each layer can be defined, and then it can be compiled into an executable program module.

[0048] Step S33232: The key optimization region image fusion unit is used to perform region fusion analysis on the first dual-channel superimposed key optimization region image group to obtain a first key optimization region superimposed fused image. Specifically, the first dual-channel superimposed key optimization region image group is format-converted and preprocessed according to the input requirements of the key optimization region image fusion unit, such as adjusting the image size and number of channels. The preprocessed image group is input into the key optimization region image fusion unit, and the fusion unit is started to run, so that it processes the image according to the preset fusion algorithm. For example, for a fusion unit based on multi-scale transformation, the image is decomposed into sub-band images of different scales, and then fusion processing is performed on each sub-band image, such as weighted average fusion of low-frequency sub-bands and fusion of absolute value maximization of high-frequency sub-bands; for a fusion unit based on deep learning, the image features are automatically learned and fused image is generated through the forward propagation process of the network. Finally, the fusion unit outputs the fused first key optimization region superimposed fused image. During processing, the intermediate outputs of the key optimization region image fusion unit (such as feature maps of each layer and sub-band images at each scale) can be monitored to understand the execution of the fusion process. Finally, the first key optimization region overlay fused image output by the key optimization region image fusion unit is obtained for preliminary quality checks and evaluations to determine whether it meets basic fusion requirements. For example, checking whether the image is complete and whether there are obvious fusion errors (such as color distortion, target loss, etc.).

[0049] Step S33233: Based on the fusion constraints and the first key optimization region overlay fused image, the parameters of the key optimization region image fusion machine are updated and optimized. The updated key optimization region image fusion machine is used as a dual-path overlay key optimization region image fusion sub-strategy, obtaining the dual-path overlay key optimization region image fusion sub-strategy set. Specifically, using preset fusion quality requirements (such as image clarity, target integrity, color consistency, etc.) as fusion constraints, an image quality assessment algorithm is used to assess the quality of the first key optimization region overlay fused image, calculating the values ​​of its various quality indicators. Based on the assessment results and optimization objectives, an optimization algorithm is used to adjust and optimize the parameters of the key optimization region image fusion machine. For example, in a deep learning-based fusion machine, the fused image quality assessment index can be used as a loss function, the output result is calculated through forward propagation, and the gradient is calculated through backpropagation, using gradient descent to update the network parameters; for a multi-scale transformation-based fusion machine, the image quality index can be used as the optimization objective function, and a genetic algorithm is used to search for the optimal parameter combination in the parameter space. During the optimization process, parameters are continuously iterated and updated, and the fusion generator is re-evaluated after each update until the quality of the fused image meets the preset fusion constraints or the maximum number of iterations is reached. Finally, the optimized key optimization region image fusion generator is recorded as a sub-strategy for dual-path overlay key optimization region image fusion, and together with other sub-strategies, it constructs a set of dual-path overlay key optimization region image fusion sub-strategies.

[0050] This implementation method pre-builds a key optimization area image fusion unit and updates and optimizes its parameters. It can automatically adjust the parameters of the fusion unit according to the specific characteristics of the dual-channel superimposed key optimization area image group and the fusion constraint requirements, so that the quality of the generated fused image in the key optimization area meets the preset requirements. This ensures that the image in the key area is clear, complete and accurate, and provides high-quality visual information for high-altitude operation monitoring.

[0051] In one possible implementation, step S332 further includes step S3324, extracting the first dual-path superimposed stable region image group from the set of dual-path superimposed stable region image groups, wherein the first dual-path superimposed stable region image group is any one of the dual-path superimposed stable region image groups in the set. Specifically, the storage structure and indexing method of the set of dual-path superimposed stable region image groups are determined. Based on actual needs (e.g., starting analysis from the earliest image group, starting analysis from an image group at a specific location, random selection, etc.), an index or query condition is selected to extract the corresponding first dual-path superimposed stable region image group from the set. For example, if the image group set is stored in a list arranged chronologically, the first image group can be extracted as the first dual-path superimposed stable region image group through list indexing operations.

[0052] Step S3325: Randomly extract a first pixel from the first dual-path superimposed stable region image group, and randomly iterate the first pixel according to a preset fusion iteration radius to determine a second pixel. Specifically, randomly select one pixel from the first dual-path superimposed stable region image group as the first pixel (starting point). Define a preset fusion iteration radius (e.g., a 5×5 pixel neighborhood), and randomly select a new pixel as the second pixel within the neighborhood of the first pixel according to the preset fusion iteration radius.

[0053] Step S3326: Connect the first pixel and the second pixel to construct a first overlapping and blending line segment. Using the center of the first overlapping and blending line segment as the center and the length of the first overlapping and blending line segment as the diameter, construct a neighborhood of the first overlapping and blending line segment. Specifically, calculate the straight-line distance and midpoint coordinates between the first pixel and the second pixel. Determine the radius of the circular neighborhood (i.e., half the straight-line distance) based on the straight-line distance. Using the midpoint as the center, draw a circular neighborhood covering the area between the first pixel and the second pixel.

[0054] Step S3327: Divide the neighborhood of the first overlay fusion line segment into a left neighborhood and a right neighborhood using the first overlay fusion line segment. Specifically, determine the start and end coordinates of the first overlay fusion line segment. For each pixel within the neighborhood, calculate its position (left or right) relative to the first overlay fusion line segment. Divide the pixels within the neighborhood into a left neighborhood and a right neighborhood.

[0055] Step S3328: Based on the first dual-path overlay stable region image group, perform pixel difference analysis on the left and right neighborhoods of the first overlay fusion line segment. If the pixel difference between the left and right neighborhoods of the first overlay fusion line segment is less than or equal to a preset pixel difference threshold, continue iterating on the second pixel until the image edge of the first dual-path overlay stable region image group is reached, obtaining the overlay path of the first dual-path overlay stable region image group. This overlay path is then used as a dual-path overlay stable region fusion sub-strategy, resulting in a set of dual-path overlay stable region image fusion sub-strategies. Specifically, perform difference analysis on the pixels in the left and right neighborhoods, calculating the difference between pixels. Check if the difference is less than or equal to a preset threshold (e.g., 10 grayscale value). If the condition is met, the image features in that region are considered consistent, and continue iterating on the second pixel, repeating steps S3325 to S3327 until the image edge is reached, forming a complete overlay path. The path is recorded as a dual-path overlay stable region fusion sub-strategy and collected into the dual-path overlay stable region image fusion sub-strategy set.

[0056] This implementation significantly reduces computation by employing a path-based iterative fast fusion method in stable regions. Since stable regions exhibit minimal variation, complex image analysis and processing are unnecessary; simple pixel difference analysis and iteration along the overlay path are sufficient to quickly generate the fusion result. This approach is more efficient than performing fine-grained fusion processing on the entire region, thereby improving the overall system's processing speed and real-time performance.

[0057] In one possible implementation, step S332 further includes step S3329: when the pixel difference between the left and right neighboring image groups of the first overlay fusion line segment is not less than or equal to a preset image group pixel difference threshold, the first overlay fusion line segment undergoes a preset number of angle adjustments, and image group pixel difference analysis is performed after each angle adjustment. Specifically, during pixel difference analysis, if the pixel difference in the left or right neighboring regions exceeds the preset threshold, the angle of the first overlay fusion line segment is adjusted. The angle adjustment can be achieved by rotating the line segment by a certain angle (e.g., adjusting by 5 or 10 degrees each time). After each angle adjustment, pixel difference analysis is performed again to check whether the pixel difference in the neighborhood of the adjusted line segment meets the condition.

[0058] Step S33210: If, after a preset number of angle adjustments, the pixel difference between the left and right neighboring image groups of the first overlay fused line segment is still not less than or equal to a preset pixel difference threshold, then the first pixel is selected again. Specifically, if, after a preset number of angle adjustments, the pixel difference still exceeds the threshold, the current path is considered unsuitable, and the first pixel needs to be selected again. After selecting the first pixel again, the previous steps are repeated, including random iteration, constructing the line segment neighborhood, and pixel difference analysis, until a path that meets the conditions is found or other termination conditions are met.

[0059] This implementation method, through angle adjustment and pixel reselection, can better adapt to complex structures and changes in images. In some cases, the initially selected path may have large pixel differences due to changes in image features (such as edges, texture changes, etc.). By adjusting the angle and reselecting the starting point, a more suitable path can be found, thereby improving the adaptability and robustness of the fusion path.

[0060] Step S400: The monitoring camera array is used to monitor the work site in real time, and the monitoring images are overlaid and fused based on the image overlay region fusion strategy to obtain an initial overlaid and fused image.

[0061] Specifically, during real-time monitoring, the camera array continuously acquires images at a set frame rate and transmits the image data to the monitoring server or fusion processing device via an image transmission network. On the server or device side, based on a constructed image overlay fusion strategy, real-time images from different cameras are overlaid and fused. For each camera image, according to its correspondence in the fusion strategy, it is fused with images from adjacent cameras in the overlay area. Image pixels are calculated and replaced according to a preset fusion algorithm to generate a complete initial overlaid fused image, reflecting the overall situation of the work site in real time.

[0062] Step S500: Collect environmental information, and perform enhancement analysis on the initial overlay fusion image based on the environmental information to obtain the target overlay fusion image.

[0063] Specifically, environmental sensors are installed at the work site. These sensors are used to sense and measure environmental parameters, such as light sensors to detect light intensity, temperature and humidity sensors to measure ambient temperature and humidity, and wind speed sensors to monitor wind speed. They are connected to the monitoring system to ensure that environmental information is collected in real time and transmitted to the monitoring server or processing equipment. At the server or equipment end, the environmental information data collected by the sensors is received, stored, and analyzed. Based on the type and value of the environmental information, appropriate image enhancement algorithms are applied to process the initial overlay and fused image. For example, for light information, histogram equalization and gamma correction algorithms can be used to adjust the brightness and contrast of the image to enhance its visibility under different lighting conditions; for temperature and humidity information, thermal imaging models can be used to compensate or correct the thermal features of the image, improving the ability to identify the thermal features of workers and equipment; for wind speed information, moving target detection and compensation algorithms can be used to compensate for and filter dynamic objects in the image, reducing their impact on image quality and monitoring effectiveness. Finally, the enhanced image is displayed and stored in real time as the target overlay and fused image, providing monitoring personnel with a higher quality monitoring view.

[0064] In one possible implementation, environmental information is collected, and the initial overlay fused image is enhanced and analyzed based on the environmental information to obtain a target overlay fused image. Step S500 further includes step S510, collecting environmental information according to preset environmental indicators. Specifically, the environmental indicators to be collected are determined, such as light intensity, temperature, humidity, wind speed, and noise. Appropriate environmental sensors are installed at the work site, and their connection to the monitoring system is ensured. The data acquisition frequency (e.g., once per second) and transmission method (e.g., via wired or wireless network) are set. The data acquisition program is started to collect data from the environmental sensors in real time and store it in the monitoring server or fusion processing device.

[0065] Step S520: Obtain a pre-built image enhancement analyzer. Use the image enhancement analyzer to perform enhancement analysis on the environmental information and the initial overlay fusion image to obtain the target overlay fusion image. Specifically, an image enhancement analyzer is pre-built, which can be a rule-based image processing module or a trained deep learning model. Load the pre-built image enhancement analyzer, ensuring its compatibility with other modules of the monitoring system. Input the collected environmental information and the initial overlay fusion image into the image enhancement analyzer. Based on the environmental information, automatically adjust the image processing parameters. For example, if the light intensity is low, increase the brightness and contrast of the image; if the wind speed is high, apply an image stabilization algorithm to process the initial overlay fusion image to generate the target overlay fusion image. Output the processed target overlay fusion image to the monitoring terminal or storage device.

[0066] This approach improves image visibility under different environmental conditions by dynamically adjusting image processing parameters based on environmental information.

[0067] In one possible implementation, step S500 further includes step S530, which involves scoring the image quality of the target overlay and fused image, and updating and optimizing the parameters of the image enhancement analyzer based on the image quality score.

[0068] Specifically, multiple quality metrics of the target overlay and fused image are calculated, such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and edge gradient. These metrics are compared with preset quality standards to generate a comprehensive image quality score. Based on the score, parameters that need adjustment are determined. For example, if the score indicates insufficient image brightness, brightness adjustment parameters can be increased; if edges are blurry, edge detection and sharpening parameters can be enhanced. Optimization algorithms, such as gradient descent, are used to automatically adjust these parameters to improve image quality. The updated parameters are applied to the image enhancement analyzer, and image enhancement processing is performed again. This process is repeated until the image quality score reaches a satisfactory level.

[0069] This approach ensures that the output images maintain a consistently high quality level by periodically or in real-time evaluating image quality and adjusting the parameters of the image enhancement analyzer accordingly. This method dynamically adapts to environmental changes and system performance fluctuations, guaranteeing the clarity and usability of the surveillance images.

[0070] This application embodiment employs a wide-angle camera array deployed at the work site. After time synchronization and coordination, the array performs trial sampling to acquire a set of monitoring image samples. After image overlay region identification and analysis, a fusion strategy is constructed for real-time monitoring and acquisition. An initial overlay fused image is generated according to the fusion strategy. The initial overlay fused image and independently acquired environmental information data are input into an image enhancement and analysis module. After processing, a target overlay fused image is output. These technical means solve the technical problems of low image overlay fusion quality and insufficient monitoring accuracy in existing high-altitude operation monitoring, achieving the technical effect of improving image overlay fusion quality and monitoring accuracy.

[0071] In the above text, refer to Figure 1 This paper describes in detail a method for overlaying and fusing multi-camera images in high-altitude operation monitoring according to an embodiment of the present invention. Next, we will refer to... Figure 2 A multi-camera image overlay and fusion system for high-altitude operation monitoring according to an embodiment of the present invention is described.

[0072] The multi-camera image overlay and fusion system for high-altitude operation monitoring according to embodiments of the present invention addresses the technical problems of low image overlay and fusion quality and insufficient monitoring accuracy in existing high-altitude operation monitoring systems, thereby improving both image overlay and fusion quality and monitoring accuracy. The multi-camera image overlay and fusion system for high-altitude operation monitoring includes: a monitoring camera array construction module 10, a trial sampling module 20, an image overlay region fusion strategy construction module 30, an overlay and fusion module 40, and an enhancement analysis module 50.

[0073] The monitoring camera array construction module 10 is used to deploy multiple wide-angle cameras at key locations in the work site to construct a monitoring camera array; the trial sampling module 20 is used to perform time synchronization coordination of the monitoring camera array based on the network time protocol, and to perform trial sampling using the monitoring camera array to obtain a set of monitoring camera trial sampling images; the image overlay region fusion strategy construction module 30 is used to perform independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategy based on the set of monitoring camera trial sampling images to construct an image overlay region fusion strategy; the overlay fusion module 40 is used to perform real-time monitoring of the work site using the monitoring camera array, and to overlay and fuse the monitoring images based on the image overlay region fusion strategy to obtain an initial overlay fused image; the enhancement analysis module 50 is used to collect environmental information, and to perform enhancement analysis on the initial overlay fused image based on the environmental information to obtain a target overlay fused image.

[0074] The specific configuration of the image overlay region fusion strategy construction module 30 will be described in detail below. As mentioned above, based on the set of test-sampled images from the surveillance cameras, the dual-channel and multi-channel image overlay region identification and fusion strategy independent analysis are performed to construct the image overlay region fusion strategy. The image overlay region fusion strategy construction module 30 may further include: a perspective conversion unit used to perform perspective conversion of the set of test-sampled images from the surveillance cameras in a unified spatial coordinate system using a feature point matching algorithm to obtain a converted set of test-sampled images from the surveillance cameras; an overlay heatmap identification unit used to perform overlay heatmap identification on the converted set of test-sampled images from the surveillance cameras based on the camera calibration parameters and coordinate mapping relationships of each wide-angle camera in the surveillance camera array to determine the dual-channel overlay region set and the dense overlay region set; and a fusion strategy independent analysis unit used to perform fusion strategy independent analysis based on the dual-channel overlay region set and the dense overlay region set, combined with the converted set of test-sampled images from the surveillance cameras, to construct the image overlay region fusion strategy.

[0075] Specifically, based on the dual-path overlay region set and the dense overlay region set, and combined with the set of sampled images from the conversion monitoring camera, an independent analysis of the fusion strategy is performed to construct the image overlay region fusion strategy. The independent analysis unit of the fusion strategy may further include: a region identification subunit used to identify key optimization regions and stable regions in the set of sampled images from the conversion monitoring camera based on the dual-path overlay region set and the dense overlay region set, obtaining a set of dual-path overlay key optimization region image groups, a set of dual-path overlay stable region image groups, a set of dense overlay key optimization region image groups, and a set of dense overlay stable region image groups; and a fusion strategy analysis subunit used to traverse the set of dual-path overlay key optimization region image groups. A fusion strategy analysis is performed on the set of dual-path superimposed stable region images, the set of densely superimposed key optimization region images, and the set of densely superimposed stable region images to obtain a set of fusion sub-strategies for dual-path superimposed key optimization region images, a set of fusion sub-strategies for dual-path superimposed stable region images, a set of fusion sub-strategies for densely superimposed key optimization region images, and a set of fusion sub-strategies for densely superimposed stable region images. A regional storage sub-unit is used to store the set of fusion sub-strategies for dual-path superimposed key optimization region images, the set of fusion sub-strategies for dual-path superimposed stable region images, the set of fusion sub-strategies for densely superimposed key optimization region images, and the set of fusion sub-strategies for densely superimposed stable region images in different regions, thereby constructing the image superimposed region fusion strategy.

[0076] The fusion strategy analysis subunit may further include: a first dual-path overlay key optimization region image group extraction component for extracting a first dual-path overlay key optimization region image group from the set of dual-path overlay key optimization region image groups, wherein the first dual-path overlay key optimization region image group is any one of the dual-path overlay key optimization region image groups in the set of dual-path overlay key optimization region image groups; a target recognition component for using a target detection algorithm to identify key fusion objects in the fused image to obtain a first dual-path overlay key fusion object set; and a region fusion sub-strategy analysis component for performing region fusion sub-strategy analysis in conjunction with the first dual-path overlay key optimization region image group, using the image overlay fusion quality of the first dual-path overlay key fusion object set meeting a preset quality requirement as a fusion constraint, to obtain a set of dual-path overlay key optimization region image fusion sub-strategies.

[0077] Specifically, the image overlay fusion quality of the first dual-path overlay key fusion object set must meet a preset quality requirement as a fusion constraint. Combined with the first dual-path overlay key optimization region image group, a region fusion sub-strategy analysis is performed to obtain a set of dual-path overlay key optimization region image fusion sub-strategies. The region fusion sub-strategy analysis component may further include: a key optimization region image fusion pre-construction sub-component for pre-constructing a key optimization region image fusion builder; a region fusion analysis sub-component for using the key optimization region image fusion builder to perform region fusion analysis on the first dual-path overlay key optimization region image group to obtain a first key optimization region overlay fusion image; and a parameter update optimization sub-component for updating and optimizing the parameters of the key optimization region image fusion builder based on the fusion constraint and the first key optimization region overlay fusion image, using the updated key optimization region image fusion builder as a dual-path overlay key optimization region image fusion sub-strategy to obtain the set of dual-path overlay key optimization region image fusion sub-strategies.

[0078] The fusion strategy analysis subunit may further include: a first dual-path superimposed stable region image group extraction component for extracting a first dual-path superimposed stable region image group from the set of dual-path superimposed stable region image groups, wherein the first dual-path superimposed stable region image group is any one of the dual-path superimposed stable region image groups in the set of dual-path superimposed stable region image groups; a pixel point random iteration component for randomly extracting a first pixel point from the first dual-path superimposed stable region image group and randomly iterating the first pixel point according to a preset fusion iteration radius to determine a second pixel point; a first superimposed fusion line segment neighborhood construction component for connecting the first pixel point and the second pixel point to construct a first superimposed fusion line segment, using the center of the first superimposed fusion line segment as the center and the length of the first superimposed fusion line segment as the diameter to construct a first superimposed fusion line segment neighborhood; and a neighborhood division component for using a first... The overlay fusion segment divides the neighborhood of the first overlay fusion segment into a left neighborhood and a right neighborhood. The pixel difference analysis component is used to perform pixel difference analysis on the left and right neighborhoods of the first overlay fusion segment based on the first dual-path overlay stable region image group. If the pixel difference of the left neighborhood image group and the pixel difference of the right neighborhood image group are both less than or equal to the preset pixel difference threshold, the second pixel point is iterated until the image edge of the first dual-path overlay stable region image group is reached, the overlay path of the first dual-path overlay stable region image group is obtained, and the overlay path of the first dual-path overlay stable region image group is used as the dual-path overlay stable region fusion sub-strategy to obtain the dual-path overlay stable region image fusion sub-strategy set.

[0079] The fusion strategy analysis subunit may further include: an angle adjustment component for adjusting the angle of the first overlay fusion line segment a preset number of times when the pixel difference between the left and right neighboring image groups of the first overlay fusion line segment is not less than or equal to a preset image group pixel difference threshold, and performing image group pixel difference analysis after each angle adjustment; and a first pixel reselection component for reselecting the first pixel if the pixel difference between the left and right neighboring image groups of the first overlay fusion line segment is still not less than or equal to the preset image group pixel difference threshold after a preset number of angle adjustments.

[0080] The specific configuration of the enhancement analysis module 50 will be described in detail below. As mentioned above, environmental information is collected, and enhancement analysis is performed on the initial overlay fusion image based on the environmental information to obtain the target overlay fusion image. The enhancement analysis module 50 may further include: an environmental information collection unit for collecting environmental information according to preset environmental indicators; and an enhancement analysis unit for obtaining a pre-constructed image enhancement analyzer, and using the image enhancement analyzer to perform enhancement analysis on the environmental information and the initial overlay fusion image to obtain the target overlay fusion image.

[0081] The enhancement analysis module 50 may further include: an image quality scoring unit for scoring the image quality of the target overlay fused image, and updating and optimizing the parameters of the image enhancement analyzer based on the image quality scoring results.

[0082] The multi-camera image overlay and fusion system for high-altitude operation monitoring provided in this embodiment of the invention can execute the multi-camera image overlay and fusion method for high-altitude operation monitoring provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0083] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0084] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for overlaying and fusing images from multiple cameras in high-altitude operations monitoring, characterized in that, The method includes: Multiple wide-angle cameras were deployed at key locations on the work site to form a surveillance camera array; The monitoring camera array is synchronized and coordinated in time based on the Network Time Protocol, and trial sampling is performed using the monitoring camera array to obtain a set of trial sampled images from the monitoring cameras; Based on the set of sampled images from the surveillance cameras, independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies is performed to construct an image overlay region fusion strategy. The monitoring camera array is used to monitor the work site in real time, and the monitoring images are overlaid and fused based on the image overlay region fusion strategy to obtain an initial overlaid and fused image; Collect environmental information, and perform enhancement analysis on the initial overlay and fusion image based on the environmental information to obtain the target overlay and fusion image; The independent analysis of dual-channel and multi-channel image overlay region identification and fusion strategies based on the sampled images from the surveillance cameras, and the construction of an image overlay region fusion strategy, include: The set of test images from the surveillance cameras is transformed using a feature point matching algorithm under a unified spatial coordinate system to obtain a transformed set of test images from the surveillance cameras. Based on the camera calibration parameters and coordinate mapping relationship of each wide-angle camera in the monitoring camera array, the set of sampled images from the converted monitoring cameras is subjected to superimposed heat map identification to determine the set of dual-path superimposed regions and the set of densely superimposed regions. Based on the set of dual-path overlay regions and the set of densely overlay regions, and combined with the set of sampled images from the conversion monitoring camera, the fusion strategy is independently analyzed to construct the image overlay region fusion strategy. The process of determining the set of dual-path overlay regions and the set of dense overlay regions includes: calculating the coverage area of ​​each image in the global coordinate system and determining the overlapping areas between different images to construct an overlay heatmap; on the overlay heatmap, counting the number of times each pixel is overlaid in the overlapping area, identifying the dual-path overlay regions and dense overlay regions based on the threshold of the number of overlays, and collecting them to form corresponding sets. The method involves independently analyzing the fusion strategy based on the dual-path overlay region set and the dense overlay region set, combined with the sampled images from the converted monitoring camera, to construct the image overlay region fusion strategy, including: Based on the set of dual-path overlay regions and the set of densely overlay regions, key optimization regions and stable regions are identified in the set of sampled images from the conversion monitoring camera to obtain a set of dual-path overlay key optimization region images, a set of dual-path overlay stable region images, a set of densely overlay key optimization region images, and a set of densely overlay stable region images. The fusion strategy is analyzed by traversing the set of dual-path superimposed key optimization region images, the set of dual-path superimposed stable region images, the set of densely superimposed key optimization region images, and the set of densely superimposed stable region images to obtain the set of dual-path superimposed key optimization region image fusion sub-strategies, the set of dual-path superimposed stable region image fusion sub-strategies, the set of densely superimposed key optimization region image fusion sub-strategies, and the set of densely superimposed stable region image fusion sub-strategies. The image overlay region fusion strategy is constructed by storing the set of image fusion sub-strategies for dual-path overlay key optimization regions, dual-path overlay stable regions, dense overlay key optimization regions, and dense overlay stable regions in separate regions.

2. The multi-camera image overlay and fusion method for high-altitude operation monitoring as described in claim 1, characterized in that, include: Extract the first dual-path superimposed key optimization region image group from the set of dual-path superimposed key optimization region image groups, wherein the first dual-path superimposed key optimization region image group is any dual-path superimposed key optimization region image group in the set of dual-path superimposed key optimization region image groups; The key fusion objects in the first key optimization region overlay fusion image are identified using a target detection algorithm to obtain the first dual-path overlay key fusion object set; the first key optimization region overlay fusion image is: the image obtained by performing region fusion analysis on the first dual-path overlay key optimization region image group using a key optimization region image fusion processor; Taking the image overlay fusion quality of the first dual-path overlay key fusion object set as a fusion constraint that it must meet a preset quality requirement, and combining the first dual-path overlay key optimization region image group, a region fusion sub-strategy analysis is performed to obtain a set of dual-path overlay key optimization region image fusion sub-strategies.

3. The multi-camera image overlay and fusion method for high-altitude operation monitoring as described in claim 2, characterized in that, Taking the image overlay fusion quality of the first dual-path overlay key fusion object set as a fusion constraint that it must meet a preset quality requirement, and combining the first dual-path overlay key optimization region image group, a region fusion sub-strategy analysis is performed to obtain a set of dual-path overlay key optimization region image fusion sub-strategies, including: Pre-build key optimization region image fusion; The key optimization region image fusion processor is used to perform region fusion analysis on the first dual-path superimposed key optimization region image group to obtain the first key optimization region superimposed fused image. Based on the fusion constraints and the first key optimization region overlay fused image, the parameters of the key optimization region image fusion machine are updated and optimized. The updated key optimization region image fusion machine is used as a dual-path overlay key optimization region image fusion sub-strategy to obtain the dual-path overlay key optimization region image fusion sub-strategy set.

4. The multi-camera image overlay and fusion method for high-altitude operation monitoring as described in claim 1, characterized in that, include: Extract the first dual-path superimposed stable region image group from the set of dual-path superimposed stable region image groups, wherein the first dual-path superimposed stable region image group is any dual-path superimposed stable region image group in the set of dual-path superimposed stable region image groups; The first pixel is randomly extracted from the first dual-path superimposed stable region image group, and the first pixel is randomly iterated according to the preset fusion iteration radius to determine the second pixel. Connect the first pixel and the second pixel to construct a first superimposed and blended line segment. With the center of the first superimposed and blended line segment as the center and the length of the first superimposed and blended line segment as the diameter, construct the neighborhood of the first superimposed and blended line segment. The neighborhood of the first superimposed and merged line segment is divided into the left neighborhood of the first superimposed and merged line segment and the right neighborhood of the first superimposed and merged line segment by the first superimposed and merged line segment; Based on the first dual-path superimposed stable region image group, pixel difference analysis is performed on the left and right neighborhoods of the first superimposed fusion line segment. If the pixel difference of the left and right neighborhoods of the first superimposed fusion line segment is less than or equal to a preset pixel difference threshold, the second pixel is iterated until the image edge of the first dual-path superimposed stable region image group is reached, the superimposed path of the first dual-path superimposed stable region image group is obtained, and the superimposed path of the first dual-path superimposed stable region image group is used as the dual-path superimposed stable region fusion sub-strategy to obtain the dual-path superimposed stable region image fusion sub-strategy set.

5. The multi-camera image overlay and fusion method for high-altitude operation monitoring as described in claim 4, characterized in that, include: When the pixel difference between the left neighboring image group and the right neighboring image group of the first superimposed and fused line segment is not less than or equal to a preset image group pixel difference threshold, the first superimposed and fused line segment is adjusted at an angle a preset number of times, and image group pixel difference analysis is performed after each angle adjustment. If, after a preset number of angle adjustments, the pixel difference between the left and right neighboring image groups of the first superimposed and fused line segment is still not less than or equal to the preset pixel difference threshold, then the first pixel is selected again.

6. The multi-camera image overlay and fusion method for high-altitude operation monitoring as described in claim 1, characterized in that, Collecting environmental information, and performing enhancement analysis on the initial overlay and fusion image based on the environmental information to obtain a target overlay and fusion image, including: Collect environmental information according to preset environmental indicators; A pre-constructed image enhancement analyzer is obtained, and the image enhancement analyzer is used to perform enhancement analysis on the environmental information and the initial overlay fusion image to obtain the target overlay fusion image.

7. The multi-camera image overlay and fusion method for high-altitude operation monitoring as described in claim 6, characterized in that, The image quality of the target overlay and fused image is scored, and the parameters of the image enhancement analyzer are updated and optimized based on the image quality score results.

8. A multi-camera image overlay and fusion system for high-altitude operation monitoring, characterized in that, The system is used to implement the multi-camera image overlay and fusion method for high-altitude operation monitoring as described in any one of claims 1-7, the system comprising: The surveillance camera array construction module is used to deploy multiple wide-angle cameras at key locations on the work site to construct a surveillance camera array; The trial sampling module is used to perform time synchronization and coordination of the monitoring camera array based on the network time protocol, and to perform trial sampling using the monitoring camera array to obtain a set of trial sampled images from the monitoring cameras; The image overlay region fusion strategy construction module is used to independently analyze the dual-channel and multi-channel image overlay region identification and fusion strategy based on the set of sampled images from the surveillance camera, and to construct the image overlay region fusion strategy. The overlay and fusion module is used to monitor the work site in real time using the monitoring camera array, and to overlay and fuse the monitoring images based on the image overlay area fusion strategy to obtain an initial overlay and fused image; The enhancement analysis module is used to collect environmental information and perform enhancement analysis on the initial overlay fusion image based on the environmental information to obtain the target overlay fusion image.

Images