Image fusion method based on differential thought

Abstract

The invention provides an image fusion method based on a differential thought, and the method comprises the steps: obtaining an extended and cut target image block as a processing starting point, preserving the edge and context information of an original image, carrying out the bidirectional verification matching of multi-scale feature points, improving the matching reliability, and generating an initial spliced image based on accurate geometric transformation. Quantitative analysis is carried out by dividing sub-regions and calculating a pixel mean value, finally, pixel values of the sub-regions are dynamically adjusted according to mean value differences, splicing is carried out again, and smooth optimization is directly carried out on splicing seams. Multi-scale features and bidirectional verification enhance the adaptability of the algorithm to different image qualities and scenes, a differential adjustment strategy acts on local splicing seams, ghosting and brightness discontinuity can be effectively eliminated, meanwhile, image details are kept, and then high-quality and high-efficiency image superposition is achieved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image fusion method based on the concept of differentiation. Background Technology

[0002] Image fusion technology is a processing method that combines information from multiple source images into a single image. In fields such as medical image analysis and computer vision, this technology is used to improve image quality or enhance specific features. Processing devices integrate image data through algorithms, and users require high-quality and efficient fusion results.

[0003] Feature point matching is a method for image stitching. It extracts image feature points using SIFT or ORB algorithms, establishes correspondences between images using feature descriptors, estimates geometric transformation parameters using the RANSAC algorithm, and finally completes the stitching using linear fusion or multi-band fusion algorithms.

[0004] This scheme includes preprocessing, feature matching, and pixel-level fusion steps, which result in high computational overhead and low processing efficiency. At the same time, it is easy to lose detailed features in complex texture areas, resulting in low quality. Summary of the Invention

[0005] This application provides an image fusion method based on the concept of differentiation to solve the problems of low image fusion quality and low efficiency.

[0006] This application provides an image fusion method based on the concept of differentiation, including: Obtain a target image patch, which is generated from the input image after expansion and cropping; Multi-scale feature points of the target image patch are matched to calculate geometric transformation parameters between the target image patches based on the successfully matched feature point pairs. The target image blocks are aligned according to the geometric transformation parameters to generate an initial stitched image; On the initial stitched image, a fusion evaluation area is defined with a set center. The fusion evaluation area is divided into multiple sub-regions along the first direction, and the average pixel value of the image stitching seam within the sub-regions is calculated. Based on the difference in the average pixel value between adjacent sub-regions, the pixel values ​​of the sub-regions are proportionally adjusted to obtain the adjusted sub-regions; The adjusted sub-regions are stitched together along the second direction to obtain a fused image.

[0007] In some feasible embodiments, acquiring the target image patch includes: Acquire multiple input images to be fused; Perform target detection on the input image to obtain the target region; Based on the target area, expand outward by a preset margin ratio to form the expanded area; Based on the boundary of the expanded region, an image block is cropped from the input image to obtain the target image block.

[0008] Image patches are obtained by expanding and cropping the detected target, while preserving the target's contextual information.

[0009] In some feasible embodiments, performing target detection on the input image to obtain the target region includes: The input image is fed into a neural network model to output candidate detection boxes containing location and confidence scores. The neural network model has multi-scale feature extraction capabilities. Based on a predetermined confidence threshold, the candidate detection boxes are filtered, and a non-maximum suppression algorithm is applied to remove overlapping boxes to obtain the target region.

[0010] Target detection is performed using a neural network with multi-scale feature extraction capabilities, and the accuracy and completeness of target region localization are ensured through threshold filtering and nonmaximum suppression.

[0011] In some feasible embodiments, the step of performing multi-scale feature point matching on the target image patch to calculate geometric transformation parameters between the target image patches based on the successfully matched feature point pairs includes: From each of the target image blocks, feature points and corresponding feature descriptors at multiple scales are extracted; each target image block includes at least a first image block and a second image block. Forward matching is performed between the feature points of the first image block and the feature points of the second image block to obtain the matching result; Perform reverse verification between the feature points of the second image block and the first image block to obtain the verification result; The feature point pairs that correspond to each other in the matching results and verification results are retained as the successfully matched feature point pairs; Based on the successfully matched feature point pairs, calculate the geometric transformation parameters between the target image blocks.

[0012] By extracting multi-scale feature points and performing bidirectional consistency checks through forward matching and backward verification, the reliability of feature matching is improved, thereby calculating more accurate geometric transformation parameters.

[0013] In some feasible embodiments, aligning the target image patches according to geometric transformation parameters to generate an initial stitched image includes: Spatial transformation is performed on each of the target image blocks using the geometric transformation parameters to obtain aligned image blocks; The pixel intensity of the aligned image block is fused using a maximum value selection strategy, which selects the edge regions and high-texture regions in the aligned image block. And / or, For the edge regions and high-texture regions in the aligned image patch, a weighted averaging strategy is used to perform pixel fusion to generate the initial stitched image.

[0014] On the aligned image patches, a maximum value selection strategy and a weighted averaging strategy are used for pixel fusion, which balances the preservation of salient features and smooth local transitions, thereby improving the quality of the initial stitched image.

[0015] In some feasible embodiments, the sub-region includes a first type of sub-region, a second type of sub-region, and a third type of sub-region; The step of dividing the fusion evaluation area into multiple sub-regions along the first direction includes: The fusion evaluation area is divided into a first main area, a second main area, and a third main area along the first direction; The first main region is evenly divided into multiple first-type sub-regions along the second direction, wherein the second direction is perpendicular to the first direction; The second main region is evenly divided into multiple second-type sub-regions along the second direction; The third main region is evenly divided into multiple third-type sub-regions along the second direction.

[0016] The fusion evaluation area is divided into multiple specific sub-regions, providing a structural basis for subsequent differentiated pixel value analysis and adjustment for different regions.

[0017] In some feasible embodiments, the step of proportionally adjusting the pixel values ​​of the sub-regions based on the difference in the average pixel values ​​between adjacent sub-regions to obtain the adjusted sub-regions includes: Calculate a first mean, which is the mean of pixels including the first seam within the first sub-region, where the first seam is the seam between the first sub-region and the second sub-region; Calculate a second mean and a third mean, where the second mean is the mean of pixels including the first seam within the second type of sub-region, and the third mean is the mean of pixels including the second seam within the second type of sub-region, where the second seam is the seam between the second type of sub-region and the third type of sub-region; Calculate the fourth mean, which is the mean of the pixels including the second stitching seam within the third sub-region.

[0018] By calculating the average pixel value of the seams between various sub-regions, the fusion quality differences at the boundaries of different regions are quantitatively evaluated, providing a basis for targeted adjustments.

[0019] In some feasible embodiments, the step of proportionally adjusting the pixel values ​​of the sub-regions based on the difference in the average pixel values ​​between adjacent sub-regions to obtain adjusted sub-regions includes: When the absolute difference between the first mean and the second mean exceeds the first threshold, the ratio of the first mean to the second mean is used as the first adjustment coefficient. The first adjustment coefficient is used to perform a weighted calculation on the pixel values ​​of the first type of sub-region to obtain the adjusted first type of sub-region; When the absolute difference between the third mean and the fourth mean exceeds the second threshold, the ratio of the fourth mean to the third mean is used as the second adjustment coefficient. The second adjustment coefficient is used to perform a weighted calculation on the pixel values ​​of the second type of sub-region to obtain the adjusted second type of sub-region.

[0020] Based on the difference in the average value of the seams between adjacent sub-regions, the pixel value ratio of the corresponding sub-regions is dynamically adjusted to effectively eliminate local seams and visual discontinuities.

[0021] In some feasible embodiments, stitching the adjusted sub-regions along a second direction to obtain a fused image includes: The adjusted first type of sub-regions are spliced ​​together along the first direction to obtain the first combined region; The third type of sub-regions are spliced ​​together along the first direction to obtain the second combined region; The first combined region, the second sub-region, and the second combined region are stitched together along the second direction to obtain a fused image.

[0022] The sub-regions, after differential adjustment, are combined and stitched together to integrate the local optimization effects, ultimately outputting a visually coherent and seamless fused image.

[0023] In some feasible embodiments, the steps of acquiring the target image patch and performing matching on multi-scale feature points are performed by an inference engine that has been optimized by the model and accelerated by the computational graph.

[0024] By optimizing and accelerating the inference engine to perform key steps, the processing speed of image patch acquisition and feature matching is significantly improved, meeting the requirements for high-efficiency processing.

[0025] As can be seen from the above technical solutions, this application provides an image fusion method based on the concept of differentiation. This method uses an expanded and cropped target image patch as the starting point for processing, preserving the edge and contextual information of the original image. It performs bidirectional verification matching on multi-scale feature points to improve matching reliability. After generating an initial stitched image based on accurate geometric transformation, it performs quantitative analysis by dividing sub-regions and calculating pixel mean values. Finally, it dynamically adjusts the pixel values ​​of the sub-regions based on the mean differences and re-stitches them, directly optimizing the smoothing of the stitching seams. Multi-scale features and bidirectional verification enhance the algorithm's adaptability to different image qualities and scenes. The differential adjustment strategy acts on the local stitching seams, effectively eliminating ghosting and brightness discontinuities while preserving image details, thus achieving high-quality and high-efficiency image overlay. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the image stitching process provided in an embodiment of this application; Figure 2 A flowchart illustrating the image fusion method based on the concept of differential equations provided in this application embodiment; Figure 3 This is a schematic diagram of the input image provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the division of the fusion evaluation region in an embodiment of this application; Figure 5 This is a schematic diagram of the region division provided for an embodiment of this application; Figure 6 This is a schematic diagram of region segmentation provided for an embodiment of this application; Figure 7 A set of target detection training results provided for embodiments of this application; Figure 8 This is a schematic diagram of the OCT target detection inference results provided in the embodiments of this application; Figure 9 Comparison of splicing effects provided in the embodiments of this application. Detailed Implementation

[0028] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application.

[0029] like Figure 1 As shown, in some embodiments, the image stitching method based on feature point matching typically adopts the following technical route: First, image feature points are extracted using algorithms such as SIFT, ORB, or SuperPoint; then, feature descriptors are used for feature matching to establish the correspondence between images; next, the homography matrix is ​​estimated using algorithms such as RANSAC to calculate the geometric transformation parameters between images; finally, algorithms such as linear fusion, multi-band fusion, or optimal stitching line are used to complete the image stitching.

[0030] In practice, the input image is usually preprocessed first, including size normalization and histogram equalization. Then, local features are extracted using a sliding window or block processing method. In the matching stage, the k-nearest neighbor algorithm is used to find corresponding point pairs, and mismatches are eliminated through geometric constraints. Finally, image alignment is achieved through affine transformation or perspective transformation, and pixel-level fusion is performed using fade-in and fade-out weights.

[0031] In the processing, the common approach is to crop first and then stitch together the images. Before feature extraction, the images need to be cropped or segmented, which leads to the loss of edge information and contextual features of the original images. This information loss can seriously affect the accuracy of subsequent analysis and diagnosis, especially in professional fields such as medical imaging and remote sensing images.

[0032] When processing images with significant variations in lighting and viewing angles, noticeable stitching seams, ghosting, and artifacts are easily produced. Linear fusion methods tend to exhibit brightness discontinuities at the boundaries of overlapping regions, while multi-band fusion, although improving visual effects, suffers from high computational complexity and struggles to maintain detail consistency.

[0033] Furthermore, it requires complex feature extraction, matching, and optimization steps, resulting in high computational overhead. This is especially true when processing high-resolution images, where the number of feature points is enormous, making the matching process extremely time-consuming and difficult to meet the needs of real-time applications.

[0034] It is sensitive to changes in image quality, shooting conditions, and content. In scenes with noise, blur, or repetitive textures, the feature matching success rate drops significantly, leading to stitching failure or quality degradation.

[0035] When balancing the contributions of different images, fusion algorithms often struggle to maintain overall consistency while preserving local details, especially in areas with complex textures and intricate structures, where problems such as blurred details or loss of features can easily occur.

[0036] These shortcomings severely limit the practical value of existing technologies in high-precision applications such as medical image analysis and industrial inspection. To address these issues, this application provides an image fusion method based on the concept of differential calculus, such as... Figure 2 As shown, it includes the following steps: S201: Obtain the target image block.

[0037] A target image patch is an image region obtained by performing specific processing on an input image. A target image patch preserves the target of interest and its surrounding contextual information from the original image. Obtaining a target image patch may involve target detection and extended cropping operations. In subsequent processing, the target image patch serves as a basic unit for feature matching and fusion. For example, in medical imaging, a target image patch might contain an organ region to be observed and its surrounding tissues.

[0038] Some solutions employ a process of cropping before stitching, which requires cropping or segmenting the image before feature extraction. This results in the loss of edge information and contextual features of the original image. This information loss can seriously affect the accuracy of subsequent analysis and diagnosis, especially in professional fields such as medical imaging and remote sensing images.

[0039] In this embodiment, the target image block is generated after the input image is expanded and cropped. Specifically, in some embodiments, obtaining the target image block includes: obtaining multiple input images to be fused; performing target detection on the input images to obtain the target region; expanding outwards by a preset margin ratio based on the target region to form an expanded region; and cropping an image block from the input image according to the boundary of the expanded region to obtain the target image block.

[0040] like Figure 3 As shown, the multiple input images to be fused refer to two or more original image data that need to be integrated. These multiple input images may originate from different shooting perspectives of the same scene, or from different imaging modalities or time series of the same object. The multiple input images to be fused are the starting data source for the image fusion processing flow, and their content usually has some overlap or complementarity.

[0041] For the input image, preprocessing is first performed, including converting the color space to RGB format and performing histogram equalization to enhance image contrast. Object detection is then performed on the preprocessed input image. The target region is a rectangular area containing a specific object of interest, identified and located from the input image using object detection technology. The target region is defined by the bounding box output by the detection algorithm, which defines the spatial location and extent of the target in the image. Identifying the target region is crucial for focusing processing on key elements of the image; it is a prerequisite for targeted cropping. In medical image analysis, the target region might be a lesion area or the area of ​​an image containing a specific anatomical structure.

[0042] The target detection stage uses a detection model based on a deep convolutional neural network. This model employs a feature pyramid network structure, enabling it to detect target regions of different sizes at multiple scales. During detection, a confidence threshold of 0.65 and a non-maximum suppression IOU threshold of 0.45 are set to ensure the accuracy and completeness of the detection results.

[0043] In some embodiments, performing object detection on an input image to obtain a target region includes: inputting the input image into a neural network model to output candidate detection boxes containing location and confidence levels through the neural network model; filtering the candidate detection boxes according to a predetermined confidence threshold and applying a non-maximum suppression algorithm to remove overlapping boxes to obtain the target region.

[0044] Neural network models possess multi-scale feature extraction capabilities, enabling them to effectively detect targets of vastly different sizes in images. At a single scale, small targets may be missed due to weak feature responses, while large targets may be inaccurately located due to loss of detail. Through multi-scale feature extraction, the model can simultaneously detect deep semantic features and shallow detailed features, ensuring that targets of different sizes can be consistently identified. For example, the model can employ a feature pyramid network structure, which fuses deep, high-semantic features with shallow, high-resolution features via a top-down path, generating a series of feature maps that integrate semantic and localization information for independent target prediction at various scales.

[0045] Through forward computation of the neural network model, the model outputs candidate detection boxes containing location and confidence scores. The output candidate detection boxes are a formalized representation of the regions in the image where targets exist and their categories. Each candidate detection box includes its center point coordinates, width, height, and a confidence score representing the degree of confidence that the box contains a target of a specific category. These candidate detection boxes are the model's raw, unfiltered interpretation of the image content, which includes a large number of repeated predictions for the same target and some false positives with low confidence scores.

[0046] Next, these candidate detection boxes are filtered based on a confidence threshold of 0.65. The purpose of this filtering is to initially eliminate low-quality predictions that the model itself is not very confident about, thereby reducing the interference of obviously erroneous results on subsequent processes. The predetermined confidence threshold only allows detection boxes with a confidence level higher than this value to pass, which can improve the average reliability of the retained results.

[0047] After initial screening based on confidence, the non-maximum suppression algorithm is applied to remove overlapping boxes. This algorithm is used to solve the problem of the same target being repeatedly labeled by multiple highly similar detection boxes. The non-maximum suppression algorithm calculates the cross-union ratio between each pair of retained detection boxes, that is, the ratio of the overlapping area to the union area.

[0048] When the intersection-union (IoU) ratio of a detection box with another detection box having higher confidence exceeds a preset IoU threshold, the lower-confidence detection box is suppressed. This process iterates through all boxes, ultimately ensuring that for each actual target in the image, only the detection box with the highest confidence is retained. Finally, after the above filtering and deduplication processes, the resulting set of target regions accurately identifies all objects in the input image that require further attention and processing.

[0049] After detecting the target region, a cropping operation is performed. The cropping boundary extends outward by 10% of the fill area from the detection bounding box to retain sufficient contextual information. Simultaneously, the coordinate position, size ratio, and rotation angle of each cropped region in the original image are recorded in detail to provide accurate geometric references for subsequent image registration.

[0050] S202: Perform multi-scale feature point matching on the target image patch to calculate the geometric transformation parameters between the target image patches based on the successfully matched feature point pairs.

[0051] Multi-scale feature points are key points and their descriptive information extracted from images and detected at different scales. Multi-scale feature points include the location coordinates of the feature points and feature descriptors that characterize their local texture structure.

[0052] The feature point extraction stage utilizes deep learning-based feature descriptors. A feature descriptor is a mathematical vector used to quantitatively characterize the visual attributes of a local image region centered on a feature point, such as texture, gradient, or shape. The feature descriptor compresses the high-dimensional pixel information of an image patch into a low-dimensional, discriminative numerical sequence. Feature detection is performed across multiple scales. The algorithm automatically adjusts the feature point density, controlling the number of feature points within the range of 512-1024 based on the image content complexity, thus optimizing computational efficiency while maintaining matching accuracy.

[0053] The feature matching process employs a bidirectional consistency verification mechanism. For example, feature matching is performed from image A to image B, and then reverse matching is performed from image B to image A. Only bidirectional consistent matching point pairs are retained. This process effectively eliminates false matches and improves the reliability of the matching results.

[0054] Specifically, in some embodiments, multi-scale feature points of target image blocks are matched to calculate geometric transformation parameters between target image blocks based on successfully matched feature point pairs. This includes: extracting feature points and corresponding feature descriptors at multiple scales from each target image block; the target image block includes at least a first image block and a second image block; performing forward matching between feature points of the first image block and feature points of the second image block to obtain a matching result; performing reverse verification between feature points of the second image block and feature points of the first image block to obtain a verification result; retaining feature point pairs that correspond to each other in the matching result and verification result as successfully matched feature point pairs; and calculating geometric transformation parameters between target image blocks based on the successfully matched feature point pairs.

[0055] The reason for extracting multi-scale feature points is that there may be differences in shooting distance or resolution between the image patches to be stitched together, causing the same physical feature to appear at different sizes in different images. By detecting feature points in multiple scale spaces, it can be ensured that both corner points of large structures and corners of small textures in the image have a chance to be captured and corresponding. Extracting feature descriptors is to assign a measurable identity to each feature point, enabling the computer to determine whether two points correspond to the same scene point by comparing the similarity of the descriptors. Typically, the target image patch includes at least a first image patch and a second image patch; these are two independent image units whose spatial relationship needs to be established.

[0056] After feature extraction, forward matching is performed. The purpose of forward matching is to try to find the most similar label in the second image for each labeled point in the first image. The matching algorithm calculates the distance between the descriptor of a feature point in the first image patch and the descriptors of all feature points in the second image patch, and selects the point with the smallest distance as a candidate match. The resulting matching result is a mapping list from the first image patch to the second image patch, but this list is preliminary and may contain incorrect pairings due to texture duplication and noise interference.

[0057] To eliminate these erroneous pairings, reverse verification is performed. Reverse verification checks the reciprocity of the forward matching; a correct matching relationship should be bidirectional. That is, if point A finds point B as its best match in the forward matching, then point B should also find point A as its best match in the reverse matching. Reverse verification uses the feature points of the second image patch as the query starting point, recalculates the descriptor distances from them to all feature points of the first image patch, and establishes a new mapping list. This reverse mapping list is the verification result. By comparing the forward matching results with the reverse verification results, point pairs that point to each other in both directions can be identified.

[0058] Subsequently, retaining bidirectional consistent matching point pairs can remove unstable pairings that are true in one direction but not in the other, greatly improving the reliability of the retained point pairs. These successfully matched feature point pairs constitute a high-quality, trustworthy set of correspondences, which are precise anchor points for aligning the content of two images.

[0059] Finally, based on these successfully matched feature point pairs, the geometric transformation parameters between the target image patches are calculated. Calculating these parameters is to use a mathematical model to summarize the spatial mapping pattern from the first image patch to the second. Using the coordinates of multiple pairs of matching points, a transformation matrix can be fitted, which can describe the combined effects of rotation, translation, scaling, and even perspective distortion.

[0060] S203: Align the target image patches according to the geometric transformation parameters to generate an initial stitched image.

[0061] The initial stitched image is an intermediate result image generated after preliminary alignment and fusion of multiple images. The initial stitched image includes the contents of all input images after geometric correction. There may be stitching marks or brightness differences in the overlapping areas of the initial stitched image.

[0062] The image alignment stage utilizes the geometric transformation parameters calculated in the previous stage to perform precise perspective transformation on the input image, eliminating geometric distortion caused by differences in viewpoint. A bilinear interpolation algorithm is used to ensure resampling quality and avoid loss of image details.

[0063] The fusion algorithm employs an improved maximum value fusion strategy, comparing the intensity values ​​of multiple images at the pixel level to retain the most significant image features. Simultaneously, it introduces local consistency constraints, using weighted average fusion in edge and texture-rich regions to maintain visual continuity. The fusion parameters are dynamically adjusted to suit the characteristics of different image regions: a stronger fusion intensity is used in flat areas to suppress noise, while the fusion intensity is reduced in detailed areas to preserve texture information. This adaptive fusion strategy significantly improves the overall quality of the overlaid images.

[0064] Specifically, in some embodiments, aligning target image blocks according to geometric transformation parameters to generate an initial stitched image includes: performing spatial transformation on each target image block using geometric transformation parameters to obtain aligned image blocks; fusing the pixel intensities of the aligned image blocks using a maximum value selection strategy, wherein the maximum value selection strategy is the edge regions and high-texture regions in the aligned image blocks; and / or, performing pixel fusion on the edge regions and high-texture regions in the aligned image blocks using a weighted averaging strategy to generate an initial stitched image.

[0065] Spatial transformation is a computational process that remaps the pixel positions of an image based on a mathematical model. It uses geometric transformation parameters as mapping rules to calculate a new output coordinate position for each pixel coordinate in the input image. The purpose of spatial transformation is to correct geometric differences between images, ensuring that the same point in different images has consistent coordinates in the output space. Examples include affine transformations and perspective transformations, which can achieve linear or non-linear coordinate mappings using a transformation matrix.

[0066] Aligned image blocks are image blocks whose pixel coordinate system has been adjusted to be consistent with the reference coordinate system after spatial transformation. The content of the aligned image block is the same as that of the original target image block, but the scene elements in it have been rearranged to the correct position. The aligned image block is an intermediate product in the image stitching process. The differences in rotation and translation between aligned image blocks have been eliminated in space, creating conditions for pixel-level fusion.

[0067] After obtaining the aligned image patches, the pixel fusion stage begins. The preferred strategy is the maximum value selection strategy, which fuses the pixel intensities of these aligned image patches. The rationale for this strategy is that, in multiple images, the brightest pixel at the same location often represents the ideal exposure, strongest reflection, or most prominent feature at that point. At the pixel level, the maximum value selection strategy compares the pixel intensity values ​​of all aligned image patches at the same coordinate point in parallel and simply selects the largest intensity value as the pixel value for that point in the output image. This strategy is computationally efficient and effectively preserves the most informative parts of each input image; for example, in multifocal image fusion, it preserves the pixels of the sharpest focused area in each image.

[0068] However, choosing the maximum value may produce unnatural hard boundaries in certain areas. Therefore, a weighted averaging strategy is used for pixel fusion, targeting both edge regions and high-texture regions in the aligned image patch.

[0069] The reason for using a weighted average in edge regions is that edges represent the contours of objects, and rigidly selecting the maximum value may lead to broken edge lines or step-like artifacts. By using a weighted average, edge pixels from different image blocks can transition smoothly, maintaining the continuity and naturalness of the edges.

[0070] The reason for using weighted averaging in highly textured regions is that the details and periodicity of the texture need to be blended gently to avoid texture pattern clashes or abrupt blotches caused by directly selecting the maximum value. The weighted averaging strategy assigns a weight ranging from 0 to 1 to each pixel in the overlapping region based on its distance to the boundary of its respective image patch or other features, and then calculates a weighted sum as the final pixel value.

[0071] These two strategies, maximum selection and weighted averaging, are not mutually exclusive, but can be used in combination in practice. One way to combine them is to use maximum selection to preserve significant features in most overlapping areas of the image, while specifically identifying edges and high-texture areas, and switching to the weighted averaging strategy in these local areas. This combination achieves a balance between highlighting global features and smoothing local transitions.

[0072] Finally, an initial stitched image is generated through a fusion operation. Generating the initial stitched image means combining the contents of multiple aligned image blocks into a complete and coherent image. This initial stitched image includes all valid information from the input images, has a correct geometry, and its pixel content has undergone preliminary fusion processing. However, due to differences in illumination, sensor noise, or incompletely uniform transition of fusion weights, the initial stitched image may still have slight brightness or color jumps at the stitching boundaries in some areas.

[0073] S204: On the initial stitching image, divide the fusion evaluation area with the set center.

[0074] like Figure 4 As shown, the fusion evaluation region is a specific area delineated on the initial stitched image for evaluating and optimizing the fusion quality. The fusion evaluation region is usually a regular geometric shape, such as a rectangular area based on the center of the parts to be stitched. The purpose of delineating the fusion evaluation region is to focus the optimization calculation on the main areas where there may be problems. The division of the fusion evaluation region helps to improve the efficiency and focus of subsequent processing steps.

[0075] In this embodiment, the center is set to the center of the part that needs to be stitched together, and the width of the fusion evaluation area is the width of the initial stitched image.

[0076] S205: Divide the fusion evaluation area into multiple sub-regions along the first direction, and calculate the average pixel value of the image stitching seam within each sub-region.

[0077] like Figure 5 As shown, after the fusion evaluation region is delineated, it is divided into multiple sub-regions along the first direction. Dividing into sub-regions is to achieve localized, refined analysis, breaking down a large fusion problem into a series of smaller, more manageable sub-problems. In some embodiments, the sub-regions include a first type of sub-region, a second type of sub-region, and a third type of sub-region. The sub-regions have different areas; for example, the first type of sub-region has the smallest area, and the second type of sub-region has the largest area.

[0078] The fusion assessment area is divided into multiple sub-areas along the first direction, including: dividing the fusion assessment area into a first main area, a second main area, and a third main area in sequence along the first direction; dividing the first main area into multiple first-type sub-areas along the second direction, with the second direction perpendicular to the first direction; dividing the second main area into multiple second-type sub-areas along the second direction; and dividing the third main area into multiple third-type sub-areas along the second direction.

[0079] like Figure 6 As shown, the first direction is the x-direction, and the second direction is the y-direction. The second direction defines the direction in which each main region is further divided into strips.

[0080] For example, on a preliminarily stitched panoramic X-ray image of the lungs, a rectangular fusion evaluation area is first determined at the center of the image. Then, along the vertical direction (first direction), this rectangle is cut from top to bottom into three strips of approximately equal height: the upper strip, the middle strip, and the lower strip, corresponding to the first, second, and third main regions. Next, along the horizontal direction (second direction), the upper strip is evenly cut into ten narrow vertical strips, which are the first type of sub-regions. Similarly, the middle strip is evenly cut into ten second type sub-regions, and the lower strip is evenly cut into ten third type sub-regions. Finally, the evaluation area is divided into three rows and ten columns, totaling thirty small rectangular grids, each grid being a sub-region to be analyzed.

[0081] S206: Based on the difference in the average pixel value between adjacent sub-regions, the pixel values ​​of the sub-regions are proportionally adjusted to obtain the adjusted sub-regions.

[0082] The visibility of the seam is directly related to the difference in pixel statistical characteristics between adjacent areas. By adjusting the pixel values ​​proportionally, this difference can be reduced, thereby achieving a smooth visual transition. The proportional adjustment is adaptive; areas with large differences require larger adjustments, while areas with small differences require smaller adjustments or no adjustments at all. After the adjustment is completed, a series of adjustment sub-regions are obtained, which have been optimized locally.

[0083] Specifically, before adjustment, in some embodiments, the mean value of each main region is calculated, and the pixel values ​​of the sub-regions are proportionally adjusted according to the difference in the mean pixel values ​​between adjacent sub-regions to obtain the adjusted sub-regions. This includes: calculating a first mean value, which is the mean value of pixels including the first seam within the first type of sub-region, where the first seam is the seam between the first type of sub-region and the second type of sub-region; calculating a second mean value and a third mean value, where the second mean value is the mean value of pixels including the first seam within the second type of sub-region, and the third mean value is the mean value of pixels including the second seam within the second type of sub-region, where the second seam is the seam between the second type of sub-region and the third type of sub-region; and calculating a fourth mean value, which is the mean value of pixels including the second seam within the third type of sub-region.

[0084] The first mean specifically refers to the average grayscale or color value of the pixels located within the first sub-region, immediately adjacent to the boundary line between it and the second sub-region. This boundary line is called the first seam. The first mean is calculated to quantify and evaluate the smoothness of the blending at a certain vertical local position of the transition boundary from the upper main region to the middle main region. The second, third, and fourth means are calculated similarly to the first mean.

[0085] Since the second region has a larger area and there are two seams between adjacent regions, two average values ​​are calculated for the second region.

[0086] When the absolute difference between the first mean and the second mean exceeds the first threshold, the ratio of the first mean to the second mean is used as the first adjustment coefficient; the first adjustment coefficient is used to perform a weighted calculation on the pixel values ​​of the first sub-region to obtain the adjusted first sub-region.

[0087] The first threshold is a preset numerical limit used to determine whether the difference between two means is significant. When the absolute difference exceeds the first threshold, it is considered that there is a visual discontinuity in the upper stitching seam of the local location that needs to be corrected. The setting of the first threshold depends on the data type of the image and the requirements for fusion quality.

[0088] The first and second mean values ​​represent the average pixel levels at the seam between the lower edge of the top strip and the upper edge of the middle strip in the same vertical column. If these two mean values ​​are very close, it indicates a natural transition at that local position, requiring no adjustment. If their absolute difference exceeds the first threshold, it indicates a significant jump in the pixel statistics on both sides of the seam within this narrow vertical strip range. This jump may result in a visible thin line or a sense of discontinuity. Therefore, a correction mechanism needs to be activated.

[0089] When the absolute difference exceeds the first threshold, the ratio of the first mean to the second mean is calculated and used as the first adjustment coefficient. The principle of calculating the ratio is based on a linear adjustment model. It is assumed that the discontinuity of the seam is mainly caused by a proportional difference in the overall brightness or color of the two sides. Then, multiplying the pixel values ​​of the upper strip (the first type of sub-region) by this proportional coefficient can theoretically make the statistical mean of the adjusted strip at the seam approach the mean of the middle strip. For example, if the first mean is larger than the second mean, the ratio is greater than one, multiplying by this coefficient will make the upper strip brighter overall, thus matching the brighter part of the middle strip at the boundary; if the ratio is less than one, the upper strip will be darker overall to match.

[0090] Next, a weighted calculation is performed on the pixel values ​​of the first sub-region that generated the first mean, using a first adjustment factor. The weighted calculation is a pixel-by-pixel multiplication operation; the original intensity value of each pixel within the first sub-region is multiplied by the first adjustment factor. This operation uniformly changes all pixels across the entire strip, resulting in the adjusted first sub-region. The reason for adjusting the entire sub-region, rather than just the boundary pixels, is to maintain color consistency within the sub-region and avoid generating new gradients within the strip. This adjustment is localized and targeted, applied only to sub-regions above specific vertical columns where significant differences were detected.

[0091] After evaluating and adjusting the upper boundary, the method proceeds to process the lower boundary. When the absolute difference between the third and fourth means exceeds the second threshold, the ratio of the fourth to the third means is used as the second adjustment coefficient. The second adjustment coefficient is then used to perform a weighted calculation on the pixel values ​​of the second sub-region to obtain the adjusted second sub-region.

[0092] The second threshold is similar to the first threshold, and will not be elaborated here. Furthermore, the logic for making this judgment is the same as that for the upper boundary, as described above.

[0093] For example, within the central evaluation area of ​​a generated panoramic landscape image, it has been divided into 3 rows and 10 columns of sub-regions. For the leftmost column, the first mean value at the bottom edge of the top row sub-region is calculated to be 150, and the second mean value at the top edge of the middle row sub-region is 100. The absolute difference between the two is 50, exceeding the first threshold set to 20. Therefore, a first adjustment factor of 1.5 is calculated. Then, the values ​​of all pixels in the top leftmost sub-region are multiplied by 1.5 to brighten it overall. For the same column, the third mean value at the bottom edge of the middle row sub-region is calculated to be 95, and the fourth mean value at the top edge of the bottom row sub-region is 120. The absolute difference is 25, exceeding the second threshold. Therefore, a second adjustment factor of 1.26 is calculated. Then, the values ​​of all pixels in the middle leftmost sub-region are multiplied by 1.26. Finally, the mean values ​​on both sides of the upper seam of this column tend to be consistent, the mean values ​​on both sides of the lower seam also tend to be consistent, while the pixels in the bottom sub-region remain unchanged.

[0094] S207: The adjusted sub-regions are stitched together along the second direction to obtain a fused image.

[0095] The adjusted sub-region is a sub-region that has undergone pixel value ratio adjustment. The pixel values ​​of the adjusted sub-region are weighted and modified according to the difference in the average pixel value of the seam between adjacent regions. The goal of the adjusted sub-region is to make its boundary transition smoother.

[0096] This stitching process reassembles all locally optimized sub-regions into a complete image. Stitching along the second direction ensures that the optimized local blocks can be combined in the correct spatial order. The resulting fused image maintains the integrity of the content as a whole, while eliminating stitching traces in local areas, achieving high-quality seamless fusion.

[0097] In some embodiments, adjusting sub-regions and stitching them together along a second direction to obtain a fused image includes: stitching the adjusted first type of sub-regions together along a first direction to obtain a first combined region; stitching the third type of sub-regions together along the first direction to obtain a second combined region; and stitching the first combined region, the second type of sub-regions, and the second combined region together along the second direction to obtain a fused image.

[0098] To reconstruct a complete and continuous top image region, these strips need to be reconnected in their original spatial order, spliced ​​along the first direction (usually the horizontal direction), which means merging these vertical strips from left to right so that the boundaries of adjacent strips fit closely together.

[0099] The first combined area is formed to reintegrate all the upper stripes, which have undergone local optimization, into a complete top region. The visual features of the first combined area are continuous in the horizontal direction, but there may be slight internal gradations in the vertical direction due to the different adjustment coefficients of different stripes. The second combined area is formed to integrate all the bottom stripes into a complete bottom region. The second combined area and the first combined area are located at the upper and lower ends of the fusion evaluation area in space.

[0100] Regardless of whether the pixel values ​​of the third type of sub-region have been adjusted, as the constituent units of the bottom region, they also need to be reintegrated into a complete bottom image block. These third type of sub-regions are stitched together in the horizontal direction to restore the full width of the bottom region and form the second combined region. This process ensures that the bottom region participates in the final synthesis as a whole.

[0101] Next, the first combined area, the entire undivided second main area, and the second combined area are stitched together along the second direction. Choosing to stitch these three large blocks along the second direction (usually the vertical direction) is to restore the complete structure of the image in the vertical direction.

[0102] The second principal region participates in the stitching as a whole because, in the differential adjustment step, the second type of sub-regions have been adjusted independently, but they collectively constitute the intermediate region, which is typically considered a continuous entity. The first combined region is placed on top, the original second principal region in the middle, and the second combined region at the bottom, stacked according to their spatial order in the original image. This vertical stitching operation aligns and connects the three large, internally optimized blocks at their vertical boundaries, forming a complete image that covers the original fusion evaluation region. This step ultimately generates a fused image that retains both the geometrically aligned and initially fused content within the evaluation region, while incorporating the effects of differential optimization based on local statistics.

[0103] This application integrates a high-performance TensorRT acceleration and optimization framework, achieving extreme performance through model optimization and transformation, and a dynamic inference architecture. It employs layer fusion and mixed-precision quantization techniques, achieving a 2-3x speedup inference at FP16 accuracy; it constructs a multi-CUDA streaming parallel architecture and a dynamic batch processing mechanism to achieve overlapping execution of data transmission and computation; it establishes an end-to-end inference pipeline for YOLOv11 and LightGlue models, optimizing resource utilization through context sharing and adaptive load balancing; and it integrates a real-time performance monitoring system, supporting deployment across platforms from Jetson embedded devices to data center GPUs. After optimization, the complete processing flow for 10 images can be completed within 5 seconds, representing an 8-12x speedup compared to traditional implementations, significantly improving processing efficiency while maintaining accuracy.

[0104] In some embodiments, the steps of acquiring the target image patch and performing matching on multi-scale feature points are performed by an inference engine that has been optimized by the model and accelerated by the computational graph.

[0105] The reason for using an inference engine to perform these steps is that object detection and feature extraction and matching are usually computationally intensive. They rely on complex deep learning models, and running these models directly on general computing frameworks may be inefficient and unable to meet the needs of real-time or high-throughput processing. Therefore, an inference engine is introduced to maximize the execution speed of these models on specific hardware through underlying optimization.

[0106] The "model-optimized" aspect of the inference engine involves analyzing and reconstructing the computation graph of an object detection model or feature extraction network after loading it. Optimizations may include merging multiple consecutive convolutional, batch normalization, and activation function layers into a single, more efficient computational kernel to reduce memory accesses and kernel startup overhead. Model optimization may also include precision quantization of model weights, such as converting FP32 floating-point numbers to FP16 or even INT8 integers, significantly improving computation speed and reducing memory usage with almost no loss of precision.

[0107] The computation graph acceleration of the inference engine focuses on leveraging the hardware's parallel computing capabilities. The engine compiles the optimized computation graph into a series of kernel functions highly optimized for GPUs or AI processors. It may employ dynamic batching techniques to process multiple input images simultaneously to improve hardware utilization. The engine may also utilize a multi-stream parallel architecture, allowing data transmission and model computation to overlap, reducing overall latency. Through these deep optimizations, the inference engine can execute the neural network forward propagation process in object detection and feature matching with far greater efficiency than standard frameworks, thereby significantly shortening the processing time of these two key steps.

[0108] For example, in a medical image processing system, when it is necessary to fuse ten consecutive OCT slices, the integrated TensorRT inference engine is invoked. This engine has been loaded and optimized with the YOLO object detection model and the SuperPoint feature extraction network. For the step of acquiring target image patches, the engine performs object detection inference on the ten input images in parallel, quickly outputting all target regions. For the feature matching step, the engine efficiently executes the forward propagation of the feature extraction network, quickly generating feature descriptors for all image patches. The computation time of the entire process is compressed to within a few seconds.

[0109] The method provided in this application adopts a fusion algorithm based on the idea of ​​differentiation. It does not require cropping or truncating the original image during the entire processing, and completely preserves all regions and information of the image. Compared with traditional methods, which suffer from edge information loss and effective area defects due to cropping, this application ensures that the final output image is completely consistent with the original input in terms of spatial range and pixel information.

[0110] By using differential difference analysis of local regions and an adaptive calibration mechanism, the algorithm achieves a natural transition between the target and the background. It can accurately identify and eliminate visual differences at the stitching boundary, avoiding artifacts, ghosting, and brightness discontinuities common in traditional fusion methods, and generating a complete image with a unified visual effect.

[0111] The differential fusion strategy operates at the pixel level, effectively preserving the subtle textures and edge features of the original image. Especially in applications such as medical imaging and remote sensing monitoring, where high fidelity of detail is required, this application ensures that all diagnostic details and analytical features are fully maintained, providing a reliable foundation for subsequent professional analysis.

[0112] Compared to the traditional multi-step process of "cropping and then stitching", the end-to-end differential fusion scheme of this application simplifies the processing chain, avoids the quality loss and time consumption caused by multiple resampling, and, combined with the TensorRT acceleration framework, achieves a significant improvement in processing efficiency while ensuring image quality.

[0113] The method is applicable to various image fusion scenarios, including but not limited to multi-exposure image synthesis, medical image overlay, remote sensing data fusion, and video frame enhancement. It can provide lossless and high-quality fusion effects in different fields and has important value for promotion and application.

[0114] like Figure 7 As shown, the training effect and performance of the object detection algorithm based on deep convolutional neural networks are illustrated. The horizontal axis (0-1000) represents the number of training iterations or epochs, indicating the progress of the model training process. The vertical axis (0-1.0) represents the scores of various evaluation metrics; higher values ​​indicate better model performance on that metric. Each curve shows the trend of the corresponding metric as training progresses.

[0115] Here, metrics / precision(B) represents precision, which is the percentage of real targets among all predicted targets. It measures the accuracy of the prediction results; high precision means fewer false positives (mistaking background for targets). Metrics / recall(B) represents recall, which is the percentage of real targets successfully detected by the model. It measures the coverage of detection. High recall means fewer missed targets. In this embodiment, we ensure that no key regions requiring fusion are omitted, thus ensuring information integrity.

[0116] Metrics / mAP50(B) represents the average precision, with an IoU threshold of 0.5. This is the core single comprehensive metric for object detection. It represents the average area under the precision-recall curve (AP), while mAP is the average of AP across multiple classes. The "50" here refers to an IoU threshold of 0.5, meaning a predicted bounding box is considered correct if its overlap with the ground truth bounding box exceeds 50%. In this embodiment, it directly measures the overall accuracy of object localization. This figure shows that the model has good comprehensive detection capabilities under the standard threshold.

[0117] Metrics / mAP50-95(B) represents the average accuracy, with IoU thresholds ranging from 0.5 to 0.95. This is a more stringent and comprehensive metric. It calculates the mAP at 10 different levels of stringency, with IoU thresholds from 0.5 to 0.95 (intervals of 0.05), and then takes the average. In this embodiment, it measures the robustness of the model under stringent localization requirements. A high metric indicates that the model's predicted bounding boxes not only find the target correctly but also have very precise locations. This is crucial for the "precise localization" and subsequent "seamless fusion" in this application, as deviations in the box position directly affect the stitching quality.

[0118] As shown in the figure, this application is highly efficient and reliable for both target detection and precise positioning.

[0119] like Figure 8As shown, in ophthalmic medical imaging, CNV is an abbreviation for choroidal neovascularization, an important pathological feature associated with diseases such as age-related macular degeneration, and a key detection target in OCT image analysis. Each sub-image represents an independent inference (prediction). After processing each OCT image, the model marks the lesion areas it identifies in the image with bounding boxes (though this may not be obvious in this thumbnail).

[0120] The oct value of 0.9 shown in the image represents the model's confidence level that the area within this bounding box contains a CNV lesion. For example, "0.9" means that the model has a 90% confidence level that a lesion exists here; "0.7" indicates that the confidence level is relatively low, but still exceeds the preset judgment threshold (e.g., 0.5).

[0121] As can be seen from the figure, the method provided in this application can detect a large number of different OCT slices (different patients, different locations), indicating that the model has good generalization ability and coverage, corresponding to high recall, ensuring that no lesions are missed, and supporting the goal of preserving complete information.

[0122] like Figure 9 As shown, a is the stitching effect without square correction, and b is the stitching effect with square correction. a produces obvious geometric deformation (image bending, unevenness) and detail misalignment (discontinuous layered structure), while b has a regular geometric shape (straight and square), a continuous and smooth structure, and accurate edge alignment. This shows that the square correction step effectively eliminates the geometric inconsistencies between the original images, laying a precise geometric foundation for achieving pixel-level seamless fusion.

[0123] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. An image fusion method based on the concept of differentiation, characterized in that, include: Obtain a target image patch, which is generated from the input image after expansion and cropping; Multi-scale feature points of the target image patch are matched to calculate geometric transformation parameters between the target image patches based on the successfully matched feature point pairs. The target image blocks are aligned according to the geometric transformation parameters to generate an initial stitched image; On the initial stitched image, a fusion evaluation area is defined with a set center. The fusion evaluation area is divided into multiple sub-regions along the first direction, and the average pixel value of the image stitching seam within the sub-regions is calculated. Based on the difference in the average pixel value between adjacent sub-regions, the pixel values ​​of the sub-regions are proportionally adjusted to obtain the adjusted sub-regions; The adjusted sub-regions are stitched together along the second direction to obtain a fused image.

2. The image fusion method based on the concept of differentiation according to claim 1, characterized in that, The acquisition of the target image patch includes: Acquire multiple input images to be fused; Perform target detection on the input image to obtain the target region; Based on the target area, expand outward by a preset margin ratio to form the expanded area; Based on the boundary of the expanded region, an image block is cropped from the input image to obtain the target image block.

3. The image fusion method based on the concept of differentiation according to claim 2, characterized in that, Performing target detection on the input image to obtain the target region includes: The input image is fed into a neural network model to output candidate detection boxes containing location and confidence scores. The neural network model has multi-scale feature extraction capabilities. Based on a predetermined confidence threshold, the candidate detection boxes are filtered, and a non-maximum suppression algorithm is applied to remove overlapping boxes to obtain the target region.

4. The image fusion method based on the concept of differentiation according to claim 1, characterized in that, The step of performing multi-scale feature point matching on the target image patch, and calculating geometric transformation parameters between the target image patches based on the successfully matched feature point pairs, includes: From each of the target image blocks, feature points and corresponding feature descriptors at multiple scales are extracted; each target image block includes at least a first image block and a second image block. Forward matching is performed between the feature points of the first image block and the feature points of the second image block to obtain the matching result; Perform reverse verification between the feature points of the second image block and the first image block to obtain the verification result; The feature point pairs that correspond to each other in the matching results and verification results are retained as the successfully matched feature point pairs; Based on the successfully matched feature point pairs, calculate the geometric transformation parameters between the target image blocks.

5. The image fusion method based on the concept of differentiation according to claim 1, characterized in that, The step of aligning the target image patch according to the geometric transformation parameters to generate an initial stitched image includes: Spatial transformation is performed on each of the target image blocks using the geometric transformation parameters to obtain aligned image blocks; The pixel intensity of the aligned image block is fused using a maximum value selection strategy, which selects the edge regions and high-texture regions in the aligned image block. And / or, For the edge regions and high-texture regions in the aligned image patch, a weighted averaging strategy is used to perform pixel fusion to generate the initial stitched image.

6. The image fusion method based on the concept of differentiation according to claim 1, characterized in that, The sub-regions include a first type of sub-region, a second type of sub-region, and a third type of sub-region; The step of dividing the fusion evaluation area into multiple sub-regions along the first direction includes: The fusion evaluation area is divided into a first main area, a second main area, and a third main area along the first direction; The first main region is evenly divided into multiple first-type sub-regions along the second direction, wherein the second direction is perpendicular to the first direction; The second main region is evenly divided into multiple second-type sub-regions along the second direction; The third main region is evenly divided into multiple third-type sub-regions along the second direction.

7. The image fusion method based on the concept of differentiation according to claim 6, characterized in that, The step of proportionally adjusting the pixel values ​​of the sub-regions based on the difference in average pixel values ​​between adjacent sub-regions, to obtain the adjusted sub-regions, includes: Calculate a first mean, which is the mean of pixels including the first seam within the first sub-region, where the first seam is the seam between the first sub-region and the second sub-region; Calculate a second mean and a third mean, where the second mean is the mean of pixels including the first seam within the second type of sub-region, and the third mean is the mean of pixels including the second seam within the second type of sub-region, where the second seam is the seam between the second type of sub-region and the third type of sub-region; Calculate the fourth mean, which is the mean of the pixels including the second stitching seam within the third sub-region.

8. The image fusion method based on the concept of differentiation according to claim 7, characterized in that, The step of proportionally adjusting the pixel values ​​of the sub-regions based on the difference in average pixel values ​​between adjacent sub-regions to obtain adjusted sub-regions includes: When the absolute difference between the first mean and the second mean exceeds the first threshold, the ratio of the first mean to the second mean is used as the first adjustment coefficient. The first adjustment coefficient is used to perform a weighted calculation on the pixel values ​​of the first type of sub-region to obtain the adjusted first type of sub-region; When the absolute difference between the third mean and the fourth mean exceeds the second threshold, the ratio of the fourth mean to the third mean is used as the second adjustment coefficient. The second adjustment coefficient is used to perform a weighted calculation on the pixel values ​​of the second type of sub-region to obtain the adjusted second type of sub-region.

9. The image fusion method based on the concept of differentiation according to claim 8, characterized in that, The step of stitching the adjusted sub-regions along the second direction to obtain a fused image includes: The adjusted first type of sub-regions are spliced ​​together along the first direction to obtain the first combined region; The third type of sub-regions are spliced ​​together along the first direction to obtain the second combined region; The first combined region, the second sub-region, and the second combined region are stitched together along the second direction to obtain a fused image.

10. The image fusion method based on the concept of differentiation according to claim 1, characterized in that, The steps of acquiring the target image patch and performing matching on multi-scale feature points are executed by an inference engine that has been optimized by the model and accelerated by the computational graph.

Images