A visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration

Abstract

The application discloses a visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration, and relates to the technical field of image processing and machine vision detection. The application adaptively adjusts the detection scale and fusion weight of each region according to local texture entropy, so that a high-texture-sensitive region can accurately capture millimeter-level tiny changes, and a low-texture flat region can be fully suppressed noise, thereby improving the detection precision and scene adaptability. In addition, the application extracts registration deviation and gray deviation to construct a system error feature vector, adaptively corrects the fusion feature map, and automatically estimates the confidence coefficient with the noise level, so that systematic interference is effectively eliminated, and the application can stably operate on different acquisition devices such as an intelligent bomb-hanging vehicle camera, an industrial camera and a mobile phone camera.

Description

Technical Field

[0001] This invention relates to the fields of image processing and machine vision detection technology, and in particular to a visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration. Background Technology

[0002] Image change detection technology automatically identifies changes in target location, structural morphology, and other information by comparing two images of the same area acquired at different times. It is a core technology in fields such as remote sensing monitoring and industrial visual inspection. In the operation scenario of intelligent bomb loading vehicles, the visual change detection system undertakes key tasks such as detecting surface damage on the mounting bracket, monitoring the relative displacement between the bomb and the mounting bracket, and verifying the positioning of the locking mechanism. Its detection accuracy is directly related to the safety and reliability of bomb loading operations.

[0003] Traditional image change detection methods mainly include pixel-level difference methods and fixed-scale pyramid analysis methods. Pixel-level difference methods construct a difference map by directly calculating the gray-level differences in the image and then segment the changed areas with a fixed threshold. They are extremely sensitive to non-realistic changes such as illumination changes, imaging angle shifts, and registration residuals. While fixed-scale pyramid analysis methods enhance detection robustness to some extent, their fixed scale settings cannot adaptively adjust to the complexity of local image textures. This results in subtle changes in textured areas (such as bolt holes and latch gaps) being easily smoothed out, while noise interference in flat textured areas is amplified. The inherent contradiction between detection sensitivity and noise resistance has never been effectively resolved.

[0004] To address the aforementioned issues, patent application CN110472634A discloses a change detection method based on a multi-scale deep feature difference fusion network. This method utilizes a Siamese convolutional neural network, performing internal encoding and cross-encoding on multi-scale deep features before difference fusion. However, this method employs a fixed-weight difference fusion approach, failing to adaptively adjust to the differences in representation between real changes and noise interference based on features at different scales. Furthermore, it relies on large-scale labeled data for training, resulting in a large number of model parameters and slow inference speed, making it difficult to meet the real-time requirements of field operations. Moreover, it lacks an error feedback and calibration mechanism for the imaging system, and therefore lacks the ability to actively detect and correct systemic interferences such as shooting angle shifts, equipment vibrations, and lighting fluctuations.

[0005] Furthermore, patent application CN120236177A discloses a remote sensing image change detection method and system based on a metric calibration adaptive SAM2 network. This method inserts an adapter module into a frozen SAM2 large model for efficient parameter fine-tuning, extracts multi-scale features from dual-temporal images through an encoder, and introduces a change perceptron to perform semantic measurement and change calibration on the decoded output features. However, this method still uses a deep learning model as the backbone network, requiring a large number of labeled samples for training, making it unsuitable for direct operation with only two images to be detected. Moreover, the metric calibration targets the semantic feature discrimination level, rather than correcting physical errors in the imaging system itself (such as registration bias and grayscale bias). Furthermore, this method is geared towards remote sensing image processing and is difficult to directly transfer to industrial field visual inspection scenarios. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract and title of the invention. Such simplifications or omissions shall not be used to limit the scope of the present invention.

[0007] In view of the aforementioned existing problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by this invention is that in the complex field environment of ammunition loading operations, traditional change detection methods cannot simultaneously take into account detection sensitivity and noise resistance, and have poor cross-equipment adaptability.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: a visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration, comprising: acquiring a reference image and a target image; reading pose sensing data and generating a region of interest; performing registration, noise reduction, and grayscale normalization on the two images within the region of interest to obtain a preprocessed image; dynamically generating an adaptive scale for each local block based on the local texture complexity of the reference image and the preprocessed image of the target image; extracting multi-scale features at each adaptive scale to obtain a multi-scale feature vector for each local block; and determining the effective scale of each multi-scale feature vector. The system generates dynamic fusion weights for each local block at each scale, and performs weighted fusion on each multi-scale feature vector based on the dynamic fusion weights to obtain global fusion feature maps of the reference image and the image to be inspected. It extracts the systematic error between the reference image and the image to be inspected, corrects the global fusion feature map based on the systematic error, and obtains a reference calibrated feature map and an image to be inspected calibrated feature map. Based on the calibrated feature maps and local texture complexity, it generates a differential threshold and determines the change region. Based on the change region, it generates a change mask map and outputs the location and degree information of each change region.

[0010] As a preferred embodiment of the present invention, the posture sensing data includes the joint angle of the intelligent bomb-loading vehicle's robotic arm, the displacement of the telescopic arm, and the horizontal tilt angle of the vehicle body.

[0011] In a preferred embodiment of the present invention, the region of interest (ROI) is generated as follows: based on the pose sensing data and combined with a pre-calibrated hand-eye mapping relationship, the expected pixel region of the pylon and projectile target in the current image is estimated in real time to generate an initial ROI; using the initial ROI as a search benchmark, lightweight edge feature matching is performed within the neighborhood of the search benchmark to verify whether the initial ROI contains the complete edge contour of the pylon and projectile; if the verification is successful, the initial ROI is used as the valid ROI of the current frame; if the verification fails, the position and size of the initial ROI are corrected based on the edge feature matching result to obtain the corrected ROI; when the pose sensing data changes, the estimation, verification, and correction process is repeated to ensure that the ROI is updated synchronously with the changes in the pose sensing data.

[0012] In a preferred embodiment of the present invention, the local texture complexity is calculated as follows: the preprocessed image of the reference image and the preprocessed image of the image to be inspected are divided into the same local block grid; for the same local block position, the local block texture entropy of the reference image and the local block texture entropy of the image to be inspected are calculated respectively, the texture entropy is used to characterize the local texture complexity of the corresponding local block, and the maximum value of the two is taken as the local texture complexity of the local block; the reference image and the image to be inspected use the same adaptive scale in the corresponding local blocks.

[0013] In a preferred embodiment of the present invention, the adaptive scale of each local block is dynamically generated as follows: the mean of the texture entropy of all local blocks is calculated as the mean of the global texture entropy; with a reference scale, for any local block, when the texture entropy of the local block is higher than the mean of the global texture entropy, the adaptive scale of the corresponding local block is smaller than the reference scale; when the texture entropy of the local block is lower than the mean of the global texture entropy, the adaptive scale of the corresponding local block is greater than the reference scale; when the texture entropy of the local block is equal to the mean of the global texture entropy, the adaptive scale of the corresponding local block is equal to the reference scale; the adaptive scale is determined according to an exponential mapping with the natural constant as the base: when the texture entropy is higher than the mean of the global texture entropy, it is scaled by a negative exponential; when the texture entropy is lower than the mean of the global texture entropy, it is scaled by a positive exponential; when the texture entropy is equal to the mean of the global texture entropy, the scaling factor is 1; the reference image and the image to be detected use the same adaptive scale for the corresponding local blocks.

[0014] In a preferred embodiment of the present invention, the reference scale is determined based on the ratio of the pixel density of the current acquisition device to the standard reference pixel density; the reference scale is used to unify the physical scale standards between different acquisition devices.

[0015] In a preferred embodiment of the present invention, the effectiveness of the multi-scale feature vector is measured by the ratio of the difference between the maximum and minimum values ​​of the feature vector at each scale to the mean of the feature vector; the larger the ratio, the higher the measure of effectiveness.

[0016] In a preferred embodiment of the present invention, the system error includes registration deviation and grayscale deviation between the reference image and the image to be inspected. An error feature vector is constructed based on the registration deviation and the grayscale deviation. The error feature vector is used to characterize the overall error level under the current acquisition environment. The error feature vector is formed by splicing the registration deviation component and grayscale deviation component of each local block in spatial order. Each local block corresponds to a two-dimensional error vector. The two-dimensional error vectors of all local blocks are flattened to form the error feature vector. The global fusion feature map includes the global fusion feature map of the reference image and the global fusion feature map of the image to be inspected. Based on the maximum value and mean value of the error feature vector, the global fusion feature map of the reference image and the global fusion feature map of the image to be inspected are corrected pixel by pixel in units of each local block to obtain the reference calibrated feature map and the image to be inspected calibrated feature map. For each pixel, the two-dimensional error vector of the local block to which the pixel belongs is extracted from the error feature vector. The magnitude of the two-dimensional error vector is used as the scalar correction factor of the corresponding local block. The same correction coefficient is used for each pixel in the same local block.

[0017] In a preferred embodiment of the present invention, the differential threshold is generated as follows: the mean of the texture entropy of all local blocks is calculated as the mean of the global texture entropy; a global reference threshold is used as a reference, which is determined based on the global differential statistical characteristics of the calibrated feature map. For any pixel location, when the texture entropy of the local block to which the pixel location belongs is higher than the mean of the global texture entropy, the differential threshold of the pixel location is lower than the global reference threshold; when the texture entropy of the local block to which the pixel location belongs is lower than the mean of the global texture entropy, the differential threshold of the pixel location is higher than the global reference threshold.

[0018] In a preferred embodiment of the present invention, the method for determining the change region is as follows: the difference degree of each pixel position is calculated based on the reference calibrated feature map and the feature map to be tested after calibration, and the difference degree of each pixel position is compared with the difference threshold of the corresponding pixel position. Pixel positions with a difference degree greater than the difference threshold are determined as change regions.

[0019] The beneficial effects of this invention are as follows: By constructing a closed-loop processing framework—texture-driven adaptive scale generation, feature validity-driven dynamic weight fusion, and system error feedback calibration—this invention effectively solves the core problems of traditional change detection methods, such as the difficulty in balancing detection sensitivity and noise resistance in complex scenarios of bomb loading operations, and poor cross-device adaptability. On one hand, this invention adaptively adjusts the detection scale and fusion weights of each region based on local texture entropy, ensuring that highly texture-sensitive areas maintain accurate capture of millimeter-level minute changes, while low-texture, flat areas achieve sufficient noise suppression, significantly improving detection accuracy and scene adaptability. On the other hand, this invention constructs a system error feature vector by extracting registration deviation and grayscale deviation, adaptively corrects the fusion feature map, and combines this with automatic estimation of the confidence coefficient with noise level, effectively eliminating systematic interference such as shooting angle shift, equipment vibration, and lighting differences. This ensures stable operation of this invention on different acquisition devices, such as intelligent bomb loading vehicle-mounted cameras, industrial cameras, and mobile phone cameras. These mechanisms collectively guarantee highly reliable detection in three scenarios: damage detection before bomb loading, displacement monitoring during bomb loading, and installation verification after bomb loading, providing strong visual inspection support for intelligent bomb loading operations. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a schematic diagram of image preprocessing as shown in the present invention.

[0021] Figure 2 This is a schematic diagram of dynamic feature fusion as shown in the present invention.

[0022] Figure 3 This is a schematic diagram of error feedback calibration and change determination as shown in the present invention.

[0023] Figure 4 This is a schematic diagram of adaptive multi-scale feature extraction as shown in the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0025] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the scope of protection of this invention.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0027] According to an embodiment of the present invention, a visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration includes: S1: Acquire a reference image and an image to be inspected, read pose sensing data and generate a region of interest (ROI). Within the ROI, register, denoise, and normalize the two images to obtain a preprocessed image. S2: Based on the local texture complexity of the preprocessed images of the reference and image to be inspected, dynamically generate adaptive scales for each local block. Extract multi-scale features at each adaptive scale to obtain multi-scale feature vectors for each local block. S3: Based on the effectiveness of each multi-scale feature vector, generate dynamic fusion weights for each local block at each scale. Perform weighted fusion on each multi-scale feature vector based on the dynamic fusion weights to obtain global fusion feature maps for the reference image and the image to be inspected. S4: Extract the system error between the reference and image to be inspected. Correct the global fusion feature map based on the system error to obtain a calibrated feature map for the reference and a calibrated feature map for the image to be inspected. Generate a differential threshold based on the calibrated feature map and local texture complexity to determine the change region. S5: Generate a change mask map based on the change region and output the location and degree information of each change region.

[0028] like Figure 1 The diagram shown illustrates the image preprocessing workflow provided in an embodiment of the present invention. In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention includes the following steps in step S1: Step S111: Determine the timing for acquiring the reference image and the image to be inspected based on the actual detection scenario of the ammunition loading operation.

[0029] The ammunition loading operation includes three scenarios: pre-loading inspection, in-loading monitoring, and post-loading verification. In the pre-loading inspection scenario, a standard image of the pylon in its unloaded state is collected as a reference image, and an image of the current surface state of the pylon is collected as the image to be inspected. This is used to detect whether there is damage, scratches, foreign objects, missing bolts, or installation misalignment on the pylon surface. In the in-loading monitoring scenario, an image of the ammunition in its initial contact state with the pylon is collected as a reference image, and a real-time image of the relative displacement process between the ammunition and the pylon is collected as the image to be inspected. This is used to detect the relative displacement, docking gap, and locking action status between the ammunition and the pylon in real time. In the post-loading verification scenario, a standard positioning image after ammunition loading is collected as a reference image, and an image of the current ammunition installation state is collected as the image to be inspected. This is used to determine whether the ammunition is installed in place, whether there is any slight misalignment, and whether the locking mechanism is fully closed.

[0030] It should be noted that the reference image is an image taken before the loading operation or under standard conditions, while the image to be inspected is an image taken under the current conditions. The two images are acquired synchronously through the vehicle-mounted vision system of the intelligent loading vehicle.

[0031] Step S112: Synchronously read the pose sensing data of the intelligent bomb-loading vehicle, including the joint angle of the intelligent bomb-loading vehicle's robotic arm, the displacement of the telescopic arm, and the horizontal tilt angle of the vehicle body.

[0032] The pose sensing data is used to dynamically generate the region of interest, ensuring that the detection area always follows the positional changes of the pylon and the projectile target.

[0033] Step S113: Based on the pose sensing data and combined with the pre-calibrated hand-eye mapping relationship, the expected pixel region of the hanger and projectile target in the current image is estimated in real time, generating an initial region of interest. The hand-eye mapping relationship represents the correspondence between pose sensing data and image pixel coordinates, and is obtained through pre-calibration. The calibration method can be the Tsai-Lenz two-step method or the Zhang Zhengyou hand-eye calibration method. During calibration, the pose sensing data of the robotic arm at multiple different joint angles and the pixel coordinates of a known marker point on the hanger in the image are recorded. The above multiple sets of correspondence relationships are used to establish a homogeneous transformation mapping matrix from the pose sensing data space to the image pixel space.

[0034] Specifically, when the joint angle of the robotic arm changes, the expected positions of the mounting bracket and projectile in the image shift accordingly; when the displacement of the telescopic arm changes, the scale of the expected target in the image scales accordingly; when the horizontal tilt angle of the vehicle changes, the position of the expected target in the image translates accordingly. It should be noted that the initial region of interest is a rectangular or arbitrary polygonal region containing the expected target, used to confine all subsequent image processing operations to this region.

[0035] It should be noted that the process of generating the initial region of interest based on pose sensing data constitutes the first step of the dynamic ROI update mechanism of this invention, namely the pose feedforward prediction step. By utilizing the inherent sensing signals (joint angles, extension displacement, vehicle tilt angle) of the intelligent ammunition loading vehicle's robotic arm to directly drive the prediction of the target region in the image, all subsequent image processing operations can be confined from the outset to a local area containing the pylon and the projectile target. This eliminates interference from a large background area at the source, significantly reducing the computational load required for subsequent registration, feature extraction, and change determination. Simultaneously, this feedforward prediction does not rely on the detection results of the previous frame image. Therefore, when the robotic arm undergoes rapid movement or the pose sensing data changes abruptly, the ROI can instantly follow the target position change, avoiding tracking loss due to large target displacement.

[0036] Step S114: Using the initial region of interest (ROI) as the search benchmark, perform lightweight edge feature matching (using the Canny operator to extract edges and using the Bach distance of the edge direction histogram as the similarity metric) within the neighborhood of the search benchmark (a circular area centered on the initial ROI with a radius of 10% of the current ROI's short side length, with a minimum of 10 pixels and a maximum of 50 pixels) to verify whether the initial ROI contains the complete edge contours of the hanger and the projectile. The judgment criterion is whether the pixel percentage of the closed edges in this area exceeds a preset threshold, such as 70%.

[0037] If the verification passes, the initial region of interest (ROI) is used as the valid ROI for the current frame. If the verification fails, the edge contours of the pylon and projectile detected during edge feature matching are extracted, and the minimum bounding rectangle of the edge contours is calculated. The minimum bounding rectangle is used as the corrected ROI. If edge feature matching fails to detect the complete edge contours of the pylon and projectile, the search range is expanded outward step by step with the initial ROI as the center, according to a preset expansion step size. For example, 10% of the current ROI's short side length is used as the single expansion step size, with a minimum of 10 pixels and a maximum of 50 pixels, until the complete edge contours of the pylon and projectile are detected within the expanded search range. The minimum bounding rectangle of the complete edge contour is used as the corrected ROI.

[0038] It should be noted that lightweight edge feature matching only extracts and compares the edge information of the pylon and the projectile body, without involving full-image feature extraction. Its computational cost is far lower than conventional feature matching, and it does not affect the system's real-time performance. Through the above verification and correction mechanisms, it is ensured that the region of interest accurately covers the target area under different operational states and environments.

[0039] When the pose sensing data changes, repeat the above estimation, verification and correction process to make the region of interest update synchronously with the change of pose sensing data; after each update, re-execute the local block division and all subsequent processing in step S2 within the new region of interest, and the local block division mesh of the previous frame is not inherited across frames.

[0040] The aforementioned verification and correction processes constitute the second and third stages of the dynamic ROI update mechanism of this invention: the edge verification stage and the deviation correction stage. These three stages together form a closed-loop ROI update strategy driven by pose sensing and verified by visual feedback. In this strategy, pose sensing data provides a coarse prediction of the target position, while edge feature matching provides precise verification of the target contour. The fusion of these two aspects ensures the accuracy and robustness of the ROI under dynamic operating conditions, and requires no manual intervention throughout the process. Through this closed-loop strategy, the ROI can be updated synchronously with changes in pose sensing data. After each update, local block partitioning and subsequent processing are re-executed within the new ROI range. The partitioned mesh of the previous frame is not inherited across frames, thus avoiding the impact of accumulated positioning errors on detection accuracy.

[0041] In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention further includes the following steps in step S1: Step S115: Within the effective region of interest, image registration is performed on the acquired reference image and the image to be inspected. Image registration eliminates displacement, rotation, and perspective deviations caused during the shooting process by matching feature points and geometrically aligning the two images, ensuring that the same target in the two images is located at the same pixel position, thus guaranteeing spatial consistency for subsequent change detection. It should be noted that feature point matching can employ any of the following: scale-invariant feature transformation, accelerated robust feature transformation, or directional rapid rotation simplified feature transformation; this invention is not limited in this regard. After registration, two spatially aligned images are output.

[0042] Step S116: Noise suppression is performed on the two registered images. The enhanced Lee filtering algorithm is used to reduce noise in the registered images. While preserving image edges, textures, and details, it suppresses environmental noise, imaging noise, and jitter interference, improving image clarity and feature recognition. It should be noted that the enhanced Lee filtering algorithm adaptively adjusts the filtering strength based on local statistical characteristics. Stronger smoothing is applied to flat areas to suppress noise, while weaker smoothing is applied to edge and textured areas to preserve details. Compared to traditional mean filtering or Gaussian filtering, it achieves a better balance between noise reduction and edge preservation.

[0043] Step S117: Perform grayscale normalization on the two denoised images to eliminate inconsistencies in grayscale distribution caused by differences in illumination intensity, exposure, and device imaging deviations, ensuring that the two images have a unified grayscale reference. The formula for grayscale normalization is as follows:

[0044] in, These are the pixel values ​​of the image after noise suppression. This represents the global grayscale mean of the image after noise reduction. This represents the global grayscale standard deviation of the denoised image.

[0045] Step S118: After the above registration, noise reduction, and grayscale normalization processes, the two images maintain consistency in spatial location, grayscale distribution, and noise level. The preprocessed image is then output for subsequent adaptive multi-scale feature extraction. It should be noted that background pixels outside the effective region of interest are not involved in registration, noise reduction, grayscale normalization, or any subsequent calculations, thus eliminating background interference at the source and significantly reducing the system's computational load. Furthermore, the preprocessing workflow of this invention is uniformly applicable to different acquisition devices such as intelligent armored vehicle cameras, industrial cameras, and mobile phone cameras. Grayscale normalization eliminates the main grayscale distribution differences between devices, providing a data foundation for subsequent cross-hardware universal detection. The noise and residual dynamic range differences between different devices are compensated for by the adaptive estimation mechanism of the confidence coefficient t in subsequent step S414.

[0046] like Figure 4 The diagram illustrates the adaptive multi-scale feature extraction process provided in an embodiment of the present invention. In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention includes the following steps in step S2: Step S211: Divide the preprocessed image of the reference image and the preprocessed image of the image to be inspected into multiple local blocks using the same local block partitioning grid. For the same local block location, calculate the local block texture entropy of the reference image and the local block texture entropy of the image to be inspected, and take the maximum value of the two as the local texture complexity of the local block.

[0047] Each local block has a fixed pixel width, either 16 or 32 pixels; for areas at the image boundary that are less than a full block size, the actual boundary size is taken as the effective block area.

[0048] The image range covered by each local block is defined as a local block region, that is, each region and each local block are synonymous expressions under the same division granularity.

[0049] Furthermore, the probability of occurrence of each gray level within each local block is statistically analyzed, and the texture entropy of each local block is calculated based on the probability of occurrence of each gray level. The texture entropy is used to characterize the local texture complexity of the corresponding local block.

[0050] Specifically, the formula for calculating texture entropy is:

[0051] in, The center pixel coordinates of the local block. For the first The probability of gray levels appearing. To determine the grayscale levels, for a normalized floating-point image, its value range is first quantized into 256 grayscale levels at equal intervals, and then the probability of each grayscale level is calculated. This is the minimum correction coefficient, an empirical constant, much smaller than the minimum non-zero value of the grayscale probability. It does not affect the relative magnitude of entropy values ​​and is used to avoid zero values ​​in logarithmic operations.

[0052] It should be noted that texture entropy A larger value indicates a more complex texture and richer details in the local block; texture entropy. The smaller the value, the flatter the texture and the less detail in the local block. Through the above texture entropy calculation, it is possible to accurately distinguish between areas with rich details such as bolt holes, latch gaps, and chamfered edges in the hanging area and flat background areas.

[0053] Step S212: Calculate the mean of all local block texture entropies as the mean of global texture entropy. and based on the global texture entropy average Based on the benchmark, combined with the benchmark scale Dynamically generate adaptive scale for each local block .

[0054] The formula for generating adaptive scaling is:

[0055] in, The texture entropy of the current local block. The maximum value of the entropy of all local block textures. This is a minimal correction term to prevent numerical anomalies when the denominator approaches zero.

[0056] This exponential mapping non-linearly maps texture complexity from the range of [minimum, maximum] to a scale scaling factor. The ratio of the maximum to the minimum scale of the interval is approximately This span is set based on experience to ensure sufficient scale differentiation between flat and textured areas.

[0057] in, This represents the maximum value of the entropy of all local block textures. (Base scale) The ratio of the pixel density of the current acquisition device to the standard reference pixel density is used to determine the value; the calculation formula is as follows:

[0058] in, This represents the actual pixel density of the current device's image sensor. The standard reference pixel density constant is pre-stored in the system. For example, the calibration value of the reference camera used for the first time in the current operation scenario is taken and kept fixed during subsequent operation of the same device. This is used to normalize the imaging scale deviation of different hardware. The scaling factor is fixed at 1.0 and serves as the baseline scaling factor for device-independent normalization, without introducing additional scaling bias. It's important to note that the core function of the baseline scale is to establish a device-independent normalized scale coordinate system, ensuring consistent physical accuracy standards for change detection regardless of whether it's running on a smart armored vehicle's onboard camera, an industrial camera, or a mobile phone camera. Using the adaptive scaling formula described above, for any local block, when the texture entropy of that local block is higher than the global average texture entropy, the adaptive scale of that local block... The detection scale is smaller than the baseline scale, meaning that the more complex the texture, the smaller the detection scale is used to capture minute details; when the texture entropy of the local block is lower than the global average texture entropy, the adaptive scale of the local block is greater than the baseline scale, meaning that the flatter the texture, the larger the detection scale is used to suppress noise interference; when the texture entropy of the local block is equal to the global average texture entropy, the adaptive scale of the local block is equal to the baseline scale.

[0059] The aforementioned adaptive scale generation mechanism enables adaptive adjustment to smaller scales as textures become more complex, allowing feature extraction to be performed on regions with varying texture complexities at the optimal scale.

[0060] In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention further includes the following steps in step S2: Step S213: For each local block, at the corresponding adaptive scale... The multi-scale features of the local block are extracted, including edge features, corner features, and texture features, and a local feature vector is constructed. The multi-scale features include edge features (extracted using the Canny operator), corner features (extracted using the Shi-Tomasi operator), and texture features (extracted using Local Binary Pattern (LBP)). Edge features are used to characterize the contours and boundary information of the mounting bracket and projectile; corner features are used to characterize the positional information of key structures such as bolt holes and latches; and texture features are used to characterize scratches, textures, and subtle deformations on the mounting bracket surface. It should be noted that features extracted at different scales have different representational capabilities: small-scale features are sensitive to changes in detail and can capture millimeter-level displacements and deformations; large-scale features are sensitive to changes in the overall structure and can capture global offsets and attitude changes. By simultaneously extracting the above three types of features at the optimal adaptive scale of each local block, it is ensured that subtle features in regions of different complexities can be effectively extracted, providing rich and accurate feature data for subsequent dynamic feature fusion.

[0061] Step S214: Summarize the multi-scale feature vectors of all local blocks to obtain the multi-scale feature vector set of each local block. The features extracted from each local block at its adaptive scale together constitute the multi-scale feature set. The multi-scale feature vector set is used by the subsequent dynamic feature fusion module for effectiveness evaluation and adaptive weighted fusion.

[0062] It should be noted that the extraction window size and detail richness of the feature vectors of each local block are determined by the adaptive scale of that local block: the smaller the adaptive scale, the smaller the extraction window and the richer the detail information; the larger the adaptive scale, the larger the extraction window and the more prominent the overall information. For subsequent weighted fusion, the feature vectors extracted at different scales are all normalized and resampled to a preset unified dimension. .

[0063] like Figure 2 The diagram shown illustrates the dynamic feature fusion process provided in an embodiment of the present invention. In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention includes the following steps in step S3: Step S311: Evaluate the effectiveness of the multi-scale feature vectors of each local block obtained in Step S2 at each adaptive scale, and calculate the feature discriminability of the feature vectors at each scale. Feature discriminability quantifies the feature's ability to represent minute changes, structural shifts, and subtle differences. Higher feature discriminability indicates greater sensitivity to changes in the armament status and warrants a higher fusion weight; lower feature discriminability indicates less effective information and warrants a lower fusion weight. Feature Discriminability The calculation formula is:

[0064] in, This represents the maximum value of the current feature vector. This represents the minimum value of the current feature vector. This is the global mean of the current feature vector.

[0065] It should be noted that this formula reflects the sensitivity to small changes by the dispersion of the amplitude within the feature vector: when the differences between the elements in the feature vector are large, it indicates that the feature has a strong response to local changes in the image and is highly effective; when the elements in the feature vector are relatively close, it indicates that the feature is not sensitive to changes in the image and is less effective.

[0066] The larger the ratio of the difference between the maximum and minimum values ​​to the mean, the higher the measure of effectiveness.

[0067] It should be noted that the feature vector F specifically refers to the edge, corner, and texture feature components extracted in step S213 and retained after enhanced Lee filtering in step S116, i.e., the effective feature components; when the mean of the feature vector is... When the value is 0 or less than a preset minimum threshold, the validity of the corresponding feature vector will be determined. Set to 0 to exclude it from subsequent dynamic weight calculations to avoid numerical anomalies. The preset minimum threshold is 0.001 or the minimum non-zero grayscale probability value.

[0068] Step S312: Based on the feature recognition result, adaptive dynamic weight allocation is performed on the features of different local blocks, and the allocated weights are normalized to ensure that the sum of the feature weights at each scale at the same location is 1, thus achieving the stability and consistency of feature fusion. Dynamic fusion weights The generation is as follows:

[0069] in, Current location Current scale Adaptive fusion weights of features The feature discrimination of the current location and current scale features. For the adaptive scale set at this location, in single-scale mode It degenerates into a single-element set containing only the adaptive scale of that location.

[0070] It should be noted that the higher the feature discrimination, the greater the weight assigned; the lower the feature discrimination, the smaller the weight assigned. Through the above dynamic weight allocation, features with different effectiveness at different scales are assigned different fusion weights, effective features are strengthened, and ineffective and noisy features are suppressed. Compared with the traditional fixed weight (such as 0.5, 0.3, 0.2) fusion method, it can adaptively adjust according to the image content and has stronger scene adaptability.

[0071] In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention further includes the following steps in step S3: Step S313: The multi-scale feature vectors extracted at each adaptive scale are weighted and fused according to the dynamic fusion weights generated in step S312. During the weighted fusion process, a feature dispersion feedback term is introduced simultaneously to enhance the gain of the real change region and suppress the smooth and noisy regions.

[0072] Before weighted fusion, for each scale The Canny edge features, Shi-Tomasi corner features, and LBP texture features were all L2 normalized and resampled to a uniform dimension. A unified-dimensional feature vector is obtained by concatenating [edge features, corner features, and texture features] in that order. :

[0073] in, For global fused feature maps at pixel locations The value, This represents the standard deviation of the joint feature vector obtained by concatenating all scale features at the current location. Let be the mean of the joint eigenvector; when When the value is 0 or less than the preset minimum threshold, Set to 0.

[0074] In the above formula, the feature vectors at each scale are first weighted and summed to obtain a joint feature vector. Then, the L2 norm is taken to compress the vector result into a scalar, which is used as the final fused feature value for that pixel location. For textureless regions, the L2 norm of the joint feature vector approaches 0, and the corresponding fused feature value also approaches 0.

[0075] It should be noted that, Used to reflect the degree of dispersion of the feature vector: The term is used as a non-linear adjustment factor to correct the fusion weights.

[0076] When the dispersion is high, the value of this exponent is small, and its contribution to fusion is moderately reduced, avoiding excessive amplification of the noise response by a single dispersion index; when the dispersion is low, the value of this exponent is large, preserving the basic characteristic response of the flat region, thereby achieving a balance between the real changing signal and the flat background and realizing adaptive robust fusion.

[0077] The design of the aforementioned exponential term allows real, minute changes to be further amplified during the fusion process, while noise interference is further suppressed, without the need to manually set any gain coefficient.

[0078] Step S314: Perform the weighted fusion and discrete feedback operations on all pixel locations to obtain the global fusion feature maps of the reference image. Global fusion feature map of the image to be inspected The result is output to the subsequent error feedback calibration and change determination module.

[0079] Global fusion features of reference image Global fusion feature map of the image to be inspected This method fully preserves minute change information in each region of the pylon and projectile, exhibiting a higher signal-to-noise ratio and greater sensitivity to change compared to traditional fixed-weight fusion methods. It should be noted that regions with larger eigenvalues ​​in the global fusion feature map indicate higher significance of change and thus higher confidence in subsequent change determinations; conversely, regions with smaller eigenvalues ​​indicate lower significance of change and are more likely to be classified as unchanged regions in subsequent change determinations. Furthermore, when this invention is run on different acquisition devices (intelligent projectile vehicle-mounted camera, industrial camera, mobile phone camera), since the baseline scale in step S2 has already normalized the hardware pixel density differences, the dynamic fusion weights in S3 can be adaptively generated on different devices. Except for the preset determination thresholds based on different models in step S515, no additional adjustment of algorithm parameters is needed for imaging noise and dynamic range differences between different devices; residual imaging differences introduced by different devices are uniformly compensated through the adaptive estimation of the confidence coefficient t in step S414.

[0080] like Figure 3 The diagram shown illustrates the error feedback calibration and change determination process provided in an embodiment of the present invention. In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention includes the following steps in step S4: Step S411: After obtaining the preprocessed image in step S1, the systematic error between the reference image and the image to be inspected is extracted immediately.

[0081] The systematic error includes registration deviation and grayscale deviation between the reference image and the image to be inspected. The registration deviation is obtained by statistically analyzing the difference in feature point positions between the two images after registration in step S1. This registration deviation refers to the residual sub-pixel level alignment error after registration, which is different from the gross deviation that has been eliminated before registration. It is used as a systematic error component for subsequent correction and is used to quantify the spatial errors introduced by shooting angle offset, equipment vibration, and mechanical displacement.

[0082] Gray-scale deviation is obtained by statistically analyzing the difference in gray-scale values ​​in the overlapping area after registration of two images. This gray-scale deviation refers to the gray-scale difference that remains between the two images after gray-scale normalization, and is used to compensate for the nonlinear illumination residuals that normalization could not completely eliminate.

[0083] Furthermore, an error feature vector is constructed based on the registration deviation and grayscale deviation. The error feature vector is used to characterize the overall error level under the current acquisition environment.

[0084] Specifically, error eigenvectors The construction method is as follows: the current frame is divided into local block grids identical to those in step S211. For each local block, the magnitude of the coordinate residual vector of corresponding pixels in the two registered images is taken as the registration deviation of that pixel, and the absolute value of the gray level difference between corresponding pixels in the two images after gray level normalization is taken as the gray level deviation of that pixel. The mean of the registration deviations of all pixels in the block is calculated as the registration deviation component of that block, and the mean of the gray level deviations of all pixels in the block is calculated as the gray level deviation component of that block. The two are concatenated in the order of [registration deviation component, gray level deviation component] to form the two-dimensional error vector corresponding to that local block. The two-dimensional error vectors of all local blocks are flattened in spatial order to form the global error feature vector of the current frame. Dimensions are 2× ,in This represents the total number of local blocks. Each component is a dimensionless normalized deviation value, which has been unified to the same dimension in the grayscale normalization process in step S117 and the registration process in step S115.

[0085] After each ROI update, the local block division in step S211 is regenerated with the boundary of the current valid ROI as the reference to ensure that the local block coordinates are always aligned with the ROI pixel coordinates.

[0086] Additionally, for pixels Determine the local block B. ,from Extract the two-dimensional error vector corresponding to the local block:

[0087] calculate:

[0088] As a scalar error correction factor for this local block; each pixel within the same local block uses the same... This allows for the extraction of global error feature vectors. Mapping to pixel-level correction.

[0089] Step S412: Correct the global fused feature map for errors based on system errors. Specifically, calculate the error feature vector. maximum value and mean , Each component is normalized by dividing by the global maximum error, so that the error correction magnitude at each position is scaled uniformly relative to the overall error level, avoiding overcorrection caused by local error extremes.

[0090] Error calibration is performed on the global fusion feature maps of the reference image and the image to be inspected, respectively, using the following formula:

[0091]

[0092] in, For the calibrated feature map at the pixel location The value, The feature map after calibration is located at the pixel position. The value of .

[0093] It should be noted that the above error calibration formula achieves dynamic calibration through the linkage of error feedback and error dispersion: when the system error Relative to the maximum error When the error is large, the correction term in the latter half of the formula is larger, and the correction magnitude of the global fusion feature map increases accordingly; when the system error is small, the correction magnitude decreases accordingly.

[0094] at the same time, The correction term automatically adjusts its strength based on the ratio of the mean error to the maximum error: when the error distribution is concentrated, the term is larger, resulting in stronger correction; when the error distribution is dispersed, the term is smaller, resulting in weaker correction. The entire correction process requires no manual setting of any calibration coefficients; it is entirely completed automatically by the system based on the current error state.

[0095] For example, when the mean of the system error is much smaller than the maximum error, the correction factor tends to be... It can achieve near 100% error correction; when the error distribution is extremely concentrated, the lower limit of the correction factor is approximately... This ensures that under no circumstances will there be over-correction leading to feature map distortion.

[0096] After the above error feedback calibration, a reference calibrated feature map is output. and the feature map after calibration. The reference calibration feature map and the calibration feature map to be tested have effectively eliminated the detection deviation caused by systematic errors such as shooting angle offset, equipment vibration, illumination change, and registration error.

[0097] In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention further includes the following steps in step S4: Step S413, based on reference and The difference between each pixel location is calculated using the following formula:

[0098] The difference is used to reflect the degree of change between the reference image and the image under test at a certain pixel position. The greater the difference, the more significant the change at that position before and after loading the ammunition.

[0099] Step S414: Determine the global baseline threshold based on the global statistical characteristics of the difference calculated from the reference calibrated feature map and the feature map to be tested after calibration, and automatically estimate the confidence coefficient based on the noise level of the calibrated feature map. Specifically, the mean of the global dissimilarity is calculated. and standard deviation The expression for the global baseline threshold is: .

[0100] Among them, the confidence coefficient The discrimination unit automatically estimates the noise level based on the calibrated feature map. Specifically, the ratio of the standard deviation of the global difference map to the median standard deviation of the local block difference can be used as a measure of the noise level, according to the linear mapping formula. ,in and These are the preset minimum and maximum noise measurement reference values, exemplified by 0.5 and 2.0 respectively. ,in, The standard deviation of global variability. The standard deviation of the difference values ​​of all pixels within each local block. For each of the local blocks The global median of the set of values; when Time to take ,when Time to take .

[0101] When the image noise level is high Automatically increases the threshold to suppress false detections caused by noise; when the image noise level is low... The threshold is automatically reduced to fully capture minute changes. It should be noted that the above... The estimation is entirely automated by the system based on the statistical characteristics of the current image, requiring no manual parameter setting. Through this mechanism, the confidence coefficient... The invention adapts to the noise levels of different devices and environments, enabling it to automatically adjust to the optimal judgment sensitivity when running on different acquisition devices (intelligent bomb-loading vehicle-mounted camera, industrial camera, mobile phone camera).

[0102] Step S415: Generate a differentiation threshold based on the difference degree and local texture complexity. The differentiation threshold is generated by using the global baseline threshold determined in step S414. For reference, and based on the texture entropy of each local block calculated in step S2, a differential threshold is independently generated for each pixel location. Specifically, the texture entropy value of all pixels within a local block is taken as the calculated texture entropy value of that local block; that is, for any pixel location (x, y) belonging to the same local block B, the following applies: This is the calculated texture entropy value; the texture entropy values ​​between different local blocks jump at the block boundaries without interpolation smoothing to ensure the independent adaptive characteristics of each local block. The specific formula is as follows:

[0103] in, The mean of global variability. The standard deviation of global variability. The mean of global texture entropy. The maximum value of the entropy of all local block textures. This is the texture sensitivity coefficient, ranging from 0.2 to 0.5, dynamically selected by the system based on the dispersion of the texture entropy of all local blocks in the current image. Calculate the ratio of the standard deviation of all local block texture entropy to the mean of the global texture entropy, and use the ratio as the basis for calculation. As a measure of the dispersion of texture distribution, when When > 0.6, Take 0.5; when When < 0.2, Take 0.2; when 0.2 ≤ When ≤ 0.6, follow the linear proportion formula Values.

[0104] It should be noted that, In response to global noise levels, The response to the differences in texture distribution applies to the global baseline and the local offset term respectively, and they are not coupled.

[0105] The calculations above show that for any pixel location, when the texture entropy of the local block to which the pixel belongs is higher than the global average texture entropy, the differentiation threshold for that pixel location is lower than the global baseline threshold. This means that regions with more complex textures and richer details (such as bolt holes, latch gaps, and hanger edges) use a lower threshold, thus fully releasing detection sensitivity and avoiding the loss of millimeter-level minute changes in critical areas. Conversely, when the texture entropy of the local block to which the pixel belongs is lower than the global average texture entropy, the differentiation threshold for that pixel location is higher than the global baseline threshold. This means that regions with flatter textures and less feature information (such as the hanger background area) use a higher threshold, effectively suppressing false detections caused by random noise and environmental fluctuations. Compared to the traditional single global threshold scheme, this bidirectional differentiation adjustment strategy simultaneously solves the inherent problems of missed detections in highly texture-sensitive areas and false detections in low-texture, flat areas.

[0106] Step S416: Determine the change region based on the difference threshold. Specifically, the calculated difference degree at each pixel location... Differential threshold with corresponding pixel position Each pixel is compared individually. If the difference at a certain pixel location exceeds the difference threshold for that location, then that pixel location is identified as a region of change and marked as such. Otherwise, it is determined to be a region with no change and marked as 0.

[0107] By using the aforementioned local adaptive threshold determination, it is ensured that minute changes in high-interest areas such as bracket bolt holes, latch gaps, and projectile docking surfaces can be accurately detected, while background noise is effectively suppressed, achieving high-precision and low-false-detection determination of change areas.

[0108] Optionally, after initially determining pixel-level change regions based on the differential threshold, this embodiment further filters and optimizes the initial determination results according to the change confidence level hierarchical determination mechanism and the region growth contour refinement strategy, specifically including: Perform connected component analysis on all pixels initially identified as changing regions in step S416, and divide adjacent changing pixels into independent connected components, denoted as... ,in, (Total number of connected components). For each connected component Calculate the neighborhood consistency metric separately. and edge matching degree metric .

[0109] Specifically, neighborhood consistency measurement The calculation method is as follows: extract the set of boundary pixels of the connected component. That is, a pixel within a connected component that has at least one 8-neighbor pixel that does not belong to that connected component. The proportion of boundary pixels belonging to the initially determined change region is calculated using the following formula:

[0110] in, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. This represents the total number of boundary pixels. The value ranges from 0 to 1. A higher value indicates that the boundary of the connected component is more consistent with the surrounding trend of change, and the greater the possibility that it belongs to a real change.

[0111] Edge matching degree metric The calculation method is as follows: the extracted edge feature map is compared with the current connected component. Spatial overlap comparison is performed, and the ratio of the number of overlapping pixels to the total number of pixels in the connected component is calculated using the following formula:

[0112] in, For the extracted edge feature map at pixels The binarization value at a given point is 1, which indicates that the pixel is an edge point, and 0 indicates that it is not an edge point. This represents the total number of pixels in the connected components. The value ranges from 0 to 1. A higher value indicates a higher degree of agreement between the variation area and the actual structural edge of the pylon and the missile body.

[0113] Based on the combined neighborhood consistency metric and edge matching metric, the final change confidence score for each connected component is calculated. Since neighborhood consistency reflects the spatial continuity quality of the changed region, and edge matching reflects the degree of agreement between the changed region and the edges of the image structure, both contribute equally to the judgment of true change. Therefore, an equally weighted summation method is adopted, and the calculation formula is as follows:

[0114] The value range is from 0 to 1. Based on... Values ​​are used for hierarchical determination: When the value is greater than or equal to 0.75, the connected component is marked as a reliable change region and is directly retained. When the value is less than or equal to 0.25, the connected component is determined to be an isolated pseudo-change region introduced by illumination flicker, minor jitter, or registration residual, and is therefore removed. In other cases, a secondary review process is initiated: For all pixels within the annular neighborhood formed by extending d pixels outward from the boundary of the connected component (for example, d=3), the differentiation threshold is temporarily reduced by 10%, i.e., the temporary threshold is 90% of the original differentiation threshold. For newly identified changed pixels after the threshold reduction, if the number of newly added pixels is greater than 20% of the number of pixels in the connected component itself, the newly added pixels are merged into the original connected component and the whole is identified as a changed region and retained; otherwise, the original connected component and the newly added pixels in its neighborhood are identified as pseudo-changed regions and removed.

[0115] Through the above-mentioned stratified judgment mechanism, high-confidence areas are directly retained, low-confidence areas are directly eliminated, and medium-confidence areas are given the opportunity for correction through secondary review, thus preserving the real change areas to the maximum extent while ensuring the reliability of the detection.

[0116] Furthermore, the reliable change regions retained after confidence level stratification may exhibit jagged or discontinuous boundaries due to threshold hard segmentation, and the transition zone pixels at the edges of the true change regions are often not included because their difference does not exceed the original threshold. This embodiment further employs an adaptive region growing strategy based on seed points to refine the contours of the retained change regions.

[0117] Specifically, the boundary pixels of each reliable change region are used as the seed point set, and the set is expanded outward layer by layer in the 8-neighborhood direction of the seed points. Let the coordinates of the current neighboring pixel to be determined be... The degree of difference is The differential threshold corresponding to this position is If the following growth criteria are met:

[0118] That is, the difference is lower than or equal to the current threshold, but the difference is within the tolerance range. = within 0.05 ( If the tolerance coefficient is 0.05 (for example), and the neighboring pixel has not yet been marked as any change region, then the neighboring pixel is included in the current change region, and the boundary of the region is updated to continue expanding outward. It should be noted that the above criterion is a one-way tolerance, and only transition zone pixels slightly below the original threshold are included in the growth range. Pixels that are above the threshold have already been determined as change regions in step S416 and are not processed again.

[0119] After growth in all directions has stopped, connected component analysis is performed again on the expanded regions, merging adjacent expanded regions and removing isolated noise points introduced during the growth process (areas smaller than a preset area threshold, for example, an area threshold of 5 pixels). After the above region growth and secondary connected component merging, the contour of the expanded regions is refined, and the jagged boundaries originally caused by hard threshold segmentation are corrected into smooth, continuous contours that highly match the actual edges of the image.

[0120] It should be noted that the confidence level stratification judgment eliminates the interference of spurious changes introduced by noise, illumination flicker and registration residuals at the global scale; the region growth contour refinement completes the edge transition zone of the real change area and corrects the boundary morphology at the local scale.

[0121] Both are executed sequentially, together providing accurate input data for generating a high-quality change mask map in the subsequent step S511.

[0122] In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention includes the following steps in step S5: Step S511: Based on the final change region determination result output after confidence level stratification and region growth contour refinement, generate a change mask image. The change mask image is a binary image, in which the pixel positions determined as change regions are assigned a first value (e.g., 255, white), and the pixel positions determined as no-change regions are assigned a second value (e.g., 0, black).

[0123] The change mask visually displays all areas on the pylon and missile body that have changed, allowing operators to easily identify the differences in state before, during, and after loading. It should be noted that the spatial resolution of the change mask is consistent with the pre-processed image, ensuring that the pixel positions of the changed areas can be accurately located to their corresponding physical positions in the original image.

[0124] Step S512 involves performing connected component analysis on each changed region in the change mask image, aggregating adjacent changed pixels and marking them as independent changed regions to obtain the positional information of each changed region. The connected component analysis employs the eight-neighbor connectivity criterion: if a changed pixel has adjacent changed pixels in any of the eight directions (up, down, left, right, upper left, upper right, lower left, and lower right), then these adjacent pixels are grouped into the same changed region. Through connected component analysis, scattered changed pixels are aggregated into physically meaningful changed regions, each corresponding to an independent physical change event (such as a scratch, an offset, or a gap change).

[0125] Location information includes the minimum bounding rectangle of each changed region in the image coordinate system, the coordinates of its geometric center, and the set of region contour points. It should be noted that the minimum bounding rectangle is used to quickly locate the overall extent of the changed region, the geometric center coordinates are used to characterize the average position of the changed region, and the region contour point set is used to accurately describe the boundary shape of the changed region, providing multi-dimensional location data for subsequent determination of the loaded status.

[0126] Step S513: Calculate the degree information of each changed region, including the area size, number of pixels, average change amplitude, and maximum change amplitude of the changed region. The area size is obtained by multiplying the total number of pixels within the changed region by the physical area corresponding to a single pixel. The physical area corresponding to a single pixel is determined by the product of the pixel size of the current acquisition device and the optical magnification of the lens. The pixel size is read from the device calibration parameters and is used to quantify the physical range of the change.

[0127] The number of pixels is used to characterize the size of the region of change.

[0128] The average change amplitude is obtained by measuring the difference among all pixels within the change region. The average value is obtained to reflect the average intensity of change in the region.

[0129] The maximum change amplitude is obtained by taking the difference of all pixels within the change region. The maximum value is obtained to reflect the most severe degree of change in the region.

[0130] It should be noted that the above-mentioned degree information quantifies the change status of each area in the ammunition loading operation from multiple dimensions. That is, the area size reflects the physical range of the change, the average change amplitude reflects the average significance of the change, and the maximum change amplitude reflects the peak significance of the change, providing a comprehensive quantitative basis for judging the qualification of the ammunition loading status.

[0131] Step S514: Based on the location and degree information of each changed area, generate multi-form visualization results. These multi-form visualization results include a change mask map, a change heatmap, and a magnified area annotation map. The change mask map clearly marks the boundaries and locations of all changed areas in a binary image format; the change heatmap visually displays the intensity distribution of changes in each area in pseudo-color—areas with stronger changes are darker or more warm-colored (e.g., red), while areas with weaker changes are lighter or more cool-colored (e.g., blue); the magnified area annotation map locally magnifies each changed area in the original image and overlays change contour annotations, allowing operators to visually view the specific shape and location of the changed areas in conjunction with the details of the original image. It should be noted that the change heatmap, generated by mapping the degree of difference to a pseudo-color space, can visually reflect the distribution of the degree of change in each area of ​​the pylon and projectile, facilitating operators to quickly locate key areas of interest. The magnified area annotation map crops and magnifies the smallest bounding rectangle of the changed area in the original image and overlays the change contour annotation lines, allowing operators to make precise judgments in conjunction with the details of the original image.

[0132] In some embodiments, the visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration provided by the present invention further includes the following steps in step S5: Step S515: Based on the location and severity information of each changed area, and combined with the three detection scenarios for ammunition loading operations, output the ammunition loading status determination result. Specifically: (1) In the pre-loading inspection scenario, the position information of each changed area is compared with the standard structural position of the rack: if the changed area is located in the bolt hole position, installation contact surface or other key areas on the rack surface, and the degree information (area size or maximum change amplitude) exceeds the preset damage threshold, it is determined that there is damage / scratches / foreign objects / missing bolts / installation offset on the rack surface, and an unqualified prompt and specific defect location are output; if the degree information of each changed area does not exceed the preset damage threshold, it is determined that the rack surface is in normal condition, and a qualified result is output.

[0133] (2) In the monitoring scenario of the projectile being loaded, the relative displacement and docking gap between the projectile and the mounting bracket are calculated based on the geometric center coordinates and contour point set of each changing area: the position of the projectile area detected in the current frame is compared with the position of the projectile area in the previous frame to obtain the real-time displacement of the projectile; the minimum distance between the contour of the projectile area and the contour of the mounting bracket area is taken as the docking gap value. If the relative displacement or docking gap value exceeds the preset safety threshold, it is determined that the projectile displacement is abnormal or the docking gap is out of limit, and a real-time alarm is triggered; if both the relative displacement and the docking gap value are within the safety threshold range, it is determined that the projectile is properly aligned and the locking action is in place, and a normal monitoring result is output.

[0134] (3) In the post-loading verification scenario, the geometric center coordinates of the final change area are compared with the standard position after loading the projectile, and the final offset of the projectile installation position is calculated; the degree of change of the area where the locking mechanism is located is matched with the standard change characteristics of the locking completion, and it is determined whether the locking mechanism is completely closed. If the final offset exceeds the preset installation accuracy threshold or the degree of change of the locking area does not meet the locking completion standard, it is determined that the projectile is not installed in place or the locking mechanism is not completely closed, and an unqualified result and specific deviation data are output; if the final offset is within the installation accuracy threshold range and the change characteristics of the locking area match the locking completion standard, it is determined that the projectile is installed in place and the locking mechanism is completely closed, and a qualified result is output.

[0135] It should be noted that the preset thresholds (damage threshold, safety threshold, installation accuracy threshold, locking completion standard) in the above-mentioned detection scenarios are preset by the system based on the maximum permissible offset, maximum permissible gap, and latch closing displacement specified in the ammunition loading vehicle operation manual. Different models and different mounting types can be configured with different threshold parameters, and this invention does not limit this.

[0136] Step S516 involves outputting the location information, degree information, ammunition loading status determination results, and multi-format visualization results of each changed area in a structured format. The structured output includes: outputting the geometric center coordinates, minimum bounding rectangle coordinates, area size, average change amplitude, maximum change amplitude, offset, docking gap value, and pass / fail determination results of each changed area in a standard data format (such as JSON or XML), for real-time reading and decision-making by the ammunition loading vehicle control system; simultaneously, the multi-format visualization results are output to the human-machine interface display terminal of the ammunition loading vehicle for real-time viewing and verification by ground operators.

[0137] It should be noted that the structured output format facilitates data interaction with the main control system of the intelligent bomb-loading vehicle, enabling fully automated closed-loop control; the visual output format facilitates manual review and anomaly confirmation, improving human-machine collaboration efficiency.

[0138] Step S517: Store the detection results of the current frame (including the location information, degree information and ammunition loading status determination results of the changed area) into the system's historical database as reference data and algorithm optimization basis for subsequent ammunition loading operations.

[0139] Meanwhile, in continuous monitoring scenarios with ammunition loaded, this invention employs an online hand-eye mapping calibration mechanism based on detection result feedback. This mechanism, together with the ROI edge verification correction in step S114, forms a dual-loop feedback loop. However, the two mechanisms have clearly defined roles and different time sequences: the ROI edge verification correction in step S114 performs single-frame verification of the initial ROI within the current frame, responsible for confirming the validity of the ROI in the current frame; the online calibration mechanism in this step executes across frames, feeding back the detection results of the current frame to the pose sensing mapping process of the next frame, responsible for compensating for hand-eye mapping parameter drift over long time. The two mechanisms work collaboratively at different time scales without conflicting with each other.

[0140] Specifically, the geometric center coordinates of the projectile region detected in the current frame are compared with the geometric center coordinates of the expected pixel region estimated based on the pose sensing data in step S113, and the positional deviation Δ between the two is calculated. This deviation reflects the degree of mismatch between the current offline calibrated hand-eye mapping relationship and the actual physical state, that is, the systematic deviation caused by factors such as joint wear, sensor zero drift, and load changes during long-term use of the robotic arm, which lead to the inaccuracy of calibration parameters.

[0141] The position deviation Δ is multiplied by the attenuation coefficient β (β is 0.3) as a compensation amount, which is then superimposed on the pose sensing data mapping result of the next frame to correct the expected pixel area of ​​the next frame.

[0142] It should be noted that the correction logic is as follows: Let the expected pixel coordinates obtained by mapping the current frame pose sensing data be... The actual target center obtained from the detection feedback is The corrected expected coordinates for the next frame are:

[0143] This allows the predicted location to gradually approach the actual detection location.

[0144] When the absolute value of the position deviation Δ in multiple consecutive frames (for example, 5 consecutive frames) is less than the preset stable threshold (for example, 2 pixels), it is determined that the hand-eye mapping relationship has been adaptively calibrated to the optimal state under the current working condition. The attenuation coefficient β is automatically reduced to 0.1 to reduce the superposition amplitude of the compensation amount and avoid over-correction that introduces new oscillations. When the deviation amount further converges to within 1 pixel, the compensation superposition is paused, the current mapping parameters are kept unchanged, and the compensation is reactivated when the deviation amount exceeds the stable threshold again.

[0145] When pose sensing data, especially joint angles or telescopic arm displacement, undergoes a sudden change (the change is determined by the amount of change in a single frame exceeding a preset jump threshold, for example, ±10% of the previous frame value), the historically accumulated compensation amount is immediately cleared and the deviation calculation and compensation superposition process is re-executed to avoid historical deviation data contaminating the mapping results after the change; at the same time, in the first frame after the change, the attenuation coefficient β is temporarily increased to 0.5 to speed up the response speed, and then restored to the normal value of 0.3 after the deviation amount converges quickly.

[0146] Through the aforementioned closed-loop iterative mechanism, this invention achieves online adaptive maintenance of the hand-eye mapping relationship, eliminating the need for periodic shutdowns for recalibration. This effectively overcomes the problem of decreased mapping accuracy caused by factors such as joint wear, sensor zero drift, and load variations during long-term use of the robotic arm. This ensures that the ROI prediction accuracy remains stable throughout continuous operation and automatically adapts to the optimal mapping parameters at different operational stages (unloaded, loaded, different projectile angles). The method also includes one or more processors and a memory.

[0147] The memory is used to store operable instructions that, when executed by one or more processors, cause one or more processors to perform operations. These operations include the flow of a visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in the foregoing embodiments, particularly... Figure 1 The flowchart of the method is shown.

[0148] Other aspects disclosed in the embodiments of the present invention also propose a computer-readable medium for storing software including instructions executable by one or more computers, which, upon execution, cause the one or more computers to perform operations including the flow of a visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration according to the foregoing embodiments, particularly... Figure 1 The flowchart of the method is shown.

[0149] It should be recognized that embodiments of the present invention may be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium.

[0150] The method can be implemented using standard programming techniques, including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium is configured such that the computer operates in a specific and predefined manner.

[0151] Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system; however, if required, the program can be implemented in assembly or machine language.

[0152] In any case, the language can be either compiled or interpreted.

[0153] Furthermore, for this purpose, the program can run on programmed application-specific integrated circuits.

[0154] The processes (or variations and / or combinations thereof) described herein can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. A computer program comprises multiple instructions that can be executed by one or more processors.

[0155] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices.

[0156] Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether portable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein.

[0157] Furthermore, machine-readable code, or parts thereof, can be transmitted via wired or wireless networks.

[0158] When such media includes instructions or programs that combine with a microprocessor or other data processor to implement the steps described above, the invention described herein includes these and other different types of non-transitory computer-readable storage media.

[0159] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration, characterized in that: include: Acquire a reference image and an image to be inspected, read pose sensing data and generate a region of interest, and perform registration, noise reduction and grayscale normalization on the two images within the region of interest to obtain a preprocessed image; Based on the local texture complexity of the preprocessed image of the reference image and the image to be inspected, an adaptive scale for each local block is dynamically generated, and multi-scale features are extracted at each adaptive scale to obtain the multi-scale feature vector of each local block. Based on the effectiveness of each multi-scale feature vector, dynamic fusion weights for each local block at each scale are generated. Based on the dynamic fusion weights, each multi-scale feature vector is weighted and fused to obtain the global fusion feature map of the reference image and the global fusion feature map of the image to be inspected. Extract the systematic error between the reference image and the image to be inspected, correct the global fusion feature map based on the systematic error, obtain the reference calibrated feature map and the image to be inspected calibrated feature map respectively, generate a differential threshold based on the calibrated feature map and local texture complexity, and determine the change region. A change mask map is generated based on the changed regions, and the location and degree information of each changed region are output.

2. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 1, characterized in that: The pose sensing data includes the joint angles of the intelligent bomb-loading vehicle's robotic arm, the displacement of the telescopic arm, and the vehicle's horizontal tilt angle.

3. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 2, characterized in that: The region of interest is generated as follows: Based on the pose sensing data and combined with the pre-calibrated hand-eye mapping relationship, the expected pixel area of ​​the pylon and projectile target in the current image is estimated in real time, and an initial region of interest is generated. Using the initial region of interest as a search benchmark, lightweight edge feature matching is performed within the neighborhood of the search benchmark to verify whether the initial region of interest contains the complete edge contours of the pylon and the projectile. If the verification passes, the initial region of interest is used as the valid region of interest for the current frame; if the verification fails, the position and size of the initial region of interest are corrected based on the edge feature matching result to obtain the corrected region of interest. When the pose sensing data changes, the estimation, verification, and correction process is repeated to ensure that the region of interest is updated synchronously with the changes in the pose sensing data.

4. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 1, characterized in that: The calculation method for the local texture complexity is as follows: The preprocessed images of the reference image and the preprocessed images of the image to be inspected are divided into the same local block grid. For the same local block location, the local block texture entropy of the reference image and the local block texture entropy of the image to be inspected are calculated respectively. The texture entropy is used to characterize the local texture complexity of the corresponding local block, and the maximum value of the two is taken as the local texture complexity of the local block. The reference image and the image to be inspected use the same adaptive scale for the corresponding local blocks.

5. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 4, characterized in that: The method for dynamically generating the adaptive scale of each local block is as follows: The mean of all local block texture entropies is calculated as the mean of global texture entropy; With reference to the baseline scale, for any local block, when the texture entropy of the local block is higher than the average global texture entropy, the adaptive scale of the corresponding local block is smaller than the baseline scale. When the texture entropy of a local block is lower than the average global texture entropy, the adaptive scale of the corresponding local block is greater than the baseline scale. When the texture entropy of a local block is equal to the mean of the global texture entropy, the adaptive scale of the corresponding local block is equal to the baseline scale. The adaptive scale is determined according to an exponential mapping with the natural constant as the base: when the texture entropy is higher than the global average texture entropy, it is scaled by a negative exponential scaling; when the texture entropy is lower than the global average texture entropy, it is scaled by a positive exponential scaling; and when the texture entropy is equal to the global average texture entropy, the scaling factor is 1. The reference image and the image to be inspected use the same adaptive scale for corresponding local blocks.

6. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 5, characterized in that: The reference scale is determined based on the ratio of the pixel density of the current acquisition device to the standard reference pixel density; The reference scale is used to unify the physical scale standards between different acquisition devices.

7. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 1, characterized in that: The effectiveness of the multi-scale feature vectors is measured by the ratio of the difference between the maximum and minimum values ​​of the feature vectors at each scale to the mean of the feature vectors. The larger the ratio, the higher the measure of effectiveness.

8. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 1, characterized in that: The system error includes the registration deviation and grayscale deviation between the reference image and the image to be inspected, and an error feature vector is constructed based on the registration deviation and the grayscale deviation; The error feature vector is used to characterize the overall error level under the current acquisition environment; The error feature vector is formed by splicing the registration deviation component and grayscale deviation component of each local block in spatial order. Each local block corresponds to a two-dimensional error vector. The error feature vector is formed by flattening the two-dimensional error vectors of all local blocks. The global fusion feature map includes the global fusion feature map of the reference image and the global fusion feature map of the image to be inspected; Based on the maximum and mean values ​​of the error feature vector, pixel-by-pixel error correction is performed on the global fusion feature map of the reference image and the global fusion feature map of the image to be inspected, with each local block as a unit, to obtain the reference calibrated feature map and the image to be inspected calibrated feature map. For each pixel, a two-dimensional error vector of the local block to which the pixel belongs is extracted from the error feature vector. The magnitude of the two-dimensional error vector is used as the scalar correction factor of the corresponding local block. All pixels within the same local block use the same correction coefficient.

9. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 4, characterized in that: The differential threshold is generated as follows: The mean of all local block texture entropies is calculated as the mean of global texture entropy; With reference to the global baseline threshold, which is determined based on the global difference statistical characteristics of the calibrated feature map, for any pixel location, when the texture entropy of the local block to which the pixel location belongs is higher than the global texture entropy mean, the difference threshold of the pixel location is lower than the global baseline threshold. When the texture entropy of the local block to which the pixel location belongs is lower than the global average texture entropy, the differentiation threshold of the pixel location is higher than the global baseline threshold.

10. The visual change detection method based on adaptive multi-scale feature fusion and error feedback calibration as described in claim 9, characterized in that: The method for determining the changing region is as follows: The difference degree of each pixel position is calculated based on the reference calibrated feature map and the feature map to be tested after calibration. The difference degree of each pixel position is compared with the difference threshold of the corresponding pixel position. Pixel positions with a difference degree greater than the difference threshold are determined as change regions.

Images