Image processing method based on artificial intelligence large model

Abstract

The application discloses an image processing method based on an artificial intelligence large model, and particularly relates to the technical field of image processing, and obtains a to-be-processed image, a shooting device posture, an instantaneous light supplement intensity and a target surface temperature and humidity change value, performs multi-scale decomposition on the image to obtain a global degradation layer, a local artifact layer and a suspected real texture layer, and generates an environment parameter feature vector; a replaceable area and a non-replaceable detail area are determined through correlation matching, a detail locking mask is constructed, and the artificial intelligence large model is constrained to only reconstruct in a potential space corresponding to the replaceable area; and then the detail locking mask is updated in reverse based on structural similarity checking until a target processed image is output; and the application can reduce the risk of false repair of real micro textures.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image processing method based on a large artificial intelligence model. Background Technology

[0002] In image acquisition scenarios such as nighttime inspections of industrial equipment, insulation component testing at coastal substations, and pipeline leak detection in enclosed factory buildings, inspection terminals typically need to acquire images under conditions of high humidity, strong supplemental lighting, fluctuating temperature differences on equipment surfaces, and frequent changes in shooting angles. Due to these imaging conditions, the images to be processed are prone to problems such as localized strong reflections, fogging, shadow trailing, and attenuation of fine textures. Especially in target areas such as fine discharge carbonization marks on insulator surfaces, early cracks at metal joints, and pinhole corrosion points on pipeline surfaces, the image texture scale is small, grayscale variations are weak, and they are highly similar in visual characteristics to environmental noise and reflective artifacts.

[0003] Existing image processing methods typically enhance the entire image uniformly based on denoising, dehazing, sharpening, or image inpainting models, or determine the area to be repaired solely based on the image's brightness, gradient, and edge information. These methods fail to fully utilize accompanying imaging data such as the camera's pose, instantaneous illumination intensity, and temperature and humidity variations on the target surface, making it difficult to determine whether local abnormal textures in the image originate from real target details or environmental disturbances. Especially when using large AI models for image reconstruction, these models possess strong content completion and texture generation capabilities. However, without constraints on real detail areas, they can easily mistake minor defects for artifacts and replace or smooth them, resulting in an overall improved image clarity but weakened, shifted, or even lost key defect features.

[0004] Therefore, how to combine imaging data during image processing to distinguish between replaceable regions caused by the environment and non-replaceable detail regions unrelated to environmental disturbances, and how to lock and verify non-replaceable detail regions during the reconstruction of large-scale artificial intelligence models to avoid the incorrect reconstruction of real details, is a technical problem that urgently needs to be solved in current image processing technology based on large-scale artificial intelligence models. Summary of the Invention

[0005] The purpose of this invention is to provide an image processing method based on a large artificial intelligence model to address the shortcomings of the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an image processing method based on a large artificial intelligence model, comprising: S100, acquire the image to be processed and its accompanying imaging data, the accompanying imaging data including the attitude of the shooting device, the instantaneous supplementary light intensity and the temperature and humidity change value of the target surface; S200, the image to be processed is decomposed into a multi-scale layer to obtain a global degradation layer, a local artifact layer and a suspected real texture layer, and an environmental parameter feature vector is generated based on the imaging accompanying data. S300, perform correlation matching between the environmental parameter feature vector and the local artifact layer and the suspected real texture layer respectively to determine the replaceable region caused by the environment and the non-replaceable detail region that is not related to the environmental disturbance. S400, a detail-locking mask is constructed based on the non-replaceable detail region, and the detail-locking mask, replaceable region and global degradation layer are input into the large artificial intelligence model, so that the large artificial intelligence model only performs image reconstruction in the potential space corresponding to the replaceable region to obtain the candidate processed image; S500: Perform structural similarity verification between the candidate image to be processed and the image to be processed in the non-replaceable detail region. If the verification does not meet the preset conditions, update the detail locking mask in reverse according to the offset result and repeat S400 until the target image to be processed is output.

[0007] Preferably, S100 includes: The midpoint between the start and end times of exposure of the image to be processed is determined as the image acquisition time; the shooting device posture and instantaneous fill light intensity are time-aligned according to the image acquisition time; if the sampling time is inconsistent with the image acquisition time, linear interpolation is performed using adjacent sampling values; the target surface temperature and humidity change value is generated based on the difference between the target surface temperature and the reference time and the difference between the equivalent relative humidity of the target surface.

[0008] Preferably, S200 includes: The image to be processed is converted into a brightness map, and the directional decomposition angle is determined based on the roll angle and yaw angle in the shooting device's attitude. The directional scaling coefficient is determined based on the pitch angle in the shooting device's attitude, and a multi-scale directional Gaussian kernel is constructed based on the directional decomposition angle and the directional scaling coefficient. The brightness map is convolved scale by scale using the multi-scale directional Gaussian kernel to obtain the final scale smoothing result as a global degradation layer, and a comprehensive high-frequency residual map is obtained from the high-frequency residuals of each scale.

[0009] Preferably, S200 further includes: The fill light influence factor is calculated based on the instantaneous fill light intensity, and the positive brightness abrupt change in the brightness map relative to the global degradation layer is normalized. The fill light influence factor, the normalized positive brightness abrupt change, and the comprehensive high-frequency residual map are multiplied to obtain the artifact scoring map. The local artifact layer is determined based on the artifact scoring map, and the part of the comprehensive high-frequency residual map after removing the local artifact layer is determined as the suspected real texture layer.

[0010] Preferably, generating environmental parameter feature vectors based on accompanying imaging data includes: The attitude of the shooting device is converted into sine and cosine components of roll angle, pitch angle and yaw angle respectively; the instantaneous supplementary light intensity is converted into supplementary light influence factor, and the target surface temperature change value and the target surface equivalent relative humidity change value are normalized respectively; the sine component, cosine component, supplementary light influence factor, normalized temperature and humidity change value and local artifact layer area ratio are combined into environmental parameter feature vector.

[0011] Preferably, S300 includes: Effective residual labeling is performed on the local artifact layer and the suspected real texture layer respectively, and the effective residual labeling results are divided into 8-neighborhood connected components to obtain candidate regions. The average intensity, orientation consistency and region area ratio are extracted for each candidate region to form a region response vector. The region response vector is used as a region-level input for correlation matching with the environmental parameter feature vector.

[0012] Preferably, S300 further includes: Based on the supplementary lighting influence factor, pitch angle component, target surface temperature and humidity change value, and local artifact layer area ratio in the environmental parameter feature vector, a local artifact projection vector is constructed; based on the environmental disturbance relationship opposite to the local artifact projection vector, a texture preservation projection vector is constructed; the region response vector of the same candidate region is matched with the local artifact projection vector and the texture preservation projection vector respectively by cosine projection matching to obtain the local artifact correlation value and the texture preservation correlation value.

[0013] Preferably, identifying replaceable areas caused by environmental factors and non-replaceable detail areas unrelated to environmental disturbances includes: Calculate the difference between the local artifact correlation value and the texture preservation correlation value of the same candidate region, and assign the difference to the pixels in the candidate region to obtain the environment attribution map; determine the pixels in the environment attribution map with a difference greater than 0 and located within the effective residual marker of the local artifact layer as replaceable regions; determine the pixels in the environment attribution map with a difference less than 0 and located within the effective residual marker of the suspected real texture layer as non-replaceable detail regions.

[0014] Preferably, S400 includes: Based on the maximum truncation influence range of the multi-scale directional Gaussian kernel at the first scale, the non-replaceable detail region is expanded to obtain a detail-locking mask; pixels overlapping with the detail-locking mask in the replaceable region are removed to obtain the effective reconstruction region; the image to be processed, the global degradation layer, the effective reconstruction region, and the detail-locking mask are input together into the large artificial intelligence model, so that the large artificial intelligence model generates candidate images for processing in the latent space corresponding to the effective reconstruction region.

[0015] Preferably, S500 includes: Within the non-replaceable detail region and its extended neighborhood, a structural similarity distribution map between the candidate image to be processed and the image to be processed is calculated. The position of the lowest structural similarity is determined based on the structural similarity distribution map, and the offset of the lowest structural similarity position relative to the center of the non-replaceable detail region is calculated. When the structural similarity within the non-replaceable detail region is lower than the structural similarity of its extended neighborhood, the detail locking mask is expanded based on the offset result, and S400 is repeated until the structural similarity verification meets the conditions, at which point the target image to be processed is output.

[0016] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention does not directly input the image to be processed into a large-scale artificial intelligence model for overall enhancement or repair. Instead, it first introduces the shooting device's posture, instantaneous illumination intensity, and target surface temperature and humidity changes corresponding to the same acquisition time as the image, transforming external imaging conditions into environmental parameter feature vectors. These vectors are then correlated with local artifact layers and suspected real texture layers obtained from multi-scale image decomposition. Through this method, the invention can determine the cause of anomalies in a region, rather than relying solely on pixel brightness, gradient, or noise intensity. For local bright spots or texture stretching caused by strong illumination reflection, condensation, or shooting pitch, the invention can identify them as replaceable regions caused by the environment. For weak textures such as small cracks, carbonization marks, and corrosion points unrelated to environmental disturbances, they are identified as non-replaceable detail regions. This directly solves the problem that existing large-scale image processing methods easily reconstruct real, minute defects as noise or artifacts, improving the reliability of defect detail preservation.

[0017] This invention constructs a detail-locking mask based on non-replaceable detail regions before large-scale AI model reconstruction. This mask, along with replaceable regions and a global degradation layer, is constrained to a latent space, ensuring the large model reconstructs only at the unlocked potential locations corresponding to the replaceable regions, rather than freely generating the entire image. This process allows the global degradation layer to provide references for fogging, uneven illumination, and overall blurring during reconstruction, while the detail-locking mask prevents the model from potentially modifying the representation of real detail regions. Furthermore, after generating candidate images, this invention does not directly output them but performs structural similarity checks between the non-replaceable detail regions and the image to be processed. When the structural preservation of a detail region is lower than its extended neighborhood, the detail-locking range is expanded in reverse based on the offset result of the lowest structural change location, and reconstruction is repeated. This closed-loop constraint gradually limits the impact of model reconstruction to the environmental artifact region, improving image clarity and local artifact repair quality while reducing the smoothing, offsetting, or disappearance of real detail edges. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a flowchart of the method of the present invention.

[0020] Figure 2 This is a flowchart of the structural similarity verification process of the present invention.

[0021] Figure 3 This is a flowchart of the constraint reconstruction process for the large artificial intelligence model of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1, please refer to Figure 1 and Figure 2 and Figure 3 As shown in this embodiment, the image processing method based on a large artificial intelligence model includes: S100, acquire the image to be processed and its accompanying imaging data, the accompanying imaging data including the attitude of the shooting device, the instantaneous supplementary light intensity and the temperature and humidity change value of the target surface.

[0024] In step S100, the image to be processed and its accompanying imaging data are acquired. The image to be processed is denoted as... The image to be processed is a pixel matrix formed by the target to be detected under actual acquisition conditions, represented as: ; in, Indicates the image height. Indicates the image width. R represents the number of color channels; when the image to be processed is a grayscale image... When the image to be processed is an RGB image, The image acquisition time corresponding to the image to be processed is denoted as... , Take the start time of exposure At the end of the exposure The intermediate time is calculated as follows: ; The midpoint of the exposure is used as the image acquisition time because the pixel brightness of the image to be processed is formed by the incident light throughout the exposure time. The midpoint of the exposure can serve as a unified time reference for the supplementary light intensity, attitude, and target surface temperature and humidity data, thereby avoiding deviations in subsequent correlation matching caused by different data using different times.

[0025] The accompanying imaging data is denoted as Including the posture of the shooting equipment Instantaneous fill light intensity and the temperature and humidity change values ​​of the target surface , is represented as: ; The image to be processed Accompanying data of the imaging Through the same collection number The corresponding data collection number is saved. Generated in ascending order of collection sequence, used to limit , , and The images belong to the same image acquisition result, avoiding the incorrect combination of data from different acquisition times.

[0026] The posture of the shooting device The orientation of the shooting device is used to indicate the angle state of the shooting direction when acquiring the image to be processed. Including roll angle Pitch angle and yaw angle , is represented as: ; in, This indicates the rotation angle of the image in the horizontal direction relative to the ground plane. This indicates the angle between the optical axis of the camera and the horizontal plane of the ground. This indicates the angle between the optical axis of the camera and the preset direction of travel within the horizontal plane of the ground. If the camera is in... The pose recorded at each moment is a quaternion. Then calculate according to the following formula. , and : , , ; in, It is the arctangent function in the fourth quadrant. It is an arcsine function. By uniformly converting quaternions into roll, pitch, and yaw angles, the shooting posture can participate in the generation of subsequent environmental parameter feature vectors with a clear angular quantity.

[0027] The instantaneous fill light intensity The image acquisition time The corresponding target surface illuminance value, in lx. If in If there is a constant sample value of supplementary lighting illuminance, then this sample value is directly used as... If the sampling times for the supplementary lighting illuminance are respectively and And satisfy Then linear interpolation is used for calculation. The calculation formula is: ; in, express The supplemental illuminance value at any given time. express The supplementary lighting intensity value at each moment. The above calculation correlates the supplementary lighting intensity with the image exposure time, which is used to subsequently distinguish between local artifacts caused by supplementary lighting and real textures.

[0028] The target surface temperature and humidity change value Including the target surface temperature change value Equivalent relative humidity change value of the target surface , is represented as: ; The target surface temperature change value Calculate using the following formula: ; in, express The target surface temperature at any given time. Indicates the reference time The target surface temperature. The reference time. This refers to the exposure midpoint of an image previously captured for the same target; when the current image is the first frame of the target. This refers to the time of a target surface temperature and humidity sampling completed before acquiring the first frame of the image. This limitation ensures... A defined calculation benchmark is established for each image acquisition.

[0029] Target surface equivalent relative humidity According to ambient temperature Ambient relative humidity and target surface temperature Calculated. Temperature Saturated water vapor pressure Calculate using the following formula: ; in, The unit is ℃. The unit is hPa. Represented by natural constant An exponential function with base 0.05. Equivalent relative humidity of the target surface. Calculate using the following formula: ; in, express Ambient relative humidity at any given time, expressed as %. Change in equivalent relative humidity at the target surface. Calculate using the following formula: ; The equivalent relative humidity of the target surface is used instead of the ambient relative humidity because atomization, condensation, and damp films are primarily related to the water vapor state under the target surface temperature conditions. and Together, they represent the temperature and humidity changes on the target surface, which can be used in subsequent steps to determine whether local bright spots, texture attenuation, or edge blurring are related to changes in the target surface humidity.

[0030] After completing step S100, the image to be processed is obtained. and its corresponding imaging accompanying data .in, Used to characterize changes in shooting angle Used to characterize the effect of supplemental lighting on image brightness and reflectivity. This data is used to characterize the effects of temperature and humidity changes on the target surface on fogging, condensation, and texture attenuation. The data described above serves as input to step S200, used to perform multi-scale decomposition on the image to be processed and generate environmental parameter feature vectors.

[0031] S200, the image to be processed is decomposed into a multi-scale layer to obtain a global degradation layer, a local artifact layer and a suspected real texture layer, and an environmental parameter feature vector is generated based on the imaging accompanying data.

[0032] In step S210, the image to be processed obtained in step S100 is... Convert to brightness map .when When converting to an RGB image, first normalize the pixel values ​​of the red, green, and blue channels to 0. Then calculate the brightness diagram using the following formula: ; in, Indicates pixel position, , , These represent the normalized values ​​of the red, green, and blue channels, respectively. When creating a grayscale image, normalize the grayscale values ​​to... And order: ; Brightness diagram This is used to uniformly characterize brightness degradation, local reflections, and texture residuals in images, avoiding the influence of color differences on subsequent multi-scale decomposition results.

[0033] Based on the shooting device posture obtained in step S100 Determine the directional decomposition angle The calculation formula is: ; in, Indicates the roll angle. Indicates the yaw angle. This indicates that the angles are being normalized. Range. Yaw angle represents the change in shooting direction in the horizontal plane, and roll angle represents the rotation of the image plane. Subtracting the two yields the principal direction in the brightness diagram caused by the shooting posture. Based on pitch angle... Calculate the directional scaling factor : ; in, .when hour, This indicates that the shooting direction is close to horizontal; when When it increases, Increase, used to increase the scale of the decomposition kernel along the directional decomposition angle, so that multi-scale decomposition can adapt to the texture direction compression caused by looking down or looking up.

[0034] In step S220, the angle is decomposed according to directionality. Brightness diagram Perform multi-scale decomposition. Number of decomposition scales. Determine by the following formula: ; in, Indicates the image height. Indicates the image width. This indicates rounding down. The value 32 represents 1 / 32 of the shorter side length as the boundary between global degradation and local texture. This design is based on the fact that fog, uneven illumination, and overall blur typically cover large areas, while the texture width corresponding to cracks, carbonization marks, and erosion points is usually less than 1 / 32 of the image's shorter side. This scale division reduces the probability of fine, realistic textures being incorporated into the global degradation layer.

[0035] No. Directional Gaussian kernels at various scales Determine by the following formula: , , ; in, , Indicates the directional decomposition angle coordinates Indicates the angle perpendicular to the directional decomposition. The coordinates. For the brightness map in pixels. Centered on pixels The calculated position of the neighboring pixels, and Calculate using the following formula: , ; To ensure that convolution calculations can be performed, the Gaussian kernel is truncated to... and Within the range, and normalize the truncated discrete kernel: ; in, This represents the normalized directional Gaussian kernel, with the denominator being the sum of all discrete kernel values ​​within the truncation range. This normalization ensures that the overall brightness energy remains consistent before and after convolution, avoiding brightness shifts in the global degradation layer caused by kernel truncation.

[0036] make Calculate the smoothing results and high-frequency residuals on a scale-by-scale basis: , ; in, This represents the convolution operation. Indicates the first Smoothing results at each scale, Indicates the first High-frequency residuals at each scale. The smoothing result at the last scale is determined as the global degradation layer. : ; The summation of the absolute values ​​of the high-frequency residuals at each scale yields the comprehensive high-frequency residual map. : ; in, Used to represent low-frequency degradation caused by uneven illumination, fogging, and overall blurring. Used to contain local artifacts and textures that appear to be real.

[0037] In step S230, based on the instantaneous supplementary light intensity From the comprehensive high-frequency residual plot By separating regions of abrupt brightness changes, local artifact layers can be obtained. First, based on the instantaneous supplemental light intensity... Calculate the effect factor of supplemental lighting : ; in, Represents the natural logarithm. The unit is lx. The basis for using logarithmic compression is that the enhancement of reflection caused by supplemental lighting does not increase linearly with illuminance; through the above calculation, changes in low illuminance can be preserved, and the value under high illuminance will not directly saturate to 1, thereby improving the stability of local artifact separation under different supplemental lighting conditions.

[0038] Calculate the normalized map of brightness abrupt changes : ; Calculate the normalized plot of high-frequency residuals : ; in, This value is used to prevent the denominator from being zero. It is less than the minimum effective grayscale interval of the normalized image and does not change the size relationship between pixels; it is only used to ensure that the division operation is true.

[0039] The artifact score map is calculated based on the supplementary lighting influence factor, the brightness abrupt change normalization map, and the high-frequency residual normalization map. : ; in, The larger the value, the higher the probability that the pixel simultaneously possesses strong supplementary lighting conditions, positive brightness abrupt change, and high-frequency residual, which better matches the imaging characteristics of supplementary lighting reflection artifacts.

[0040] Otsu's method was used to analyze artifact scoring maps. Determining the artifact segmentation threshold : ; in, express The value is less than pixel ratio, express The value is greater than or equal to pixel ratio, express The value is less than The average pixel value, express The value is greater than or equal to The average pixel value. This threshold is calculated from the artifact score distribution of the current image to be processed, and does not need to be specified in advance. Its technical effect is to separate high-scoring brightness abrupt regions from low-scoring texture regions according to the statistical distribution of the current image.

[0041] according to Generate artifact marker map : ; According to the artifact marker map Obtain the local artifact layer and suspected real texture layers : , ; in, This indicates content identified as a local artifact caused by a combination of fill light and abrupt changes in brightness. This represents the texture content retained after removing local artifacts from the synthesized high-frequency residual map. Because... It may still contain real-world texture defects and weak artifacts that have not been identified, hence it is called a suspected real-world texture layer.

[0042] In step S240, an environmental parameter feature vector is generated based on the accompanying imaging data. First, based on the artifact marker map... Calculate the area ratio of local artifact layers : ; in, This represents the proportion of pixels identified as local artifacts to the total number of pixels in the image to be processed.

[0043] Change in target surface temperature Equivalent relative humidity change value of the target surface Normalize: , ; in, The unit is ℃. The unit is %. The 1 in the denominator corresponds to a temperature change scale of 1°C and a relative humidity change scale of 1%, respectively, and is used to compress the value to a smaller scale while preserving the direction of change. Range. After this processing, the decrease in temperature and the increase in humidity can be included in the environmental parameter feature vector in the direction of sign, which facilitates subsequent judgment of image degradation caused by fogging, condensation or wet film.

[0044] The orientation of the shooting equipment is expressed using sine and cosine values ​​to avoid angle inconsistencies. and A numerical jump occurs nearby. This ultimately generates an environmental parameter feature vector. : ; The first six components are used to represent the shooting posture. Used to indicate the effect of supplemental lighting and Used to represent the temperature and humidity changes on the target surface Used to represent the area percentage of local artifacts in the current image.

[0045] The global degradation layer is obtained through step S200. Local artifact layer Suspected real texture layer and environmental parameter feature vectors .in, As a global degradation reference when the large artificial intelligence model performs image reconstruction in step S400; , and As input to correlation matching in step S300, it is used to determine replaceable regions caused by the environment and non-replaceable detail regions that are not related to environmental disturbances.

[0046] S300, the environmental parameter feature vector is correlated with the local artifact layer and the suspected real texture layer to determine the replaceable region caused by the environment and the non-replaceable detail region that is not related to the environmental disturbance.

[0047] In step S310, the local artifact layer obtained in step S200 is... and suspected real texture layers These are divided into connected candidate regions. To avoid mistakenly identifying convolutional residual pixels with values ​​close to 0 as candidate regions, they are first divided into... and Perform normalization processing. For any layer to be partitioned... ,in for or Normalization layer Calculate using the following formula: ; in, Indicates pixel position, This is used to prevent the denominator from being 0.

[0048] According to the normalization layer The effective residual threshold is determined using the maximum inter-class variance method. The calculation formula is: ; in, express The value is less than pixel ratio, express The value is greater than or equal to pixel ratio, express The value is less than The average pixel value, express The value is greater than or equal to The average pixel value. This threshold is calculated from the pixel distribution of the current layer and is not pre-specified; its function is to separate effective high-frequency residuals from convolutional residuals that are close to 0, thereby improving the stability of connected candidate regions.

[0049] according to Generate effective residual labeling map : ; when At that time, the effective residual marker map of local artifacts was obtained. ;when At that time, a residual marker map of what appears to be a true texture was obtained. Subsequently, they respectively... and Perform 8-neighbor connectivity partitioning to obtain a set of local artifact candidate regions. and a set of candidate regions for suspected real textures 8-neighbor connectivity means that the current pixel and its adjacent pixels in the top, bottom, left, right, and four diagonal directions that have a value of 1 are grouped into the same candidate region.

[0050] For any candidate region Extract average intensity Consistency of direction and the proportion of regional area To form a regional response vector .when When originating from a local artifact layer, let ;when When it originates from a suspected real texture layer, make Average intensity Calculate using the following formula: ; in, Indicates candidate region The number of pixels within.

[0051] Consistency of direction Calculated based on the concentration of gradient directions within the candidate region. For pixels... ,definition: , The horizontal and vertical gradients are respectively: , ; in, Indicates the image width. This represents the image height. Using the above method, the gradient of boundary pixels is calculated using adjacent valid pixels, avoiding undefined calculations at image boundaries.

[0052] gradient magnitude and gradient direction They are respectively: , ; Consistency of direction Calculate using the following formula: ; Among them, adopt This is to make the directions different The textures are considered to be in the same direction. The closer it is to 1, the more concentrated the texture direction is within the candidate region; The closer it is to 0, the more dispersed the texture direction is within the candidate region.

[0053] Regional area percentage Calculate using the following formula: ; Finally, candidate regions were obtained. Region response vector: ; In step S320, the environmental parameter feature vector obtained in step S200 is used. For each region response vector Perform projection matching. The environmental parameter feature vector is: ; in, Indicates the influence factor of supplemental lighting. This represents the normalized value of the target surface temperature change. This represents the normalized value of the equivalent relative humidity change on the target surface. This indicates the area percentage of the local artifact layer.

[0054] First, calculate the attitude influence based on the environmental parameter feature vector. And the influence of wet film : , ; in, Used to characterize the effect of pitch angle on reflection and perspective compression; when the equivalent relative humidity of the target surface increases and the target surface temperature decreases... Increase An increase indicates a stronger impact of condensation, wet film, or fogging on imaging. Because... and The range of values ​​is ,therefore The range of values ​​is .

[0055] according to , , and Constructing local artifact projection vectors and texture-preserving projection vector : , ; in, Used to represent the environmental characteristics that together cause local artifacts due to strong supplementary lighting, attitude effects, and wet film effects; This is used to represent environmental features that, contrary to the aforementioned environmental perturbations, tend to be preserved as realistic textures. Both are 3D vectors, and are related to the region response vector. The dimensions are consistent, enabling projection matching.

[0056] For each candidate region Calculate the local artifact correlation values ​​respectively. And texture preservation related values : , ; in, Represents the dot product of vectors. This represents the L2 norm of a vector. It is calculated simultaneously over the same candidate region. and This ensures that subsequent difference comparisons occur on the same region's response vector, avoiding inconsistencies in the comparison objects caused by calculations only on a local artifact layer or only on one side of a suspected real texture layer.

[0057] In step S330, based on the local artifact correlation values ​​of the same candidate region Values ​​related to texture preservation The difference between them generates an environmental attribution map. For any candidate region First, calculate the regional attribution difference. : ; Then, the region attribution difference is assigned to each pixel within the candidate region to obtain the environment attribution map. : ; in, This represents the union of all candidate regions. If This indicates that the correlation between the candidate region at that location and the local artifact projection vector is higher than its correlation with the texture-preserving projection vector; if This indicates that the correlation between the candidate region at that location and the texture-preserving projection vector is higher than its correlation with the local artifact projection vector; if This indicates that the location was not included in the candidate region or that the two correlations are equal.

[0058] In step S340, based on the environmental attribution map Identify replaceable areas caused by environmental disturbances and non-replaceable detail areas unrelated to environmental perturbations. Replaceable area mask. Determine by the following formula: ; Non-replaceable detail area mask Determine by the following formula: ; in, This is a map showing the effective residual markers for local artifacts. This is a residual marker map for suspected real texture. The boundary values ​​are set to 0 because... The difference between two correlation values ​​that are from the same candidate region is 0, which means that the correlation between local artifacts and the correlation between texture preservation are equal; values ​​greater than 0 and values ​​less than 0 correspond to the magnitude of the two correlations, and are not subject to artificially set thresholds.

[0059] Through the above processing, pixels located in the local artifact layer with a positive environmental attribution difference are identified as replaceable regions, indicating that these regions are more likely to be caused by a combination of lighting, pose, and temperature and humidity changes on the target surface, and are suitable for reconstruction by a large artificial intelligence model in subsequent steps; pixels located in the suspected real texture layer with a negative environmental attribution difference are identified as non-replaceable detail regions, indicating that these regions have a low correlation with environmental disturbances and should remain unchanged in subsequent image reconstruction.

[0060] After completing step S300, a replaceable area mask is obtained. and non-replaceable detail area mask The replaceable area mask Non-replaceable detail area mask and the global degradation layer obtained in step S200 Together, they serve as input to step S400, constraining the large AI model to reconstruct the image within the latent space corresponding to the replaceable regions, and limiting its modification of non-replaceable detail regions.

[0061] S400, a detail-locking mask is constructed based on the non-replaceable detail region, and the detail-locking mask, replaceable region and global degradation layer are input into the large artificial intelligence model, so that the large artificial intelligence model can only reconstruct the image in the potential space corresponding to the replaceable region to obtain the candidate processing image.

[0062] In step S400, based on the non-replaceable detail region mask obtained in step S300... Construct a detail-locked mask and combine it with an alternative region mask. and global degradation layer The large artificial intelligence model is constrained to reconstruct images within the potential space corresponding to the replaceable regions, thus obtaining candidate processed images.

[0063] First, based on the non-replaceable detail area mask Generate detail-locked mask Since the multi-scale decomposition in step S200 uses a directional Gaussian kernel, and the first scale has the greatest impact on fine textures, the detail locking radius is determined based on the maximum influence range of the directional Gaussian kernel in the first scale. In the first scale: , ; in, This refers to the directional scaling factor obtained from the pitch angle in step S200. (Detailed locking radius) Determine by the following formula: ; in, This indicates rounding up. The basis for using 3 times the standard deviation is that the directional Gaussian kernel in step S200... By truncating within a certain range, true details may be affected by high-frequency residual diffusion within that range. Including this range in the locked area can reduce the likelihood that crack endpoints, fine carbide marks, or corrosion edges will be altered during reconstruction.

[0064] Detail Lock Mask Calculate using the following formula: ; in, This indicates the pixel position in the image to be processed. This indicates the pixel position within a non-replaceable detail region. When... If there is no pixel with a value of 1, let .

[0065] Then, based on the replaceable region mask and detail locking mask Obtain an effective reconstruction area mask : ; in, This indicates that the pixel belongs to a replaceable region and is not locked for detail, so it can participate in the reconstruction; This indicates that the pixel will not participate in the reconstruction. This calculation prioritizes the preservation of non-replaceable detail regions when they are adjacent to or overlap with replaceable regions.

[0066] For global degradation layer Normalization is performed to obtain the normalized global degradation layer. : ; in, for The minimum value in, for The maximum value in, This is used to prevent the denominator from being zero. After normalization... It is used to represent the degree of fogging, uneven illumination, and overall blurring in the image to be processed, so that large artificial intelligence models can refer to the global brightness degradation state when reconstructing replaceable areas.

[0067] Image to be processed Normalized global degradation layer Effective reconstruction of regional masks and detail locking mask By concatenating channels, the conditional input is obtained. : ; in, This indicates splicing by channel. When When it is an RGB image, It includes 3 image channels, 1 global degradation layer channel, 1 effective reconstruction region channel, and 1 detail locking channel; when When it is a grayscale image, It includes one image channel, one global degradation layer channel, one effective reconstruction region channel, and one detail lock channel.

[0068] The large-scale AI model is a pre-trained latent space reconstruction model with image inpainting capabilities, and its parameters have been determined before executing this method. This step does not change the model parameters. The large-scale AI model includes encoding functions. Potential reconstruction function and decoding function First, process the image. Encode to obtain the initial latent representation : ; set up The size is ,in, For potential representation of height, For the potential representation width, This represents the number of potential channels. For potential spatial locations... Its corresponding image pixel set Defined as: ; in, and These are the height and width of the image to be processed, respectively. This indicates rounding down to the nearest integer.

[0069] The effective reconstruction region mask and detail-locking mask are mapped to the latent space to obtain the latent reconstruction weights. and potential lock mark : , ; in, Represents a set of pixels The number of pixels in the image. This indicates the percentage of reconstructable pixels in the image corresponding to the potential location; This indicates that the potential location covers at least one detail-locked pixel.

[0070] Further, the revised potential reconstruction weights were obtained. : ; This calculation ensures that as long as a potential location covers a detail-locked pixel, the reconstruction weight of that potential location is 0, thereby preventing the same representation unit in the latent space from simultaneously affecting both replaceable and non-replaceable detail regions.

[0071] Large-scale artificial intelligence models based on conditional input Initial latent representation and the corrected potential reconstruction weights Generate candidate latent representations : ; Then, by limiting the update range of the latent space according to the following formula, the constrained latent representation is obtained. : ; in, This indicates element-wise multiplication. When At that time, the potential location was completely preserved. ;when At that time, the potential location receives a latent representation generated by a large AI model according to the proportion of reconstructable pixels. This enables the reconstruction of only the latent space corresponding to replaceable regions that is not locked in detail.

[0072] For constrained potential representations Decode the image to obtain the reconstructed image. : ; Finally, utilize the effective reconstruction area mask. Reconstruct the image With the image to be processed Pixel-level synthesis is performed to obtain candidate images for processing. : ; in, This indicates the image channel number. For grayscale images, For RGB images, Through this synthesis method, candidate images are processed. Only the effectively reconstructed regions come from the large artificial intelligence model; the remaining regions retain the original pixel content of the image to be processed. The candidate image for processing... As input to step S500, it is used to perform structural similarity verification with the image to be processed in the non-replaceable detail region.

[0073] S500: Perform structural similarity verification between the candidate image to be processed and the image to be processed in the non-replaceable detail region. If the verification does not meet the preset conditions, update the detail locking mask in reverse according to the offset result and repeat S400 until the target image to be processed is output.

[0074] In step S500, the candidate reconstructed image obtained by the large artificial intelligence model in step S400 is denoted as... ,in, Indicates the first The second reconstruction and verification process, initially taking The candidate images used in this step are for structural similarity verification, referring to candidate reconstructed images that have not yet undergone final pixel-level synthesis. After passing the verification, the target image is synthesized based on the effective reconstructed region mask. The reason for this setting is that if the non-replaceable detail regions are directly replaced with the original pixels of the image to be processed, the structural similarity of that region will always be equal to 1, making it impossible to determine whether the reconstruction influence of the large artificial intelligence model in the latent space has invaded the non-replaceable detail regions.

[0075] In step S510, the non-replaceable detail region mask obtained in step S300 is used. To determine the range, from candidate processed images and the image to be processed Extract the corresponding detail image patches. If If there are no pixels with a value of 1, it means that the current image has not identified any non-replaceable detail regions, and the current candidate processing result obtained in step S400 is directly output as the target processing image.

[0076] When the image is an RGB image, and Convert to a brightness map, the calculation formula is: , ;in, Indicates pixel position, RGB channel values ​​are all normalized to When the image is a grayscale image, set the brightness map to the corresponding grayscale map.

[0077] Masks for non-replaceable detail areas Perform 8-neighbor connectivity partitioning to obtain a set of non-replaceable detail regions. For any non-replaceable detail region With the region and its outward expansion The range of one pixel is used as the range of detail image blocks. ,in This refers to the detail locking radius determined by the influence range of the directional Gaussian kernel in step S400. (Extension) The basis for using a single pixel is that the high-frequency residual in step S200 may be affected by real details within this range, and using the same radius can cover the reconstruction offset position that may occur at the edge of the detail.

[0078] exist Inside, in pixels Calculate the structural similarity distribution map centered on the center Calculation window For Centered on, with side length as The set of pixels; when the window exceeds the image boundary, only the valid pixels within the image are used. The structural similarity calculation formula is: ; in, and They are respectively and In the window The average value within, and The two are in the window respectively within variance, For both in the window Covariance within; , Since the brightness value has been normalized to The above constants correspond to 1% and 3% of the brightness stabilization, respectively, to prevent the denominator from being 0, while maintaining sensitivity to subtle texture changes.

[0079] In step S520, the location with the lowest structural similarity is determined based on the structural similarity distribution map. For non-replaceable detail regions... The lowest position is determined by the following formula: ; When multiple pixels have the same minimum structural similarity, the pixel farthest from the region center is selected; if multiple pixels still exist, the pixel with the smallest x-coordinate is selected; if the x-coordinates are still the same, the pixel with the smallest y-coordinate is selected. This rule ensures that the lowest position has a unique result.

[0080] The center of the non-replaceable detail area is calculated using the following formula: , ; Detailed offset results include offset vectors and offset distance The calculation formula is: , ; Here, the offset vector represents the direction of the location of the maximum structural change in the candidate image relative to the center of the non-replaceable detail region, and the offset distance represents the degree to which the location of the change deviates from the center of the region.

[0081] In step S530, it is determined whether the structural similarity check meets the preset conditions. For non-replaceable detail regions... To build its extended neighborhood : ; in, Represents pixels arrive The Euclidean distance to the nearest pixel within the neighborhood. If If no valid pixels are found, then the verification result for this non-replaceable detail area is deemed to meet the conditions.

[0082] when When valid pixels exist, calculate the mean structural similarity within the non-replaceable detail region and the mean structural similarity within the extended neighborhood, respectively: , ; If any non-replaceable detail region satisfies: If the structural similarity check does not meet the preset conditions, then the structural similarity check is deemed not to meet the preset conditions; if all non-replaceable detail regions meet the conditions: If the structural similarity check meets the preset conditions, it is determined that the structural similarity check satisfies the preset conditions. These preset conditions are not fixed empirical thresholds, but rather the relative stability relationship between non-replaceable detail regions and their surrounding neighborhoods within the same image. The design principle is that non-replaceable detail regions should have a higher or at least no lower degree of structural preservation than their surrounding reconstructable influence regions; if their structural similarity is lower than that of the surrounding neighborhoods, it indicates that the reconstruction influence has intruded into the true detail regions.

[0083] In step S540, when the structural similarity check does not meet the preset conditions, the detail locking mask is updated in reverse based on the offset result. Here, "reverse update" means feeding back the check result obtained in S500 to step S400 to expand the detail locking range in the next reconstruction, and does not mean expanding in space along the opposite direction of the offset vector.

[0084] For non-replaceable detail areas that do not meet the preset conditions ,like Calculate the unit offset vector: And generate an offset-locked extended region. : ;in, This represents the distance variable along the offset direction. This represents the Euclidean distance. The extended region extends from the center of the non-replaceable detail region along the direction of the lowest structural similarity, and continues beyond that lowest point. Each pixel is used to lock the propagation direction most likely to be affected by potential spatial reconstruction.

[0085] like Then, a circle with a radius of 1 is generated with the center of the non-replaceable detail region as the center. Extended area: ; Incorporate all expanded areas that do not meet the preset conditions into the current detail-locked mask. The updated detail lock mask is obtained. : ; Subsequently The detail-locked mask in step S400 is replaced, the effective reconstruction region mask is recalculated, and a new candidate image is generated. Since each update only adds pixels with a value of 1 in the detail-locked mask and does not reduce already locked pixels, the reconstructable area shrinks successively.

[0086] To ensure the repetitive process can be terminated, the maximum number of repetitions is set. Determine by the following formula: ; in, and These represent the height and width of the image to be processed, respectively. This upper limit is determined by the image's diagonal length and the detail lock radius used for each expansion, meaning that even with incremental expansion along the longest distance of the image, potentially affected areas can be covered within a finite number of expansions. If... If the pre-check meets the preset conditions, the repetition stops; if the condition is met... If the preset conditions are still not met, then the mask of the currently effective reconstruction area will be expanded outwards to include the non-replaceable detail area. All overlapping pixels are set to 0, and the final image compositing is completed.

[0087] Once the verification is satisfied, the candidate images that passed the verification will be processed. With the image to be processed Mask of effective reconstruction area Pixel-level synthesis is performed to obtain the target image. : ; in, This indicates the image channel number. Through this process, the target image is processed using pixels generated by a large AI model only in the valid reconstructed regions that pass the structural similarity check, preserving the original details of the image to be processed in non-replaceable detail regions and within the influence range of its extended locking.

[0088] This invention first acquires the image to be processed, along with the image capture device's posture, instantaneous illumination intensity, and target surface temperature and humidity changes at the same acquisition time. It then establishes a correspondence between the image's inherent brightness degradation information and imaging condition information. Next, it performs multi-scale decomposition on the image to be processed, separating a global degradation layer representing overall fogging, uneven illumination, and blurriness, as well as a local artifact layer and a suspected real texture layer representing local reflections and high-frequency textures. Subsequently, it uses the accompanying imaging data to generate environmental parameter feature vectors and performs correlation matching with the local artifact layer and the suspected real texture layer to determine which areas are more likely to be caused by environmental factors such as illumination, posture, or wet film, and which areas are more likely to belong to real details. Based on this, areas caused by environmental factors are treated as replaceable areas, and real detail areas are constructed as detail-locking masks, constraining the large AI model to reconstruct only within the potential space corresponding to the replaceable areas. Finally, by verifying the structural similarity between the candidate processed image and the original image to be processed within the non-replaceable detail areas, the detail-locking range is reversed until the real details are not weakened or shifted before outputting the target processed image. This improves image clarity while avoiding the large model mistakenly repairing small defects in texture.

[0089] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An image processing method based on a large-scale artificial intelligence model, characterized in that, include: S100, acquire the image to be processed and its accompanying imaging data, the accompanying imaging data including the attitude of the shooting device, the instantaneous supplementary light intensity and the temperature and humidity change value of the target surface; S200, the image to be processed is decomposed into a multi-scale layer to obtain a global degradation layer, a local artifact layer and a suspected real texture layer, and an environmental parameter feature vector is generated based on the imaging accompanying data. S300, perform correlation matching between the environmental parameter feature vector and the local artifact layer and the suspected real texture layer respectively to determine the replaceable region caused by the environment and the non-replaceable detail region that is not related to the environmental disturbance. S400, a detail-locking mask is constructed based on the non-replaceable detail region, and the detail-locking mask, replaceable region and global degradation layer are input into the large artificial intelligence model, so that the large artificial intelligence model only performs image reconstruction in the potential space corresponding to the replaceable region to obtain the candidate processed image; S500: Perform structural similarity verification between the candidate image to be processed and the image to be processed in the non-replaceable detail region. If the verification does not meet the preset conditions, update the detail locking mask in reverse according to the offset result and repeat S400 until the target image to be processed is output.

2. The image processing method based on a large artificial intelligence model according to claim 1, characterized in that, S100 includes: The midpoint between the start and end times of exposure of the image to be processed is determined as the image acquisition time; the shooting device posture and instantaneous fill light intensity are time-aligned according to the image acquisition time; if the sampling time is inconsistent with the image acquisition time, linear interpolation is performed using adjacent sampling values; the target surface temperature and humidity change value is generated based on the difference between the target surface temperature and the reference time and the difference between the equivalent relative humidity of the target surface.

3. The image processing method based on a large artificial intelligence model according to claim 2, characterized in that, S200 includes: The image to be processed is converted into a brightness map, and the directional decomposition angle is determined based on the roll angle and yaw angle in the shooting device's attitude. The directional scaling coefficient is determined based on the pitch angle in the shooting device's attitude, and a multi-scale directional Gaussian kernel is constructed based on the directional decomposition angle and the directional scaling coefficient. The brightness map is convolved scale by scale using the multi-scale directional Gaussian kernel to obtain the final scale smoothing result as a global degradation layer, and a comprehensive high-frequency residual map is obtained from the high-frequency residuals of each scale.

4. The image processing method based on a large artificial intelligence model according to claim 3, characterized in that, The S200 further includes: The fill light influence factor is calculated based on the instantaneous fill light intensity, and the positive brightness abrupt change in the brightness map relative to the global degradation layer is normalized. The fill light influence factor, the normalized positive brightness abrupt change, and the comprehensive high-frequency residual map are multiplied to obtain the artifact scoring map. The local artifact layer is determined based on the artifact scoring map, and the part of the comprehensive high-frequency residual map after removing the local artifact layer is determined as the suspected real texture layer.

5. The image processing method based on a large artificial intelligence model according to claim 4, characterized in that, The environmental parameter feature vector generated based on the accompanying imaging data includes: The attitude of the shooting device is converted into sine and cosine components of roll angle, pitch angle and yaw angle respectively; the instantaneous supplementary light intensity is converted into supplementary light influence factor, and the target surface temperature change value and the target surface equivalent relative humidity change value are normalized respectively; the sine component, cosine component, supplementary light influence factor, normalized temperature and humidity change value and local artifact layer area ratio are combined into environmental parameter feature vector.

6. The image processing method based on a large artificial intelligence model according to claim 5, characterized in that, The S300 includes: Effective residual labeling is performed on the local artifact layer and the suspected real texture layer respectively, and the effective residual labeling results are divided into 8-neighborhood connected components to obtain candidate regions. The average intensity, orientation consistency and region area ratio are extracted for each candidate region to form a region response vector. The region response vector is used as a region-level input for correlation matching with the environmental parameter feature vector.

7. The image processing method based on a large artificial intelligence model according to claim 6, characterized in that, The S300 also includes: Based on the supplementary lighting influence factor, pitch angle component, target surface temperature and humidity change value, and local artifact layer area ratio in the environmental parameter feature vector, a local artifact projection vector is constructed; based on the environmental disturbance relationship opposite to the local artifact projection vector, a texture preservation projection vector is constructed; the region response vector of the same candidate region is matched with the local artifact projection vector and the texture preservation projection vector respectively by cosine projection matching to obtain the local artifact correlation value and the texture preservation correlation value.

8. The image processing method based on a large artificial intelligence model according to claim 7, characterized in that, The identification of replaceable areas caused by environmental factors and non-replaceable detailed areas unrelated to environmental disturbances includes: Calculate the difference between the local artifact correlation value and the texture preservation correlation value of the same candidate region, and assign the difference to the pixels in the candidate region to obtain the environment attribution map; determine the pixels in the environment attribution map with a difference greater than 0 and located within the effective residual marker of the local artifact layer as replaceable regions; determine the pixels in the environment attribution map with a difference less than 0 and located within the effective residual marker of the suspected real texture layer as non-replaceable detail regions.

9. The image processing method based on a large artificial intelligence model according to claim 8, characterized in that, The S400 includes: Based on the maximum truncation influence range of the multi-scale directional Gaussian kernel at the first scale, the non-replaceable detail region is expanded to obtain a detail-locking mask; pixels overlapping with the detail-locking mask in the replaceable region are removed to obtain the effective reconstruction region; the image to be processed, the global degradation layer, the effective reconstruction region, and the detail-locking mask are input together into the large artificial intelligence model, so that the large artificial intelligence model generates candidate images for processing in the latent space corresponding to the effective reconstruction region.

10. The image processing method based on a large artificial intelligence model according to claim 9, characterized in that, The S500 includes: Within the non-replaceable detail region and its extended neighborhood, a structural similarity distribution map between the candidate image to be processed and the image to be processed is calculated. The position of the lowest structural similarity is determined based on the structural similarity distribution map, and the offset of the lowest structural similarity position relative to the center of the non-replaceable detail region is calculated. When the structural similarity within the non-replaceable detail region is lower than the structural similarity of its extended neighborhood, the detail locking mask is expanded based on the offset result, and S400 is repeated until the structural similarity verification meets the conditions, at which point the target image to be processed is output.

Images