Image fusion visual detection positioning method

Abstract

The application discloses an image fusion visual detection positioning method, and belongs to the field of industrial visual detection. The method first synchronously acquires multi-band reflection signals of a workpiece covered with a light-transmitting refractive medium, and performs radiation calibration; secondly, based on the difference in spectral attenuation characteristics of different bands, the absolute physical thickness spatial distribution of the medium is calculated by inversion; subsequently, the thickness distribution is used to solve the optical path distortion vector field in combination with the dispersion refractive parameter, and the original signal is compensated by reverse projection, so that the physical pixel deviation caused by the medium refraction is completely eliminated; finally, the transmittance adaptive fusion model is constructed with the thickness distribution as a variable, the real physical edge is extracted for high-precision spatial positioning, and the thickness limit is linked to perform an out-of-tolerance early warning. The application eliminates the positioning deviation and false edge interference caused by the oil film refraction from the physical optical level, and greatly improves the precision and stability of industrial detection under complex working conditions.

Description

Technical Field

[0001] This invention belongs to the field of industrial visual inspection technology, and in particular relates to an image fusion visual inspection and positioning method. Background Technology

[0002] In the textile machinery manufacturing industry, the machining precision of core components such as rollers, rings, and travelers directly determines the quality of yarn spinning. Image-based visual inspection technology is widely used to achieve high-precision dimensional measurement and defect detection. However, on actual industrial production lines, the surfaces of freshly machined metal parts are usually coated with a layer of anti-oxidation oil film for protection against oxidation.

[0003] This rust-preventive oil film, acting as a light-transmitting and refractive medium, forms a liquid film of varying thickness on the workpiece surface. When the light source of the vision inspection system shines on the workpiece surface, severe physical refraction and dispersion occur as the light penetrates the oil film and is reflected back to the camera from the underlying metal. Due to the randomness of the oil film thickness distribution, this refraction causes the edge pixels received on the camera target surface to deviate from their true physical positions, producing a microscopic "lens distortion" effect; at the same time, the edges of the oil film are prone to aggregate, forming high-contrast false edges.

[0004] Traditional image fusion or visual positioning techniques typically only involve grayscale transformations at the software algorithm level, such as histogram equalization, Gaussian filtering, or simple edge sharpening. These known techniques cannot identify and eliminate optical path offsets caused by the physical refraction of the medium, nor can they distinguish between the pseudo-edges of oil films and the true edges of metal from an optical mechanism perspective. This leads to problems such as positional shifts, dimensional misjudgments, and misreporting of oil stains as surface defects when performing sub-pixel positioning with oil films, severely limiting the accuracy and yield of online full inspection of textile machinery parts. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an image fusion visual detection and localization method that solves the aforementioned problems.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an image fusion visual detection and localization method, comprising:

[0007] S10. Acquire multi-band reflection signals of the test component covered by the light-transmitting and refractive medium, perform radiometric calibration based on the reference target, and resolve the calibration texture matrix and calibration penetration matrix.

[0008] S20. Based on the spectral attenuation characteristics of the light-transmitting and refractive medium, and combined with the calibration texture matrix and calibration transmission matrix, the medium thickness distribution model is calculated by inversion.

[0009] S30. Based on the medium thickness distribution model and the interface dispersion and refraction parameters, calculate the dual-channel optical path distortion vector field;

[0010] S40. The dual-channel optical path distortion vector field is used to perform back projection correction on the calibration texture matrix and calibration penetration matrix respectively to obtain the texture compensation matrix and penetration compensation matrix under a unified reference coordinate system.

[0011] S50. Construct a transmittance adaptive fusion model based on the medium thickness distribution model, fuse the texture compensation matrix and the penetration compensation matrix to calculate the target positioning coordinates, and combine the medium thickness distribution model to perform positioning output and over-limit warning.

[0012] Based on the above technical solutions, the present invention also provides the following optional technical solutions:

[0013] Further technical solutions: The spectral attenuation characteristics mentioned in step S20 are specifically: the single-pass physical absorption attenuation coefficient of the light-transmitting refractive medium for visible light and the single-pass physical absorption attenuation coefficient for near-infrared light; The interface dispersion and refractive parameters mentioned in step S30 are specifically: the absolute refractive index of the light-transmitting refractive medium for the visible light band and the near-infrared band, the physical resolution coefficient of the imaging plane, and the spatial incident angle between the imaging principal optical axis and the surface normal of the component under test.

[0014] Further technical solution: The radiation calibration based on the reference target described in step S10 has the following mapping logic:

[0015] Using the pre-calibrated system dark current noise floor intensity of each band as the system zero-position reference, a calibration response scale factor is constructed based on the average calibration intensity of the corresponding band and the known inherent physical reflectivity. The original reflection intensity collected by the photoelectric sensor at each pixel coordinate is mapped by a linear bias correction operator based on the system zero-position reference and the calibration response scale factor, and the true relative reflection intensity of each band at the corresponding pixel coordinate is output after calibration.

[0016] Further technical solution: The solution logic of the medium thickness distribution model in step S20 is as follows:

[0017] Based on the true relative reflectance intensity of the calibration penetration matrix and the inherent reflectance of near-infrared light, and the true relative reflectance intensity of the calibration texture matrix and the inherent reflectance of visible light, a cross-band reflectance feature is constructed at the corresponding pixel coordinates.

[0018] A differential absorption coefficient is constructed based on the single-pass physical absorption attenuation coefficient of the light-transmitting and refractive medium for visible and near-infrared light.

[0019] The cross-band reflectance characteristic is input into the logarithmic attenuation mapping model, and the initial physical thickness is calculated by combining the differential absorption coefficient and the reflection round-trip transmission factor. The initial physical thickness is then input into the non-negative boundary truncation operator to remove the negative distortion caused by the shot noise at the bottom layer of the sensor, and finally the absolute physical thickness of the medium at the corresponding pixel coordinates is output.

[0020] Further technical solution: The solution logic for the dual-channel optical path distortion vector field in step S30 is as follows:

[0021] Using the spatial incident angle between the imaging principal optical axis and the surface normal of the component under test, and the absolute physical refractive index in the corresponding band as input parameters, a dispersive refraction spatial operator characterizing the cross-medium deflection characteristics of light is constructed.

[0022] The absolute offset amplitude is obtained by nonlinearly distorting the quotient of the absolute physical thickness of the medium and the physical resolution coefficient of the imaging plane using the dispersive refraction spatial operator.

[0023] Simultaneously, a direction sign determination function is constructed based on the relative position of the spatial incident angle on both sides of the surface normal; the direction sign determination function is used to assign a vector direction to the absolute offset amplitude, and the apparent pixel offset of optical path distortion at the corresponding pixel coordinates of each band is output independently.

[0024] Further technical solution: The correction logic for projecting the dual-channel signal onto the unified reference coordinate system in step S40 is as follows:

[0025] Extract the absolute azimuth angle of the imaging back ray corresponding to the pixel on the projection plane as input, and construct an orthogonal projection decomposition operator;

[0026] The orthogonal projection decomposition operator is used to decompose the apparent pixel offset vector of optical path distortion into horizontal distortion compensation components and vertical distortion compensation components.

[0027] Based on the discrete horizontal and vertical coordinates of the original sensor acquisition matrix, reverse translation resampling mapping is performed by subtracting the corresponding horizontal distortion compensation components and vertical distortion compensation components respectively, thereby obtaining the true horizontal and true vertical coordinates mapped to a unified reference coordinate system.

[0028] Further technical solution: The logic for executing the positioning output and over-limit warning in step S50 is as follows:

[0029] First, the absolute physical thickness of the medium on the unified physical grid is extracted. Then, combined with the spatial incident angle and the absolute physical refractive index of the medium for the visible light band, the true oblique optical path length of light propagating unidirectionally within the medium is calculated through geometric deflection path mapping.

[0030] Next, using the actual oblique optical path length and the single-pass physical absorption attenuation coefficient of the medium for visible light as independent variables, an exponential attenuation weight allocation function is constructed. This function is used to perform complementary weighted fusion calculation on the texture compensation matrix and the penetration compensation matrix aligned to a unified reference coordinate system, generating a unified mesh fusion signal intensity output by the transmittance adaptive fusion model.

[0031] Subsequently, within the discrete pixel connected set of the feature to be tested in a unified reference coordinate system, the gray-scale centroid spatial positioning operator is executed with the unified grid fusion signal intensity as the feature distribution weight to calculate the target positioning horizontal and vertical coordinates.

[0032] Finally, the extreme value traversal interception logic is executed to traverse the connected set to extract the thickness limit value. When it is determined that the thickness limit value meets the set safety threshold, the horizontal and vertical coordinates of the target positioning are allowed to be output; otherwise, the output is blocked and an optical distortion over-limit warning is triggered.

[0033] An image fusion visual inspection and positioning system for textile machinery parts includes:

[0034] The multimodal acquisition and calibration module is used to acquire multi-band reflection signals of the test component covering the light-transmitting and refractive medium, perform radiometric calibration based on the reference target, and parse out the calibration texture matrix and calibration penetration matrix.

[0035] The medium distribution inversion module is used to invert and calculate the medium thickness distribution model based on the spectral attenuation characteristics of the light-transmitting and refractive medium, combined with the calibration texture matrix and the calibration transmission matrix.

[0036] The optical path distortion calculation module is used to calculate the dual-channel optical path distortion vector field based on the medium thickness distribution model and the interface dispersion and refraction parameters.

[0037] The coordinate projection compensation module is used to perform back projection correction on the calibration texture matrix and calibration penetration matrix respectively using the dual-channel optical path distortion vector field to obtain the texture compensation matrix and penetration compensation matrix under a unified reference coordinate system.

[0038] The fusion early warning module is used to construct a transmittance adaptive fusion model based on the medium thickness distribution model, fuse the texture compensation matrix and the penetration compensation matrix to calculate the target positioning coordinates, and perform positioning output and over-limit early warning in combination with the medium thickness distribution model.

[0039] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the above-described image fusion visual detection and localization method.

[0040] This invention provides an image fusion visual detection and localization method, which has the following advantages compared with the prior art:

[0041] 1. This invention abandons the traditional pure digital image processing approach and innovatively utilizes the physical absorption differences of different optical bands in a medium to reverse-engineer the absolute thickness distribution of the light-transmitting medium on the workpiece surface. Based on this thickness distribution and optical dispersion parameters, the optical path deflection vector of each pixel is directly calculated, and refraction distortion is eliminated through reverse resampling mapping. This mechanism corrects the positional offset caused by the oil film from the root of physical optics, achieving accurate restoration of the real physical edge at the micrometer level;

[0042] 2. Existing multimodal fusion methods often employ globally fixed weights or local weights based on image contrast, which cannot adapt to drastic changes in oil film morphology. This invention directly uses the inverted physical thickness of the medium as the sole independent variable for fusion weight allocation. It automatically increases the weight of bands with strong penetration characteristics in thicker medium regions and automatically increases the weight of bands with high resolution characteristics in thinner medium regions. This fusion logic driven by "real physical quantities" completely eliminates light and shadow fluctuations caused by uneven oil film thickness and effectively suppresses the generation of false edges.

[0043] 3. This invention does not merely use medium thickness for coordinate compensation, but further transforms it into an early warning indicator for industrial sites. While performing grayscale centroid spatial positioning, the system simultaneously extracts the extreme thickness values ​​within the feature region. When the oil film thickness exceeds the physical limit of the optical compensation algorithm, the system can proactively block potentially distorted positioning output and trigger an early warning. This mechanism effectively prevents forced positioning misjudgments under extreme conditions, ensuring the absolute reliability of the detection system in complex production environments. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the process of the present invention.

[0045] Figure 2 This is the data flow mapping diagram of the algorithm of this invention.

[0046] Figure 3 This is a structural diagram of an image fusion visual detection and localization system. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0049] Please see Figures 1 to 3 An image fusion visual detection and localization method, provided in one embodiment of the present invention, includes:

[0050] S10. Acquire multi-band reflection signals of the test component covered by the light-transmitting and refractive medium, perform radiometric calibration based on the reference target, and resolve the calibration texture matrix and calibration penetration matrix.

[0051] S20. Based on the spectral attenuation characteristics of the light-transmitting and refractive medium, and combined with the calibration texture matrix and calibration transmission matrix, the medium thickness distribution model is calculated by inversion.

[0052] S30. Based on the medium thickness distribution model and the interface dispersion and refraction parameters, calculate the dual-channel optical path distortion vector field;

[0053] S40. The dual-channel optical path distortion vector field is used to perform back projection correction on the calibration texture matrix and calibration penetration matrix respectively to obtain the texture compensation matrix and penetration compensation matrix under a unified reference coordinate system.

[0054] S50. Construct a transmittance adaptive fusion model based on the medium thickness distribution model, fuse the texture compensation matrix and the penetration compensation matrix to calculate the target positioning coordinates, and combine the medium thickness distribution model to perform positioning output and over-limit warning.

[0055] The spectral attenuation characteristics mentioned in step S20 are specifically: the single-pass physical absorption attenuation coefficient of the light-transmitting refractive medium for visible light and the single-pass physical absorption attenuation coefficient for near-infrared light; the interface dispersion and refractive parameters mentioned in step S30 are specifically: the absolute refractive index of the light-transmitting refractive medium for the visible light band and the near-infrared band, the physical resolution coefficient of the imaging plane, and the spatial incident angle between the imaging principal optical axis and the surface normal of the component under test.

[0056] For ease of understanding, the following explains some key terms in this embodiment:

[0057] A light-transmitting and refractive medium refers to a substance that allows light to pass through and be refracted. In this embodiment, it specifically refers to a liquid film such as an anti-rust oil film covering the surface of textile machinery parts. This medium has an uneven thickness distribution and different refractive and absorption characteristics for different wavelengths of light, which is the main cause of positioning errors in visual inspection.

[0058] Multi-band reflection signals refer to optical signals reflected from the surface of the component under test and collected at different wavelengths (such as visible light and near-infrared light). By collecting multi-band signals, physical information about the workpiece surface under different spectral responses can be obtained, providing a data foundation for subsequent media characteristic analysis and image fusion.

[0059] Radiometric calibration is an image processing technique designed to correct the raw light signal acquired by an image sensor, eliminating the effects of sensor response inhomogeneities, dark current noise, and variations in illumination conditions. By comparing the raw digital quantized value with a benchmark target of known reflectivity, it converts the original value into a physically meaningful true relative reflectance, ensuring the accuracy of subsequent calculations.

[0060] The calibration texture matrix and calibration penetration matrix are image data matrices obtained after radiometric calibration, corresponding to different wavelength bands. The calibration texture matrix typically refers to image data in the visible light band, mainly reflecting macroscopic features such as color and texture of the workpiece surface. The calibration penetration matrix typically refers to image data in the near-infrared light band. Because near-infrared light has strong penetrating power through certain media, this matrix can reflect more information about the workpiece beneath the medium.

[0061] A medium thickness distribution model is a mathematical model, calculated to describe the thickness distribution of the light-transmitting and refractive medium at various points on the surface of the component under test. This model is crucial for understanding and correcting visual distortions caused by the medium, and its accuracy directly affects subsequent optical path distortion calculations and image correction effects.

[0062] The dual-channel optical path distortion vector field refers to the apparent pixel offset calculated separately for the visible and near-infrared light bands due to light passing through the translucent and refractive medium. This vector field describes, in vector form, the degree and direction to which each pixel in the image deviates from its true physical position due to refraction and dispersion effects, and serves as the basis for backprojection correction.

[0063] A unified reference coordinate system is a defined and calibrated common reference coordinate system used to align image data from different bands and correct distortions. Mapping all image data and calculation results to this unified coordinate system can eliminate geometric deviations caused by different sensors or optical paths, ensuring the accuracy of image fusion and positioning.

[0064] The transmittance adaptive fusion model is an image fusion algorithm that dynamically adjusts the fusion weights based on the thickness distribution of the medium. This model utilizes the differences in transmittance of the medium to different wavelengths of light to perform weighted fusion of multi-band images after distortion correction, aiming to highlight the true features of the workpiece and suppress medium artifacts, thereby generating a clearer and more accurate fused image.

[0065] Positioning output and over-limit warning refer to the system's judgment of the medium thickness based on a preset safety tolerance threshold after calculating the target's positioning coordinates. If the medium thickness exceeds the threshold, the positioning output is blocked and a warning is triggered to prevent unreliable positioning results from being output when the medium distortion is too large, thus ensuring the reliability and safety of the detection.

[0066] The following example will provide a more detailed explanation of the above technical solution:

[0067] Imagine a textile machinery parts production line where visual inspection and positioning of freshly processed roller workpieces is required. The roller surface is coated with a layer of unevenly thick anti-rust oil film, which serves as the light-transmitting and refractive medium.

[0068] First, in step S10, the detection system activates a multi-band camera, such as a visible light camera and a near-infrared camera, to photograph the roller workpiece covered with an oil film, acquiring its multi-band reflection signals. Before actually photographing the workpiece, the system first photographs a standard gray board with known reflectivity and performs radiometric calibration based on the known reflectivity of the gray board and the original grayscale values ​​acquired by the camera. Through this calibration process, the original digital quantization values ​​are converted into physically meaningful real relative reflection intensities. Subsequently, the calibration data in the visible light band is resolved into a calibration texture matrix, while the calibration data in the near-infrared band is resolved into a calibration penetration matrix. For example, the visible light image clearly shows the reflection and texture of the oil film surface, while the near-infrared image can partially penetrate the oil film, showing the outline of the roller metal substrate.

[0069] Next, in step S20, the system uses the pre-determined one-way physical absorption attenuation coefficients of the oil film for visible and near-infrared light, combined with the acquired calibration texture matrix and calibration penetration matrix, to invert and calculate the dielectric thickness distribution model of the oil film on the roller surface. This model represents the actual physical thickness of the oil film at various points on the roller surface in the form of a two-dimensional image. For example, by comparing the relative attenuation differences of visible and near-infrared light in the oil film, the distribution of thicker and thinner regions of the oil film can be inferred.

[0070] Subsequently, in step S30, based on the medium thickness distribution model and preset interface dispersion and refraction parameters (including the absolute refractive index of the oil film for visible and near-infrared light, the physical resolution coefficient of the imaging plane, and the spatial incident angle between the camera's principal optical axis and the normal to the roller surface), the system calculates the dual-channel optical path distortion vector field. This vector field accurately describes the offset and direction of the imaging position of visible and near-infrared light on the camera target surface relative to their actual physical position after penetrating the oil film due to the refraction and dispersion effects of the oil film. For example, in areas with thicker oil films, the value of the optical path distortion vector field will be larger, indicating a more significant pixel offset.

[0071] Further, in step S40, the system utilizes the dual-channel optical path distortion vector field to perform back-projection correction on the calibration texture matrix and calibration penetration matrix, respectively. This correction process remaps each pixel that has shifted due to oil film refraction back to its true physical position in a unified reference coordinate system. In this way, the geometric distortion caused by the oil film is eliminated, enabling precise spatial alignment of the visible light and near-infrared light images. The corrected visible light image forms the texture compensation matrix, and the corrected near-infrared image forms the penetration compensation matrix. For example, roller edges that were originally blurred or misaligned due to oil film refraction become clear and accurately positioned after correction.

[0072] Finally, in step S50, the system constructs a transmittance adaptive fusion model based on the medium thickness distribution model. This model dynamically adjusts the fusion weights of the texture compensation matrix and the penetration compensation matrix according to the local thickness of the oil film. For example, in areas with thinner oil films, the weight of the visible light image may be higher to preserve rich surface textures; while in areas with thicker oil films, the weight of the near-infrared image may be higher to utilize its penetrating power to reveal the underlying metal features. Through this adaptive fusion, a fused image is generated that preserves the surface texture information of the workpiece while effectively suppressing oil film artifacts and distortions. Subsequently, the system calculates the target positioning coordinates of the roller on this fused image, for example, by determining the center position of the roller using the gray-scale centroid method. Simultaneously, the system traverses the oil film thickness distribution model to find the maximum thickness value. If the maximum thickness value exceeds the system's preset safety tolerance threshold, the positioning output stream is blocked and an optical distortion over-limit warning is triggered to avoid misjudgment when the positioning result is unreliable due to excessively thick oil film. This closed-loop logic ensures the positioning accuracy and reliability even in the presence of an oil film.

[0073] Based on the above examples, the image fusion visual detection and localization method proposed in this embodiment demonstrates significant technical contributions. In traditional textile machinery component inspection, as described in the background section, when a rust-preventive oil film is attached to the roller surface, existing image fusion or visual localization techniques often only involve grayscale transformations at the software algorithm level, such as histogram equalization or simple edge sharpening. These methods cannot fundamentally solve the problems of optical path shift and dispersion caused by the oil film as a light-transmitting and refractive medium.

[0074] In contrast, this embodiment introduces the acquisition and radiometric calibration of multi-band reflection signals (step S10), providing accurate raw data for subsequent physical model establishment. More importantly, this embodiment can inversely calculate the medium thickness distribution model (step S20), enabling the system to quantify the actual thickness variation of the oil film on the workpiece surface. For example, in the roller detection example above, the system no longer merely sees a blurry image, but can accurately determine the thickness of the oil film at various points on the roller surface.

[0075] Furthermore, this embodiment calculates the dual-channel optical path distortion vector field based on the medium thickness distribution model and interface dispersion and refraction parameters (step S30), and uses this vector field to perform back-projection correction on the calibration texture matrix and calibration penetration matrix (step S40). This series of steps directly solves the "lens distortion" effect caused by the oil film at the optical physics level, accurately correcting pixels that have deviated from their true positions due to refraction and dispersion back to the unified reference coordinate system. This contrasts sharply with traditional methods that only enhance the image without eliminating physical distortion. In the roller example, this means that the roller edges that were originally misaligned due to the oil film can accurately reflect their true physical positions after correction, thus avoiding size misjudgment.

[0076] Furthermore, this embodiment constructs a transmittance adaptive fusion model based on the medium thickness distribution model to fuse the corrected image (step S50). This fusion method is not a simple image superposition, but rather dynamically adjusts the weights of different waveband images according to the local thickness of the oil film, thereby effectively suppressing oil film artifacts and highlighting the true features of the workpiece in the fused image. For example, in areas with thicker oil films, the system relies more on the penetration information of near-infrared images, while in areas with thinner oil films, it utilizes the rich texture of visible light images. Finally, the positioning output and over-limit warning are executed in conjunction with the medium thickness distribution model, ensuring timely warning and blocking of unreliable positioning results when the oil film distortion is too large. This fundamentally solves the problems of traditional methods misreporting oil stains as surface defects and the system's tendency to shift positions when performing sub-pixel positioning with an oil film, significantly improving the accuracy and yield of online full inspection of textile machinery parts.

[0077] Preferably, the radiometric calibration based on the reference target in step S10 has the following mapping logic:

[0078] Using the pre-calibrated system dark current noise floor intensity of each band as the system zero reference, a calibration response scale factor is constructed using the average calibration intensity of the corresponding band and the known inherent physical reflectivity. The original reflection intensity collected by the photoelectric sensor at each pixel coordinate is mapped by a linear bias correction operator based on the system zero reference and the calibration response scale factor, and the true relative reflection intensity of each band at the corresponding pixel coordinate is output after calibration.

[0079] Specifically, the specific calculation formula for the radiation calibration based on the reference target in step S10 is as follows:

[0080]

[0081] In the formula, Indicates band channel In the discrete coordinates of the sensor The true relative reflection intensity after calibration (when the band channel) The corresponding output is the calibration texture matrix. When the band channel The corresponding output is the calibration penetration matrix. , Indicates band channel In coordinates The raw reflection intensity collected by the photoelectric sensor (in digital quantization units) ), Indicates band channel The pre-calibrated system dark current noise floor intensity is the system zero-point reference (unit: ), Indicates band channel The average calibration intensity collected from a standard reference target under the same illumination environment, in units of... ), This indicates that the standard reference target is in the band channel. The known inherent physical reflectivity of the given area This indicates the calibration response scaling factor. Linear bias correction operator.

[0082] in, Indicates band channel In the discrete coordinates of the sensor The true relative reflection intensity at this calibrated location is the result of correction and normalization, accurately reflecting the reflection characteristics of an object in this band and providing reliable input for subsequent image processing and analysis. For example, when the band channel... Visible light band ( When ), its output is the calibrated texture matrix. When the band channel Near-infrared band ( When ), its output is the calibration penetration matrix. . Indicates band channel In coordinates The raw reflection intensity collected by the photoelectric sensor is a digital quantization unit directly output by the sensor. The value contains target reflection information as well as various noises and system biases. Indicates band channel The pre-calibrated system dark current noise floor intensity, i.e. the system zero-point reference, is the signal generated by the sensor when there is no light input. It is superimposed on the original reflection intensity, so it needs to be pre-measured and subtracted from the original signal to eliminate its influence. This value can be obtained by collecting sensor data in a completely dark environment and taking the average value, or by setting it using calibration data provided by the sensor manufacturer. Indicates band channel The average calibration intensity is collected from a standard reference target under the same lighting conditions. The standard reference target is a reference object with a known reflectivity. By measuring its response under specific lighting conditions, it can be used to calibrate the sensor's sensitivity to changes in lighting. This value is usually obtained by taking the average of multiple measurements on a standard white or gray board under the same lighting conditions as the object under test, or by using a device such as an integrating sphere for precise measurement. This indicates that the standard reference target is in the band channel. The known inherent physical reflectance of a target at a specific wavelength is a physical property inherent to a standard reference target at a specific wavelength, and is usually provided by the target manufacturer or measured by a high-precision spectrometer. This represents the calibration response scaling factor, which is used to convert the sensor's digital quantization output into a physically meaningful reflectance value. It takes into account the sensor's response characteristics and illumination conditions. This represents the linear bias correction operator, which achieves linear correction of the original sensor data by subtracting the dark current, dividing by the reference intensity, and multiplying by the reference reflectivity, thereby obtaining an accurate true relative reflection intensity.

[0083] The solution in this application accurately processes the multi-band reflection signal obtained in step S10 by introducing the specific radiometric calibration calculation formula mentioned above. This formula first calculates the original reflection intensity... Subtract the system dark current noise level This effectively eliminated the inherent noise of the sensor under no-light conditions, ensuring the zero-point accuracy of the signal. Subsequently, the corrected signal was divided by the average calibration intensity acquired from a standard reference target under the same lighting conditions. And multiplied by the standard reference target in the band channel The known intrinsic physical reflectivity This achieves compensation for sensor response nonlinearity and ambient light fluctuations. This normalization process, based on known physical reflectivity, enables different frequency band channels... Sensor output below It can accurately represent the true relative reflection intensity of the tested component in the corresponding wavelength band. The resulting calibration texture matrix... And calibration penetration matrix The data quality and accuracy have been significantly improved, providing a solid foundation for the accurate inversion of the medium thickness distribution model in the subsequent step S20, thereby ensuring the reliability and accuracy of the entire image fusion visual detection and positioning method.

[0084] The following is a specific example. As a concrete implementation, when inspecting textile machinery parts, an industrial camera equipped with visible light and near-infrared filters can be used as a photoelectric sensor. Before radiometric calibration, multiple frames are first acquired in complete darkness, and the average pixel value of each band channel is calculated as the system's dark current noise floor intensity. Next, under the same lighting conditions as the actual test, a standard gray card with known reflectivity (as the standard reference target) was placed at the test location, and multiple frames of images were acquired. The average pixel value of each band channel was calculated as the average calibration intensity. The known inherent physical reflectances of the standard gray board in the visible and near-infrared bands are respectively... and When the original reflection intensity of the component under test is actually collected. Then, radiometric calibration can be performed using the above formula. For example, for the visible light band, the sensor's coordinates... The original intensity collected at the location is The true relative reflection intensity after calibration Will pass The calculation yields the result. Similarly, for the near-infrared band, the calculation can be performed. These have been precisely calibrated and The calibration texture matrix and the calibration penetration matrix will be used as inputs into the subsequent calculation of the medium thickness distribution model.

[0085] Through the above technical solution, this application can effectively solve the problems of system dark current, ambient light fluctuations, and sensor response nonlinearity in the original reflection signal. By introducing a precise radiometric calibration calculation formula, the original sensor data is systematically corrected, eliminating non-physical deviations. This ensures that the acquired calibration texture matrix and calibration penetration matrix can truly and accurately reflect the physical reflection characteristics of the component under test. This high-precision input data significantly improves the accuracy of subsequent medium thickness distribution model inversion, thereby ensuring the calculation accuracy of the dual-channel optical path distortion vector field. Ultimately, this makes the output of target positioning coordinates more accurate and reliable, and can trigger optical distortion over-limit warnings more promptly and accurately, thus improving the robustness and reliability of the entire detection and positioning method.

[0086] Preferably, the solution logic of the medium thickness distribution model in step S20 is as follows:

[0087] Based on the true relative reflectance intensity of the calibration penetration matrix and the inherent reflectance of near-infrared light, and the true relative reflectance intensity of the calibration texture matrix and the inherent reflectance of visible light, a cross-band reflectance feature is constructed at the corresponding pixel coordinates.

[0088] A differential absorption coefficient is constructed based on the single-pass physical absorption attenuation coefficient of the light-transmitting and refractive medium for visible and near-infrared light.

[0089] The cross-band reflectance characteristic is input into the logarithmic attenuation mapping model, and the initial physical thickness is calculated by combining the differential absorption coefficient and the reflection round-trip transmission factor. The initial physical thickness is then input into the non-negative boundary truncation operator to remove the negative distortion caused by the shot noise at the bottom layer of the sensor, and finally the absolute physical thickness of the medium at the corresponding pixel coordinates is output.

[0090] Specifically, the specific calculation formula for the medium thickness distribution model in step S20 is as follows:

[0091]

[0092] In the formula, Representing coordinates Absolute physical thickness of the medium after correction by the nonnegative truncation operator (unit: ), This is a non-negative truncation operator (used to forcibly remove the non-physically meaningful negative thickness caused by shot noise at the sensor's bottom layer, preventing logical reversal in the subsequent transmission model), and constant 2 is the round-trip transmission factor corresponding to the reflective optical path. and These represent the one-way physical absorption attenuation coefficients of the light-transmitting and refractive media for visible and near-infrared light, respectively, with units of [missing information]. ), and These represent the calibration penetration matrix and calibration texture matrix in coordinates, respectively. The true relative reflection intensity at that location, and These represent the inherent reflectivity of the physical metal substrate of the component to visible light and near-infrared light, respectively. This represents the characteristic quantity of cross-band reflectance. Represents the differential absorption coefficient. This represents the non-negative boundary truncation operator.

[0093] Among them, the medium thickness distribution model This model is used to quantify the spatial physical thickness distribution of the light-transmitting and refractive medium covering the surface of the component under test. Its importance lies in the fact that the thickness of the medium is a key parameter affecting the propagation path and attenuation of light within it. Accurately obtaining its spatial distribution is crucial for subsequent precise correction of optical distortion and high-precision positioning. This model is represented in the form of a two-dimensional image, with each pixel... This corresponds to a specific thickness value, thus forming a continuous thickness field. Non-negative truncation operator. To ensure that the calculated medium thickness is always non-negative, in actual measurements, due to the inherent shot noise of the photoelectric sensor or other measurement errors, the thickness value obtained by logarithmic inversion may have a small negative value. These negative values ​​do not physically exist. If not corrected, they will lead to logical reversal or calculation errors in the subsequent transmission model. Therefore, this operator effectively eliminates non-physical negative values ​​by forcing all calculation results less than zero to zero, ensuring the physical rationality of the thickness data and the stability of subsequent calculations. The constant 2, as the round-trip transmission factor corresponding to the reflective optical path, reflects the propagation characteristics of light in the reflective detection optical path. In reflective detection, the incident light first passes through the light-transmitting refractive medium to reach the surface of the component to be measured, then reflects from the surface of the component, and passes through the medium again to return to the sensor. Therefore, the light undergoes two one-way propagation in the medium, i.e., a "round-trip" process. This factor ensures that the thickness calculation formula can accurately reflect the total optical path actually traversed by the light in the medium. and These two coefficients represent the one-way physical absorption attenuation coefficients of the light-transmitting and refractive medium for visible and near-infrared light, respectively. They characterize the inherent absorption capacity of the light-transmitting and refractive medium for the visible and near-infrared light bands. They are physical properties of the medium material itself and represent the degree of attenuation of light intensity per unit length of the medium for the corresponding wavelength band. Since the absorption and attenuation characteristics of light in different wavelength bands are different in the medium, the difference between these two coefficients (i.e., differential absorption) is the physical basis for realizing thickness inversion calculation based on spectral differences. and These represent the calibration penetration matrix and calibration texture matrix in coordinates, respectively. The two matrices, representing the true relative reflection intensity at a given location and obtained after radiometric calibration, are image data reflecting the surface reflection characteristics of the tested component under uniform illumination conditions. Among them, It mainly reflects the surface texture information of components under the medium, while Because near-infrared light has stronger penetrability to many media, it can better reflect the true reflection intensity of components under the medium. It is the direct input data for thickness inversion calculation, providing information on the intensity of light after penetrating the medium at different wavelengths. and These two parameters represent the inherent reflectivity of the physical metal substrate of the component to visible and near-infrared light, respectively. They represent the inherent reflectivity of the surface of the component (e.g., the metal substrate) to visible and near-infrared light when there is no medium covering it. They are physical properties of the substrate material and are used as reference values ​​to eliminate the influence of substrate reflectivity differences on thickness calculation, allowing thickness inversion to focus more on the attenuation effect of the medium itself. (Cross-band reflectivity characteristic quantity) By performing ratio calculations on the calibrated reflectance intensities of different wavelength bands and normalizing them in conjunction with the inherent reflectivity of the substrate, spectral difference information directly related to the medium thickness can be effectively extracted. This ratio calculation can mitigate the influence of non-medium factors such as uneven illumination and sensor gain, highlighting the differential attenuation effect of the medium on light of different wavelength bands, making it a sensitive indicator for inverting medium thickness. Differential absorption coefficient It is a quantitative representation of the difference in absorption and attenuation capabilities of visible light and near-infrared light in a light-transmitting and refractive medium. It is the core parameter in the medium thickness inversion formula and directly determines the logarithmic relationship between the reflection intensity ratio and the medium thickness. Only when the medium has significant differential absorption of light in these two bands can the thickness of the medium be accurately inverted by measuring the reflection intensity ratio.

[0094] This application's solution achieves accurate inversion of the thickness of the light-transmitting and refractive medium by introducing a specific calculation formula for the medium thickness distribution model. This method first utilizes the calibration texture matrix of the visible light band obtained through radiometric calibration. Calibration penetration matrix in the near-infrared band Combining the inherent reflectivity of the physical metal substrate of the components to visible and near-infrared light and Cross-band reflectivity characteristics were constructed. This characteristic quantity can effectively reflect the relative differences in light intensity attenuation at different wavelengths after light passes through the medium. Subsequently, the single-pass physical absorption attenuation coefficients of the transmissive and refracting medium for visible and near-infrared light are used. and The resulting differential absorption coefficient The aforementioned cross-band reflectivity characteristics are converted into physical quantities proportional to the medium thickness through logarithmic operations. Considering that light rays pass through the medium twice in a reflective optical path, a constant of 2 is introduced as a correction for the reflective optical path's round-trip transmission factor. Finally, to eliminate the non-physically meaningful negative thickness caused by shot noise at the sensor's bottom layer, a non-negative truncation operator is employed. The calculation results are corrected to ensure that the output medium thickness distribution model $h(x,y)$ is physically reasonable and valid. Through the above calculation process, the thickness distribution of the medium on the surface of the component under test can be accurately quantified, providing accurate input for the subsequent calculation of the dual-channel optical path distortion vector field based on the thickness distribution, thereby ensuring the overall accuracy and robustness of image fusion visual detection and positioning.

[0095] As a specific implementation method, it is assumed that the surface of the component under test is covered with a layer of transparent lubricating oil, which has different absorption and attenuation characteristics for visible and near-infrared light. First, multi-band reflection signals from the lubricated component are acquired, and radiometric calibration based on a reference target is performed to resolve the calibration texture matrix. And calibration penetration matrix For example, images can be acquired simultaneously using a visible light camera and a near-infrared camera, and radiometric calibration can be performed using a standard grayscale target to obtain the results for each pixel. The visible and near-infrared reflectance intensities were calibrated. Secondly, the one-way physical absorption attenuation coefficient of the lubricating oil for visible light was measured or consulted beforehand. and the single-pass physical absorption attenuation coefficient for near-infrared light And the visible light of the metal substrate of the components. and near-infrared light The inherent reflectivity, for example, can be measured using a spectrometer on a lubricating oil sample and by testing the reflectivity of exposed component surfaces. These parameters are then substituted into the calculation formula of the medium thickness distribution model. For each pixel in the image... The system will determine its corresponding and Values, and preset values , , , Calculate the thickness of the medium at that point. During this calculation process, if the calculated thickness of a pixel is negative due to noise interference, for example, -0.01mm, then the non-negative truncation operator... This will be corrected to 0mm to ensure the physical validity of the thickness value. Ultimately, a two-dimensional image reflecting the thickness distribution of lubricating oil on the surface of the component is obtained, where each pixel value represents the thickness of the lubricating oil at that location.

[0096] Through the above technical solution, this application provides an accurate and robust method for calculating the thickness distribution model of a medium. This method, by introducing specific mathematical formulas, fully utilizes the differential absorption characteristics of the translucent and refracting medium for different wavelengths of light, achieving a quantitative inversion of the medium thickness. Crucially, the application of a non-negative truncation operator effectively avoids non-physically meaningful negative thickness values ​​caused by factors such as sensor noise, thus ensuring the physical rationality of the thickness calculation results and the stability of subsequent processing. This makes the obtained medium thickness distribution model more accurate and reliable, providing a solid foundation for subsequent optical path distortion correction based on this model, and significantly improving the overall accuracy and reliability of image fusion visual inspection and positioning of textile machinery parts.

[0097] Preferably, the solution logic for the dual-channel optical path distortion vector field in step S30 is as follows:

[0098] Using the spatial incident angle between the imaging principal optical axis and the surface normal of the component under test, and the absolute physical refractive index in the corresponding band as input parameters, a dispersive refraction spatial operator characterizing the cross-medium deflection characteristics of light is constructed.

[0099] The absolute offset amplitude is obtained by nonlinearly distorting the quotient of the absolute physical thickness of the medium and the physical resolution coefficient of the imaging plane using the dispersive refraction spatial operator.

[0100] Simultaneously, a direction sign determination function is constructed based on the relative position of the spatial incident angle on both sides of the surface normal; the direction sign determination function is used to assign a vector direction to the absolute offset amplitude, and the apparent pixel offset of optical path distortion at the corresponding pixel coordinates of each band is output independently;

[0101] Specifically, the specific calculation formula for the dual-channel optical path distortion vector field in step S30 is as follows:

[0102]

[0103] In the formula, Indicates band channel In coordinates The apparent pixel offset due to optical path distortion caused by dispersion and refraction (unit: According to the channel sign Independent output visible light distortion field With near-infrared distortion field , The direction sign determination function (the rule is set as follows: when the principal optical axis of the image is incident on the left side of the surface normal). When incident from the right side ), The spatial angle of incidence (in radians, dimensionless) between the principal optical axis of the imaging and the normal to the surface of the part under test. Represents the absolute physical thickness of the medium (unit: ). ), The physical resolution coefficient of the imaging plane (unit: ) ), This indicates that the light-transmitting and refractive medium is in the corresponding wavelength channel. The absolute physical refractive index below This represents the spatial operator for dispersion and refraction.

[0104] Specifically, The parameters are represented in the image sensor coordinates. At this point, due to the dispersion and refraction of light as it passes through the translucent and refractive medium, an apparent pixel offset occurs on the sensor plane, resulting in an offset for a specific wavelength channel. Independently calculated, for example, in the visible light band, a distortion field will be generated. The near-infrared band will produce another distortion field. Its function is to accurately quantify image distortion caused by the medium, providing basic data for subsequent image correction. This is a direction sign determination function used to determine the direction of optical path distortion offset. For example, when the principal optical axis of the image is incident on the left side of the surface normal, the function outputs -1; when it is incident on the right side, it outputs 1. This function ensures that the directionality of distortion correction is consistent with the actual physical phenomenon, avoiding secondary distortion caused by incorrect correction direction. The parameter represents the spatial incident angle between the imaging principal optical axis and the surface normal of the part under test. This angle is a key geometric factor that affects the degree of light refraction at the medium interface. The larger the incident angle, the greater the degree of light deflection, which leads to greater optical path distortion. This angle can be obtained by pre-calibration or by calculation through the system geometric model. Parameters are represented in coordinates The absolute physical thickness of the light-transmitting and refractive medium. The thickness of the medium directly affects the distance light travels within it and the degree of refraction. The thicker the medium, the longer the path light travels, and the more significant the accumulated refraction effect, resulting in greater optical path distortion. This thickness value is provided by the medium thickness distribution model in step S20 above. The parameter represents the physical resolution coefficient of the imaging plane, and its unit is meters per pixel (m / pixel). It converts distance units in the physical world into pixel units on an image sensor, through... It can accurately map the calculated physical optical path distortion to the pixel offset on the image sensor, thereby achieving precise pixel-level correction. The parameters represent the light-transmitting and refractive medium in the corresponding wavelength channel. The absolute physical refractive index is used because, due to the dispersive properties of the medium, the refractive index of light in different wavelength bands is different in the same medium. For example, the refractive index of the visible light band is different. Refractive index in the near-infrared band Unlike other parameters, this one is the core of calculating the dispersive refraction effect, ensuring independent and accurate quantification of light distortion at different wavelengths. The expression is called the dispersive refraction space operator, which is used to quantify the lateral displacement of light rays caused by refraction when passing through the interface of a medium. This operator takes into account the influence of the incident angle and the refractive index of the medium on the deflection of light rays and is a key component for accurately calculating optical path distortion.

[0105] This application's solution addresses the image distortion problem caused by the light-transmitting and refractive medium by introducing the calculation of a dual-channel optical path distortion vector field. Firstly, based on the medium thickness distribution model obtained in the preceding steps... This model provides precise thickness information of the medium at each spatial location. Secondly, it incorporates pre-determined interface dispersion and refraction parameters, including the spatial angle of incidence between the imaging principal optical axis and the surface normal of the component under test. Physical resolution coefficient of the imaging plane and the absolute refractive index of the light-transmitting and refractive medium in the visible and near-infrared bands. These parameters collectively describe the refraction behavior of light at the interface of the medium and the dispersion characteristics of the medium. By substituting these parameters into a specific physical model formula, namely the calculation formula for the dual-channel optical path distortion vector field, the system can accurately calculate the apparent pixel offset caused by dispersion and refraction at each pixel point for each band channel (visible light and near-infrared). This calculation formula considers multiple physical factors such as medium thickness, incident angle, medium refractive index, and sensor resolution, ensuring the accuracy of distortion transformation. Specifically, the formula uses a dispersion refraction spatial operator to convert the physical deflection of light within the medium into pixel offset on the image sensor, and utilizes a direction sign determination function. Determine the offset direction. By calculating independent distortion vector fields for the visible and near-infrared bands respectively, this scheme can accurately capture the differential distortions caused by dispersion effects in different bands of light. This provides a precise physical basis for subsequent back projection correction that unifies image data from different bands into the same reference coordinate system, thereby significantly improving the accuracy of image fusion and target localization.

[0106] As a specific implementation method, the calculation of the dual-channel optical path distortion vector field can be achieved as follows: First, the medium thickness distribution model... This can be obtained by inverting the differential absorption characteristics of visible and near-infrared light in the medium through analysis in step S20 above. Secondly, the spatial incident angle... The physical resolution coefficient of the imaging plane can be obtained by measuring the angle between the imaging principal optical axis and the surface normal of the part under test using a high-precision angle sensor during system installation, or by pre-setting it through a 3D CAD model. The physical resolution coefficient of the imaging plane can be accurately calculated or calibrated based on the selected image sensor pixel size and the focal length and working distance of the optical lens. (In some preferred embodiments, the physical resolution coefficient of the imaging plane is obtained by direct measurement calibration. The specific calibration process is as follows: First, a standard measuring tool with known high-precision physical dimensions (such as a glass ruler with precision scale or a standard gauge block) is precisely placed horizontally at the actual calibration working distance of the detection system, i.e., the physical height of the surface of the textile component to be measured, to ensure that the reference plane of the measuring tool is strictly parallel to the photosensitive chip plane of the photoelectric sensor. Then, the camera is controlled to acquire an image of the standard measuring tool under the same light source environment as the actual detection. Next, a feature boundary with a known physical length on the measuring tool is selected in the acquired image, and the discrete pixel coordinates of the physical boundary in the image are calculated using a sub-pixel edge extraction algorithm. Then, the actual pixel distance between the two points in the image is calculated based on the extracted sub-pixel coordinates. Finally, the physical resolution coefficient of the imaging plane is solved by the following physical resolution coefficient model formula, combined with the known real physical distance between the two points of the standard measuring tool.) In the formula, This represents the physical resolution coefficient of the imaging plane, in units of... ; This represents the known physical distance between two feature boundaries on the standard gauge, in units of... ; This represents the true pixel distance measured in the corresponding image using a sub-pixel algorithm, in units of... Obtaining the resolution coefficient through this direct physical mapping method can eliminate microscopic dispersion and focal length errors caused by the actual light source wavelength to the greatest extent, ensuring the absolute accuracy of inverse optical distortion compensation. (The absolute refractive index of the transmissive and refractory medium in the visible and near-infrared bands.) ,For example and This information can be obtained by consulting the material property data sheet of the medium, or by experimental determination using precision optical measuring equipment such as a spectroscopic ellipsometer. Direction sign determination function. This can be determined based on the system's geometric layout, such as when the camera is positioned on a specific side of the object being measured, by pre-setting the value to positive or negative. Substituting these specific values ​​into the formula allows for the determination of the value at each pixel. The corresponding visible light distortion field was calculated at that location. and near-infrared distortion field .

[0107] Through the above technical solution, this application can accurately quantify the dispersive and refractive distortions caused by the light-transmitting and refractive medium to different wavelengths of light. By considering multiple physical factors such as medium thickness, incident angle, medium refractive index, and sensor resolution, and by calculating independent distortion vector fields for the visible and near-infrared bands respectively, this solution effectively solves the problem of inaccurate positioning caused by neglecting or simplifying the optical distortion model in traditional methods. This precise distortion quantification provides a solid physical foundation for subsequent image back-projection correction, ensuring high-precision alignment of image data from different bands in a unified reference coordinate system. This significantly improves the quality of image fusion and the accuracy and reliability of final target positioning, and provides more accurate input for over-limit early warning.

[0108] Preferably, the correction logic for projecting the dual-channel signal onto the unified reference coordinate system in step S40 is as follows:

[0109] Extract the absolute azimuth angle of the imaging back ray corresponding to the pixel on the projection plane as input, and construct an orthogonal projection decomposition operator;

[0110] The orthogonal projection decomposition operator is used to decompose the apparent pixel offset vector of optical path distortion into horizontal distortion compensation components and vertical distortion compensation components.

[0111] Based on the discrete horizontal and vertical coordinates of the original sensor acquisition matrix, reverse translation resampling mapping is performed by subtracting the corresponding horizontal distortion compensation components and vertical distortion compensation components respectively, thereby obtaining the true horizontal and true vertical coordinates mapped to a unified reference coordinate system.

[0112] Specifically, the correction formula for projecting the dual-channel signal onto the unified reference coordinate system in step S40 is as follows:

[0113]

[0114]

[0115] In the formula, and These represent the band channels. The original sensor pixels are back-projected and mapped to their true horizontal and vertical coordinates in a unified reference coordinate system (all units are...). ), and Represents the discrete horizontal and vertical coordinates of the original sensor acquisition matrix (units are...). ), Indicates band channel Independent optical path distortion apparent pixel offset (in units) ), The absolute azimuth angle (in radians, dimensionless) of the imaging backlight ray corresponding to a pixel on the projection plane is represented by the output of the correction step, which is mapped to a unified physical grid. The texture compensation matrix on With the penetration compensation matrix and the thickness field of synchronous mapping , , This represents the orthogonal projection decomposition operator. This represents the horizontal distortion compensation component. This represents the vertical distortion compensation component. , This represents the reverse translation resampling mapping.

[0116] This correction formula is used to map the pixel coordinates of the original sensor, which are affected by optical distortion, to a unified, distortion-free physical reference coordinate system through back projection. Its function is to eliminate the dispersion and refraction distortion caused by the light-transmitting and refractive medium, and ensure that the image data of different bands are accurately aligned in space. This provides an accurate spatial reference for subsequent image fusion and target localization. The correction formula can be derived based on the principles of geometric optics, or it can be fitted by a pre-established distortion model. and These two parameters represent the band channel. raw sensor pixels After distortion correction, the true horizontal and true vertical coordinates in a unified reference coordinate system represent the corrected physical position, reflecting the actual spatial position of the pixel in the absence of distortion. and These two parameters represent the discrete horizontal and vertical coordinates in the raw sensor acquisition matrix, i.e., the raw pixel positions without any correction. This parameter represents the band channel. In coordinates The apparent pixel offset is the optical path distortion caused by dispersion and refraction. It quantifies the deviation between the position of the light received by the sensor and the actual physical position caused by the refraction effect of the medium. This parameter represents the absolute azimuth angle of the imaging backlight ray corresponding to the pixel on the projection plane. It describes the path direction of the light ray from the object surface through the medium and optical system, finally reaching the sensor pixel. This azimuth angle can be obtained through camera intrinsic and extrinsic parameter calibration, or it can be obtained through real-time calculation of ray tracing. The correction step output is mapped to a unified physical mesh. The texture compensation matrix on With the penetration compensation matrix and the thickness field of synchronous mapping This means that after processing with the above correction formula, the original calibration texture matrix, calibration penetration matrix, and medium thickness field are resampled and aligned to a unified physical mesh. Texture compensation matrix and penetration compensation matrix These are corrected visible and near-infrared image data, which are spatially precisely aligned. The thickness field is mapped synchronously. This indicates the thickness information of the medium on the same physical grid. , These two parameters represent the orthogonal projection decomposition operator, which is used to decompose optical path distortion into apparent pixel offsets. Decompose it into horizontal and vertical components so that they can be used separately. and The coordinates are corrected. , These two parameters represent the horizontal distortion compensation component and the vertical distortion compensation component, respectively. They are calculated based on the optical path distortion offset and the azimuth angle of the light rays and are used to accurately correct the positional deviation of the original pixels in the horizontal and vertical directions. , These two expressions represent reverse translation resampling mapping, which maps the original pixel to its actual physical location in a unified reference coordinate system by subtracting the corresponding distortion compensation component from the original pixel coordinates.

[0117] The solution in this application introduces a dual-channel optical path distortion vector field. The absolute azimuth angle of the imaging backlight corresponding to the pixel on the projection plane Accurately calculate each original sensor pixel. Distortion compensation components in the horizontal and vertical directions. Specifically, optical path distortion appears as a pixel offset. First, the operator is decomposed using orthogonal projection. and Decomposed into horizontal distortion compensation components and vertical distortion compensation components Subsequently, these compensation components were obtained from the discrete horizontal coordinates of the original sensor matrix. and vertical coordinates Subtracting from the middle, and then obtaining the band channel through reverse shift resampling mapping. The original sensor pixels are mapped to their true horizontal coordinates in a unified reference coordinate system after back-projection calculation. perpendicular coordinates to the true coordinates This process is essentially a modification of the original calibration texture matrix. , calibration penetration matrix and medium thickness distribution model Spatial resampling and correction are performed to accurately map and align them from their respective distorted sensor coordinate systems to a unified physical grid. This outputs the texture compensation matrix. Penetration compensation matrix and the thickness field of synchronous mapping In this way, image data from different bands are aligned with high precision in space, eliminating dispersion and refraction distortion caused by the light-transmitting refractive medium. This provides an accurate and consistent spatial reference for subsequent image fusion and target localization, effectively solving the problem of spatial misalignment caused by optical distortion in the original image data.

[0118] As a specific implementation method, the above-mentioned technical means can be implemented with reference to the following example. After obtaining the dual-channel optical path distortion vector field... The absolute azimuth angle of the imaging back ray on the projection plane Then, the system can be configured for each band channel. (For example, visible light and near-infrared bands) Construct a distortion correction lookup table or directly apply the correction formula described above. For each pixel acquired by the original sensor... First, according to its corresponding and The compensation amounts in the horizontal and vertical directions can be calculated. For example, for the visible light band, the compensation amounts can be calculated. and Then, the original pixel coordinates Subtracting these compensation amounts yields the corrected physical coordinates. Since the corrected coordinates are usually not integers, the system can use image resampling techniques, such as bilinear interpolation or nearest neighbor interpolation, to resample the original calibrated texture matrix. Pixel values ​​are mapped to a unified physical grid. Texture compensation matrix on Similarly, the calibration penetration matrix for the near-infrared band... Perform the same correction and resampling to obtain the penetration compensation matrix. In addition, the medium thickness distribution model Simultaneous resampling is also required to ensure uniform physical grid. Obtain aligned thickness field In this way, all relevant data will be precisely aligned to the same spatial reference.

[0119] Through the above technical solution, this application can accurately eliminate the dispersion and refraction distortion and spatial misalignment caused by the light-transmitting and refractive medium on multi-band image data. By back-projecting and correcting the original sensor data of different bands to a unified physical reference coordinate system, high-precision spatial alignment of the texture compensation matrix and the penetration compensation matrix is ​​ensured. This precise alignment is the basis for subsequent image fusion and target localization, significantly improving the accuracy and reliability of detection and localization, avoiding misjudgments or positioning deviations caused by distortion, thus providing a more solid data foundation for the visual inspection of textile machinery parts.

[0120] Preferably, the logic for executing the positioning output and over-limit warning in step S50 is as follows:

[0121] First, the absolute physical thickness of the medium on the unified physical grid is extracted. Then, combined with the spatial incident angle and the absolute physical refractive index of the medium for the visible light band, the true oblique optical path length of light propagating unidirectionally within the medium is calculated through geometric deflection path mapping.

[0122] Next, using the actual oblique optical path length and the single-pass physical absorption attenuation coefficient of the medium for visible light as independent variables, an exponential attenuation weight allocation function is constructed. This function is used to perform complementary weighted fusion calculation on the texture compensation matrix and the penetration compensation matrix aligned to a unified reference coordinate system, generating a unified mesh fusion signal intensity output by the transmittance adaptive fusion model.

[0123] Subsequently, within the discrete pixel connected set of the feature to be tested in a unified reference coordinate system, the gray-scale centroid spatial positioning operator is executed with the unified grid fusion signal intensity as the feature distribution weight to calculate the target positioning horizontal and vertical coordinates.

[0124] Finally, the extreme value traversal interception logic is executed to traverse the connected set to extract the thickness limit value. When it is determined that the thickness limit value meets the set safety threshold, the horizontal and vertical coordinates of the target positioning are allowed to be output. Otherwise, the output is blocked and an optical distortion over-limit warning is triggered.

[0125] Specifically, the logical decision-making closed loop for executing the positioning output and over-limit warning in step S50 is as follows:

[0126] First, by extracting the physical mesh Thickness field The actual physical distance of a single-pass oblique optical path through the medium is calculated, and the transmittance adaptive fusion model is constructed.

[0127]

[0128] Among them, the actual physical distance of the single-pass oblique optical path The coupling calculation formula is as follows:

[0129]

[0130] Secondly, feature regions are extracted by combining the fused signals under a unified reference coordinate system. The target positioning coordinates within the area, where the target positioning coordinates include the horizontal coordinates. with vertical coordinates :

[0131]

[0132]

[0133] Finally, traverse the aforementioned feature regions. Finding extreme points (extreme value traversal interception logic) If and only if the extreme point Less than or equal to the system-set safety tolerance threshold If the target location coordinates are displayed correctly, the positioning output stream is blocked and an optical distortion over-limit warning is triggered.

[0134] In the formula, This represents the intensity of the unified mesh fused signal output by the transmittance adaptive fusion model. This represents the true oblique optical path length of light propagating unidirectionally within a medium (unit: ). The constant 2 is the round-trip transmission factor corresponding to the reflective optical path. This represents the single-pass physical absorption attenuation coefficient of a medium for visible light (unit: ). ), It represents the absolute physical refractive index of a medium in the visible light band. The spatial angle of incidence (in radians). Represents the absolute physical thickness of the medium on the physical grid (unit: ). ), and These represent the intensity values ​​of the texture compensation matrix and the penetration compensation matrix aligned on a uniform physical mesh, respectively. and Represent the discrete horizontal and vertical coordinates of the unified reference coordinate system (unit: ), This represents the discrete connected set of pixels representing the feature to be measured in a unified reference coordinate system. and These represent the output target positioning horizontal and vertical coordinates (units are both...). ), and All are physical quantities representing thickness limits and safety thresholds (unit: ...). ), Represents the geometric deflection path mapping. and their corresponding complementary terms This represents the exponentially decaying weight allocation function. This indicates complementary weighted fusion calculation. and This represents the grayscale centroid spatial positioning operator.

[0135] The transmittance adaptive fusion model aims to dynamically adjust the fusion weights of images in different wavelength bands based on the physical properties of the medium. It can be constructed by deriving a physical optics model, such as considering light attenuation in the medium based on the Lambert-Beer law, or by using data-driven methods, such as machine learning algorithms, to learn the optimal fusion weights for different medium thicknesses. The actual physical distance of the single-pass oblique optical path is also mentioned. This is the actual path length of light propagating in the medium. It depends not only on the vertical thickness of the medium but also on the angle of incidence of the light and the refractive index of the medium. It is a key parameter for accurately calculating light attenuation. The target positioning coordinates... and This is used to accurately identify the spatial location of specific feature points on the component under test. The calculation method can employ grayscale centroid method, template matching method, or feature point detection algorithm, etc. The logical decision-making closed loop is a key link in ensuring the reliability of the system output. It avoids positioning errors caused by medium anomalies through real-time monitoring and early warning of medium thickness. The extreme point... This represents the maximum thickness of the medium within the characteristic region and is an important indicator for assessing the risk of optical distortion. The safety tolerance threshold... This is a preset, acceptable maximum thickness limit for the medium, which can be set according to the actual application scenario, optical system performance, and positioning accuracy requirements, for example, determined through experimental calibration or theoretical calculation. The positioning output and the over-limit warning are two possible outcomes of this closed loop; the former provides normal positioning data, while the latter prevents inaccurate outputs and issues a warning when a potential risk is detected.

[0136] The solution in this application uses a medium thickness distribution model. With respect to the optical parameters of the medium (such as refractive index) and spatial incident angle Coupling allows for precise calculation of the true physical distance of a single-path oblique light beam within a medium. Based on this actual optical path, a transmittance adaptive fusion model was constructed. This model utilizes an exponential decay weight allocation function, which is based on the actual distance the light travels in the medium and the absorption attenuation coefficient of the medium for visible light. Dynamically adjust the visible light texture compensation matrix With near-infrared penetration compensation matrix The fusion weights are used to generate a unified mesh fusion signal strength. This adaptive fusion mechanism enables the fused image to more accurately reflect the true characteristics of the measured component, effectively overcoming the impact of medium thickness variations on image quality. Subsequently, a gray-scale centroid spatial positioning operator is used to analyze the fused signal. Extracting feature regions Target positioning coordinates within and To further ensure the reliability of positioning, the system traverses the feature region before outputting the positioning coordinates. Inner thickness field Find the extreme point of the medium thickness. .like Exceeding the preset safety tolerance threshold If the thickness of the medium is deemed insufficient, the system will determine that the positioning result may be unreliable. In this case, the system will block the positioning output stream and trigger an optical distortion over-limit warning, thereby avoiding erroneous positioning information caused by excessively thick medium or excessive optical distortion. The entire process forms a complete closed loop from physical modeling to image fusion, and then to positioning output and safety verification, ensuring the accuracy and reliability of visual inspection and positioning of textile machinery parts in complex media environments.

[0137] As a specific implementation method, it is conceivable to perform visual inspection and positioning on a textile machinery bearing covered with transparent lubricating oil. First, the system has obtained a calibrated visible light texture compensation matrix through multi-band imaging and radiometric calibration. and near-infrared penetration compensation matrix and the thickness distribution model of the lubricating oil layer. To accurately integrate this information, the system will utilize... The refractive index of lubricating oil to visible light and the spatial incident angle between the imaging principal optical axis and the normal to the bearing surface. The actual physical distance of a single-way oblique optical path in the lubricating oil layer was calculated. Next, based on And the single-pass physical absorption attenuation coefficient of lubricating oil for visible light Construct an adaptive fusion model for transmittance, and then... and Perform weighted fusion to generate a fused signal. For example, if A larger value indicates a thicker lubricating oil layer, resulting in severe visible light attenuation. Therefore, the weight of the near-infrared image will be increased accordingly to better penetrate the oil layer. Then, in the signal fusion... The system identifies specific features on the bearing (such as bolt holes) and calculates their target positioning coordinates. and Before outputting these coordinates, the system checks the maximum thickness of the lubricating oil layer in the bolt hole area. .if Exceeding the preset safety tolerance threshold (For example, if the system design specifies that the positioning accuracy cannot be guaranteed when the lubricating oil thickness exceeds a certain value), the system will not output the positioning coordinates of the bolt hole, but will trigger an over-limit warning, prompting the operator that the lubricating oil layer is too thick and may need to be cleaned or re-inspected.

[0138] Through the above technical solution, this application effectively solves the problems of inaccurate image information fusion and insufficient reliability of positioning results in the visual inspection and positioning of textile machinery parts under the coverage of a light-transmitting and refractive medium. This solution, by introducing a medium thickness distribution model and calculating the actual propagation path of light in the medium, achieves physical model-driven adaptive fusion of multi-band images, significantly improving the quality of the fused image and the accuracy of target positioning. Simultaneously, the integrated logical judgment closed-loop and over-limit warning mechanism can monitor the medium thickness in real time and promptly block inaccurate positioning outputs when the thickness is abnormal, effectively avoiding misjudgments caused by excessive optical distortion. This greatly enhances the reliability and safety of the entire inspection and positioning system, ensuring high-precision and high-reliability inspection and positioning of textile machinery parts in actual production environments.

[0139] An image fusion visual inspection and positioning system for textile machinery parts includes:

[0140] The multimodal acquisition and calibration module is used to acquire multi-band reflection signals of the test component covering the light-transmitting and refractive medium, perform radiometric calibration based on the reference target, and parse out the calibration texture matrix and calibration penetration matrix.

[0141] The medium distribution inversion module is used to invert and calculate the medium thickness distribution model based on the spectral attenuation characteristics of the light-transmitting and refractive medium, combined with the calibration texture matrix and the calibration transmission matrix.

[0142] The optical path distortion calculation module is used to calculate the dual-channel optical path distortion vector field based on the medium thickness distribution model and the interface dispersion and refraction parameters.

[0143] The coordinate projection compensation module is used to perform back projection correction on the calibration texture matrix and calibration penetration matrix respectively using the dual-channel optical path distortion vector field to obtain the texture compensation matrix and penetration compensation matrix under a unified reference coordinate system.

[0144] The fusion early warning module is used to construct a transmittance adaptive fusion model based on the medium thickness distribution model, fuse the texture compensation matrix and the penetration compensation matrix to calculate the target positioning coordinates, and perform positioning output and over-limit early warning in combination with the medium thickness distribution model.

[0145] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the above-described image fusion visual detection and localization method.

[0146] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0147] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An image fusion visual detection and localization method, characterized in that, include: S10. Acquire multi-band reflection signals of the test component covered by the light-transmitting and refractive medium, perform radiometric calibration based on the reference target, and resolve the calibration texture matrix and calibration penetration matrix. S20. Based on the spectral attenuation characteristics of the light-transmitting and refractive medium, and combined with the calibration texture matrix and calibration transmission matrix, the medium thickness distribution model is calculated by inversion. S30. Based on the medium thickness distribution model and the interface dispersion and refraction parameters, calculate the dual-channel optical path distortion vector field; S40. The dual-channel optical path distortion vector field is used to perform back projection correction on the calibration texture matrix and calibration penetration matrix respectively to obtain the texture compensation matrix and penetration compensation matrix under a unified reference coordinate system. S50. Construct a transmittance adaptive fusion model based on the medium thickness distribution model, fuse the texture compensation matrix and the penetration compensation matrix to calculate the target positioning coordinates, and combine the medium thickness distribution model to perform positioning output and over-limit warning.

2. The image fusion visual detection and localization method according to claim 1, characterized in that, The spectral attenuation characteristics mentioned in step S20 are specifically: the single-pass physical absorption attenuation coefficient of the light-transmitting refractive medium for visible light and the single-pass physical absorption attenuation coefficient for near-infrared light; the interface dispersion and refractive parameters mentioned in step S30 are specifically: the absolute refractive index of the light-transmitting refractive medium for the visible light band and the near-infrared band, the physical resolution coefficient of the imaging plane, and the spatial incident angle between the imaging principal optical axis and the surface normal of the component under test.

3. The image fusion visual detection and localization method according to claim 2, characterized in that... The radiometric calibration based on the reference target described in step S10 has the following mapping logic: Using the pre-calibrated system dark current noise floor intensity of each band as the system zero-position reference, a calibration response scale factor is constructed based on the average calibration intensity of the corresponding band and the known inherent physical reflectivity. The original reflection intensity collected by the photoelectric sensor at each pixel coordinate is mapped by a linear bias correction operator based on the system zero-position reference and the calibration response scale factor, and the true relative reflection intensity of each band at the corresponding pixel coordinate is output after calibration.

4. The image fusion visual detection and localization method according to claim 2, characterized in that, The solution logic for the medium thickness distribution model in step S20 is as follows: Based on the true relative reflectance intensity of the calibration penetration matrix and the inherent reflectance of near-infrared light, and the true relative reflectance intensity of the calibration texture matrix and the inherent reflectance of visible light, a cross-band reflectance feature is constructed at the corresponding pixel coordinates. A differential absorption coefficient is constructed based on the single-pass physical absorption attenuation coefficient of the light-transmitting and refractive medium for visible and near-infrared light. The cross-band reflectance characteristic is input into the logarithmic attenuation mapping model, and the initial physical thickness is calculated by combining the differential absorption coefficient and the reflection round-trip transmission factor. The initial physical thickness is then input into the non-negative boundary truncation operator to remove the negative distortion caused by the shot noise at the bottom layer of the sensor, and finally the absolute physical thickness of the medium at the corresponding pixel coordinates is output.

5. The image fusion visual detection and localization method according to claim 4, characterized in that, The solution logic for the dual-channel optical path distortion vector field in step S30 is as follows: Using the spatial incident angle between the imaging principal optical axis and the surface normal of the component under test, and the absolute physical refractive index in the corresponding band as input parameters, a dispersive refraction spatial operator characterizing the cross-medium deflection characteristics of light is constructed. The absolute offset amplitude is obtained by nonlinearly distorting the quotient of the absolute physical thickness of the medium and the physical resolution coefficient of the imaging plane using the dispersive refraction spatial operator. Simultaneously, a direction sign determination function is constructed based on the relative position of the spatial incident angle on both sides of the surface normal; the direction sign determination function is used to assign a vector direction to the absolute offset amplitude, and the apparent pixel offset of optical path distortion at the corresponding pixel coordinates of each band is output independently.

6. The image fusion visual detection and localization method according to claim 4, characterized in that, The correction logic for projecting the dual-channel signal onto the unified reference coordinate system in step S40 is as follows: Extract the absolute azimuth angle of the imaging back ray corresponding to the pixel on the projection plane as input, and construct an orthogonal projection decomposition operator; The orthogonal projection decomposition operator is used to decompose the apparent pixel offset vector of optical path distortion into horizontal distortion compensation components and vertical distortion compensation components. Based on the discrete horizontal and vertical coordinates of the original sensor acquisition matrix, reverse translation resampling mapping is performed by subtracting the corresponding horizontal distortion compensation components and vertical distortion compensation components respectively, thereby obtaining the true horizontal and true vertical coordinates mapped to a unified reference coordinate system.

7. The image fusion visual detection and localization method according to claim 3, characterized in that, The logic for executing the positioning output and over-limit warning in step S50 is as follows: First, the absolute physical thickness of the medium on the unified physical grid is extracted. Then, combined with the spatial incident angle and the absolute physical refractive index of the medium for the visible light band, the true oblique optical path length of light propagating unidirectionally within the medium is calculated through geometric deflection path mapping. Next, using the actual oblique optical path length and the single-pass physical absorption attenuation coefficient of the medium for visible light as independent variables, an exponential attenuation weight allocation function is constructed. This function is used to perform complementary weighted fusion calculation on the texture compensation matrix and the penetration compensation matrix aligned to a unified reference coordinate system, generating a unified mesh fusion signal intensity output by the transmittance adaptive fusion model. Subsequently, within the discrete pixel connected set of the feature to be tested in a unified reference coordinate system, the gray-scale centroid spatial positioning operator is executed with the unified grid fusion signal intensity as the feature distribution weight to calculate the target positioning horizontal and vertical coordinates. Finally, the extreme value traversal interception logic is executed to traverse the connected set to extract the thickness limit value. When it is determined that the thickness limit value meets the set safety threshold, the horizontal and vertical coordinates of the target positioning are allowed to be output; otherwise, the output is blocked and an optical distortion over-limit warning is triggered.

8. An image fusion visual inspection and positioning system for textile machinery parts, characterized in that, include: The multimodal acquisition and calibration module is used to acquire multi-band reflection signals of the test component covering the light-transmitting and refractive medium, perform radiometric calibration based on the reference target, and parse out the calibration texture matrix and calibration penetration matrix. The medium distribution inversion module is used to invert and calculate the medium thickness distribution model based on the spectral attenuation characteristics of the light-transmitting and refractive medium, combined with the calibration texture matrix and the calibration transmission matrix. The optical path distortion calculation module is used to calculate the dual-channel optical path distortion vector field based on the medium thickness distribution model and the interface dispersion and refraction parameters. The coordinate projection compensation module is used to perform back projection correction on the calibration texture matrix and calibration penetration matrix respectively using the dual-channel optical path distortion vector field to obtain the texture compensation matrix and penetration compensation matrix under a unified reference coordinate system. The fusion early warning module is used to construct a transmittance adaptive fusion model based on the medium thickness distribution model, fuse the texture compensation matrix and the penetration compensation matrix to calculate the target positioning coordinates, and perform positioning output and over-limit early warning in combination with the medium thickness distribution model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image fusion visual detection and localization method as described in any one of claims 1 to 7.

Images