A multi-color LED illumination based co-localization analysis multi-channel image automatic registration method and system

Abstract

The application provides a multi-color LED illumination-based co-localization analysis multi-channel image automatic registration method and system, relates to the image processing technical field, and obtains the driving current parameters corresponding to each illumination channel and the structural jitter parameters of the acquisition device in the face image acquisition process, and the correlation is the face acquisition state parameter; according to the face acquisition state parameter, a preset model is used to analyze the chromatic dispersion displacement component and the structural offset component; the two are nonlinearly fitted to determine the estimated registration deviation; the position fluctuation interval is determined according to the fluctuation amplitude, and the local search area is delimited in the phase correlation diagram; the sub-pixel offset coordinates are positioned and extracted in the local search area; the sub-pixel offset coordinates are determined as the final registration offset, high-precision automatic registration of multi-channel images can be realized, and the reliability and efficiency of co-localization analysis are significantly improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for automatic registration of multi-channel images based on co-localization analysis using multi-color LED illumination. Background Technology

[0002] Colocalization analysis multispectral imaging is an important technique in fields such as dermatology and biomedical testing. It acquires images of the same target frame by frame by switching multicolor LED illumination channels of different wavelengths, and then synthesizes the multi-channel images into a multispectral image to analyze physiological information such as the distribution of components, pigment deposition, and blood vessel distribution on the surface and deep layers of the target.

[0003] In existing technologies, multi-color LED lighting acquisition systems suffer from pixel-level or even sub-pixel-level offsets between images acquired through different lighting channels due to minute time intervals in the lighting spectrum switching and image sensor acquisition processes. Furthermore, these offsets are influenced by factors such as slight movement of the object under test, mechanical vibration of the acquisition device, and differences in refractive index when light of different wavelengths passes through lenses. This results in registration errors. Directly superimposing or fusing these offset multi-channel images leads to problems such as blurred edges, false-color stripes, and detail distortion in the synthesized image, severely impacting the accuracy of subsequent co-localization analysis.

[0004] Existing image registration methods largely rely on global search and matching based on image content features. Feature point extraction-based methods require extracting and matching stable feature points from each channel image; however, these methods are less robust in scenes with weak texture and sparse features, such as facial skin. While phase-correlation-based frequency domain registration methods are computationally efficient, they are susceptible to noise, background artifacts, and brightness variations caused by spectral differences during global peak search, leading to misidentified peaks and distorted registration results. Furthermore, none of these methods fully utilize the physical state information of the lighting system and the acquisition device itself, making it difficult to balance registration accuracy and computational efficiency under complex acquisition conditions.

[0005] Therefore, how to achieve high-precision and robust automatic image registration in multispectral imaging scenarios of co-location analysis with multicolor LED lighting is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for automatic registration of multi-channel images based on co-location analysis using multi-color LED lighting, in order to solve the problem in the prior art that it is difficult to balance registration accuracy and computational efficiency under complex acquisition conditions.

[0007] To address the aforementioned technical problems, in a first aspect, this application provides a method for automatic registration of multi-channel images based on co-localization analysis using multi-color LED illumination, comprising: The driving current parameters and structural jitter parameters of the acquisition device corresponding to each illumination channel are obtained during the facial image acquisition process, and correlated as facial acquisition state parameters of the corresponding image frame. Based on the facial acquisition state parameters, the dispersion shift component caused by spectral differences and the structural offset component caused by the mechanical movement of the acquisition device are analyzed by a preset model. The dispersion shift component and the structural offset component are nonlinearly fitted to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image. Based on the fluctuation amplitude in the facial acquisition status parameters, the position fluctuation range for the estimated registration deviation is determined, and a local search area is delineated in the phase correlation map with the position fluctuation range as the boundary. Within the local search area, the intensity peak point in the phase correlation map is located, and the corresponding sub-pixel offset coordinates are extracted; The subpixel offset coordinates are determined as the final registration offset of each channel facial image relative to the reference channel facial image.

[0008] Optionally, the step of resolving the dispersion shift component caused by spectral differences based on the facial acquisition state parameters using a preset model includes: The driving current parameters of each illumination channel are extracted from the facial acquisition state parameters, and the algebraic difference between the driving current parameters and the driving current parameters of the reference channel is calculated. Substituting the algebraic difference into the pre-calibrated photoelectric response function, the center wavelength drift value of the corresponding illumination channel is calculated; The center wavelength drift value is input into the lens dispersion polynomial for algebraic operation to solve for the spatial offset value in the pixel coordinate system, which is used as the dispersion displacement component.

[0009] Optionally, the step of analyzing the structural offset component generated by the mechanical movement of the acquisition device based on the facial acquisition state parameters using a preset model includes: Extract the structural jitter parameters of adjacent moments from the facial acquisition state parameters to obtain the relative displacement value of the acquisition device in physical space; By combining the magnification of the acquisition system and the unit size of the image sensor, the relative displacement values ​​in the physical space are converted to pixel coordinates to obtain the corresponding pixel displacement values, which are used as structural offset components.

[0010] Optionally, the step of nonlinearly fitting the dispersion shift component and the structural offset component to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image includes: Extract the projection values ​​of the dispersion displacement component and the structure offset component onto the horizontal and vertical axes of the pixel coordinate system; Calculate the nonlinear weighting coefficient based on the magnitude of the structural offset component; The nonlinear weighting coefficients are multiplied by the projected values ​​of the horizontal and vertical axes respectively, and then vectored together to output the predicted registration deviation, which is composed of the predicted values ​​of the horizontal and vertical axes.

[0011] Optionally, before determining the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, the method further includes: Frequency domain transformation is performed on the facial images of each channel and the facial image of the reference channel, and the complex values ​​after transformation are multiplied by conjugate to extract the pure phase information of the multiplication result; Perform an inverse frequency domain transform on the pure phase information to output the phase correlation map in the spatial domain.

[0012] Optionally, determining the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, and delineating a local search area in the phase correlation map using the positional fluctuation range as the boundary, includes: Extract the numerical deviation of the structural jitter parameter from the facial acquisition state parameters, and determine the numerical deviation as the fluctuation amplitude; The corresponding pixel radius value is calculated by associating the fluctuation amplitude with a preset offset redundancy, and is used as the position fluctuation range. In the phase correlation diagram, a local search area is defined with the coordinates corresponding to the estimated registration deviation as the center and the position fluctuation range as the boundary.

[0013] Optionally, locating the intensity peak point in the phase correlation map within the local search area includes: Extract the intensity value of each pixel within the local search area one by one; Compare all extracted intensity values, select the pixel with the highest intensity value, and use the pixel with the highest intensity value as the integer peak point.

[0014] Optionally, extracting the corresponding sub-pixel offset coordinates includes: Using the integer-level peak point as the center, extract the intensity values ​​of a preset number of neighboring pixels; Substitute the intensity values ​​of the integer-level peak points and the neighboring pixels into the two-dimensional surface equation to perform surface fitting, and solve the coordinate solution of the two-dimensional surface equation where the first derivative is zero. Extract the continuous spatial values ​​corresponding to the coordinate solutions as sub-pixel offset coordinates.

[0015] Optionally, after determining the sub-pixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image, the method further includes: Based on the horizontal and vertical values ​​of the final registration offset, pixel-level interpolation sampling is performed on the facial images of each channel; The corresponding channel pixel values ​​of the facial images in each channel after interpolation sampling are extracted and compared with the corresponding channel facial images in the reference channel. The values ​​are then algebraically added and combined to form a multispectral facial image.

[0016] Secondly, this application provides a co-location analysis multi-channel image automatic registration system based on multi-color LED illumination, comprising: The acquisition module is used to obtain the driving current parameters of each illumination channel and the structural jitter parameters of the acquisition device during the facial image acquisition process, and associate them with the facial acquisition status parameters of the corresponding image frame. The analysis module is used to analyze the dispersion shift component caused by spectral differences and the structural offset component caused by the mechanical movement of the acquisition device based on the facial acquisition state parameters and a preset model. The fitting module is used to perform nonlinear fitting between the dispersion shift component and the structural offset component to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image. The delineation module is used to determine the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, and to delineate a local search area in the phase correlation map with the positional fluctuation range as the boundary. The positioning module is used to locate the intensity peak point in the phase correlation map within the local search area and extract the corresponding subpixel offset coordinates. The determination module is used to determine the subpixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image.

[0017] The multi-channel image automatic registration method based on multi-color LED lighting provided in this application correlates the driving current parameters of the lighting channels and the structural jitter parameters of the acquisition device as facial acquisition state parameters, fully utilizing the physical state information of the acquisition system itself to provide a priori basis for subsequent registration. Based on these state parameters, the dispersion shift component caused by spectral differences is analyzed using photoelectric response and lens dispersion models, and the structural offset component caused by device jitter is analyzed using a mechanical motion model, thus clearly decomposing the causes of the offset at the physical level. Subsequently, the two types of components are nonlinearly fitted to obtain the predicted registration error, providing a reliable initial estimate for registration.

[0018] Furthermore, by combining the fluctuation amplitude in the state parameters to determine the position fluctuation range, and using this to delineate the local search area in the phase correlation map, the search range is narrowed, reducing the interference of noise and spurious peaks on the global search, making the localization of intensity peak points more robust. Finally, fine offset coordinates are extracted through sub-pixel level surface fitting to obtain a high-precision final registration offset. Thus, the method provided in this application, which uses physical model prediction to guide the registration search and frequency domain phase correlation and sub-pixel fitting to finely correct the coordinates, achieves high-precision and robust automatic registration in multi-color LED lighting acquisition scenarios with weak textures and significant spectral differences, improving the quality and reliability of multispectral imaging in co-localization analysis.

[0019] This application also discloses a co-location analysis multi-channel image automatic registration system based on multi-color LED lighting, which can achieve the same technical effects as described above. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of a multi-channel image automatic registration method based on multi-color LED lighting co-localization analysis provided in this application embodiment; Figure 2 A schematic diagram of a multi-channel image acquisition device for co-localization analysis of multi-color LED lighting provided in this application embodiment; Figure 3 A schematic diagram illustrating a facial capture state parameter parsing process provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the delineation of a local search region in a phase correlation graph, provided as an embodiment of this application; Figure 5 This is a structural diagram of a multi-channel image automatic registration system based on multi-color LED lighting for co-localization analysis, provided in an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] This application discloses an automatic registration method for multi-channel images based on co-location analysis using multi-color LED lighting, which improves the accuracy and robustness of multi-channel image registration.

[0024] See Figure 1 The present application provides a flowchart of a multi-channel image automatic registration method based on multi-color LED illumination co-localization analysis, as shown in the embodiments. Figure 1 As shown, the method includes steps S101 to S106, and each step will be described in detail below with reference to the relevant figures.

[0025] S101: Obtain the driving current parameters and structural jitter parameters of the acquisition device corresponding to each illumination channel during the facial image acquisition process, and associate them as facial acquisition state parameters of the corresponding image frame.

[0026] Among them, the co-location analysis multi-channel image acquisition device refers to an optical imaging device that uses multiple LEDs of different wavelengths as illumination sources and performs frame-by-frame imaging of the object under test by switching the LEDs of different channels in a time-division manner. The composition of the acquisition device can be found in [reference needed]. Figure 2 .

[0027] See Figure 2 The present application provides a schematic diagram of the structure of a multi-channel image acquisition device for co-location analysis of multi-color LED lighting, as shown in the embodiment. Figure 2 As shown, the device includes a multi-color LED lighting module 10, an optical lens 20, an image sensor 30, an inertial measurement unit 40, and a processing unit 50.

[0028] The multi-color LED lighting module 10 integrates multiple LEDs with different center wavelengths, such as blue, green, red, and near-infrared channels. Each LED channel is driven by an independent driving circuit to control its lighting time and luminous intensity. The optical lens 20 is used to focus the light reflected from the object under test onto the image sensor 30. The image sensor 30 is used to convert the light signal into an electrical signal and output a digital image. The inertial measurement unit 40 is used to measure the spatial attitude change of the acquisition device in real time. The processing unit 50 is connected to the multi-color LED lighting module 10, the image sensor 30, and the inertial measurement unit 40, respectively, and is used to receive driving current, structural jitter, and image data and perform corresponding processing operations.

[0029] The driving current parameter refers to the current applied when driving the LED of the corresponding lighting channel to emit light. This parameter is directly read by the processing unit 50 from the feedback port of the LED driving circuit. Since the center wavelength of LED emission will drift slightly with the change of driving current, the change in the value of the driving current parameter can reflect the shift of the actual emission spectrum of the corresponding channel. The structural jitter parameter refers to the quantified value of the slight spatial displacement or attitude change caused by factors such as hand-holding, mechanical vibration, and airflow during the shooting process. It is usually acquired synchronously by the inertial measurement unit 40 at the exposure time of each frame of image, including the linear displacement increment in three directions and the angular deflection increment.

[0030] In practice, the multi-color LED lighting module 10 sequentially illuminates each LED channel according to a pre-set timing sequence. During the illumination of each channel, the image sensor 30 acquires a frame of facial image corresponding to the wavelength. During the exposure of each frame, the processing unit 50 synchronously records the driving current value corresponding to that frame and the structural jitter value output by the inertial measurement unit 40 at the exposure time of that frame. The driving current parameter and the structural jitter parameter are associated and bound to the facial acquisition status parameter of that frame image according to the frame number. For example, the data is stored as structured data in the format of four fields: frame number, channel identifier, driving current, and structural jitter, as a record.

[0031] This step integrates the electrical parameters of the lighting side with the displacement parameters of the mechanical side into facial acquisition state parameters, providing complete prior input data for analyzing the causes of offset from a physical perspective, and avoiding the problem of insufficient robustness of registration methods that rely solely on image content in weak texture scenes.

[0032] After obtaining the facial acquisition state parameters, it is necessary to further analyze the two types of physical causes that lead to image shift between channels. Since the driving current in the state parameters corresponds to the difference in the illumination spectrum, and the structural jitter corresponds to the difference in the mechanical motion of the device, the two cause dispersion displacement and structural shift, respectively. Therefore, S102 is specifically divided into two parallel branches: analyzing the dispersion displacement component and analyzing the structural shift component.

[0033] S102: Based on the facial acquisition status parameters, the dispersion displacement component caused by spectral differences and the structural offset component caused by the mechanical movement of the acquisition device are analyzed through a preset model.

[0034] First, we will introduce the analysis of the dispersion shift component. The dispersion shift component refers to the pixel-level displacement caused by the difference in the center wavelength of the emission spectrum of different LED channels and the different refractive indices when the light passes through the optical lens 20, resulting in the shift in the imaging position of the same object point in different channel images.

[0035] In practice, this process includes three sub-steps. First, the driving current parameters of the current channel and the driving current parameters of the reference channel are extracted from the facial acquisition state parameters, and the algebraic difference between the two is calculated. The reference channel is typically selected as a channel with a central wavelength, such as the green light channel.

[0036] Then, the algebraic difference is substituted into the pre-calibrated photoelectric response function to calculate the center wavelength drift value of the corresponding lighting channel. The photoelectric response function is a univariate function that describes the relationship between the center wavelength of the LED and the driving current. It is generally in the form of a linear or low-order polynomial. Its specific coefficients are obtained by calibrating the actual center wavelength of each channel under different currents using a spectrometer and then using least squares fitting before leaving the factory.

[0037] For example, the specific expression for the photoelectric response function can be: ,in, This represents the center wavelength drift value. The slope coefficient, Here, represents the intercept coefficient; when using a more precise second-order polynomial form, the specific expression for the photoelectric response function can be: ,in , , These are the coefficients to be calibrated. In practical applications, the algebraic difference obtained in the previous sub-step S102 is directly substituted into the above expression to calculate the center wavelength drift value of the current channel. ;coefficient , or , , The value can be determined based on the actual calibration results of the spectrometer before it leaves the factory.

[0038] Finally, the center wavelength drift value is input into the lens dispersion polynomial for algebraic operations to solve for the lateral and longitudinal spatial offset values ​​in the pixel coordinate system, which are then used as the dispersion displacement components. The lens dispersion polynomial is a polynomial function describing the relationship between the deviation of the focusing position of the optical lens 20 for different wavelengths of light and the wavelength. It is generally in the form of a second-order polynomial, and its coefficients are obtained by imaging a uniform grid calibration plate under illumination at multiple wavelengths, measuring the pixel offset of the grid corner points in each wavelength image relative to the reference wavelength image, and fitting the data using the least squares method.

[0039] For example, the specific expression for the lens dispersion polynomial can be taken in the following second-order form: , ,in and These are the dispersion displacement components on the horizontal and vertical axes of the pixel coordinate system, respectively. , , , These are the polynomial coefficients to be calibrated. In practical applications, the center wavelength drift value obtained in the previous sub-step is used as the coefficient. Substituting these values ​​into the horizontal and vertical expressions above and performing algebraic operations, we can obtain the projection values ​​of the dispersion shift components on the horizontal and vertical axes; coefficients to The value of can be determined based on the actual fitting results of the calibration plate imaging experiment.

[0040] The reason for using the above photoelectric response function and lens dispersion polynomial two-stage conversion to obtain the dispersion shift component after extracting the driving current parameters is that the true emission spectrum of the LED channel is difficult to measure directly, while the driving current is an electrical parameter that can be directly read in the system. By passing the current change to the spectral change and then to the pixel offset through the calibrated function chain, the imaging offset caused by spectral differences can be accurately predicted without adding spectral measurement hardware.

[0041] Next, we will discuss the analysis of the structure offset component. After completing the analysis of the dispersion shift component, since device jitter can also cause image shifts, it is necessary to analyze the structure offset component simultaneously. The structure offset component refers to the numerical value of the change in pixel position of the same object point in the image caused by the spatial displacement of the acquisition device during the interval between adjacent frame captures.

[0042] In specific implementation, firstly, the structural jitter parameters of adjacent moments (i.e., the current channel frame moment and the reference channel frame moment) in the facial acquisition state parameters are extracted. The difference between the two is used to obtain the relative displacement value of the acquisition device in physical space, with the unit being millimeters, including components in both the horizontal and vertical directions. Then, combining the magnification of the acquisition system and the unit pixel size of the image sensor 30, the relative displacement value in physical space is converted to the pixel coordinate system to obtain the corresponding pixel displacement value, which is used as the structural offset component. This conversion is essentially a simple proportional operation of multiplying the physical displacement by the magnification and then dividing by the unit pixel size.

[0043] For example, the specific expression for the conversion is: , ,in and These are the structural offset components on the horizontal and vertical axes of the pixel coordinate system, respectively. The optical magnification of the acquisition system. This refers to the pixel size of the image sensor 30. In practical applications, it is obtained by subtracting the structural jitter parameters at adjacent time points. and By substituting the values ​​into the two expressions above and performing multiplication and division operations, the projection values ​​of the structural offset components on the horizontal and vertical axes can be obtained; the values ​​of magnification M and unit pixel size p are determined by the optical design of the acquisition device and the hardware specifications of the image sensor 30.

[0044] For the overall analysis process of S102, please refer to [link / reference]. Figure 3 See also Figure 3 This application provides a schematic diagram of a facial acquisition state parameter parsing process, as shown in the embodiment. Figure 3 As shown, two parallel analytical branches are derived from the facial acquisition state parameters: the left branch starts from the driving current parameters and goes through three stages in sequence: algebraic difference calculation, photoelectric response function mapping, and lens dispersion polynomial operation to obtain the dispersion displacement component; the right branch starts from the structural jitter parameters and goes through two stages in sequence: adjacent time difference calculation and magnification and pixel size conversion to obtain the structural offset component; the two branches finally merge for the next step of nonlinear fitting.

[0045] By extracting electrical and mechanical parameters from the state parameters and mapping them to the pixel coordinate system through independent calibration models, the two causes of dispersion and jitter that were originally coupled in image offset can be decoupled, providing two clear input components for subsequent nonlinear fitting.

[0046] After obtaining the dispersive displacement component and the structural offset component in S102, these two types of physical components need to be combined into a total offset estimate that can guide subsequent searches. Since the two types of components have different dominance under different acquisition conditions—for example, when the device jitter is small, the offset mainly comes from spectral differences, while when the device jitter is large, the offset mainly comes from mechanical motion—direct linear addition cannot accurately reflect the true offset. Therefore, a nonlinear method that adapts to amplitude needs to be used for fusion.

[0047] S103: Perform nonlinear fitting between the dispersion shift component and the structural offset component to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image.

[0048] Nonlinear fitting refers to the adaptive combination of two types of displacement components using a weighting function related to the displacement amplitude, so that the structural offset component dominates when the jitter amplitude is large, and the dispersive displacement component dominates when the jitter amplitude is small, thus better reflecting the coupling relationship between the two types of factors in the actual imaging process.

[0049] In practice, the projection values ​​of the dispersion displacement component and the structural offset component onto the horizontal and vertical axes of the pixel coordinate system are first extracted, that is, the two displacement vectors are decomposed into horizontal and vertical components respectively. Then, a nonlinear weighting coefficient is calculated based on the magnitude of the structural offset component, where the magnitude of the structural offset component is obtained by taking the square root of the sum of the squares of its horizontal and vertical components.

[0050] The nonlinear weighting coefficients are adaptively switched using the Sigmoid function. When the structural offset amplitude is large, the weighting coefficients approach 1; when the structural offset amplitude is small, the weighting coefficients approach 0. The scale and threshold parameters in the Sigmoid function are pre-calibrated constants. For example, the scale parameter can be set to 0.5, and the threshold parameter can be set to 2 pixels, depending on the specific application scenario. Finally, the components are fused using the nonlinear weighting coefficients. The estimated registration deviations in the horizontal and vertical axes satisfy the following relationships: , In the formula, These are non-linear weighting coefficients. , These are the structural offset components in the horizontal and vertical directions, respectively. , These are the dispersion displacement components in the horizontal and vertical directions, respectively. , The registration deviation is estimated for the horizontal and vertical axes, respectively, and the final output is the estimated registration deviation composed of the estimated values ​​for the horizontal and vertical axes.

[0051] By introducing nonlinear weights, adaptive fusion of the two types of physical components is achieved, making the predicted registration error more representative under various acquisition conditions, and providing accurate center coordinates for locating the search area in the phase correlation map.

[0052] After obtaining the estimated registration bias in S103, although an initial offset value based on physical parameters is available, errors exist in calibration parameters such as the photoelectric response function, lens dispersion polynomial, and magnification. Furthermore, not all causes of offset are covered by the physical model. Therefore, this initial value still needs fine-tuning using image content information. A phase correlation map is a frequency domain registration tool that can centrally express the translation between two images in peak form, making it very suitable for fine-tuning the estimated bias. Therefore, a phase correlation map needs to be constructed before proceeding to S104.

[0053] Before determining the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, the method further includes: performing frequency domain transformation on the facial images of each channel and the facial image of the reference channel respectively, and multiplying the complex values ​​after transformation by conjugate to extract the pure phase information of the multiplication result; performing inverse frequency domain transformation on the pure phase information to output the phase correlation map in the spatial domain.

[0054] A phase correlation map is a spatial intensity distribution map generated by the Fourier transform phase correlation principle, which reflects the amount of translation between two images. The coordinates of its peak points correspond to the pixel-level offset between the two images.

[0055] In practice, firstly, two-dimensional fast Fourier transforms are performed on both the reference channel image and the current channel image to obtain two complex frequency domain matrices. Then, the complex conjugates of the current channel frequency domain matrix and the reference channel frequency domain matrix are multiplied element-wise to obtain the cross-power spectrum. Next, the cross-power spectrum is divided by its own amplitude to extract pure phase information, resulting in a normalized cross-power spectrum. Finally, a two-dimensional inverse fast Fourier transform is performed on the normalized cross-power spectrum to obtain the phase correlation map in the spatial domain. Ideally, the phase correlation map exhibits significant peaks at the true offset coordinates of the two images.

[0056] The reason for constructing a phase correlation map first instead of directly using the estimated registration deviation as the final result is that the estimated registration deviation only depends on the calibration values ​​of physical parameters, and there may be residual deviations caused by calibration errors or unmodeled factors. Therefore, it is necessary to use the image content information contained in the phase correlation map for fine correction. The peak position provided by the phase correlation map can reflect the true offset between the two images with pixel or even sub-pixel accuracy. The two complement each other to balance robustness and accuracy.

[0057] After constructing the phase correlation map, performing a global peak search directly on the entire phase correlation map is susceptible to interference from noise, edge effects, and false peaks introduced by brightness variations caused by spectral differences, leading to distorted search results. Considering that S103 has already obtained a relatively reliable estimated registration error, and that the fluctuation amplitude in the state parameters can reflect the uncertainty range of the estimated error, a local region can be defined in the phase correlation map as the search range, centered on the estimated registration error and bounded by the pixel radius converted from the fluctuation amplitude.

[0058] S104: Based on the fluctuation amplitude in the facial acquisition status parameters, determine the position fluctuation range for the estimated registration deviation, and delineate a local search area in the phase correlation diagram with the position fluctuation range as the boundary.

[0059] The position fluctuation range refers to a pixel radius value centered on the estimated registration deviation coordinates, representing the possible deviation range. This range reflects the propagation result of the uncertainty of state parameters in the pixel coordinate system during the acquisition process.

[0060] In the specific implementation, the numerical deviation of the structural jitter parameter in the facial acquisition state parameters is first extracted. The numerical deviation is specifically the standard deviation or range of the jitter signal acquired by the inertial measurement unit 40 during the frame exposure. The numerical deviation is determined as the fluctuation amplitude. Then, the fluctuation amplitude is associated with the preset offset redundancy to calculate the corresponding pixel radius value, which is used as the position fluctuation range.

[0061] The offset redundancy is a pre-set amplification factor, such as a redundancy range corresponding to 3 times the standard deviation, which can be set according to the stability of the actual acquisition environment. The conversion process is to first multiply the fluctuation amplitude by the offset redundancy to obtain the redundancy displacement value in physical space, and then convert it to pixel coordinates based on the amplification factor and unit pixel size. Finally, in the phase correlation map, the local search area is delineated with the coordinates corresponding to the estimated registration deviation as the center and the position fluctuation range as the boundary.

[0062] The results of the above local search area delineation can be found in [reference needed]. Figure 4 See also Figure 4 This application provides a schematic diagram of local search region delineation in a phase correlation graph, as shown in the embodiments below. Figure 4 As shown, within the global coordinate range of the phase correlation map, the estimated registration deviation obtained from S103 corresponds to a coordinate point P as the center, and the pixel radius R calculated from S104 is used as the radius to define a circular region S around point P. This circular region is the local search region, and pixels outside the region do not participate in the subsequent peak search. Figure 4 The diagram also schematically marks the relative positions of the integer-level peak points and sub-pixel offset coordinates that need to be located subsequently within the local search area S.

[0063] By limiting the search range to a reasonable area predicted by the physical model, the interference of spurious peaks caused by noise, edge effects or spectral differences in the phase correlation diagram on peak localization can be effectively avoided, improving the accuracy and robustness of peak search, while also reducing computational complexity compared to global search.

[0064] After defining the local search region in S104, the next step is to determine the peak points on the phase correlation map that truly reflect the offset between the two images within that region. However, due to the discreteness of the phase correlation map, there is a quantization error of no more than half a pixel between the maximum value point on the integer pixel grid and the true peak value. To meet the requirements of high-precision registration, further sub-pixel refinement is needed based on the integer-level peak points.

[0065] S105: Within the local search area, locate the intensity peak point in the phase correlation map and extract the corresponding subpixel offset coordinates.

[0066] Integer-level peak points refer to the points where the intensity of the phase correlation map is at its maximum on an integer pixel grid. Subpixel offset coordinates refer to continuous coordinate values ​​with a precision higher than one pixel, obtained by fitting a continuous surface to the discrete intensity values ​​near the peak.

[0067] In specific implementation, the process of locating integer-level peak points is as follows: The intensity values ​​of each pixel within the local search area are extracted one by one; then, all extracted intensity values ​​are compared, and the pixel with the highest intensity value is selected as the integer-level peak point. This process can be found in [reference needed]. Figure 4 The locations of integer-level peak points within the local search region S.

[0068] After obtaining the integer-level peak points, the sub-pixel offset coordinates are further extracted. The specific process is as follows: Using the integer-level peak points as the center, extract the intensity values ​​of a predetermined number of neighboring pixels, for example, pixels within a 3×3 or 5×5 neighborhood window; substitute the intensity values ​​of the integer-level peak points and neighboring pixels into a two-dimensional surface equation for surface fitting. The two-dimensional surface equation is a parabolic equation in quadratic polynomial form, and the coefficients are solved using the least squares method; solve for the coordinate solutions where the first derivative of the two-dimensional surface equation is zero, i.e., take the partial derivatives with respect to the horizontal and vertical axes and set them equal to zero, obtaining a system of linear equations about the horizontal and vertical coordinates. Solving this system yields the coordinates of the extreme points in continuous space; extract the corresponding continuous space values ​​as the sub-pixel offset coordinates.

[0069] The reason for performing surface fitting on top of integer peak points is that the true peak of the phase correlation map often does not fall precisely at integer pixel positions. Directly using integer coordinates will introduce a quantization error of up to half a pixel, while two-dimensional surface fitting can improve the offset estimation accuracy to the sub-pixel level.

[0070] After S105 completes the extraction of subpixel offset coordinates, these coordinates not only reflect the global correctness of the physical model's prediction, but also achieve fine correction with the help of image content, so they can be directly used as the final output.

[0071] S106: Determine the subpixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image.

[0072] In this step, the subpixel offset coordinates obtained in S105 are directly output as the final registration offset of the current channel face image relative to the reference channel face image in the horizontal and vertical directions, for use in subsequent image transformations.

[0073] In a preferred embodiment, after determining the sub-pixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image, the method further includes: performing pixel-level interpolation sampling on each channel facial image based on the horizontal and vertical values ​​of the final registration offset; extracting the corresponding channel pixel values ​​of each channel facial image and the reference channel facial image after the interpolation sampling operation, performing algebraic addition, and combining them into a multispectral facial image.

[0074] Pixel-level interpolation sampling refers to the process of performing reverse mapping sampling on each channel image based on the final registration offset to achieve sub-pixel-level image alignment. In practice, bilinear interpolation or bicubic interpolation algorithms can be used: for any pixel coordinate in the reference channel image, its corresponding sampling coordinate in the current channel image is calculated as that coordinate plus the final registration offset. If the sampling coordinate is not an integer, the interpolation intensity at that position is obtained by weighted summation based on the intensity values ​​of its neighboring integer pixels.

[0075] For example, taking bilinear interpolation as an example, let the sampling coordinates be... It falls between four adjacent integer pixels, and the four adjacent integer pixels are respectively , , , The corresponding strength values ​​are respectively , , , The specific expression for bilinear interpolation is: ,in and These are the sampling coordinates relative to the top-left integer pixel point. Decimal offsets in the horizontal and vertical directions This is the interpolation intensity of the current channel image at the reference pixel coordinates (x, y) after alignment.

[0076] After interpolation and resampling of all non-reference channel images, each channel image is spatially precisely aligned with the reference channel image. Then, the aligned pixel values ​​of each channel are algebraically added together as components of the corresponding channel in the multispectral image, resulting in a multispectral facial image with multiple channels. The algebraic addition of channel components to form a multispectral image does not involve summing the intensity values ​​of each channel to obtain a scalar grayscale image. Instead, it involves stacking the aligned two-dimensional intensity matrices of each channel in a pre-defined channel order into a multidimensional array. Each pixel coordinate (x, y) in this multidimensional array stores a multidimensional vector composed of the combined intensity values ​​of each LED channel, with each component of the vector corresponding to the reflection intensity of an LED channel.

[0077] For example, if the acquisition device includes four LED channels—blue, green, red, and near-infrared—the resulting multispectral facial image is an image data structure with four channel components. Each pixel location is composed of a combination of four independent channel intensity values, which can be used to extract any one channel or any combination of channels for subsequent colocalization analysis. In this way, the final synthesized multispectral facial image has no visible edge misalignment or false-color stripes between the channels and can be directly used for subsequent colocalization analysis, such as pigment distribution analysis and vascular network analysis.

[0078] The multi-channel image automatic registration method based on multi-color LED lighting provided in this application associates the driving current parameters of the lighting channels and the structural jitter parameters of the acquisition device as facial acquisition state parameters, making full use of the physical state information of the acquisition system itself to provide a priori basis for subsequent registration. Based on these state parameters, the dispersion shift component caused by spectral differences is analyzed through photoelectric response and lens dispersion models, and the structural offset component caused by device jitter is analyzed through mechanical motion models, thus clearly decomposing the causes of the offset at the physical level. Subsequently, the two types of components are nonlinearly fitted to obtain the estimated registration deviation, providing a reliable initial estimate for registration.

[0079] Furthermore, by combining the fluctuation amplitude in the state parameters to determine the position fluctuation range, and using this to delineate the local search area in the phase correlation map, the search range is narrowed, and the interference of noise and spurious peaks on the global search is reduced, making the localization of intensity peak points more robust. Finally, fine offset coordinates are extracted through sub-pixel level surface fitting to obtain a high-precision final registration offset.

[0080] Therefore, the method provided in this application embodiment uses physical model prediction to guide the registration search and frequency domain phase correlation and sub-pixel fitting to finely correct the coordinates, achieving high-precision and robust automatic registration in multi-color LED lighting acquisition scenarios with weak texture and significant spectral differences, thereby improving the quality and reliability of co-localization analysis multispectral imaging.

[0081] The following describes an automatic registration system for multi-channel images based on multi-color LED lighting for co-localization analysis, provided by an embodiment of this application. The automatic registration system for multi-channel images based on multi-color LED lighting described below can be referred to in conjunction with the automatic registration method for multi-channel images based on multi-color LED lighting described above.

[0082] See Figure 5 This application provides a structural diagram of a multi-channel image automatic registration system based on multi-color LED illumination for co-localization analysis, as shown in the embodiments below. Figure 5As shown, the system includes an acquisition module 100, an analysis module 200, a fitting module 300, a delineation module 400, a positioning module 500, and a determination module 600. The modules are connected sequentially according to the steps in the aforementioned method embodiment. Specifically, the output of the acquisition module 100 is connected to the input of the analysis module 200, the output of the analysis module 200 is connected to the input of the fitting module 300, the output of the fitting module 300 is connected to the input of the delineation module 400, the output of the delineation module 400 is connected to the input of the positioning module 500, and the output of the positioning module 500 is connected to the input of the determination module 600. Each module will be described in detail below.

[0083] The acquisition module 51 is used to acquire the driving current parameters of each illumination channel and the structural jitter parameters of the acquisition device during the facial image acquisition process, and associate them with the facial acquisition status parameters of the corresponding image frame.

[0084] The analysis module 52 is used to analyze the dispersion shift component caused by spectral differences and the structural offset component caused by the mechanical movement of the acquisition device based on the facial acquisition state parameters and a preset model.

[0085] The fitting module 53 is used to perform nonlinear fitting between the dispersion shift component and the structural offset component to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image.

[0086] The delineation module 54 is used to determine the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, and to delineate a local search area in the phase correlation map with the positional fluctuation range as the boundary.

[0087] The positioning module 55 is used to locate the intensity peak point in the phase correlation map within the local search area and extract the corresponding subpixel offset coordinates.

[0088] The determination module 56 is used to determine the subpixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image.

[0089] The multi-channel image automatic registration system based on multi-color LED lighting provided in this application acquires complete physical state parameters through the acquisition module 51. The analysis module 52 maps the physical parameters into image offset components, which are then fused by the fitting module 53 to obtain a preliminary estimated deviation. Subsequently, the delineation module 54 and the positioning module 55 complete peak search and sub-pixel fitting within the local search area of ​​the phase correlation map. Finally, the determination module 56 outputs a high-precision registration offset. Each module guides the coordinate search with physical model estimation, constrains the search area with structural rules, and refines the final offset with sub-pixel fitting. This achieves high-precision and robust automatic registration in multi-color LED lighting acquisition scenarios with weak texture and significant spectral differences, improving the synthesis quality and analysis reliability of multi-channel images in co-localization analysis.

[0090] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0091] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0092] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for automatic registration of multi-channel images based on co-localization analysis using multi-color LED illumination, characterized in that, include: The driving current parameters and structural jitter parameters of the acquisition device corresponding to each illumination channel are obtained during the facial image acquisition process, and correlated as facial acquisition state parameters of the corresponding image frame. Based on the facial acquisition state parameters, the dispersion shift component caused by spectral differences and the structural offset component caused by the mechanical movement of the acquisition device are analyzed by a preset model. The dispersion shift component and the structural offset component are nonlinearly fitted to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image. Based on the fluctuation amplitude in the facial acquisition status parameters, the position fluctuation range for the estimated registration deviation is determined, and a local search area is delineated in the phase correlation map with the position fluctuation range as the boundary. Within the local search area, the intensity peak point in the phase correlation map is located, and the corresponding sub-pixel offset coordinates are extracted; The subpixel offset coordinates are determined as the final registration offset of each channel facial image relative to the reference channel facial image; The step of resolving the dispersion shift component caused by spectral differences based on the facial acquisition state parameters using a preset model includes: The driving current parameters of each illumination channel are extracted from the facial acquisition state parameters, and the algebraic difference between the driving current parameters and the driving current parameters of the reference channel is calculated. Substituting the algebraic difference into the pre-calibrated photoelectric response function, the center wavelength drift value of the corresponding illumination channel is calculated; The center wavelength drift value is input into the lens dispersion polynomial for algebraic operation to solve for the spatial offset value in the pixel coordinate system, which is used as the dispersion displacement component. The step of analyzing the structural offset component generated by the mechanical movement of the acquisition device based on the facial acquisition state parameters using a preset model includes: Extract the structural jitter parameters of adjacent moments from the facial acquisition state parameters to obtain the relative displacement value of the acquisition device in physical space; By combining the magnification of the acquisition system and the unit size of the image sensor, the relative displacement values ​​in the physical space are converted to pixel coordinates to obtain the corresponding pixel displacement values, which are used as structural offset components.

2. The method according to claim 1, characterized in that, The step of performing nonlinear fitting between the dispersion shift component and the structural offset component to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image includes: Extract the projection values ​​of the dispersion displacement component and the structure offset component onto the horizontal and vertical axes of the pixel coordinate system; Calculate the nonlinear weighting coefficient based on the magnitude of the structural offset component; The nonlinear weighting coefficients are multiplied by the projected values ​​of the horizontal and vertical axes respectively, and then vectored together to output the predicted registration deviation, which is composed of the predicted values ​​of the horizontal and vertical axes.

3. The method according to claim 1, characterized in that, Before determining the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition status parameters, the method further includes: Frequency domain transformation is performed on the facial images of each channel and the facial image of the reference channel, and the complex values ​​after transformation are multiplied by conjugate to extract the pure phase information of the multiplication result; Perform an inverse frequency domain transform on the pure phase information to output the phase correlation map in the spatial domain.

4. The method according to claim 1, characterized in that, The step of determining the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, and delineating a local search area in the phase correlation map using the positional fluctuation range as the boundary, includes: Extract the numerical deviation of the structural jitter parameter from the facial acquisition state parameters, and determine the numerical deviation as the fluctuation amplitude; The corresponding pixel radius value is calculated by associating the fluctuation amplitude with a preset offset redundancy, and is used as the position fluctuation range. In the phase correlation diagram, a local search area is defined with the coordinates corresponding to the estimated registration deviation as the center and the position fluctuation range as the boundary.

5. The method according to claim 1, characterized in that, Locating the intensity peak point in the phase correlation map within the local search area includes: Extract the intensity value of each pixel within the local search area one by one; Compare all extracted intensity values, select the pixel with the highest intensity value, and use the pixel with the highest intensity value as the integer peak point.

6. The method according to claim 5, characterized in that, The extraction of the corresponding sub-pixel offset coordinates includes: Using the integer-level peak point as the center, extract the intensity values ​​of a preset number of neighboring pixels; Substitute the intensity values ​​of the integer-level peak points and the neighboring pixels into the two-dimensional surface equation to perform surface fitting, and solve the coordinate solution of the two-dimensional surface equation where the first derivative is zero. Extract the continuous spatial values ​​corresponding to the coordinate solutions as sub-pixel offset coordinates.

7. The method according to claim 1, characterized in that, After determining the subpixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image, the method further includes: Based on the horizontal and vertical values ​​of the final registration offset, pixel-level interpolation sampling is performed on the facial images of each channel; The corresponding channel pixel values ​​of the facial images in each channel after interpolation sampling are extracted and compared with the corresponding channel facial images in the reference channel. The values ​​are then algebraically added and combined to form a multispectral facial image.

8. A multi-channel image automatic registration system based on multi-color LED illumination for co-localization analysis, used to execute the multi-channel image automatic registration method based on multi-color LED illumination for co-localization analysis as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to obtain the driving current parameters of each illumination channel and the structural jitter parameters of the acquisition device during the facial image acquisition process, and associate them with the facial acquisition status parameters of the corresponding image frame. The analysis module is used to analyze the dispersion shift component caused by spectral differences and the structural offset component caused by the mechanical movement of the acquisition device based on the facial acquisition state parameters and a preset model. The fitting module is used to perform nonlinear fitting between the dispersion shift component and the structural offset component to determine the estimated registration deviation of each channel facial image relative to the reference channel facial image. The delineation module is used to determine the positional fluctuation range for the estimated registration deviation based on the fluctuation amplitude in the facial acquisition state parameters, and to delineate a local search area in the phase correlation map with the positional fluctuation range as the boundary. The positioning module is used to locate the intensity peak point in the phase correlation map within the local search area and extract the corresponding subpixel offset coordinates. The determination module is used to determine the subpixel offset coordinates as the final registration offset of each channel facial image relative to the reference channel facial image.

Images