A physiological time series signal and skin image cross-modal feature extraction method

Abstract

The application discloses a physiological time sequence signal and skin image cross-modal feature extraction method, relates to the technical field of data processing, and has the following steps: outputting corresponding mapping conditions in response to a preset mapping relationship between a physiological sensing channel and a skin image; extracting a feature calibration block corresponding to the physiological sensing channel position from the target skin image according to the mapping conditions; obtaining signal feature data of the physiological time sequence signal, obtaining a physiological time sequence feature sequence according to the signal feature data and a modal registration correction parameter; obtaining a fusion cross-modal feature according to the physiological time sequence feature sequence and a coupling correlation factor; determining a disturbance source and outputting corresponding feature sets based on the comparison result of the fusion cross-modal feature and a preset reference skin texture threshold value and whether the skin texture distortion feature belongs to the feature calibration block, so that the reliability and the explainability of the skin physiological state detection result are improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a method for cross-modal feature extraction of physiological time-series signals and skin images. Background Technology

[0002] Traditional processing methods lack a unified mapping benchmark, making it impossible to pinpoint the effective imaging area corresponding to the physiological sensing channel in the skin image. They can only perform global operations on the entire target skin image, which not only significantly increases hardware computing power consumption but also introduces image edge clutter interference. It is difficult to extract standardized image patches with clear boundaries that match the sensing acquisition range, and there is a lack of reliable feature calibration carriers.

[0003] Existing technologies lack in-depth utilization of long-term historical data, relying solely on single-frame, single-acquisition data to complete feature calculations. They cannot quantify the long-term coupling interference patterns between skin optical grayscale and physiological temporal amplitude, cannot construct historical modal difference sequences recording changes in skin optical offset, have difficulty distinguishing between inherent offsets in device hardware and numerical fluctuations caused by changes in the skin's own state, cannot quantify the mutual restraint coupling strength between the two types of modalities, and lack a unified quantitative benchmark for interference suppression.

[0004] Traditional multimodal data fusion processes lack hierarchical correction processing, directly splicing and fusing the original physiological time-series signals with skin image features without performing spatial offset compensation or noise reduction and smoothing correction on the physiological time-series signals. Hardware spatial offset and instantaneous acquisition noise of the original signals will continue to be superimposed into the fusion result. At the same time, there is no standardized scale adaptation process, and the sampling scale of physiological time-series signals and the pixel scale of skin images cannot be unified. The features generated by the final fusion have problems of dimensional misalignment and error superposition, making it difficult to synchronously and realistically reflect the skin texture state and physiological time-series change patterns. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method for cross-modal feature extraction of physiological time-series signals and skin images.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for cross-modal feature extraction of physiological time-series signals and skin images includes the following steps: The corresponding mapping conditions are output in response to the preset mapping relationship between the physiological sensing channel and the skin image. Based on the mapping conditions, feature calibration blocks corresponding to the physiological sensing channel locations are extracted from the target skin image; Historical skin optical grayscale data and historical physiological temporal amplitude data affected by skin grayscale coupling were statistically analyzed over historical acquisition periods. Based on the historical skin optical grayscale data and historical physiological temporal amplitude data, a historical modal difference sequence was obtained. The coupling correlation factor is obtained based on the coupling correlation trend between the feature-labeled blocks, the historical modality difference sequence, and the historical mapping distance value. Acquire signal feature data of physiological time-series signals, and obtain physiological time-series feature sequences based on the signal feature data and modal registration correction parameters; The fused cross-modal features are obtained based on physiological temporal feature sequences and coupling correlation factors; Based on the comparison results of the fused cross-modal features and the preset benchmark skin texture threshold, and whether the skin texture distortion features belong to the feature calibration block, the source of the disturbance is determined and the corresponding feature set is output.

[0007] Preferably, the corresponding mapping conditions are output in response to the preset mapping relationship between the physiological sensing channel and the skin image, specifically including the following steps: A global matching benchmark between physiological sensing channels and skin images is pre-constructed. Spatial alignment results are obtained by calibrating the spatial dimensions of physiological sensing channels and skin images based on the global matching benchmark. Spatial matching constraints are generated based on spatial alignment results, and the overlapping sections of the boundaries between physiological sensing channels and skin image imaging areas are identified based on spatial matching constraints. Constraint verification is performed on the overlapping boundary sections to obtain the verification judgment result, and mapping conditions are generated based on the verification judgment result.

[0008] Preferably, the feature calibration block corresponding to the physiological sensing channel position is extracted from the target skin image according to the mapping conditions, specifically including the following steps: Based on the mapping conditions, global pixel-level spatial registration of the target skin image is performed to generate a registered complete skin image; The initial image cropping region is obtained based on the registered complete skin image and the direction of the skin surface texture. The cropped area of ​​the initial image is regularized to obtain a regularized image patch. After identifying the boundaries of skin basal pigment distribution based on regular image patches, the texture partitions are divided into hierarchical regions. The overlapping image regions of the sensing acquisition path are selected based on multi-level texture partitioning, and feature calibration blocks are generated by integrating the overlapping image regions.

[0009] Preferably, the initial image cropping region is obtained based on the registered complete skin image and the skin surface texture direction, specifically including the following steps: The skin spatial range covered by the physiological sensor channel is determined based on the registered complete skin image; The initial image cropping area is defined by following the direction of the skin's surface texture.

[0010] Preferably, the historical modal difference sequence is obtained based on historical skin optical grayscale data and historical physiological temporal amplitude data, specifically including the following steps: The numerical offset is obtained based on historical skin optical grayscale data and historical physiological temporal amplitude data; The persistent offset components of the skin's optical state are obtained based on the numerical offset, and the persistent offset components are arranged sequentially to form a continuously arranged historical modal difference sequence.

[0011] Preferably, the coupling correlation factor is obtained based on the coupling correlation trend between the feature-labeled blocks, the historical mode difference sequence, and the historical mapping distance value, specifically including the following steps: Extract the global pixel grayscale distribution of the feature calibration block, retrieve the fluctuation trend of the historical modal difference sequence, and extract the continuous change trend of the historical mapping distance value. Determine the strength of the linkage between the grayscale distribution state, fluctuation trend, and continuous change pattern of pixels across the entire domain; Based on the strength of the linkage changes, the synchronous change interval is obtained, and the coupling correlation factor is obtained by processing the offset amplitude of the synchronous change interval.

[0012] Preferably, the method further includes the following steps: Preset modal registration and correction parameters based on the original acquired parameters; Image reconstruction processing, performed with modal registration correction parameters on skin images, yields skin texture distortion detection results. Skin texture distortion features are obtained based on the skin texture distortion detection results.

[0013] Preferably, obtaining the physiological time-series feature sequence based on the signal feature data and modal registration correction parameters specifically includes the following steps: Spatial offset compensation is performed on the signal feature data using modal registration correction parameters to generate compensated time series data. Denoising time series data is generated based on the compensated time series data. The denoising time series data is then continuously segmented and smoothed to generate physiological time series feature sequences.

[0014] Preferably, the fused cross-modal features are obtained based on the physiological time-series feature sequences and coupling correlation factors, specifically including the following steps: The physiological time-series feature sequences are subjected to interference processing based on coupling correlation factors to obtain the time-series base features after interference suppression. Modal collaborative intermediate features are obtained based on the gray-level correlation weights of temporal basis features and coupling correlation factors after interference suppression; Scale-adaptive features are obtained by performing scale processing on the spatial arrangement features corresponding to the mapping conditions of the intermediate features of modal collaboration. The scale-adaptive features are aggregated with the skin texture information corresponding to the feature-labeled blocks to generate fused cross-modal features.

[0015] Preferably, based on the comparison results of the fused cross-modal features and the preset benchmark skin texture threshold, and whether the skin texture distortion features belong to the feature calibration block, the source of the disturbance is determined and the corresponding feature set is output, specifically including the following steps: Calculate the feature values ​​of each spatial point within the fused cross-modal features and the point texture deviation value between the feature values ​​and the preset benchmark skin texture threshold, statistically analyze the distribution of points with deviation values ​​greater than zero and less than or equal to zero, and generate global texture deviation distribution statistical results. Determine whether each segment of the skin texture distortion feature falls within the spatial coordinate range of the feature calibration block, and generate an attribution determination result; Based on the statistical results and attribution determination results of the global texture deviation distribution, when points with deviation values ​​greater than zero are concentrated in texture segments belonging to the feature calibration block, the source of disturbance is determined to be an internal disturbance; when points with deviation values ​​greater than zero are concentrated in texture segments belonging to the outside of the feature calibration block, the source of disturbance is determined to be an external disturbance; when points with positive and negative deviation values ​​are evenly distributed in the internal and external segments, the source of disturbance is determined to be a modal cross disturbance. Based on the determined type of disturbance source, corresponding spatial points and feature values ​​are selected to generate a feature set of the corresponding type.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention generates standardized mapping conditions based on a pre-defined mapping relationship between physiological sensors and skin images. This allows for the extraction of feature calibration blocks corresponding to the sensor locations, locking in the effective skin detection area from the source and significantly eliminating invalid pixel data outside the imaging range. By combining the feature calibration blocks, historical modal difference sequences, and historical mapping distance values, a coupling correlation factor is calculated to quantify the stable coupling interference components between skin optical grayscale and physiological temporal amplitude. This distinguishes between long-term stable modal shifts and instantaneous random acquisition noise, providing a quantitative correction benchmark for signal interference suppression and avoiding feature distortion caused by mutual interference between the two types of modal data. Through modal registration correction parameters, signal spatial shift compensation and noise reduction segmented smoothing are completed to obtain a standardized physiological temporal feature sequence. Combined with the coupling correlation factor, interference suppression, modal collaboration, scale adaptation, and cross-modal feature aggregation operations are performed layer by layer, fully integrating two heterogeneous dimensions of information: temporal physiological changes and skin texture distortion, thereby improving the reliability and interpretability of skin physiological state detection results. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for extracting cross-modal features from physiological time-series signals and skin images, provided by an embodiment of the present invention. Figure 2 This invention provides a flowchart illustrating the generation of mapping conditions in a method for cross-modal feature extraction of physiological time-series signals and skin images. Figure 3 This is a schematic diagram illustrating the process of generating feature calibration blocks in a cross-modal feature extraction method for physiological time-series signals and skin images, provided by an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the process of obtaining the initial image cropping region in a method for cross-modal feature extraction of physiological time-series signals and skin images, as provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0021] Reference Figures 1-4 As shown.

[0022] The embodiments further illustrate the cross-modal feature extraction method for physiological time-series signals and skin images proposed in this invention.

[0023] A method for cross-modal feature extraction of physiological time-series signals and skin images includes the following steps: The corresponding mapping conditions are output in response to the preset mapping relationship between the physiological sensing channel and the skin image. Based on the mapping conditions, feature calibration blocks corresponding to the physiological sensing channel locations are extracted from the target skin image; Historical skin optical grayscale data and historical physiological temporal amplitude data affected by skin grayscale coupling were statistically analyzed over historical acquisition periods. Based on the historical skin optical grayscale data and historical physiological temporal amplitude data, a historical modal difference sequence was obtained. The coupling correlation factor is obtained based on the coupling correlation trend between the feature-labeled blocks, the historical modality difference sequence, and the historical mapping distance value. Acquire signal feature data of physiological time-series signals, and obtain physiological time-series feature sequences based on signal feature data and modal registration correction parameters; The fused cross-modal features are obtained based on physiological temporal feature sequences and coupling correlation factors; Based on the comparison results of the fused cross-modal features and the preset benchmark skin texture threshold, and whether the skin texture distortion features belong to the feature calibration block, the source of the disturbance is determined and the corresponding feature set is output.

[0024] The fused cross-modal features contain the fused feature values ​​corresponding to each spatial point in the image. The preset benchmark skin texture threshold is a fixed constant pre-set based on healthy standard skin samples. The formula for calculating the feature difference of a single point is: Point texture deviation value = Single spatial point fused cross-modal feature value - Preset benchmark skin texture threshold. By traversing all image spatial points covered by the fused cross-modal features and substituting them into the formula, the difference is solved. The total number of points with texture deviation values ​​greater than 0 and the total number of points with texture deviation values ​​less than or equal to 0 are counted to generate a global texture deviation distribution statistical result. This statistical result is a complete comparison result between the fused cross-modal features and the preset benchmark skin texture threshold, which can quantitatively characterize the distribution range and deviation magnitude of the skin texture features in the entire image from the standard benchmark.

[0025] To determine whether skin texture distortion features belong to a feature calibration block, a second core judgment criterion is generated. The global segmented texture distortion difference of the skin texture distortion feature and the spatial boundary coordinates of the corresponding segmented texture are used. The feature calibration block stores its own horizontal and vertical coordinate extreme values. The horizontal attribution relationship is determined as follows: Horizontal attribution flag value = minimum horizontal coordinate of the segmented texture - minimum horizontal coordinate of the feature calibration block. If the horizontal attribution flag value is greater than or equal to 0, and the maximum horizontal coordinate of the segmented texture is less than or equal to the maximum horizontal coordinate of the feature calibration block, then the horizontal interval of the segmented texture is determined to fall within the calibration block. Vertical attribution flag value = minimum vertical coordinate of the segmented texture - minimum vertical coordinate of the feature calibration block. If the vertical attribution flag value is greater than or equal to 0, and the maximum vertical coordinate of the segmented texture is less than or equal to the maximum vertical coordinate of the feature calibration block, then the vertical interval of the segmented texture is determined to fall within the calibration block. When a single texture segment satisfies both the horizontal and vertical attribution conditions, the skin texture distortion feature of that segment is determined to belong to the feature labeling block. If any single dimension does not meet the conditions, the distortion feature of that segment is determined to belong to the outside of the block. The attribution labeling is completed by traversing all texture segments one by one, and the attribution labeling results of all segments are summarized to form a complete attribution determination conclusion.

[0026] If most points with texture deviation values ​​greater than 0 are concentrated in texture segments marked as internal, the source of the disturbance is determined to be an intrinsic disturbance caused by physiological texture distortion of the skin itself. If most points with texture deviation values ​​greater than 0 are concentrated in texture segments marked as external, the source of the disturbance is determined to be an extrinsic disturbance caused by image acquisition edge clutter. If points with positive and negative texture deviations are evenly distributed in internal and external segments, the source of the disturbance is determined to be modal cross-disturbance caused by the coupling shift between physiological temporal signals and skin optical modalities.

[0027] The system classifies, filters, and integrates cross-modal features and skin texture distortion features across the entire domain to generate dedicated feature sets for each perturbation type. For intrinsic perturbation markers, all spatial points belonging to the internal domain with texture deviation values ​​greater than 0 are selected. The cross-modal feature values ​​of these points are extracted and sequentially concatenated with the corresponding segmental skin texture distortion differences to form the intrinsic perturbation feature set. For extrinsic perturbation markers, all spatial points belonging to the external domain with texture deviation values ​​greater than 0 are selected. The cross-modal feature values ​​of these points are extracted and sequentially concatenated with the corresponding segmental skin texture distortion differences to form the extrinsic perturbation feature set. For modal cross-perturbation markers, feature values ​​of all positive and negative deviation points belonging to both the internal and external domains are extracted evenly and integrated according to spatial arrangement to form the modal cross-perturbation feature set.

[0028] After responding to the preset mapping relationship between the physiological sensing channel and the skin image, the corresponding mapping conditions are output, specifically including the following steps: A global matching benchmark between physiological sensing channels and skin images is pre-constructed. Spatial alignment results are obtained by calibrating the spatial dimensions of physiological sensing channels and skin images based on the global matching benchmark. Spatial matching constraints are generated based on spatial alignment results, and the overlapping sections of the boundaries between physiological sensing channels and skin image imaging areas are identified based on spatial matching constraints. Constraint verification is performed on the overlapping boundary sections to obtain the verification judgment result, and mapping conditions are generated based on the verification judgment result.

[0029] A global matching baseline is pre-established, storing basic coordinate transformation operations. The x-coordinate of the target image pixel = x-coordinate of the baseline image pixel × scale conversion coefficient + lateral offset + rotation correction value; the y-coordinate of the target image pixel = y-coordinate of the baseline image pixel × scale conversion coefficient + lateral offset + rotation correction value. The rotation correction value is converted from radians to pixel displacement before being used in the calculation. Real-time acquired raw data from the physiological sensor channel and raw skin image data are input into the global matching baseline for alignment calibration. During the calculation, the spatial position difference between the physiological sensor acquisition points and the skin image pixel points is compared pixel by pixel. The scale conversion coefficient, lateral offset, lateral offset, and rotation correction value are continuously iterated and adjusted until the spatial position difference between corresponding points of the two types of data converges to a preset minimum difference threshold. After the iteration terminates, a stable spatial alignment result is output. The spatial alignment result includes a complete physiological sensor point coordinate sequence and a skin image pixel matrix after unified coordinate transformation.

[0030] To generate spatial matching constraints and identify overlapping sections, spatial matching constraints are generated based on the spatial alignment results after unifying coordinates. These constraints are then used to divide the overlapping sections of the two types of data imaging regions. When generating spatial matching constraints, the set of boundary coordinates of the complete imaging coverage area of ​​the physiological sensor channel within the spatial alignment results is extracted, along with the set of boundary coordinates of the complete imaging coverage area of ​​the skin image. The total number of overlapping pixels is calculated as: Total number of pixels in the physiological sensor region + Total number of pixels in the skin image region - Total number of pixels after merging the two regions. This total number of overlapping pixels is used as a quantitative indicator of the spatial matching constraint. This, along with the overlapping range of the region boundary coordinates and the difference in the region center coordinates, forms the complete spatial matching constraint. The spatial matching constraint clearly defines the numerical range for determining the effective overlapping region. Regions whose calculation results fall within the corresponding numerical range are considered valid matching regions. After generating complete spatial matching constraints, the quantitative indicators within the constraints are used to traverse all boundary coordinate points within the spatial alignment result. The coordinate difference between the boundary points of the physiological sensing channel and the boundary points of the skin image is compared point by point. When the coordinate difference is less than the preset coordinate difference threshold, the point is determined to be an overlapping boundary point. The continuously distributed overlapping boundary points are then connected in sequence to obtain a complete boundary overlapping section. The boundary overlapping section is a continuous boundary area where the physiological sensing acquisition range and the skin image shooting range completely overlap.

[0031] Constraint verification is performed on overlapping boundary sections. This verification operation retrieves all quantized indicators within the spatial matching constraints generated in the second layer. The total number of overlapping pixels, the length of overlap at the boundary coordinates, and the difference in the region center coordinates, calculated in real-time for the overlapping boundary sections, are compared with the standard numerical ranges within the spatial matching constraints. The indicator deviation value is calculated as: Real-time indicator value of the overlapping boundary section - Median value of the spatial matching constraint standard (e.g., pixels, pixel length). After calculating the indicator deviation values ​​for all indicators, the number of indicators exceeding the allowable deviation threshold is counted to form the verification judgment result. A compliance judgment state is when all indicator deviation values ​​do not exceed the allowable deviation threshold, indicating that the spatial matching accuracy between the current physiological sensing channel and the skin image meets the system standard. An abnormal judgment state is when any indicator deviation value exceeds the allowable deviation threshold, indicating that there is an accuracy defect in the spatial matching of the two types of data. Corresponding mapping conditions are generated according to different judgment states. The mapping conditions are used to define the corresponding conversion rules between physiological sensor values ​​and skin image pixel features. If the verification judgment result is a compliant judgment state, a high-precision direct mapping condition is generated: skin image pixel feature value = physiological sensor acquisition value × feature conversion weight + basic correction value. If the verification judgment result is an abnormal judgment state, a compensation correction mapping condition is generated: skin image pixel feature value = physiological sensor acquisition value × feature conversion weight + basic correction value + index deviation compensation value. Index deviation compensation value = first type index deviation value × first weight coefficient + second type index deviation value × second weight coefficient + third type index deviation value × third weight coefficient.

[0032] Based on the mapping conditions, feature calibration blocks corresponding to the physiological sensing channel locations are extracted from the target skin image, specifically including the following steps: Based on the mapping conditions, global pixel-level spatial registration of the target skin image is performed to generate a registered complete skin image; The initial image cropping region is obtained based on the registered complete skin image and the direction of the skin surface texture, specifically including the following steps: The skin spatial range covered by the physiological sensor channel is determined based on the registered complete skin image; The initial image cropping area is defined along the skin's surface texture; The cropped area of ​​the initial image is regularized to obtain a regularized image patch. After identifying the boundaries of skin basal pigment distribution based on regular image patches, the texture partitions are divided into hierarchical regions. The overlapping image regions of the sensing acquisition path are selected based on multi-level texture partitioning, and feature calibration blocks are generated by integrating the overlapping image regions.

[0033] Based on the mapping conditions, the target skin image undergoes global pixel-level spatial registration, ultimately outputting a registered complete skin image. Utilizing the standardized transformation features between physiological sensor data and skin image pixels, misalignment between pixel coordinates within a single target skin image and physiological sensor acquisition coordinates is eliminated, achieving binding between each pixel location and its corresponding sensor acquisition point in the entire image. The mapping conditions store pixel transformation calculation relationships adapted to the matching accuracy of the current device. The registered pixel x-coordinate = original target image pixel x-coordinate × pixel scaling factor + horizontal mapping compensation value; the registered pixel y-coordinate = original target image pixel y-coordinate × pixel scaling factor + vertical mapping compensation value. Both horizontal and vertical mapping compensation values ​​are uniformly expressed in pixels, and both types of compensation values ​​are directly taken from pre-stored parameters within the mapping conditions. If the pre-verification result is an abnormal judgment state, error compensation calculation needs to be superimposed. The complete conversion formula after superimposed compensation is: after registration, pixel x-coordinate = original target image pixel x-coordinate × pixel scaling factor + horizontal mapping compensation value + horizontal deviation compensation amount; after registration, pixel y-coordinate = original target image pixel y-coordinate × pixel scaling factor + vertical mapping compensation value + vertical deviation compensation amount. Traverse all pixel points inside the target skin image, and substitute them one by one into the corresponding conversion formula to complete the coordinate remapping. After all pixel points have completed the coordinate update, they are stitched together to form a complete skin image with no coordinate misalignment and corresponding registration between pixels and sensor points.

[0034] Using the registered complete skin image and the skin surface texture direction as joint input, the initial image cropping region is extracted step by step. The registered complete skin image is used to lock the skin space covered by the physiological sensor channel. The pixel coordinate set of all bound sensor acquisition points within the registered image is extracted, and the boundary extrema of the coordinate set are calculated. The spatial range boundary determination formula is: minimum x-coordinate of the region = minimum x-coordinate of all bound pixels; maximum x-coordinate of the region = maximum x-coordinate of all bound pixels; minimum y-coordinate of the region = minimum y-coordinate of all bound pixels; maximum y-coordinate of the region = maximum y-coordinate of all bound pixels. A rectangular basic range is selected using these four extrema values. This rectangular basic range represents the skin space completely covered by the physiological sensor channel. Image pixels outside this extrema range are all determined as invalid regions not covered by the sensor. The initial image cropping region is delineated along the skin surface texture direction. Texture gradient extraction is performed on the registered complete skin image. The single pixel texture gradient value = horizontal texture gray-level difference + vertical texture gray-level difference. The horizontal texture gray-level difference = the difference between adjacent left and right pixels. The result of subtracting the gray values ​​of individual pixels is the vertical texture gray value, which is the result of subtracting the gray values ​​of adjacent pixels. After calculating the texture gradient value pixel by pixel, the pixels with consecutive and similar gradient values ​​are concatenated to form texture direction lines. The texture direction lines are used as the segmentation boundary to cut the locked skin space range, and the continuous image area extending along the texture lines is retained. This continuous area is the initial image cropping area. Regularization processing is performed on the initial image cropping area to generate regular image areas. Regularization processing includes two types of operations: boundary smoothing and size normalization. The boundary smoothing operation formula is: smoothed boundary pixel x-coordinate = sum of the x-coordinates of the three adjacent boundary pixels ÷ 3, smoothed boundary pixel y-coordinate = sum of the y-coordinates of the three adjacent boundary pixels ÷ 3. The size normalization operation formula is: the number of horizontal pixels in the normalized area = the number of horizontal pixels of the fixed reference, the number of vertical pixels in the normalized area = the number of vertical pixels of the fixed reference. Pixels exceeding the reference size are cropped, and areas that do not reach the reference size are supplemented with blank boundary pixels. After eliminating jagged edges through boundary smoothing and unifying the area size through size normalization, a regular image area with smooth edges and uniform size is output.

[0035] Based on the identification of skin basal pigment distribution boundaries in regular image regions, multi-level texture partitioning is performed based on pigment boundaries. Multiple independent texture regions are split according to differences in skin pigment concentration, distinguishing skin sub-regions with different skin tones. Basal pigment feature values ​​of each pixel within the regular image region are extracted. These pigment feature values ​​are calculated by weighting the gray levels of the red, green, and blue channels of the image. The formula for calculating pigment features is: Single pixel basal pigment value = Red channel gray value × Red channel weight + Green channel gray value × Green channel weight + Blue channel gray value × Blue channel weight. Multi-level pigment gradient thresholds are defined according to pigment value ranges. Pixels corresponding to different gradient thresholds are sequentially marked, and consecutively distributed pixels under the same gradient threshold are concatenated to form a single-level pigment distribution boundary. Multiple gradient boundaries together constitute the complete skin basal pigment distribution boundary. Regular image regions are segmented using each level of pigment distribution boundary as a dividing line. Each consecutive image sub-region enclosed by a separate boundary is separately divided into a first-level texture partition. After all segments are completed, multi-level texture partitions are obtained. Different levels of texture partitions correspond to independent skin regions with clearly distinguishable pigment depths.

[0036] Using multi-level texture partitions as the filtering medium, overlapping image regions matching the sensor acquisition path are selected. All overlapping image regions are integrated to generate the final feature calibration blocks. The subdivided regions after hierarchical splitting are then re-integrated into standardized blocks suitable for physiological feature analysis. Each texture partition is traversed, and all pixels within it are compared with the coordinate range of the physiological sensor acquisition path. The formula for determining region overlap is: Number of overlapping pixels in a partition = Total number of pixels within the partition whose x-coordinate falls within the horizontal extreme range of the sensor path and whose y-coordinate falls within the vertical extreme range of the sensor path. A minimum overlapping pixel threshold is set. When the number of overlapping pixels in a single-level texture partition exceeds the minimum overlapping pixel threshold, the partition is determined to be an overlapping image region of the sensor acquisition path. Partitions that do not meet the quantity condition are discarded and not included in subsequent integration calculations. After completing the overlap determination of all partitions, all the filtered overlapping image regions are summarized. Pixel stitching and integration operations are performed on adjacent overlapping image regions. The stitching and integration formula is: the horizontal extreme value of the integrated block = the minimum and maximum values ​​of the horizontal extreme values ​​of all overlapping regions; the vertical extreme value of the integrated block = the minimum and maximum values ​​of the vertical extreme values ​​of all overlapping regions. Complete and continuous regions are selected according to the integrated horizontal and vertical extreme values. The unified region after integration is the feature calibration block. All pixels inside the block meet the core conditions of conforming to the natural texture of the skin, matching the sensing acquisition range, and uniform distribution of the base pigment. It is directly used for the joint calibration of physiological sensing data and skin image features.

[0037] The historical modal difference sequence is obtained based on historical skin optical grayscale data and historical physiological temporal amplitude data, specifically including the following steps: The numerical offset is obtained based on historical skin optical grayscale data and historical physiological temporal amplitude data; The persistent offset components of the skin's optical state are obtained based on the numerical offset, and the persistent offset components are arranged sequentially to form a continuously arranged historical modal difference sequence.

[0038] Skin optical grayscale data and physiological temporal amplitude data are matched one-to-one according to the same time node. Before the operation, a dimensionless normalization conversion coefficient is set to complete the unit alignment. Normalized skin grayscale value = original skin optical grayscale value ÷ maximum grayscale range, normalized physiological amplitude value = original physiological temporal amplitude value ÷ maximum amplitude range. The maximum grayscale range is fixed at 255 grayscale levels, and the maximum amplitude range is determined by the sensor hardware parameters. The units are uniformly converted to millivolts before the division operation is completed. After the conversion, both types of values ​​are converted into dimensionless values ​​in the range of 0 to 1, and the difference comparison operation can be directly carried out.

[0039] After normalizing all time-series samples, the offset difference of a single time point is calculated for each pair of data corresponding to a time node. The basic difference at a single time point = normalized skin gray value - normalized physiological amplitude value. The numerical offset = the sum of the basic differences at a single time point for all time-series points ÷ the total number of valid time-series samples. The total number of valid time-series samples is the number of time nodes where the two types of data can be completely paired. Finally, the mean result is calculated as the numerical offset in the global dimension. This value can characterize the constant numerical deviation between the skin optical characteristics and the physiological sensing amplitude throughout the historical period.

[0040] Based on the numerical offset decomposition, the persistent offset components corresponding to the optical state of the skin are extracted. The persistent offset components of all time periods are arranged sequentially according to the acquisition time to construct a continuously arranged historical modal difference sequence. The segmented local numerical offset is calculated for each segment of the continuous historical time series. The segmented offset fluctuation value = segmented local numerical offset - global numerical offset. The segmented offset fluctuation value represents the deviation caused by temporary fluctuations within a single time series. The persistent offset component of a single time period = global numerical offset - segmented offset fluctuation value. This formula can eliminate the interference caused by short-term random fluctuations and retain the optical offset components that are stable and unchanged over a long period of time. Each continuous historical time series will have a set of independent persistent offset components calculated. The values ​​of persistent offset components corresponding to different time periods are independent of each other, and there will be no confusion caused by the reuse of feature values.

[0041] Based on the coupling correlation trend between feature-labeled blocks, historical modality difference sequences, and historical mapping distance values, the coupling correlation factor is obtained, specifically including the following steps: Extract the global pixel grayscale distribution of the feature calibration block, retrieve the fluctuation trend of the historical modal difference sequence, and extract the continuous change trend of the historical mapping distance value. Determine the strength of the linkage between the grayscale distribution state, fluctuation trend, and continuous change pattern of pixels across the entire domain; Based on the strength of the linkage changes, the synchronous change interval is obtained, and the coupling correlation factor is obtained by processing the offset amplitude of the synchronous change interval.

[0042] The algorithm iterates through all pixels within the feature calibration block, counting the total number of pixels in different grayscale intervals. The global grayscale mean is calculated as: (Number of pixels in each grayscale interval × Sum of corresponding grayscale levels) ÷ Total number of pixels in the feature calibration block. This global grayscale mean represents the grayscale distribution of pixels in the current block. Each time-series sampling moment generates an independent global grayscale mean. To analyze the fluctuation trend of the historical modal difference sequence, the complete historical modal difference sequence is retrieved, and the fluctuation amplitude is calculated for each adjacent time-series node. The formula for calculating the fluctuation amplitude of a single node is: Single-segment modal fluctuation amplitude = Current time persistent offset component - Previous time persistent offset component. This numerical sequence completely records the rise and fall patterns of values ​​within the sequence, thus representing the fluctuation trend of the historical modal difference sequence. To address the continuous change trend of historical mapping distance values, multiple sets of mapping coordinate difference data within the historical time period are retrieved. The formula for calculating the mapping distance difference between adjacent moments is: Single segment mapping change amplitude = Current moment mapping distance value - Previous moment mapping distance value. All mapping distance values ​​are normalized, and the set of continuous amplitude values ​​obtained after normalization represents the continuous change trend of historical mapping distance values.

[0043] The system quantifies the strength of the linkage among three factors: the overall pixel grayscale distribution, the fluctuation trend of historical modal difference sequences, and the continuous change trend of historical mapping distance values. It also quantifies the matching degree of synchronous rise and fall changes of the three types of features and performs synchronous temporal alignment processing on the three types of temporal feature data to ensure that each set of data corresponds to the same skin detection time node. The synchronous change correlation between each pair of features is calculated for each set of synchronous temporal samples. The formula for calculating the correlation of a single set of temporal dual features is: Single temporal correlation = Sum of the results of multiplying the normalized values ​​of the two types of features ÷ Geometric mean of the sum of the squares of the feature values ​​of a single temporal sample. All correlation values ​​are fixed in the range of -1 to +1. The closer the value is to +1, the more synchronous and directional the two types of features are. The closer the value is to -1, the more synchronous and directional the two types of features are. The closer the value is to -1, the more synchronous and directional the two types of features are. The closer the value is to -1, the more synchronous and directional the two types of features are. The closer the value is to -0, the more synchronous and directional the two types of features are. After completing the pairwise correlation calculations of the three sets, the weighted sum is used to obtain the overall linkage strength. The formula for calculating the overall linkage strength is: Linkage Change Strength Index = Gray-level Modal Correlation × First Weight Coefficient + Gray-level Mapping Correlation × Second Weight Coefficient + Modal Mapping Correlation × Third Weight Coefficient. The final calculated linkage change strength index is a quantitative representation of the overall linkage change strength of the three types of features. The higher the index value, the more unified the synchronous change trend of the three types of data.

[0044] The effective time-series intervals are filtered based on the strength index of linkage changes. Then, a fusion calculation is performed on the feature offset amplitudes within the filtered intervals to generate a unified coupling correlation factor. When performing the synchronous change interval filtering operation, a preset linkage strength benchmark threshold is used. Only continuous time-series segments with a linkage strength index greater than the benchmark threshold are retained as synchronous change intervals. Time-series segments with an index lower than the benchmark threshold are determined to be invalid intervals with asynchronous feature changes and are directly discarded and not included in subsequent calculations. For the retained synchronous change intervals, the original offset amplitudes of three types of features corresponding to all time-series nodes within the interval are extracted. The three types of offset amplitudes are global grayscale offset amplitude, modal fluctuation offset amplitude, and mapping distance offset amplitude. The formula for calculating the interval average offset amplitude is: Interval Comprehensive Offset Amplitude = Average Global Grayscale Offset Amplitude × Grayscale Influence Coefficient + Average Modal Fluctuation Offset Amplitude × Modal Influence Coefficient + Average Mapping Distance Offset Amplitude × Mapping Influence Coefficient, corresponding to the proportion of influence of the three types of features on the skin's optical-physiological coupling relationship, respectively. The target coupling correlation factor = Interval Comprehensive Offset Amplitude × Interval Average Linkage Change Strength Index represents the coupling and linkage relationship between skin optical pixel features, long-term modal offset, and spatial mapping error.

[0045] It also includes the following steps: Preset modal registration and correction parameters based on the original acquired parameters; Image reconstruction processing, performed with modal registration correction parameters on skin images, yields skin texture distortion detection results. Skin texture distortion features are obtained based on the skin texture distortion detection results.

[0046] Based on the original acquisition parameters, the preset configuration of modal registration and correction parameters is completed. The original acquisition parameters include the lens scaling parameters of the image acquisition hardware, the imaging lateral offset reference, the imaging longitudinal offset reference, and the optical sensor modal grayscale correction coefficient. The scale correction item = original lens scaling parameter × global modal equalization coefficient. The lateral coordinate correction item = original imaging lateral offset reference × spatial registration compensation coefficient. The longitudinal coordinate correction item = original imaging longitudinal offset reference × spatial registration compensation coefficient. The grayscale modal correction item = original optical sensor modal grayscale correction coefficient × optical modal balance coefficient.

[0047] The modal registration and correction parameters are retrieved, and a global image reconstruction process is performed on the input original skin image. The output is the skin texture distortion detection result. The correction parameters correct scale deviation, coordinate offset, and optical grayscale deviation caused by hardware issues during image acquisition. Simultaneously, the degree of distortion in the skin texture within the image is quantified, generating a global quantized distortion detection result. Image reconstruction operations are divided into coordinate remapping and grayscale modal correction operations. The formula for calculating the reconstructed pixel x-coordinate is: Reconstructed pixel x-coordinate = Original image pixel x-coordinate × Scale correction term + Horizontal coordinate correction term. The formula for calculating the reconstructed pixel y-coordinate is: Reconstructed pixel y-coordinate = Original image pixel y-coordinate × Scale correction term + Vertical coordinate correction term. All pixels in the original skin image are traversed and substituted into the formulas to update the coordinates, resulting in an intermediate image matrix with corrected spatial positions. The grayscale modal correction operation is performed based on the grayscale modal correction term. The reconstructed pixel grayscale value = Original pixel grayscale value + Grayscale modal correction term, with a value range constrained between 0 and 255. Values ​​outside this range are truncated to the range boundary. Extract the set of skin texture line coordinates in the reconstructed image, compare the standard undistorted texture reference coordinates segment by segment, and calculate the single-segment texture distortion difference = the sum of the reconstructed texture point coordinates - the sum of the standard texture point coordinates. Summarize and store the single-segment texture distortion difference corresponding to all texture segments in the image to form a set containing the quantified values ​​of texture distortion at every point in the entire image.

[0048] Using skin texture distortion detection results as the data source, standardized skin texture distortion features are extracted through hierarchical statistical operations. The dispersed segmented distortion differences are aggregated and quantified to extract multiple independent distortion features that characterize the overall distortion state of the image. During computation, basic statistical operations are performed on all single-segment texture distortion differences within the detection results. The global average distortion amplitude = the sum of all single-segment texture distortion differences ÷ the total number of texture segments in the image; this value represents the average severity of texture distortion in the entire skin image. Extreme values ​​of distortion differences within the detection results are extracted: maximum local distortion = the maximum value among all single-segment texture distortion differences, and minimum local distortion = the minimum value among all single-segment texture distortion differences; these two values ​​characterize the fluctuation boundaries of distortion in local image regions. The percentage of abnormal distortion = the number of texture segments with distortion differences exceeding the distortion threshold ÷ the total number of texture segments in the image. The global average distortion amplitude, maximum local distortion, minimum local distortion, and percentage of abnormal distortion are quantified and combined to constitute the skin texture distortion features.

[0049] The physiological time-series feature sequence is obtained based on signal feature data and modality registration correction parameters, specifically including the following steps: Spatial offset compensation is performed on the signal feature data using modal registration correction parameters to generate compensated time series data. Denoising time series data is generated based on the compensated time series data. The denoising time series data is then continuously segmented and smoothed to generate physiological time series feature sequences.

[0050] Using modal registration and correction parameters as the correction benchmark, global spatial offset compensation is performed on the original signal feature data to generate compensated time-series data. This eliminates signal spatial coordinate offset errors caused by hardware assembly and imaging modal differences during the physiological sensing acquisition stage. The modal registration and correction parameters include scale correction terms, lateral spatial compensation terms, and longitudinal spatial compensation terms. Each time-series sample within the original signal feature data has the original signal abscissa, original signal ordinate, and original signal amplitude. The compensated signal abscissa = original signal abscissa × scale correction term + lateral spatial compensation term; the compensated signal ordinate = original signal ordinate × scale correction term + longitudinal spatial compensation term; and the compensated signal amplitude = original signal amplitude × scale correction term. All corrected time-series samples are linearly arranged according to the original acquisition time sequence to form compensated time-series data.

[0051] First, the noise reduction time series data generation operation is performed. Moving weighted noise reduction is performed on consecutive adjacent time series samples within the compensated time series data. A fixed noise reduction window length is set, and the window contains a fixed number of consecutive time series samples. The formula for calculating the noise reduction amplitude of a single sample is: Noise reduction signal amplitude = first group of compensated amplitude within the window × first noise reduction weight + second group of compensated amplitude within the window × second noise reduction weight until the last sample in the window. The sum of all noise reduction weight coefficients equals 1. All weight coefficients are dimensionless fixed constants. The window slides sequentially through all compensated time series samples. Each window slide corresponds to the generation of a set of noise reduction amplitude values. All noise reduction values ​​are arranged and integrated in time series to form noise reduction time series data. This set of data has filtered out numerical jumps caused by instantaneous random noise during the acquisition process. After generating the denoised time series data, a continuous segmented smoothing operation is performed. The complete denoised time series data is divided into multiple continuous independent time series segments according to a fixed number of samples. For all samples within each time series segment, a mean smoothing operation is performed. The segmented smoothing reference amplitude is calculated as the sum of all denoised amplitudes within the segment divided by the total number of samples in the segment. The segmented smoothing reference amplitude is used to replace all original denoised amplitude values ​​within the corresponding segment, thus completing the single-segment time series smoothing correction. After all segments have been smoothed and replaced in sequence, all segments are reassembled and integrated in chronological order to form a physiological time series feature sequence.

[0052] The fused cross-modal features are obtained based on physiological time-series features and coupling correlation factors, specifically including the following steps: The physiological time-series feature sequences are subjected to interference processing based on coupling correlation factors to obtain the time-series base features after interference suppression. Modal collaborative intermediate features are obtained based on the gray-level correlation weights of temporal basis features and coupling correlation factors after interference suppression; Scale-adaptive features are obtained by performing scale processing on the spatial arrangement features corresponding to the mapping conditions of the intermediate features of modal collaboration. The scale-adaptive features are aggregated with the skin texture information corresponding to the feature-labeled blocks to generate fused cross-modal features.

[0053] Global interference suppression processing is performed on the physiological temporal feature sequence based on the coupling correlation factor. After the operation, interference-suppressed temporal base features are generated. Multimodal linkage coupling quantification index is used to offset the cross-interference generated between the skin optical modality and the temporal physiological sensing modality, and pure temporal base data unaffected by cross-modal clutter is removed. The linkage coupling strength between skin image and physiological temporal signal is characterized by the formula: single-node temporal base amplitude = original physiological temporal amplitude - original physiological temporal amplitude × coupling correlation factor × interference attenuation coefficient. All temporal nodes in the physiological temporal feature sequence are traversed and substituted into the formula to complete the amplitude correction. All corrected temporal nodes are arranged continuously in the original acquisition time order to form the interference-suppressed temporal base features. This set of data removes the cross-interference components caused by changes in skin optical texture and retains only the basic temporal base input of the physiological signal itself.

[0054] Using the temporal base features after interference suppression and the coupling correlation factor as dual input data sources, a weighted fusion operation is performed based on their corresponding gray-level correlation weights to obtain modal collaborative intermediate features. This binds the pure physiological temporal base features with the skin's optical coupling linkage laws, generating transitional intermediate features that simultaneously carry both temporal physiological changes and skin optical linkage laws. The gray-level correlation weights of the temporal base and the coupling factor are extracted separately, and the sum of these two weights equals 1, used to allocate the proportion of the two types of input data in the fusion operation. The single-node collaborative intermediate value = single-node temporal base amplitude × temporal base gray-level correlation weight + coupling correlation factor × coupling factor gray-level correlation weight, where the coupling factor = original coupling correlation factor × reference voltage conversion constant. All temporal nodes of the interference-suppressed temporal base features are traversed and substituted into the fusion formula to obtain an ordered and complete set of values.

[0055] The mapping conditions for modal collaborative intermediate feature matching are extracted. The spatial arrangement features stored within the mapping conditions are retrieved to perform global scale normalization. After the operation, scale-adapted features are output, unifying the scale standards of the modal collaborative intermediate features and the skin image pixel space system, eliminating dimensional bias caused by the mismatch between the temporal signal sampling scale and the image pixel scale. The spatial arrangement features corresponding to the mapping conditions include horizontal scale reference coefficients and vertical scale reference coefficients. The pre-adaptation horizontal coordinate pixel value = original temporal horizontal coordinate × step size pixel conversion constant; the pre-adaptation vertical coordinate pixel value = original temporal vertical coordinate × step size pixel conversion constant; the scale-adapted horizontal coordinate = pre-adaptation horizontal coordinate pixel value × horizontal scale reference coefficient; the scale-adapted vertical coordinate = pre-adaptation vertical coordinate pixel value × vertical scale reference coefficient; the scale-adapted amplitude = single-node collaborative intermediate value × horizontal scale reference coefficient × vertical scale reference coefficient. The coordinates and amplitudes of all temporal nodes of the modal collaborative intermediate features are sequentially substituted into three sets of scale correction formulas to complete the unified scale correction across all dimensions. All corrected temporal nodes are then combined in an orderly manner to form scale-adapted features.

[0056] The scale-adaptive features and the skin texture information corresponding to the feature-labeled blocks are fused globally to generate a fused cross-modal feature. This integrates the temporal physiological sensing dimension and the skin optical texture dimension to form a unified feature set that can characterize the temporal changes in skin physiological function and the optical state of skin texture. The scale-adaptive features include two types of quantitative information: temporal spatial coordinates and temporal physiological amplitude. The skin texture information includes two types of quantitative information: texture gray-level mean and texture distortion amplitude. Dimensionless normalization is performed on these two features. Normalized temporal amplitude = scale-adaptive amplitude ÷ maximum temporal amplitude range; normalized texture gray level = texture gray-level mean ÷ maximum gray level range; normalized distortion amplitude = texture distortion amplitude ÷ image pixel baseline side length; single-point fused feature value = normalized temporal amplitude × temporal fusion weight + normalized texture gray level × gray level fusion weight + normalized distortion amplitude × distortion fusion weight. The temporal fusion weight, gray level fusion weight, and distortion fusion weight are preset fixed dimensionless constants, and their sum equals 1. According to the coordinate matching feature of all spatial points within the scale adaptation feature, the skin texture information of the same coordinate in the feature calibration block is entered into the fusion formula one by one to complete the aggregation of point features. The fusion values ​​of all spatial points are fully summarized in the spatial arrangement order of the image to form the fused cross-modal feature.

[0057] In this embodiment, the steps of determining the source of disturbance and outputting the corresponding feature set based on the comparison results of the fused cross-modal features and the preset benchmark skin texture threshold, and whether the skin texture distortion features belong to the feature calibration block are described in detail.

[0058] First, the generated fused cross-modal features are acquired. These features are fused feature numerical matrices that correspond one-to-one with spatial points in the target skin image. A preset baseline skin texture threshold is set as a fixed constant pre-determined based on healthy standard skin samples. This constant is determined by the statistical mean of texture features from a large number of healthy skin samples, and the unit is the same as the fused feature values. For each spatial point (pixel) within the fused cross-modal features, the point texture deviation value is calculated using the following formula.

[0059] After traversing all spatial points to solve for the difference, the total number of points with texture deviation values ​​greater than 0 and the total number of points with texture deviation values ​​less than or equal to 0 are counted to generate a global texture deviation distribution statistical result. This statistical result includes a set of points with positive deviation, a set of points with negative deviation, and their respective distribution coordinate ranges, which are used to quantify the distribution area and deviation magnitude of skin texture features from the standard baseline within the entire image.

[0060] Secondly, the skin texture distortion features obtained in the previous step are retrieved. These features include the distortion difference of the global segmented texture and the spatial boundary coordinates of the corresponding segmented texture (defined by the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate). Simultaneously, the spatial coordinate extreme values ​​of the feature-labeled blocks (minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate of the block) are obtained. For each texture segment, horizontal and vertical attribution determinations are performed sequentially.

[0061] Lateral attribution determination: Calculate the lateral attribution marker value = minimum x-coordinate of the segmented texture. The minimum horizontal coordinate of the feature-labeled block. If the horizontal attribution value is ≥ 0, and the maximum horizontal coordinate of the segmented texture is ≤ the maximum horizontal coordinate of the feature-labeled block, then the horizontal interval of the segmented texture is determined to fall inside the feature-labeled block; otherwise, it is determined to be outside.

[0062] Vertical attribution determination: Calculate the vertical attribution marker value = minimum ordinate of segmented texture. The minimum ordinate of the feature-labeled block. If the vertical assignment value is ≥ 0, and the maximum ordinate of the segmented texture is ≤ the maximum ordinate of the feature-labeled block, then the vertical interval of the segmented texture is determined to fall inside the feature-labeled block; otherwise, it is determined to be outside.

[0063] When a texture segment satisfies both the horizontal and vertical attribution conditions, the skin texture distortion feature of that segment is determined to belong to the feature labeling block (marked as "internal"); if any one dimension does not meet the conditions, it is determined not to belong to the feature labeling block (marked as "external"). All texture segments are traversed segment by segment to generate the attribution label for each texture segment, and the results are summarized to form a complete attribution determination.

[0064] Next, combining the statistical results of the global texture deviation distribution with the attribution determination results, the source of the disturbance is determined according to the following rules: Intrinsic disturbance determination: If the number of points with texture deviation values ​​greater than 0 (i.e., positive deviation) that exceed a preset proportion threshold (e.g., 70%) is concentrated in the texture segment marked as "internal", then the disturbance source is determined to be intrinsic disturbance caused by the skin's own physiological texture distortion. External disturbance determination: If the number of points with texture deviation values ​​greater than 0 that exceed a preset proportion threshold (e.g., 70%) is concentrated in the texture segment marked as "external", then the disturbance source is determined to be external disturbance caused by image acquisition edge clutter or external environmental interference. Modal cross-perturbation determination: If the positive and negative points of the texture deviation value are evenly distributed in the "inner" and "outer" segments (i.e., the proportion of positive deviation points in the inner and outer segments does not exceed the preset proportion threshold, and the distribution of negative deviation points is also not obviously biased), then the perturbation source is determined to be modal cross-perturbation caused by the coupling shift between physiological time sequence signals and skin optical modality.

[0065] Finally, based on the determined type of disturbance source, a feature set generation operation is performed: If it is determined to be an internal disturbance, all spatial points belonging to "internal" and with a texture deviation value greater than 0 are selected. The fused cross-modal feature value and the corresponding segmented skin texture distortion difference value of each point are extracted. The two are then sequentially spliced ​​together according to the spatial coordinate order to form an internal disturbance feature set. If it is determined to be an external disturbance, all spatial points belonging to "external" and with a texture deviation value greater than 0 are selected. The fused cross-modal feature values ​​and the corresponding segmented skin texture distortion difference values ​​of each point are extracted and sequentially spliced ​​according to spatial coordinate order to form an external disturbance feature set. If it is determined to be a modal cross-perturbation, then the positive and negative deviation points belonging to "internal" and "external" are uniformly extracted (parts of each can be taken proportionally), and the fused cross-modal feature values ​​and texture distortion differences of the corresponding points are extracted. All data are integrated according to the spatial arrangement order to form a modal cross-perturbation feature set.

[0066] The corresponding feature sets are output for subsequent skin physiological state assessment or abnormality diagnosis modules. This embodiment, through refined attribution determination and bias statistics, can accurately distinguish different sources of disturbance, effectively improving the reliability and interpretability of skin physiological state detection results.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for cross-modal feature extraction of physiological time-series signals and skin images, characterized in that, Includes the following steps: The corresponding mapping conditions are output in response to the preset mapping relationship between the physiological sensing channel and the skin image. Based on the mapping conditions, feature calibration blocks corresponding to the physiological sensing channel locations are extracted from the target skin image; Historical skin optical grayscale data and historical physiological temporal amplitude data affected by skin grayscale coupling were statistically analyzed over historical acquisition periods. Based on the historical skin optical grayscale data and historical physiological temporal amplitude data, a historical modal difference sequence was obtained. The coupling correlation factor is obtained based on the coupling correlation trend between the feature-labeled blocks, the historical modality difference sequence, and the historical mapping distance value. Acquire signal feature data of physiological time-series signals, and obtain physiological time-series feature sequences based on the signal feature data and modal registration correction parameters; The fused cross-modal features are obtained based on physiological temporal feature sequences and coupling correlation factors; Based on the comparison results of the fused cross-modal features and the preset benchmark skin texture threshold, and whether the skin texture distortion features belong to the feature calibration block, the source of the disturbance is determined and the corresponding feature set is output.

2. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 1, characterized in that, After responding to the preset mapping relationship between the physiological sensing channel and the skin image, the corresponding mapping conditions are output, specifically including the following steps: A global matching benchmark between physiological sensing channels and skin images is pre-constructed. Spatial alignment results are obtained by calibrating the spatial dimensions of physiological sensing channels and skin images based on the global matching benchmark. Spatial matching constraints are generated based on spatial alignment results, and the overlapping sections of the boundaries between physiological sensing channels and skin image imaging areas are identified based on spatial matching constraints. Constraint verification is performed on the overlapping boundary sections to obtain the verification judgment result, and mapping conditions are generated based on the verification judgment result.

3. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 1, characterized in that, Based on the mapping conditions, feature calibration blocks corresponding to the physiological sensing channel locations are extracted from the target skin image, specifically including the following steps: Based on the mapping conditions, global pixel-level spatial registration of the target skin image is performed to generate a registered complete skin image; The initial image cropping region is obtained based on the registered complete skin image and the direction of the skin surface texture. The cropped area of ​​the initial image is regularized to obtain a regularized image patch. After identifying the boundaries of skin basal pigment distribution based on regular image patches, the texture partitions are divided into hierarchical regions. The overlapping image regions of the sensing acquisition path are selected based on multi-level texture partitioning, and feature calibration blocks are generated by integrating the overlapping image regions.

4. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 3, characterized in that, The initial image cropping region is obtained based on the registered complete skin image and the direction of the skin surface texture, specifically including the following steps: The skin spatial range covered by the physiological sensor channel is determined based on the registered complete skin image; The initial image cropping area is defined by following the direction of the skin's surface texture.

5. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 1, characterized in that, The historical modal difference sequence is obtained based on historical skin optical grayscale data and historical physiological temporal amplitude data, specifically including the following steps: The numerical offset is obtained based on historical skin optical grayscale data and historical physiological temporal amplitude data; The persistent offset components of the skin's optical state are obtained based on the numerical offset, and the persistent offset components are arranged sequentially to form a continuously arranged historical modal difference sequence.

6. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 5, characterized in that, Based on the coupling correlation trend between feature-labeled blocks, historical modality difference sequences, and historical mapping distance values, the coupling correlation factor is obtained, specifically including the following steps: Extract the global pixel grayscale distribution of the feature calibration block, retrieve the fluctuation trend of the historical modal difference sequence, and extract the continuous change trend of the historical mapping distance value. Determine the strength of the linkage between the grayscale distribution state, fluctuation trend, and continuous change pattern of pixels across the entire domain; Based on the strength of the linkage changes, the synchronous change interval is obtained, and the coupling correlation factor is obtained by processing the offset amplitude of the synchronous change interval.

7. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 1, characterized in that, It also includes the following steps: Preset modal registration and correction parameters based on the original acquired parameters; Image reconstruction processing, performed with modal registration correction parameters on skin images, yields skin texture distortion detection results. Skin texture distortion features are obtained based on the skin texture distortion detection results.

8. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 7, characterized in that, The physiological time-series feature sequence is obtained based on the signal feature data and modal registration correction parameters, specifically including the following steps: Spatial offset compensation is performed on the signal feature data using modal registration correction parameters to generate compensated time series data. Denoising time series data is generated based on the compensated time series data. The denoising time series data is then continuously segmented and smoothed to generate physiological time series feature sequences.

9. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 8, characterized in that, The fused cross-modal features are obtained based on physiological time-series features and coupling correlation factors, specifically including the following steps: The physiological time-series feature sequences are subjected to interference processing based on coupling correlation factors to obtain the time-series base features after interference suppression. Modal collaborative intermediate features are obtained based on the gray-level correlation weights of temporal basis features and coupling correlation factors after interference suppression; Scale-adaptive features are obtained by performing scale processing on the spatial arrangement features corresponding to the mapping conditions of the intermediate features of modal collaboration. The scale-adaptive features are aggregated with the skin texture information corresponding to the feature-labeled blocks to generate fused cross-modal features.

10. The method for cross-modal feature extraction of physiological time-series signals and skin images according to claim 9, characterized in that, Based on the comparison results of fused cross-modal features and preset benchmark skin texture thresholds, and whether skin texture distortion features belong to the feature calibration block, the source of disturbance is determined and the corresponding feature set is output. The specific steps include: Calculate the feature values ​​of each spatial point within the fused cross-modal features and the point texture deviation value between the feature values ​​and the preset benchmark skin texture threshold, statistically analyze the distribution of points with deviation values ​​greater than zero and less than or equal to zero, and generate global texture deviation distribution statistical results. Determine whether each segment of the skin texture distortion feature falls within the spatial coordinate range of the feature calibration block, and generate an attribution determination result; Based on the statistical results and attribution determination results of the global texture deviation distribution, when points with deviation values ​​greater than zero are concentrated in texture segments belonging to the feature calibration block, the source of disturbance is determined to be an internal disturbance; when points with deviation values ​​greater than zero are concentrated in texture segments belonging to the outside of the feature calibration block, the source of disturbance is determined to be an external disturbance; when points with positive and negative deviation values ​​are evenly distributed in the internal and external segments, the source of disturbance is determined to be a modal cross disturbance. Based on the determined type of disturbance source, corresponding spatial points and feature values ​​are selected to generate a feature set of the corresponding type.

Images