A Method for Detecting the Pasteurization of Amorphophallus bulbils Based on Multimodal Image Acquisition Technology

By combining multimodal image acquisition technology with multispectral and thermal imaging, a spatiotemporal descriptor for the gelatinization process was constructed, which solved the problem of missing information in the gelatinization detection of bulbils and konjac, and achieved high-precision automated detection.

CN123072003APending Publication Date: 2026-11-10KUNMING UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611047199.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-15
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In existing technologies, the detection of gelatinization in bulbils and konjac relies on manual observation or single-modal image acquisition, which makes it difficult to achieve precise quantification of the gelatinization process and results in information loss, failing to accurately reflect the distribution of gelatinization state.

Method used

By employing multimodal image acquisition technology, combined with multispectral and thermal imaging, and through the cross-gradient fusion of frequency domain phase consistency and spatial domain texture energy, a spatiotemporal descriptor of the gelatinization process is constructed to achieve automated determination of gelatinization completion.

Benefits of technology

It achieves high-precision, automated detection of the gelatinization process of bulbils and konjac, accurately reflects the distribution of gelatinization state, and improves the accuracy and robustness of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123072003A_ABST
    Figure CN123072003A_ABST
Patent Text Reader

Abstract

This invention discloses a method for detecting the gelatinization of konjac bulbils based on multimodal image acquisition technology, belonging to the field of food processing detection technology. The method includes: acquiring multispectral image sequences and thermal imaging image sequences of konjac bulbils slices at multiple preset sampling times to form a multimodal image sequence; performing a joint spatial-to-frequency domain transformation on the multimodal image sequence to extract the frequency domain phase consistency feature map and spatial domain texture energy distribution map of the konjac bulbils slices at each sampling time; performing pixel-by-pixel cross-gradient fusion of the frequency domain phase consistency feature map and the spatial domain texture energy distribution map to generate a fused feature tensor; constructing a spatiotemporal descriptor of the gelatinization process of the konjac bulbils slices based on the temporal change trajectory of each pixel in the fused feature tensor; and determining the degree of gelatinization completion of the konjac bulbils slices at the current sampling time based on the projection position of the spatiotemporal descriptor of the gelatinization process in a preset gelatinization state discrimination space.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of food processing and testing technology, specifically to a method for detecting the gelatinization of bulbils and konjac based on multimodal image acquisition technology. Background Technology

[0002] In the processing of konjac bulbils, accurate detection of the degree of gelatinization is one of the key factors determining the quality of the final product. Currently, the determination of the gelatinization state of konjac bulbils mainly relies on visual observation by operators or contact-based humidity and viscosity measurements. Visual judgment is heavily influenced by subjective experience, resulting in poor consistency and making it difficult to achieve precise quantification of the gelatinization process. Contact-based measurement methods can only obtain single-point or local information, failing to reflect the gelatinization state distribution across the entire konjac slice surface, and the measurement process itself may interfere with the gelatinization environment. To overcome these limitations, some studies have begun to explore the use of single-modal image acquisition techniques for gelatinization detection, such as using only a visible light camera to capture the apparent color changes of konjac slices, or using only thermal imaging to obtain surface temperature distribution. The single-modal information source means that such methods can only capture one aspect of the gelatinization process. Color and texture changes reflect the combined appearance effects of surface moisture migration and starch gelation, while temperature distribution is related to the endothermic and exothermic processes of gelatinization. These two aspects are complementary in their physical origins, but existing methods have failed to effectively integrate multimodal information. Meanwhile, gelatinization is a typical spatiotemporal dynamic process, and the gelatinization rate varies at different spatial locations of konjac slices. Simple frame-by-frame image analysis cannot construct the continuous evolution trajectory of pixel-level features in the temporal dimension, losing the important criterion of differences in gelatinization progress in different regions. The problem this method aims to solve is how to extract and fuse two complementary representations—frequency domain phase consistency and spatial domain texture energy—from multimodal image sequences, thereby tracking temporal changes at the pixel level and forming descriptors that can characterize the spatiotemporal evolution of gelatinization, so as to achieve accurate determination of the gelatinization completion of bulbil konjac. Summary of the Invention

[0003] This invention provides a method for detecting gelatinization of bulbils and konjac based on multimodal image acquisition technology. The purpose is to achieve cross-gradient fusion of frequency domain phase consistency features and spatial domain texture energy features obtained from bulbils and konjac slice images through the collaborative acquisition and joint transformation of multispectral and thermal imaging. Based on the fused feature tensor, a spatiotemporal descriptor reflecting the pixel-level gelatinization process is constructed, and finally, the degree of gelatinization completion is automatically and accurately determined in a preset gelatinization state discrimination space.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a method for detecting the gelatinization of konjac bulbils based on multimodal image acquisition technology, comprising the following steps: acquiring multispectral image sequences and thermal imaging image sequences of konjac bulbils slices at multiple preset sampling times to form a multimodal image sequence; performing a joint transformation from the spatial domain to the frequency domain on the multimodal image sequence to extract the frequency domain phase consistency feature map and the spatial domain texture energy distribution map of the konjac bulbils slices at each sampling time; performing pixel-by-pixel cross-gradient fusion of the frequency domain phase consistency feature map and the spatial domain texture energy distribution map to generate a fused feature tensor; constructing a spatiotemporal descriptor of the gelatinization process of the konjac bulbils slices based on the temporal change trajectory of each pixel in the fused feature tensor; and determining the degree of gelatinization completion of the konjac bulbils slices at the current sampling time according to the projection position of the spatiotemporal descriptor of the gelatinization process in a preset gelatinization state discrimination space.

[0005] As a preferred technical solution of the present invention, the step of acquiring a multimodal image sequence specifically includes: controlling a multispectral camera to sequentially acquire reflectance images of konjac bulbils slices in the visible light band, near-infrared band, and short-wave infrared band at multiple preset sampling times, and arranging them in the order of sampling times to generate a multispectral image sequence; simultaneously controlling a thermal imaging camera to acquire surface thermal radiation distribution images of konjac bulbils slices at the same preset sampling time, and arranging them in the order of sampling times to generate a thermal imaging image sequence; establishing a timestamp correspondence between the multispectral images acquired at the same sampling time and the thermal imaging images to form multimodal image pairs, and arranging all multimodal image pairs in sequence according to the order of sampling times to combine them into a multimodal image sequence. This synchronous acquisition method can comprehensively capture the correlation changes in the spectral and thermal properties of the slices during the gelatinization process, avoid temporal misalignment, and ensure data fusion accuracy.

[0006] Preferably, the step of performing a joint spatial-domain to frequency-domain transformation on the multimodal image sequence includes: sequentially extracting multimodal image pairs at each sampling time from the multimodal image sequence; performing a two-dimensional discrete Fourier transform on the multispectral image in the multimodal image pair to obtain a multispectral frequency domain amplitude spectrum and a multispectral frequency domain phase spectrum; performing a two-dimensional discrete cosine transform on the thermal imaging image to obtain a thermal imaging frequency domain coefficient matrix; calculating the phase consistency metric on the multispectral frequency domain phase spectrum, extracting the phase consistency response value of each pixel, and generating a frequency domain phase consistency feature map. This feature map is insensitive to changes in illumination and contrast and can stably reflect changes in structural edge intensity caused by blurring; performing an inverse transform on the thermal imaging frequency domain coefficient matrix to reconstruct a thermal imaging spatial domain reconstructed image, and calculating the contrast statistics and energy statistics of the local neighborhood gray-level co-occurrence matrix of each pixel in the reconstructed image; and normalizing and superimposing the contrast statistics and energy statistics to generate a spatial domain texture energy distribution map. This distribution map can finely depict the dynamic evolution of thermal field texture during the blurring process. By jointly extracting frequency domain phase features and spatial domain texture energy features, the gelatinization state can be perceived in multiple dimensions, improving the detection sensitivity of early microscopic changes.

[0007] As a further preferred approach, during pixel-by-pixel cross-gradient fusion, the phase consistency gradient magnitude and direction of each pixel in the frequency domain phase consistency feature map are calculated to generate a phase consistency gradient field; the texture energy gradient magnitude and direction of each pixel in the spatial domain texture energy distribution map are calculated to generate a texture energy gradient field; the phase consistency gradient magnitude and texture energy gradient magnitude at the same pixel location are weighted and summed to obtain the fusion gradient magnitude of that pixel; the fusion gradient direction of that pixel is determined based on the cosine of the angle between the phase consistency gradient direction and the texture energy gradient direction; the fusion gradient magnitude and fusion gradient direction of each pixel are combined into a fusion feature vector, and the fusion feature vectors of all pixels are arranged according to their pixel spatial positions to generate a fusion feature tensor. This cross-gradient fusion method can enhance the gradient response consistency between modes during the blurring process, effectively suppress single-modal noise, and highlight the real change areas.

[0008] The preferred steps for constructing a spatiotemporal descriptor of the gelatinization process based on the fusion feature tensor include: extracting the fusion gradient magnitude sequence and fusion gradient direction sequence of each pixel at all preset sampling times from the fusion feature tensor to form the temporal change trajectory of that pixel; calculating the difference in fusion gradient magnitude and fusion gradient direction between adjacent sampling times in the temporal change trajectory of each pixel to generate the temporal difference trajectory of that pixel; performing spatial neighborhood clustering on the temporal difference trajectories of all pixels, grouping pixels with consistent temporal difference trajectory change trends into the same change cluster; statistically analyzing the spatial distribution density of pixels within each change cluster and the average change rate of the temporal difference trajectory to generate a cluster descriptor for each change cluster; and combining and encoding the cluster descriptors of all change clusters according to spatial positional relationships and temporal order to form a spatiotemporal descriptor of the gelatinization process of konjac bulb slices. Using pixel-aggregated change clusters as units for spatiotemporal description not only compresses redundant information but also expresses the expansion density and dynamic intensity of the gelatinization front through spatial density and average change rate.

[0009] When performing spatial neighborhood clustering, the preferred approach is as follows: The amplitude and direction change sequences of the temporal differential trajectory of each pixel are used as the trajectory feature vector of that pixel; the dynamic time-warped distance between the trajectory feature vectors of adjacent pixels is calculated as the trajectory similarity between adjacent pixels; adjacent pixels with trajectory similarity less than a preset similarity threshold are connected as spatial adjacency edges, and connected regions are marked based on all spatial adjacency edges, marking the set of interconnected pixels as the initial change cluster; the mean vector of the trajectory feature vectors of all pixels within each initial change cluster is calculated, and this mean vector is used as the cluster center vector of that initial change cluster; the initial change clusters are iteratively merged, and in each iteration, two initial change clusters whose Euclidean distance between their cluster center vectors is less than a preset merging threshold are merged into a new change cluster, until the Euclidean distance between the cluster center vectors of any two change clusters is not less than the preset merging threshold, obtaining the final change cluster partitioning result. Using dynamic time-warped distance can tolerate the temporal scaling caused by non-uniform changes in bleaching, making the clustering results more closely resemble the actual bleaching diffusion pattern.

[0010] When generating the cluster descriptor for each variation cluster, a preferred approach includes: counting the total number of pixels contained in each variation cluster; calculating the average spatial distance between each pixel in the variation cluster and its nearest neighbor pixels; using the reciprocal of the average spatial distance as the spatial distribution density of the variation cluster; extracting the temporal difference trajectory of all pixels within each variation cluster; calculating the sum of the absolute values ​​of the amplitude changes of the temporal difference trajectory of each pixel over all sampling times; dividing this sum by the total number of sampling intervals to obtain the average temporal change rate of the pixel; calculating the arithmetic mean of the average temporal change rates of all pixels within the variation cluster, using this as the average change rate of the temporal difference trajectory of the variation cluster; combining the spatial distribution density and the average change rate of the temporal difference trajectory into a density-rate tuple, and using this tuple as the cluster descriptor for the variation cluster. This tuple compactly characterizes the aggregation state and the intensity of dynamic evolution of the fading variation centers.

[0011] The preferred steps for encoding the cluster descriptors of the change clusters into a spatiotemporal descriptor of the fading process are as follows: First, obtain the spatial distribution density of each change cluster; sort all change clusters according to their spatial distribution density from high to low to obtain a spatial density sorting sequence; second, obtain the average change rate of the temporal differential trajectory of each change cluster; arrange the average change rates of the temporal differential trajectory according to the order of sampling times to obtain a temporal rate sorting sequence; third, cross-align the cluster identifier of each change cluster in the spatial density sorting sequence with the cluster identifier of the corresponding change cluster in the temporal rate sorting sequence to establish a mapping relationship between the change clusters in the spatial and temporal dimensions; fourth, concatenate the cluster descriptors of each change cluster into a spatiotemporal joint coding segment according to the mapping relationship; fifth, connect all the spatiotemporal joint coding segments of all change clusters sequentially according to the order of the spatial density sorting sequence to generate a spatiotemporal descriptor of the fading process. The descriptor generated in this way fully preserves the spatial structural features and temporal evolution patterns of the fading process, exhibiting high recognizability.

[0012] The preferred steps for determining the degree of gelatinization completion based on the spatiotemporal descriptor of the gelatinization process include: mapping the spatiotemporal descriptor of the gelatinization process to a preset gelatinization state discrimination space to obtain a projected coordinate vector, wherein the gelatinization state discrimination space is a metric space pre-constructed based on the spatiotemporal descriptor of the gelatinization process and its corresponding gelatinization completion label of historical konjac bulb germ gelatinization samples; calculating the Euclidean distance between the projected coordinate vector and the center vectors of the ungelatinized region, the partially gelatinized region, and the fully gelatinized region in the gelatinization state discrimination space; performing interval constraint normalization on the obtained distance values, and using the normalized distance values ​​as the final distance values; selecting the gelatinization state region corresponding to the minimum value among the final distance values ​​as the gelatinization state region at the current sampling time; obtaining the gelatinization completion interval corresponding to the gelatinization state region, and determining the specific value of the gelatinization completion degree of the konjac bulb germ slice at the current sampling time within the gelatinization completion interval based on the relative position of the projected coordinate vector within the gelatinization state region. By comparing projection and normalized distance in the constructed discrimination space, complex spatiotemporal descriptors can be transformed into accurate continuous blurring completion values, avoiding the subjectivity caused by fixed threshold judgment and significantly improving the accuracy and robustness of detection.

[0013] In the detection of gelatinization of bulbils and konjac, the technical solution of this invention makes full use of the complementary advantages of multispectral and thermal imaging in the frequency and spatial domains. Through cross-gradient fusion and spatiotemporal dynamic modeling, it continuously senses gelatinization changes from multiple dimensions. On this basis, it uses metric space projection matching to achieve quantitative determination of gelatinization completion, effectively realizing high-timeliness and high-precision automated detection of the gelatinization process.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0015] By simultaneously extracting frequency domain phase consistency feature maps and spatial domain texture energy distribution maps from multimodal image sequences and performing pixel-by-pixel cross-gradient fusion of the two, a multidimensional characterization of the gelatinization state of bulbil-konjac slices was achieved. The frequency domain phase consistency feature map captures local phase changes caused by microstructural deformation of konjac tissue during gelatinization, is insensitive to changes in illumination and contrast, and stably reflects the degree of boundary preservation during the gel network formation stage. The spatial domain texture energy distribution map quantifies the changes in surface roughness and uniformity caused by gelatinization through the contrast and energy statistics of the local gray-level co-occurrence matrix. By weighted fusion of the gradient fields of the two maps and using the cosine of the gradient direction angle to determine the synthesis direction, the resulting fused feature tensor simultaneously integrates frequency domain structural invariance information and spatial statistical texture information. This highlights subtle transition regions in the gelatinization process that are difficult to discern with the naked eye, effectively amplifying subtle differences in the gelatinization state at the feature level, thus overcoming the deficiency of single-modal features in resolving the mid-stage of gelatinization. Based on the fused feature tensor, the fused gradient magnitude and direction sequence of each pixel at all sampling times are extracted to form a temporal change trajectory, which in turn generates a temporal difference trajectory. Spatial neighborhood clustering is then performed on the temporal difference trajectories of all pixels to form change clusters. Dynamic temporal warping distance is used to measure the similarity of trajectories of adjacent pixels. Pixels with consistent temporal change trends are grouped into the same change cluster. The spatial distribution density and average change rate of the temporal difference trajectory are statistically analyzed using clusters as units and encoded as a spatiotemporal descriptor of the blurring process. This processing method transforms discrete pixel-level change information into regional change patterns with spatial connectivity and temporal evolution consistency. It can accurately distinguish edge-priority blurring regions, center-lagging blurring regions, and uniform blurring regions in the blurring process, resulting in more significant separation of samples with different blurring completion levels when projected into the discrimination space. Meanwhile, since the density-rate descriptor of the variation cluster directly characterizes the synchronicity and speed of the gelatinization process within the region, even when faced with individual differences in different batches of konjac in the gelatinization state discrimination space, the degree of gelatinization completion can still be stably determined based on the distance between the projection position and the center vector of the known gelatinization state region, thus improving the anti-interference ability and generalization of the detection method. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1 This is a flowchart of the method for detecting gelatinization of bulbils and konjac;

[0018] Figure 2 This is a flowchart of the multimodal image sequence acquisition process for sliced ​​bulbils of konjac.

[0019] Figure 3 This is a flowchart of the generation process for the frequency domain phase consistency feature map and the spatial domain texture energy distribution map;

[0020] Figure 4 It is a flowchart of the process of generating a fused feature tensor by fusing the frequency domain phase consistency feature and the spatial domain texture energy distribution with cross-gradient fusion;

[0021] Figure 5 This is a flowchart of the spatiotemporal descriptor construction process of the gelatinization process of bulbil konjac slices;

[0022] Figure 6 This is a flowchart of the method for generating a spatiotemporal descriptor for the gelatinization process of bulbil konjac slices and determining the degree of gelatinization completion. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See Figure 1 This invention provides a method for detecting the gelatinization of Amorphophallus bulbifera based on multimodal image acquisition technology. The method includes: acquiring multispectral image sequences and thermal imaging image sequences of Amorphophallus bulbifera slices at multiple preset sampling times to form a multimodal image sequence; performing a joint spatial-domain to frequency-domain transformation on the multimodal image sequence to extract the frequency-domain phase consistency feature map and spatial-domain texture energy distribution map of the Amorphophallus bulbifera slices at each sampling time; performing pixel-by-pixel cross-gradient fusion of the frequency-domain phase consistency feature map and the spatial-domain texture energy distribution map to generate a fused feature tensor; constructing a spatiotemporal descriptor of the gelatinization process of the Amorphophallus bulbifera slices based on the temporal change trajectory of each pixel in the fused feature tensor; and determining the degree of gelatinization completion of the Amorphophallus bulbifera slices at the current sampling time according to the projection position of the spatiotemporal descriptor of the gelatinization process in a preset gelatinization state discrimination space.

[0025] Example 1:

[0026] In specific implementation, please refer to Figure 2 The process of acquiring multispectral image sequences and thermal imaging image sequences of bulbils-konjac slices at multiple preset sampling times to form a multimodal image sequence is achieved through four parts: multispectral camera control, thermal imaging camera synchronization control, establishment of timestamp correspondence, and sequence combination.

[0027] During multispectral image sequence acquisition, the multispectral camera is positioned directly above the sliced ​​konjac bulbils, with the lens optical axis perpendicular to the surface to be measured. Preset sampling times are configured in the acquisition control unit of the multispectral camera at predetermined time intervals. The number of preset sampling times is related to the expected monitoring duration of the gelatinization process and the sampling frequency. At each preset sampling time, the multispectral camera sequentially switches to the visible light band, near-infrared band, and short-wave infrared band, triggering imaging in each band to obtain the reflectance images of the visible light band, near-infrared band, and short-wave infrared band corresponding to the current sampling time. Band switching within the multispectral camera is achieved through filter wheel rotation or a combination of a beam splitter and multiple imaging sensors to ensure strict spatial registration of the reflectance images of the three bands at the same sampling time. The reflectance images of the three bands corresponding to one sampling time are combined into a set of multispectral image frames. The sets of multispectral image frames acquired at each preset sampling time are arranged in chronological order to generate a multispectral image sequence. Each frame of a multispectral image sequence carries a timestamp information of the acquisition time. The timestamp information is provided by the hardware clock inside the multispectral camera and records the start time of the imaging exposure.

[0028] During the acquisition of thermal imaging image sequences, the thermal imaging camera and the multispectral camera are fixed together on the same acquisition bracket, with the thermal imaging camera's field of view covering the same target area of ​​the Amorphophallus bulbifera slice. The thermal imaging camera is connected to the synchronization signal output interface of the multispectral camera via an external trigger signal line. When the multispectral camera begins exposure at each preset sampling time, the synchronization signal is sent to the thermal imaging camera, triggering it to acquire a frame of surface thermal radiation distribution image of the Amorphophallus bulbifera slice at the same sampling time. The response delay of the thermal imaging camera is pre-calibrated and compensated for by adjusting the advance of the trigger signal relative to the multispectral camera's exposure time, ensuring that the surface thermal radiation distribution image generated by the thermal imaging camera is temporally consistent with the multispectral image frame acquired by the multispectral camera. The surface thermal radiation distribution image acquired by the thermal imaging camera is also marked with the timestamp information of the acquisition time, provided by the internal hardware clock of the thermal imaging camera and synchronized with the timestamp of the multispectral camera. All surface thermal radiation distribution images are arranged in the order of the preset sampling times to generate a thermal imaging image sequence.

[0029] When establishing the timestamp correspondence, multispectral image frames and surface thermal radiation distribution images acquired at the same preset sampling time are extracted. The multispectral image frame contains reflectance images in the visible light band, near-infrared band, and short-wave infrared band, while the surface thermal radiation distribution image is a single-channel thermal radiation value image. By comparing the timestamps of the multispectral image frame and the surface thermal radiation distribution image at the same preset sampling time, it is determined that they belong to the same preset sampling time. Based on the condition that the timestamps are equal or the difference is less than the preset synchronization tolerance, the multispectral image frame and the surface thermal radiation distribution image are associated to form a set of multimodal image pairs. In terms of data structure, the multimodal image pair is represented by packaging the three band reflectance images of the multispectral image frame and the surface thermal radiation distribution image into the same data unit. A set of multimodal image pairs is generated for each preset sampling time.

[0030] When combining the images into a multimodal image sequence, all multimodal image pairs are arranged sequentially according to the preset sampling time order. The multimodal image sequence corresponds to the increasing sampling time with index order. The 0th element in the multimodal image sequence records the multimodal image pair at the first preset sampling time, the 1st element records the multimodal image pair at the second preset sampling time, and so on. The final multimodal image sequence is used as the input for the subsequent joint transformation step.

[0031] Example 2:

[0032] In specific implementation, please refer to Figure 3 A joint spatial-to-frequency domain transformation is performed on the multimodal image sequence to extract the frequency-domain phase consistency feature map and spatial-domain texture energy distribution map of the bulbil-konjac slices at each sampling time. This is achieved through the following steps:

[0033] The multimodal image pairs corresponding to each preset sampling time are sequentially extracted from the multimodal image sequence. The multimodal image sequence is arranged in chronological order of sampling time, and each multimodal image pair contains a visible light band reflectance image, a near-infrared band reflectance image, a short-wave infrared band reflectance image, and a surface thermal radiation distribution image acquired at the same preset sampling time. When extracting the multimodal image pairs, the visible light band reflectance image, the near-infrared band reflectance image, and the short-wave infrared band reflectance image are used together as a multispectral image, and the surface thermal radiation distribution image is used as a thermal imaging image.

[0034] A two-dimensional discrete Fourier transform (DFT) is performed on the multispectral image. Each band reflectance image in the multispectral image is considered as a two-dimensional spatial domain signal, and a DFT is performed on each band reflectance image. The DFT is calculated as follows: for a band reflectance image of size M×N, f(x,y), where x is the pixel row index (x=0,1,…,M-1) and y is the pixel column index (y=0,1,…,N-1), the complex frequency domain function F(u,v) is calculated:

[0035]

[0036] in: Represents the complex frequency domain coefficients after the two-dimensional discrete Fourier transform. This is the frequency index for the row direction in the frequency domain. , Frequency index in the column direction of the frequency domain. , The imaginary unit satisfies , is the base of the natural logarithm. This represents the total number of rows in the band reflectance image. This represents the total number of columns in the band reflectance image. The amplitude was extracted to obtain the multispectral frequency domain amplitude spectrum. Phase extraction yields multispectral frequency domain phase spectrum ,in This indicates the complex argument operator. Two-dimensional discrete Fourier transform and amplitude / phase extraction operations are performed on the visible light band reflectance image, near-infrared band reflectance image, and short-wave infrared band reflectance image in the multispectral image. The multispectral frequency domain phase spectra generated from the three band reflectance images are stacked in a pixel-aligned manner to form a unified multispectral frequency domain phase spectrum representation.

[0037] A two-dimensional discrete cosine transform is performed on the thermal imaging image. The surface thermal radiation distribution image is considered as a two-dimensional spatial domain signal, where the pixel values ​​represent the thermal radiation intensity at the corresponding location. The two-dimensional discrete cosine transform is applied to the surface thermal radiation distribution image using the following formula:

[0038]

[0039] in: The image represents the distribution of surface thermal radiation. The pixel value of the location, These are the coefficients of the two-dimensional discrete cosine transform. This is the row index for the surface thermal radiation distribution image. For the column index of the surface thermal radiation distribution image, This is the frequency index for the row direction in the frequency domain. Frequency index in the column direction of the frequency domain. This represents the total number of rows in the surface thermal radiation distribution image. This represents the total number of columns in the surface thermal radiation distribution image. and As the normalization factor, when hour ,when hour ,when hour ,when hour All Corresponding two-dimensional discrete cosine transform coefficients By constructing a matrix, we obtain the thermal imaging frequency domain coefficient matrix.

[0040] In practical implementation, phase consistency measurement is calculated on the multispectral frequency domain phase spectrum, and the phase consistency response value of each pixel is extracted to generate a frequency domain phase consistency feature map. For each pixel location in the multispectral frequency domain phase spectrum, local phase analysis is performed at different scales and directions at that pixel location. A set of bandpass filters is constructed, which have different scale and direction selection characteristics in the frequency domain. For each scale and direction, the local phase value under the corresponding filter response is calculated, and the degree of local phase consistency is statistically analyzed. The phase consistency measurement is given by the following formula for each pixel. Phase consistency response value :

[0041]

[0042] In the formula for phase consistency response value This is a scale index; the total number of scales is [number missing]. , The value of is an integer ranging from 3 to 6; Here is the direction index, and the total number of directions is... , The value is 6, corresponding to six directions: 0°, 30°, 60°, 90°, 120°, and 150°. For the first The first scale Pixels in each direction The frequency spread weight is calculated based on the concentration of the filter response near the dominant frequency. For the first The first scale Pixels in each direction The local amplitude is determined by the multispectral frequency domain amplitude spectrum and the response of the directional filter; For the first The first scale Pixels in each direction The local phase deviation function, with a value range of It is used to measure the deviation between the local phase and the average phase; For the first The first scale The phase deviation threshold in each direction is a fixed proportion of the local amplitude mean in the corresponding scale direction, with a scale factor of 0.2. This indicates half-wave rectification operation. When the internal expression is positive, it takes the positive value; otherwise, it takes zero. The numerical stability constant has a value of 1 × 10⁻ 6 In the denominator For the first The first scale Pixels in each direction The local amplitude is consistent with the meaning of the same symbol in the molecule. Phase consistency response values ​​are calculated for all pixels to form a frequency domain phase consistency feature map with the same spatial size as the multispectral image. The phase consistency response value is stored at each pixel location in the frequency domain phase consistency feature map. .

[0043] In practice, the thermal imaging frequency domain coefficient matrix is ​​reconstructed through inverse transformation to obtain a reconstructed thermal imaging spatial domain image. The contrast and energy statistics of the local neighborhood gray-level co-occurrence matrix of each pixel in the reconstructed thermal imaging spatial domain image are then calculated. Low-frequency sub-band coefficients from the thermal imaging frequency domain coefficient matrix are selected, and these low-frequency sub-band coefficients are determined by satisfying… and The coefficients constitute, where Take as , Take as This involves retaining the energy concentration coefficients of the top 25% of the row-direction and column-direction frequency indices. After setting the remaining high-frequency coefficients to zero, a two-dimensional inverse discrete cosine transform (ICC) is performed. The form of the ICC is symmetrical to that of the two-dimensional discrete cosine transform, and the calculation result forms the reconstructed thermal imaging spatial domain image. For each pixel in the reconstructed thermal imaging spatial domain image, a local neighborhood window centered on that pixel is determined, with a fixed size of 32×32 pixels. Within this local neighborhood window, the gray-level co-occurrence matrix (GLCM) is calculated, with the displacement parameter of the GLCM set to... This means that the number of gray levels is compressed to 16 for horizontally adjacent pixels. A contrast statistic is calculated from the gray-level co-occurrence matrix (GLCM). The contrast statistic reflects the degree of difference in gray values ​​within a local neighborhood; it is calculated as the sum of the products of the squares of the differences between each matrix element and its row and column indices. An energy statistic is calculated from the GLCM. The energy statistic reflects the uniformity of gray-level distribution within a local neighborhood; it is calculated as the sum of the squares of all matrix elements. Both contrast and energy statistics are generated for each pixel.

[0044] Contrast statistics and energy statistics are normalized and superimposed to generate a spatial domain texture energy distribution map. First, the contrast statistics are normalized by taking the maximum value of the contrast statistics of all pixels in the thermal imaging spatial domain reconstructed image as [the value is then defined as 'max']. The minimum value is denoted as The contrast statistics for each pixel are denoted as Normalized contrast statistic The energy statistics are also normalized, and the maximum value among all pixel energy statistics is denoted as... The minimum value is denoted as The energy statistics of each pixel are denoted as Normalized energy statistics The normalized contrast statistics and normalized energy statistics are superimposed pixel-by-pixel to obtain the spatial domain texture energy distribution map. Position value All pixels This constructs a spatial domain texture energy distribution map. The spatial domain texture energy distribution map has the same spatial size as the frequency domain phase consistency feature map and is used for subsequent cross-gradient fusion.

[0045] Example 3:

[0046] In specific implementation, please refer to Figure 4 The frequency domain phase consistency feature map and the spatial domain texture energy distribution map are fused pixel by pixel through cross-gradient fusion to generate a fused feature tensor, which is achieved in the following way.

[0047] The phase consistency gradient magnitude and direction are calculated for each pixel in the frequency domain phase consistency feature map to generate a phase consistency gradient field. The frequency domain phase consistency feature map is treated as a two-dimensional scalar field, and the phase consistency gradient is calculated for each pixel using the central difference operator. The pixel coordinates are then used to calculate the gradient. For example, among which This is the row index of the frequency domain phase consistency feature map. The column index of the frequency domain phase consistency feature map, and the row direction component of the phase consistency gradient. The phase consistency gradient column direction component is obtained by differentiating adjacent pixel values ​​in the row direction of the frequency domain phase consistency feature map. The phase consistency feature map is obtained by differentiating adjacent pixel values ​​along the column direction in the frequency domain. The phase consistency gradient magnitude is calculated from the row and column direction components. The calculation method is as follows Phase-consistent gradient direction Determined by the following formula: when When it is not zero, ;when equal to zero and When greater than zero, ;when equal to zero and When less than zero, ;when and When both are zero, Set to zero. Calculate the phase consistency gradient magnitude and direction for all pixels in the frequency domain phase consistency feature map, generating a phase consistency gradient field. The phase consistency gradient field is stored at each pixel location. binary tuple.

[0048] The texture energy gradient magnitude and direction are calculated for each pixel in the spatial domain texture energy distribution map to generate a texture energy gradient field. The spatial domain texture energy distribution map is treated as a two-dimensional scalar field. For each pixel in the spatial domain texture energy distribution map, the texture energy gradient is calculated using a central difference operator of the same size as that used in phase-consistent gradient calculation. The pixel coordinates are then used to calculate the gradient. Represents the row and column indices of the spatial domain texture energy distribution map, and the row direction component of the texture energy gradient. The texture energy gradient column direction component is obtained by differing the values ​​of adjacent pixels in the row direction of the spatial domain texture energy distribution map. The texture energy gradient magnitude is obtained by subtracting the values ​​of adjacent pixels along the column direction from the spatial domain texture energy distribution map. The calculation formula is: Texture energy gradient direction The determination rule is: when When it is not zero, ;when equal to zero and When greater than zero, ;when equal to zero and When less than zero, ;when and When all are zero, Set to zero. Calculate the texture energy gradient magnitude and direction for all pixels in the spatial domain texture energy distribution map, generating a texture energy gradient field. The texture energy gradient field is stored at each pixel location. binary tuple.

[0049] The weighted sum of the gradient magnitudes of the phase-consistency gradient field and the texture energy gradient field at the same pixel location yields the fused gradient magnitude for that pixel. The phase-consistency gradient field and the texture energy gradient field are pre-registered spatially, based on the fact that the spatial dimensions of the frequency-domain phase-consistency feature map and the spatial-domain texture energy distribution map are the same and their row and column indices correspond one-to-one. At the same pixel location... Read the phase-consistent gradient magnitude from the phase-consistent gradient field Read the texture energy gradient magnitude from the texture energy gradient field. Fusion gradient magnitude Calculated using the following formula:

[0050]

[0051] in: pixel position The magnitude of the fusion gradient at the point is dimensionless. The phase consistency gradient magnitude weighting coefficient has a range of values. , The method of obtaining the value is determined by maximizing the mutual information between the gradient magnitude distribution of the frequency domain phase consistency feature map of historical samples and the annotation of the blurring process. The value is selected when the mutual information reaches its maximum value. The value is ; The weighting coefficients for the texture energy gradient magnitude. and Satisfying Relationships ,Depend on After determination, the calculation is obtained. ; pixel position Phase consistency gradient magnitude at the location; pixel position The magnitude of the texture energy gradient at that location.

[0052] The fusion gradient direction of a pixel is determined based on the cosine of the angle between the phase consistency gradient direction and the texture energy gradient direction. At the same pixel location... Read the phase-consistent gradient direction from the phase-consistent gradient field Read the texture energy gradient direction from the texture energy gradient field Calculate the cosine of the angle between the phase consistency gradient direction and the texture energy gradient direction. The calculation formula is: Cosine value of the included angle The range of values ​​is The magnitude of the cosine of the included angle reflects the degree to which the two gradient fields are aligned at that pixel location. (Fusing gradient directions) It is determined by a weighted combination of the cosine of the included angle and the two gradient directions, and the calculation method is as follows:

[0053]

[0054] Where: weighting coefficient and The value of is the same as that used in the calculation of the fusion gradient magnitude, which are respectively and .

[0055] In one alternative implementation, when the cosine value of the included angle... When it is greater than zero, it indicates that the angle between the phase consistency gradient direction and the texture energy gradient direction is... Within the interval, the two gradient directions tend to be consistent, and the gradient directions are merged. Pick and The weighted arithmetic mean result; when the cosine value of the included angle When the gradient is less than or equal to zero, it indicates that the two gradient directions differ significantly, and the gradient directions should be merged. The gradient direction of the gradient field is determined by taking the larger value between the phase consistency gradient magnitude and the texture energy gradient magnitude.

[0056] The magnitude and direction of the fused gradient for each pixel are combined to form a fused feature vector. These fused feature vectors are then arranged according to their spatial positions to generate a fused feature tensor. (For pixel positions...) fusion of feature vectors It is a two-dimensional vector, and the first dimension stores the fused gradient magnitude. The second-dimensional component stores the fused gradient direction. ,Right now Index all pixel rows. from to Column index from to Fusion feature vector after traversal Arranged in order of pixel spatial location, among which This represents the total number of rows in the frequency domain phase consistency feature map. The total number of columns in the frequency domain phase consistency feature map is arranged into a three-dimensional data structure, with the first dimension corresponding to the row index. The second dimension corresponds to the column index. The third dimension stores the two components of the fused feature vector; this three-dimensional data structure is called the fused feature tensor. The size of the fused feature tensor is... .

[0057] Example 4:

[0058] In specific implementation, please refer to Figure 5 Based on the temporal variation trajectory of each pixel in the fusion feature tensor, a spatiotemporal descriptor for the gelatinization process of bulbil konjac slices is constructed, which is achieved in the following way:

[0059] The fused gradient magnitude sequence and fused gradient direction sequence of each pixel at all preset sampling times are extracted from the fused feature tensor to form the temporal variation trajectory of that pixel. A fused feature tensor is generated at each preset sampling time, and the fused feature tensors at all preset sampling times are arranged in chronological order to form a fused feature tensor time series. For pixel positions... Iterate through the fused feature tensors at all preset sampling times, starting from the first... Read the fused feature vector from the fused feature tensor at each preset sampling time. ,in This is the index for the preset sampling time. , The total number of preset sampling times. For the first Pixel position at each preset sampling time The fusion gradient magnitude, For the first Pixel position at each preset sampling time The fusion gradient direction. All The fused gradient magnitude at each preset sampling time is indexed. Arrange in ascending order to obtain pixel positions. fused gradient magnitude sequence All The fusion gradient direction at each preset sampling time is indexed. Arrange in ascending order to obtain pixel positions. fused gradient direction sequence The merged gradient magnitude sequence and the merged gradient direction sequence together constitute the pixel location. The temporal variation trajectory is obtained. Extraction is performed on all pixel positions in the fused feature tensor, generating a set of temporal variation trajectories for each pixel position.

[0060] Calculate the difference in fused gradient magnitude and fused gradient direction between adjacent sampling times in the temporal variation trajectory of each pixel to generate the temporal difference trajectory of that pixel. For pixel location... The fused gradient magnitude sequence is used to calculate the fused gradient magnitude difference between adjacent sampling times. ,in The range is to , co-generated Each fused gradient magnitude difference value, sorted by index The gradient magnitude difference sequence is formed by ascending order. For pixel locations... The fused gradient direction sequence is used to calculate the difference between fused gradient directions between adjacent sampling times. The calculation method is to... and The difference is mapped to Within the interval, the mapping method is to add or subtract the difference. Integer multiples of make the mapping result fall on Within the range, The range is to , co-generated Each fused gradient direction difference value, by index The gradient magnitude difference sequence is formed by merging the gradient direction difference sequence in ascending order. The merged gradient magnitude difference sequence and the merged gradient direction difference sequence together constitute the pixel location. The temporal difference trajectory.

[0061] Spatial neighborhood clustering is performed on the temporal difference trajectories of all pixels, grouping pixels with consistent temporal difference trajectory trends into the same change cluster. The fused gradient magnitude difference sequence and fused gradient direction difference sequence of each pixel are used as the trajectory feature vector for that pixel. Pixel location. Trajectory feature vector The construction method is to fuse the gradient magnitude difference sequence. In the difference sequence of each element and the fused gradient direction The elements are joined together end to end to form a dimension. The vector for the pixel position. adjacent pixel positions Adjacent relationship is defined as belong The 8-neighborhood, i.e. , ,and Calculate the feature vectors of the trajectories of adjacent pixels. and The dynamic temporal warping distance is used as the trajectory similarity between adjacent pixels. The dynamic temporal warping distance is calculated by constructing a matrix of size [size missing]. The cumulative distance matrix, row index in the cumulative distance matrix correspond The Middle The sub-vector composed of the fused gradient magnitude difference and the fused gradient direction difference at each difference time step, column index. correspond The Middle The sub-vectors are composed of the difference in fused gradient magnitude and the difference in fused gradient direction at each difference time. The distance between the sub-vectors is measured by Euclidean distance. The elements of the cumulative distance matrix are obtained recursively through dynamic programming. The dynamic time warped distance is taken as the final element value of the lower right corner of the cumulative distance matrix.

[0062] Connect adjacent pixels whose trajectory similarity is less than a preset similarity threshold to form spatial adjacency edges. The preset similarity threshold is set to... The preset similarity threshold is taken as follows: The basis is that, in the temporal difference trajectory dataset of historical bulbil konjac gelatinization samples, the distribution of trajectory similarity between pixels within the same gelatinization state region is statistically analyzed, and the distribution is taken as the first... Percentiles serve as the boundary values ​​distinguishing between differences within a region of gelatinization and differences between regions. These boundary values ​​are calculated as follows: For each pair of adjacent pixel positions, if the trajectory similarity is less than... A spatial adjacency edge is established between two pixel locations, indicating that the two pixel locations share a consistent trajectory change trend. Based on all spatial adjacency edges, connected component labeling is performed on the pixels, and the set of interconnected pixels is marked as the initial change cluster. Connected component labeling is completed by traversing the graph structure formed by spatial adjacency edges using either depth-first search or breadth-first search, with each connected component corresponding to an initial change cluster.

[0063] For each initial change cluster, calculate the mean vector of the trajectory feature vectors of all pixels within the initial change cluster, and use this mean vector as the cluster center vector of the initial change cluster. Let the initial change cluster... Include The pixel positions are set as follows: Cluster center vector The calculation formula is: The initial variant clusters are iteratively merged. In each iteration, the Euclidean distance between the cluster center vectors of any two initial variant clusters is calculated. Two initial variant clusters whose Euclidean distance is less than a preset merging threshold are merged into a new variant cluster. The preset merging threshold is set to... The preset merging threshold is set to The basis for this analysis is that, in historical samples of konjac bulbils undergoing gelatinization, the spatiotemporal distribution characteristics of the cluster center vectors of the initial change clusters during the gelatinization process were statistically analyzed. The valley points of the inter-cluster distance histogram were used as the critical distances to distinguish different gelatinization process modes. The valley points were determined to be located at... .

[0064] The iterative merging process continues. In each iteration, all current change clusters are traversed, the Euclidean distance between the cluster center vectors of each pair of change clusters is calculated, and the change cluster pair with the smallest Euclidean distance is selected. If the smallest Euclidean distance is less than a preset merging threshold, the process is considered complete. Merge this pair of variant clusters, recalculate the cluster center vector of the merged variant cluster, and proceed to the next iteration; if the minimum Euclidean distance is not less than the preset merging threshold... The iterative merging process terminates, yielding the final variant cluster partitioning result. In the final variant cluster partitioning result, the Euclidean distance between the cluster center vectors of any two variant clusters is not less than a preset merging threshold. .

[0065] The spatial distribution density of pixels within each variation cluster and the average rate of change of the temporal difference trajectory are statistically analyzed to generate a cluster descriptor for each variation cluster. For each variation cluster... Statistical change clusters The total number of pixels contained is denoted as Calculate the variation cluster The average spatial distance between each pixel and its nearest neighbor. (Based on pixel position) In changing clusters Taking the example in the image, its nearest neighbor pixels are a variation cluster. Except for pixel position In addition to, and pixel position The pixel position with the smallest Euclidean distance. spatial distance Calculate the variation cluster All pixels The arithmetic mean of the spatial distances is obtained. The reciprocal of the average spatial distance is used as the cluster of variation. Spatial distribution density The calculation formula is: .

[0066] Extracting variation clusters Temporal difference trajectories of all pixels within the range. For variation clusters... Inner pixel position Its fused gradient magnitude difference sequence is , The range from to Calculate pixel position The sum of the absolute values ​​of the amplitude changes of the time-series differential trajectory over all sampling times The total number of sampling intervals is Pixel position Average rate of change over time Calculate the variation cluster The arithmetic mean of the temporal change average rates of all pixels within the range is used as the change cluster. Average rate of change of time-series difference trajectory The calculation formula is: Spatial distribution density and the average rate of change of time-series differential trajectory Combinations of density-rate binary groups of variable clusters And the density-rate binary is used as the variation cluster. Cluster descriptors.

[0067] In practice, the formula only involves the calculation of the Euclidean distance between the center vectors of the changing clusters. The Euclidean distance is calculated as the square root of the sum of the squares of the differences in the corresponding dimensions of the two vectors, and will not be listed separately.

[0068] The cluster descriptors of all variation clusters are combined and encoded according to their spatial location and temporal sequence to form the spatiotemporal descriptor of the gelatinization process of the Amorphophallus bulbifera slice. The spatial distribution density of each variation cluster is obtained, and all variation clusters are sorted from high to low spatial distribution density to obtain a spatial density sorting sequence. In the spatial density sorting sequence, the variation cluster indices are arranged in descending order of spatial distribution density. The average change rate of the temporal differential trajectory of each variation cluster is obtained, and the average change rate of the temporal differential trajectory is arranged according to the chronological order of sampling time to obtain a temporal rate sorting sequence. The temporal rate sorting sequence is constructed as follows: for each variation cluster, the pixel positions it contains correspond to a specific region of the Amorphophallus bulbifera slice in space. The average change rate of the temporal differential trajectory of this specific region is expanded in ascending order of sampling time index to form the temporal rate sequence of that variation cluster. The temporal rate sequences of all variation clusters are combined according to their variation cluster indices to form the temporal rate sorting sequence. The cluster identifier of each variation cluster in the spatial density sorting sequence is cross-aligned with the cluster identifier of the corresponding variation cluster in the temporal rate sorting sequence to establish a mapping relationship between the variation clusters in the spatial and temporal dimensions. According to the mapping relationship, the cluster descriptors of each variation cluster are concatenated into a spatiotemporal joint coding segment. The spatiotemporal joint coding segment contains a cluster identifier, spatial distribution density, average change rate of temporal differential trajectory, and a compact representation of the spatial location range of pixels within the variation cluster. All spatiotemporal joint coding segments of all variation clusters are sequentially concatenated according to the spatial density sorting sequence to generate the spatiotemporal descriptor of the gelatinization process. The data structure of the spatiotemporal descriptor of the gelatinization process is a variable-length structure, and its length depends on the number of variation clusters in the current bulbil-konjac slice. Each segment in the spatiotemporal descriptor of the gelatinization process corresponds to the spatial distribution density and average change rate of temporal differential trajectory information of one variation cluster.

[0069] Example 5:

[0070] In specific implementation, please refer to Figure 6 The cluster descriptors of all variation clusters are combined and encoded according to their spatial position and temporal sequence to form the spatiotemporal descriptor of the gelatinization process of bulbil-konjac slices, which is achieved in the following way:

[0071] Obtain the spatial distribution density of each variation cluster. The spatial distribution density of the variation cluster is stored in the cluster descriptor as the first component of a density-rate tuple. Sort all variation clusters in descending order of spatial distribution density. The sorting operation involves comparing the spatial distribution density values ​​of any two variation clusters, placing the variation cluster with the larger spatial distribution density value at the beginning of the sequence and the variation cluster with the smaller spatial distribution density value at the end of the sequence. If the spatial distribution densities of two variation clusters are equal, further compare the average change rate of their temporal difference trajectories, placing the variation cluster with the smaller average change rate of its temporal difference trajectories first. After sorting, a spatial density sorted sequence is obtained. Each element in the spatial density sorted sequence records the cluster identifier and spatial distribution density value of the variation cluster, and the sequence length is the total number of variation clusters. The spatial density sorting sequence reflects the difference in the degree of aggregation of structural changes in different spatial regions of Amorphophallus bulbifera slices during the gelatinization process. Clusters of changes with higher spatial density correspond to regions where the spatial distribution of pixels in the Amorphophallus bulbifera slices is more compact, while clusters of changes with lower spatial density correspond to regions where the spatial distribution of pixels is more sparse.

[0072] Obtain the average rate of change of the temporal differential trajectory for each variation cluster. The average rate of change of the temporal differential trajectory for each variation cluster is stored in the cluster descriptor as the second component of a density-rate tuple. Arrange the average rates of change of the temporal differential trajectory according to the order of sampling times to obtain a time rate sorting sequence. The time rate sorting sequence is arranged as follows: for each variation cluster, all preset sampling time indices are... from to In ascending order, index for each sampling time. A temporal differential trajectory average rate of change is corresponding to a value obtained by taking the arithmetic mean of the temporal average rates of change of all pixels within the change cluster at that sampling time, forming a temporal rate sequence for that change cluster. All temporal rate sequences of change clusters are then organized according to their cluster identifiers and combined into a temporal rate sorting sequence. Each row of the sorted temporal rate sequence corresponds to a change cluster, and each column corresponds to a preset sampling time. The row order and spatial density sorting sequence have not yet been established as a correspondence.

[0073] The cluster identifiers of each variation cluster in the spatial density sorting sequence are cross-aligned with the corresponding cluster identifiers in the time rate sorting sequence to establish a mapping relationship between the variation clusters in the spatial and temporal dimensions. The cross-alignment operation uses the cluster identifier as the matching key, traversing each cluster identifier in the spatial density sorting sequence, finding rows with the same cluster identifier in the time rate sorting sequence, and associating the row index in the time rate sorting sequence with the sequence position in the spatial density sorting sequence. After cross-alignment, each variation cluster simultaneously has a spatial density sorting position and a data reference in the time rate sequence, establishing a mapping relationship between the variation clusters in the spatial and temporal dimensions.

[0074] According to the mapping relationship, the cluster descriptors of each variation cluster are concatenated into a spatiotemporal joint coding segment. For a variation cluster, the spatiotemporal joint coding segment contains four fields: the first field is the cluster identifier of the variation cluster, encoded using integer; the second field is the spatial distribution density of the variation cluster, encoded using 32-bit floating-point numbers; the third field is the average rate of change of the temporal differential trajectory of the variation cluster in all... The sequence consists of a set of preset sampling times, with each element encoded using a 32-bit floating-point number. The fourth field represents the spatial location range of the variation cluster, expressed as the smallest bounding rectangle of all pixel positions within the variation cluster, recording four integer values: the row index of the top-left corner, the column index of the top-left corner, the height of the rectangle, and the width of the rectangle. The four fields are concatenated sequentially to form a fixed-structure spatiotemporal joint coding segment.

[0075] The spatiotemporal joint encoded segments of all variation clusters are concatenated sequentially according to the spatial density sorting sequence to generate the spatiotemporal descriptor of the scrambling process. The concatenation method is as follows: starting with the variation cluster corresponding to the first element of the spatial density sorting sequence, the spatiotemporal joint encoded segment of that variation cluster is written to the output buffer. Then, the spatiotemporal joint encoded segment of the variation cluster corresponding to the second element of the spatial density sorting sequence is appended to the output buffer, and so on, until the spatiotemporal joint encoded segments of all variation clusters in the spatial density sorting sequence have been written to the output buffer. The complete byte sequence in the output buffer constitutes the spatiotemporal descriptor of the scrambling process. The total length of the spatiotemporal descriptor of the scrambling process is... The total number of variation clusters The length of each spatiotemporal joint coding segment determines the cluster descriptor information of each variation cluster, which is read sequentially segment by segment during parsing.

[0076] In practice, the degree of gelatinization of the bulbils and konjac slices at the current sampling time is determined by the projection position of the spatiotemporal descriptor of the gelatinization process in the preset gelatinization state discrimination space, which is achieved in the following way.

[0077] The spatiotemporal descriptor of the gelatinization process is mapped to a pre-defined gelatinization state discrimination space to obtain a projected coordinate vector. The pre-defined gelatinization state discrimination space is pre-constructed based on the spatiotemporal descriptor of the gelatinization process of historical konjac bulb germ gelatinization samples and their corresponding gelatinization completion labels. The gelatinization state discrimination space is constructed by collecting historical konjac bulb germ gelatinization samples. Each historical konjac bulb germ gelatinization sample corresponds to a spatiotemporal descriptor of the gelatinization process and a manually labeled gelatinization completion label. The gelatinization completion label is a vector located at... Real values ​​within the interval This indicates that it is not gelatinized. The first value represents complete gelatinization, and the median value represents partial gelatinization. The spatiotemporal descriptors of the gelatinization process for all historical Amorphophallus bulbifera gelatinization samples are vectorized. This vectorization involves expanding the spatial distribution density of each change cluster and the average rate of change of the temporal difference trajectory in the spatiotemporal descriptor of the gelatinization process into a one-dimensional vector. Principal component analysis is performed on the vectorized set of historical gelatinization process spatiotemporal descriptors, extracting the top principal component directions with a cumulative contribution rate of 95%. These directions form the basis vectors of the gelatinization state discrimination space, denoted by . , Determined by the cumulative contribution rate threshold. The spatiotemporal descriptor of the gelatinization process of the current bulbil-konjac slice is expanded into a one-dimensional vector using the same vectorization method, and projected onto the basis vectors of the gelatinization state discrimination space to obtain the projected coordinate vector. ,in For the projected coordinate vector at the th The coordinate values ​​along the directions of each principal component. .

[0078] Calculate the distance between the projected coordinate vector and the center vectors of multiple gelatinized state regions in the gelatinization state discrimination space. The gelatinization state discrimination space is pre-divided into three gelatinized state regions: ungelatinized region, partially gelatinized region, and fully gelatinized region. The center vector of the ungelatinized region is obtained by selecting gelatinization completion labels from historical bulbil konjac gelatinization samples. The projected coordinate vectors of the samples within the interval are used to calculate the arithmetic mean of these projected coordinate vectors, which yields the center vector of the unbleached region. The method for obtaining the center vector of the partially gelatinized region is to screen gelatinization completion labels from historical bulbil konjac gelatinization samples. The projected coordinate vectors of the samples within the interval are used to calculate the arithmetic mean of these projected coordinate vectors, which yields the center vector of the partially blurred region. The method for obtaining the center vector of the fully gelatinized region is to screen gelatinization completion labels from historical bulbil-konjac gelatinization samples. The projected coordinate vectors of the samples within the interval are used to calculate the arithmetic mean of these projected coordinate vectors, which yields the center vector of the completely blurred region. .

[0079] With projected coordinate vector With the center vector of the unbleached region The Euclidean distance between them is used as the first distance value. The calculation formula is: . Using projected coordinate vectors With the center vector of the partially gelatinized region The Euclidean distance between them is used as the second distance value. The calculation formula is: . Using projected coordinate vectors With the center vector of the fully gelatinized region The Euclidean distance between them is used as the third distance value. The calculation formula is: .

[0080] The first, second, and third distance values ​​are each subjected to interval constraint normalization. The calculation method for interval constraint normalization is as follows: Let... This is the original distance value. , or One of them, the final distance value after normalization It is given by the following formula:

[0081]

[0082] in: This represents the final distance value after interval constraint normalization, dimensionless, and its range is [value missing]. ; This represents the original distance value to be normalized. , or ; Indicates taking the first distance value Second distance value and the third distance value The minimum value in; Indicates taking the first distance value Second distance value and the third distance value The maximum value in; The normalized smoothing factor is The value is For the first distance value Perform interval constraint normalization to obtain the normalized first distance value. For the second distance value Perform interval constraint normalization to obtain the normalized second distance value. For the third distance value Perform interval constraint normalization to obtain the normalized third distance value. . , and This serves as the final distance value used to select the minimum distance.

[0083] The region of gelatinized state corresponding to the minimum final distance value is selected as the region of gelatinized state with the minimum distance to the projected coordinate vector. Comparison. , and The magnitude of the value, if If the smallest value is selected from the three final distance values, the unbleached region is chosen as the bleached region at the current sampling time; if If the smallest value is selected from the three final distance values, then the partially blurred region is chosen as the blurred state region at the current sampling time; if The region with the smallest final distance value is selected as the region of complete gelatinization at the current sampling time.

[0084] Obtain the gelatinization completion interval corresponding to the selected gelatinized region in the gelatinization state discrimination space. The gelatinization completion interval corresponding to the ungelatinized region is... The gelatinization completion range corresponding to the partially gelatinized regions is: The complete gelatinization region corresponds to the gelatinization completion range as follows: Based on the relative position of the projected coordinate vector within the selected gelatinization region, the specific value of the gelatinization completion degree of the bulbil-konjac slice at the current sampling time within the gelatinization completion degree interval is determined. Taking a partial gelatinization region as an example, the projected coordinate vector... The relative position within the partially gelatinized region is determined by... With the center vector of this region The similarity measure, combined with the gelatinization completion interval Linear interpolation is performed to obtain specific values. If the unbleached region is selected, the values ​​are within the interval... Internal basis The specific value is determined by interpolation based on the distance ratio to the boundary and center vector of the ungelatinized region. If a fully gelatinized region is selected, the value is determined within the interval... Internal basis The specific value is determined by interpolation based on the distance ratio between the boundary and center vector of the fully gelatinized region. The specific value of gelatinization completion is a real number, and this real number is output as the gelatinization completion determination result of the bulbil-konjac slice at the current sampling time.

[0085] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for detecting the gelatinization of bulbils and konjac based on multimodal image acquisition technology, characterized in that, The method includes: Multispectral image sequences and thermal imaging image sequences of slices of Amorphophallus bulbifera at multiple preset sampling times were obtained to form a multimodal image sequence; A joint spatial-to-frequency domain transformation is performed on the multimodal image sequence to extract the frequency domain phase consistency feature map and spatial domain texture energy distribution map of the bulbils-konjac slices at each sampling time. The frequency domain phase consistency feature map and the spatial domain texture energy distribution map are fused pixel by pixel through cross-gradient fusion to generate a fused feature tensor. Based on the temporal change trajectory of each pixel in the fusion feature tensor, a spatiotemporal descriptor of the gelatinization process of bulbil konjac slices is constructed. Based on the projection position of the spatiotemporal descriptor of the gelatinization process in the preset gelatinization state discrimination space, the degree of gelatinization completion of the bulbil konjac slice at the current sampling time is determined.

2. The method for detecting the gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 1, characterized in that, The step of obtaining multispectral image sequences and thermal imaging image sequences of bulbil konjac slices at multiple preset sampling times to form a multimodal image sequence specifically includes: The multispectral camera is controlled to sequentially acquire reflectance images of bulbils and konjac slices in the visible light band, near-infrared band, and short-wave infrared band at multiple preset sampling times, and the images are arranged in the order of sampling times to generate a multispectral image sequence. The thermal imaging camera is synchronously controlled to acquire surface thermal radiation distribution images of bulbils and konjac slices at the same preset sampling time, and the images are arranged in the order of sampling time to generate a thermal imaging image sequence. Establish a timestamp correspondence between multispectral images and thermal imaging images acquired at the same sampling time to form multimodal image pairs; All multimodal image pairs are arranged sequentially according to the order of sampling time, and combined into a multimodal image sequence.

3. The method for detecting the gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 2, characterized in that, The steps of performing a joint spatial-to-frequency domain transformation on the multimodal image sequence to extract the frequency-domain phase consistency feature map and spatial-domain texture energy distribution map of the bulbil-konjac slices at each sampling time specifically include: Multimodal image pairs at each sampling time are sequentially extracted from the multimodal image sequence, and two-dimensional discrete Fourier transform is performed on the multispectral images in the multimodal image pairs to obtain the multispectral frequency domain amplitude spectrum and the multispectral frequency domain phase spectrum. A two-dimensional discrete cosine transform is performed on the thermal imaging images in the multimodal image pair to obtain the thermal imaging frequency domain coefficient matrix; The phase consistency measurement of the multispectral frequency domain phase spectrum is calculated, the phase consistency response value of each pixel is extracted, and a frequency domain phase consistency feature map is generated. The thermal imaging frequency domain coefficient matrix is ​​reconstructed by inverse transformation to obtain a thermal imaging spatial domain reconstructed image, and the contrast statistics and energy statistics of the local neighborhood gray-level co-occurrence matrix of each pixel in the thermal imaging spatial domain reconstructed image are calculated. The contrast statistics and energy statistics are normalized and superimposed to generate a spatial domain texture energy distribution map.

4. The method for detecting gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 3, characterized in that, The step of performing pixel-by-pixel cross-gradient fusion of the frequency domain phase consistency feature map and the spatial domain texture energy distribution map to generate a fused feature tensor specifically includes: Calculate the phase consistency gradient magnitude and phase consistency gradient direction of each pixel in the frequency domain phase consistency feature map to generate a phase consistency gradient field; Calculate the texture energy gradient magnitude and texture energy gradient direction of each pixel in the spatial domain texture energy distribution map to generate a texture energy gradient field; The phase consistency gradient field and the texture energy gradient field at the same pixel location are weighted and summed to obtain the fused gradient magnitude of that pixel. The fusion gradient direction of the pixel is determined based on the cosine of the angle between the phase consistency gradient direction and the texture energy gradient direction. The magnitude and direction of the fused gradient of each pixel are combined into a fused feature vector, and the fused feature vectors of all pixels are arranged according to their spatial positions to generate a fused feature tensor.

5. The method for detecting gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 4, characterized in that, The steps for constructing a spatiotemporal descriptor of the gelatinization process of bulbil konjac slices based on the temporal change trajectory of each pixel in the fused feature tensor specifically include: Extract the fusion gradient magnitude sequence and fusion gradient direction sequence of each pixel at all preset sampling times from the fusion feature tensor to form the temporal change trajectory of the pixel. Calculate the difference in fused gradient magnitude and the difference in fused gradient direction between adjacent sampling times in the temporal variation trajectory of each pixel to generate the temporal difference trajectory of that pixel; Spatial neighborhood clustering is performed on the temporal difference trajectories of all pixels, and pixels with consistent temporal difference trajectory change trends are grouped into the same change cluster; The spatial distribution density of pixels within each change cluster and the average rate of change of the temporal differential trajectory are statistically analyzed to generate a cluster descriptor for each change cluster. The cluster descriptors of all variation clusters are combined and encoded according to their spatial position and temporal sequence to form the spatiotemporal descriptor of the gelatinization process of bulbil konjac slices.

6. The method for detecting gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 5, characterized in that, The step of performing spatial neighborhood clustering on the temporal difference trajectories of all pixels, and grouping pixels with consistent temporal difference trajectory trends into the same change cluster, specifically includes: The amplitude change sequence and direction change sequence of the temporal differential trajectory of each pixel are used as the trajectory feature vector of that pixel; Calculate the dynamic time-warped distance between the trajectory feature vectors of adjacent pixels, and use it as the trajectory similarity between adjacent pixels; Connect adjacent pixels whose trajectory similarity is less than a preset similarity threshold as spatial adjacency edges; Based on all spatial adjacent edges, the pixels are marked as connected regions, and the set of mutually connected pixels is marked as the initial change cluster; For each initial change cluster, calculate the mean vector of the trajectory feature vectors of all pixels within the cluster, and use the mean vector as the cluster center vector of that initial change cluster; The initial change clusters are iteratively merged. In each iteration, two initial change clusters whose Euclidean distance between their cluster center vectors is less than a preset merging threshold are merged into a new change cluster. This process continues until the Euclidean distance between the cluster center vectors of any two change clusters is not less than the preset merging threshold, thus obtaining the final change cluster partitioning result.

7. The method for detecting gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 6, characterized in that, The step of generating a cluster descriptor for each change cluster by statistically analyzing the spatial distribution density of pixels within each change cluster and the average rate of change of the temporal differential trajectory specifically includes: The total number of pixels contained in each variation cluster is counted, the average spatial distance between each pixel in the variation cluster and its nearest neighbor pixel is calculated, and the reciprocal of the average spatial distance is used as the spatial distribution density of the variation cluster. Extract the temporal difference trajectory of all pixels in each change cluster, calculate the sum of the absolute values ​​of the amplitude changes of the temporal difference trajectory of each pixel over all sampling times, and then divide by the total number of sampling times to obtain the average temporal change rate of that pixel. Calculate the arithmetic mean of the temporal change average rate of all pixels within the change cluster, and use it as the temporal difference trajectory average change rate of that change cluster. The spatial distribution density and the average rate of change of the temporal differential trajectory are combined into a density-rate binary tuple of the variation cluster, and the density-rate binary tuple is used as the cluster descriptor of the variation cluster.

8. The method for detecting the gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 7, characterized in that, The step of combining and encoding the cluster descriptors of all variation clusters according to their spatial position and temporal sequence to form the spatiotemporal descriptor of the gelatinization process of bulbil konjac slices specifically includes: Obtain the spatial distribution density of each variation cluster, sort all variation clusters according to their spatial distribution density from high to low, and obtain the spatial density sorting sequence; The average rate of change of the temporal differential trajectory of each change cluster is obtained, and the average rate of change of the temporal differential trajectory is arranged in the order of sampling time to obtain the time rate sorting sequence. Cross-align the cluster identifier of each variation cluster in the spatial density sorting sequence with the cluster identifier of the corresponding variation cluster in the time rate sorting sequence to establish a mapping relationship between variation clusters in the spatial and time dimensions. The cluster descriptors of each variation cluster are concatenated into a spatiotemporal joint coding segment according to the mapping relationship described above; The spatiotemporal joint coding segments of all variation clusters are sequentially concatenated according to the spatial density sorting sequence to generate a spatiotemporal descriptor of the slurrying process.

9. The method for detecting the gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 8, characterized in that, The step of determining the degree of gelatinization completion of the bulbil konjac slice at the current sampling time based on the projection position of the spatiotemporal descriptor of the gelatinization process in the preset gelatinization state discrimination space specifically includes: The spatiotemporal descriptor of the gelatinization process is mapped to a preset gelatinization state discrimination space to obtain a projected coordinate vector. The gelatinization state discrimination space is a metric space pre-constructed based on the spatiotemporal descriptor of the gelatinization process and its corresponding gelatinization completion label of historical bulb konjac gelatinization samples. Calculate the distance between the projected coordinate vector and the center vector of multiple gelatinized state regions in the gelatinized state discrimination space, wherein the multiple gelatinized state regions include ungelatinized regions, partially gelatinized regions, and completely gelatinized regions; The region of gelatinization with the smallest distance from the projected coordinate vector is selected as the region of gelatinization at the current sampling time. Obtain the gelatinization completion interval corresponding to the selected gelatinization state region in the gelatinization state discrimination space; Based on the relative position of the projected coordinate vector within the selected gelatinization state region, the specific value of the gelatinization completion degree of the bulbil konjac slice at the current sampling time within the gelatinization completion degree range is determined.

10. The method for detecting gelatinization of bulbils and konjac based on multimodal image acquisition technology according to claim 9, characterized in that, The step of calculating the distance between the projected coordinate vector and the center vectors of multiple blurred state regions in the blurred state discrimination space specifically includes: The coordinate values ​​of the center vectors of the ungelatinized region, the partially gelatinized region, and the fully gelatinized region are read from the gelatinization state discrimination space, respectively. The Euclidean distance between the projected coordinate vector and the center vector of the ungelatinized region is used as the first distance value; the Euclidean distance between the projected coordinate vector and the center vector of the partially gelatinized region is used as the second distance value; and the Euclidean distance between the projected coordinate vector and the center vector of the fully gelatinized region is used as the third distance value. The first distance value, the second distance value, and the third distance value are respectively subjected to interval constraint normalization. The normalized first distance value, the second distance value, and the third distance value are used as the final distance value for selecting the minimum distance. The region of gelatinization corresponding to the minimum value among the final distance values ​​is selected as the region of gelatinization with the smallest distance from the projected coordinate vector.