A real-time monitoring method for ginkgo leaf extract solution state based on image processing

Abstract

The application discloses a kind of based on image processing's ginkgo leaf extract state real-time monitoring method, it is related to ginkgo leaf extraction liquid technical field.The method steps include: constructing multiple light source imaging environment and shooting ginkgo leaf extract image;Through U-net segmentation model, obtain extraction liquid mask, and obtain extraction liquid foreground image in combination with ginkgo leaf extract image;Through adaptive threshold method, extraction liquid foreground image is segmented, and extraction liquid block diagram is obtained, and after optimization through watershed boundary optimization method, optimized extraction liquid block diagram is obtained;Through the boundary probability model of double distance change, the block boundary probability value is calculated, the block gradient amplitude is calculated by Sobel operator, and the block boundary confidence is obtained in combination with block boundary probability value and block gradient amplitude, to determine block boundary pixel;Collect the boundary gradient feature vector sequence of block boundary pixel, input to extraction liquid evolution trend model, output to obtain oxidation risk index, and oxidation risk index is greater than preset threshold value, then early warning is carried out.

Description

Technical Field

[0001] This invention relates to the field of ginkgo leaf extract technology, specifically a method for real-time monitoring of the state of ginkgo leaf extract based on image processing. Background Technology

[0002] Ginkgo leaf extract, a core raw material in the pharmaceutical, health product, and cosmetic fields, relies heavily on the content and stability of its active ingredients, such as flavonoids and terpene lactones, for its quality. In industrial production processes, the extract is susceptible to oxidative degradation from preparation and storage to purification, influenced by factors such as oxygen exposure, temperature fluctuations, and light radiation. This degradation manifests as a gradual color change from light yellow to brownish-yellow and then dark brown, accompanied by an irreversible decrease in the content of active ingredients. Therefore, dynamic monitoring of the extract's oxidation state is crucial for ensuring product quality and controlling production losses.

[0003] In existing technologies, ginkgo leaf extract is usually sent to a laboratory where high-performance liquid chromatography (HPLC) is used to detect the content of active ingredients such as flavonoids. The oxidation state of the ginkgo leaf extract is determined based on the detection results, and the quality of the ginkgo leaf extract is judged based on the oxidation state.

[0004] However, determining the oxidation state of ginkgo extract through laboratory analysis has a significant detection lag and lacks the ability to track the oxidation state of the extract in real time during the production process. This makes it difficult to adjust the process parameters in the production process in a timely manner, resulting in batches of ginkgo leaf extract products being substandard and resources being wasted. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a real-time monitoring method for the state of Ginkgo biloba extract based on image processing, thereby resolving the problems existing in the background technology.

[0006] To achieve the above objectives, the present invention provides a method for real-time monitoring of the state of Ginkgo biloba leaf extract based on image processing, comprising the following steps:

[0007] Step S1: Construct a multi-light source imaging environment, place the ginkgo leaf extract in the multi-light source imaging environment for imaging, and obtain an image of the ginkgo leaf extract;

[0008] Step S2: Construct a U-net segmentation model. Input the image of the ginkgo leaf extract into the U-net segmentation model and output the extract mask. By combining the image of the ginkgo leaf extract and the extract mask, obtain the foreground image of the extract.

[0009] Step S3: Perform threshold segmentation on the foreground image of the extract using an adaptive thresholding method to obtain an extract block map; perform boundary optimization on the extract block map using a watershed boundary optimization method to obtain an optimized extract block map;

[0010] Step S4: Calculate the boundary probability of the optimized extract block map using a boundary probability model based on dual-distance variation to obtain the block boundary probability value; perform edge detection on the optimized extract block map using the Sobel operator to obtain the block gradient magnitude; calculate the block boundary confidence by combining the block boundary probability value and the block gradient magnitude; and determine the block boundary of the optimized extract block map based on the block boundary confidence value to obtain the block boundary pixels.

[0011] Step S5: By performing pixel gradient change analysis on the boundary pixels of the block, the boundary gradient feature vector is obtained; the boundary gradient feature vectors within a preset time period are collected to obtain a boundary gradient feature vector sequence.

[0012] Step S6: Construct an evolution trend model for the extract. Input the boundary gradient feature vector sequence into the evolution trend model for the extract and output an oxidation risk index. Compare the oxidation risk index with a preset threshold. If the index is greater than the preset threshold, issue an early warning, thereby realizing the status monitoring of the Ginkgo biloba extract.

[0013] Preferably, the construction of the multi-light source imaging environment, in which the ginkgo leaf extract is placed under the multi-light source imaging environment for imaging to obtain an image of the ginkgo leaf extract, includes the following specific steps:

[0014] A full-spectrum adjustable light source was set up at the monitoring point of the Ginkgo biloba extract production line to form a uniform and stable composite lighting environment. A camera was installed at the monitoring point, and the camera lens was calibrated so that it was directly facing the imaging area of ​​the monitoring point and the focus and white balance were adjusted. After the light source and camera were turned on, the light source parameters were kept unchanged, and the Ginkgo biloba extract was placed in the imaging area of ​​the monitoring point for shooting, and finally the image of the Ginkgo biloba extract was obtained.

[0015] Preferably, the step of constructing the U-net segmentation model and returning the cloud server parameter set to the communication board includes the following steps:

[0016] The U-net segmentation model consists of: an input layer, an encoder, a bottleneck layer, a decoder, and an output layer.

[0017] The input layer is used to receive images of ginkgo leaf extract and normalize the pixel value range of the ginkgo leaf extract images from [0,255] to [0,1].

[0018] The bottleneck layer consists of two consecutive 3×3 convolutions with the same padding method and a ReLU activation function.

[0019] The decoder includes: upsampling stage 1, upsampling stage 2, upsampling stage 3, and upsampling stage 4;

[0020] Output layer: 1×1 convolution with stride of 1, same padding, followed by a sigmoid activation function, and finally outputs an extract mask.

[0021] Preferably, obtaining the foreground image of the extract by combining the image of the ginkgo leaf extract and the extract mask includes the following steps:

[0022] The foreground image of the extract is obtained by combining the image of the ginkgo leaf extract with the extract mask:

[0023]

[0024] in, The pixel value of pixel (h,w) in channel c of the foreground image of the extractant. This represents the pixel value (h,w) of the ginkgo leaf extract image in channel c. Let be the pixel value of pixel (h, w) in the extraction liquid mask on channel c, where c is the channel index, h is the height index, and w is the width index. It is an empty value.

[0025] Preferably, the step of performing threshold segmentation on the foreground image of the extract using an adaptive thresholding method to obtain an extract block map includes the following steps:

[0026] The saturation channel S in the foreground image of the extract is segmented using an adaptive thresholding method: First, the saturation histogram is calculated.

[0027]

[0028] in, This represents the statistical value of pixel value k in the saturation channel. The saturation of pixel (h, w) in the extract block image, k is a statistical variable value, round() is the rounding function, h is the height index, w is the width index, H is the maximum height, and W is the maximum width.

[0029] Calculate the between-class variance:

[0030]

[0031] in, Here, AQ represents the inter-class variance, AQ represents the proportion of pixels in the stable region, and LQ represents the proportion of pixels in the oxidized region. The average saturation in the stable region, The average saturation of the oxidation zone is denoted as T, and the candidate threshold is denoted as T.

[0032] By iterating through the values ​​of T from 1 to 255, the inter-class variance corresponding to each T is calculated. Solve for the optimal threshold The optimal threshold is the value of T that maximizes the inter-class variance.

[0033] Through the optimal threshold The foreground image of the extract is segmented to obtain a block image of the extract:

[0034]

[0035] in, The label for the pixel (h,w) in the extract block image. The saturation of pixel (h,w) in the extract block image. This is the optimal threshold.

[0036] Preferably, the step of optimizing the extract block map using the watershed boundary optimization method to obtain an optimized extract block map includes the following specific steps:

[0037] The boundary optimization of the extract block map is performed using the watershed boundary optimization method: First, the extract block map is... Performing a Euclidean distance transformation yields the distance matrix D:

[0038]

[0039] Where D(h,w) represents the minimum Euclidean distance from pixel (h,w) to pixels in the stable region, h is the height index, and w is the width index. For the pixel height index of the stable region, Index of the pixel width of the stable region;

[0040] Based on the distance matrix D, seed points for the oxidation region are defined as follows:

[0041]

[0042] in, Let D(h,w) represent the seed point in the oxidation region, and let D(h,w) represent the minimum Euclidean distance from pixel (h,w) to pixels in the stable region. This represents the maximum value of the distance matrix D across all pixels;

[0043] Define the seed point of the stable region , ;

[0044] Construct a marker matrix based on the seed points of the oxidation region and the seed points of the stable region. :

[0045]

[0046] in, Let h be the value of pixel (h, w) in the marker matrix, where h is the height index and w is the width index;

[0047] Based on the tag matrix Watershed segmentation is performed using distance gradient:

[0048]

[0049] in, The watershed label for pixel (h,w) () represents the standard watershed algorithm function. This refers to the value corresponding to height h and width w in the marker matrix. Represents the gradient of the distance matrix;

[0050] The labels for the optimized binary image are:

[0051]

[0052] in, To optimize the label of pixel (h,w) in the binary image, The watershed label for pixel (h,w).

[0053] Preferably, the step of calculating the boundary probability of the optimized extract block map using a boundary probability model based on dual-distance variation to obtain the block boundary probability value includes the following specific steps:

[0054] The boundary probability of the optimized extract block map is calculated using a boundary probability model based on dual-distance variation, yielding the block boundary probability values:

[0055]

[0056] in, Let (h,w) be the block boundary probability value. Let (h,w) be the distance from pixel (h,w) to the nearest oxidation core region. Let (h,w) be the distance from pixel (h,w) to the nearest stable region. This is the scale parameter, which defaults to 2.

[0057] Preferably, the step of performing edge detection on the optimized extract block map using the Sobel operator to obtain the block gradient magnitude includes the following specific steps:

[0058] Edge detection within a 3×3 neighborhood is performed on the saturation channel of the pixel (h,w) in the optimized extract block image using the Sobel operator to obtain the block gradient magnitude:

[0059] ;

[0060] ;

[0061] ;

[0062] in, Let (h,w) be the horizontal gradient component of pixel (h,w). Let (h,w) be the vertical gradient component of pixel (h,w). Let be the block gradient magnitude of pixel (h, w), where i is the horizontal offset and j is the vertical offset. For the horizontal direction operator of the Sobel operator, For the Sobel operator in the vertical direction, Let be the saturation value of pixel (h,w) with a horizontal offset of i and a vertical offset of j.

[0063] Preferably, the step of calculating the block boundary confidence score by combining the block boundary probability value and the block gradient magnitude includes the following specific steps:

[0064] After normalizing the standard deviation of the block gradient magnitude of pixel (h,w) within a 3×3 neighborhood, the block boundary confidence score is calculated by combining it with the block boundary probability value.

[0065]

[0066] in, For the confidence level of the block boundary, Let (h,w) be the block boundary probability value. This represents the block gradient magnitude of pixel (h,w), where k is an adjustment coefficient, with a default value of 5.

[0067] Preferably, the step of constructing the extract evolution trend model, inputting the boundary gradient feature vector sequence into the extract evolution trend model, and outputting the oxidation risk index includes the following specific steps:

[0068] The extract evolution trend model includes a time series input layer, an LSTM feature encoder, a GRU sequence prediction decoder, and a risk index calculation module;

[0069] Risk Index Calculation Module: This module receives the 5×3D prediction matrix output from the decoder and calculates the oxidation risk index step-by-step.

[0070]

[0071] in, This is the oxidation risk index at time step t+k. Let X be the eigenvalue at the (t+k)th time step. This is the reference value for the Xth feature. denoted as the weight coefficient of the Xth feature, BD as the global edge ambiguity, SM as the global gradient smoothness, and TS as the global saturation transition speed.

[0072] This invention provides a real-time monitoring method for the state of Ginkgo biloba leaf extract based on image processing, involving machine learning and deep learning technologies, which has the following beneficial effects:

[0073] (1) By combining the image of the Ginkgo biloba extract and the extract mask, the foreground image of the extract is obtained. The mask generated by U-Net segmentation accurately identifies the foreground region (pixel value 1) and the background region (pixel value 0) such as the pipe wall and the observation window in the image. After expanding the mask to three channels and aligning it with the original image, a conditional assignment operation is performed. The final foreground image of the extract only contains the visual information of the target extract region, removing the interference signals of non-target regions in the imaging environment. This provides a clean and noise-free data foundation for subsequent state analysis based on the color and texture features of the extract region (such as adaptive threshold segmentation).

[0074] (2) By combining the block boundary probability value and the block gradient magnitude, the block boundary confidence is calculated. The block boundary probability value is based on the dual distance transformation model, which quantifies the spatial distance difference between the pixel and the oxidation area and the stable area (the smaller the distance difference, the higher the probability), reflecting the geometric position probability of the pixel on the boundary. The block gradient magnitude captures the local change intensity of the saturation channel through the Sobel operator, which characterizes the visual gradient features of the boundary area. The fusion of the two can complement each other to utilize spatial location information and image gradient information, effectively filter out noise pixels with low probability or low gradient, and ensure that only pixels with high confidence are judged as real boundary pixels.

[0075] (3) By combining the adaptive thresholding method and the watershed boundary optimization method, the accuracy and robustness of the segmentation of the oxidation and stable regions in the Ginkgo biloba extract image were significantly improved. The adaptive thresholding method is based on the principle of maximizing the inter-class variance of the saturation channel and automatically determines the optimal segmentation threshold to achieve the initial division of the oxidation and stable regions. On this basis, the watershed boundary optimization method reconstructs a physically reasonable continuous boundary through distance transformation and morphological seed point positioning, which effectively overcomes the jagged artifacts and noise sensitivity problems that are easily generated in the transition region by traditional thresholding. It provides a clear and distinct binary mask basis for subsequent boundary feature extraction and oxidation state analysis, and is a key preprocessing step for achieving accurate quantitative monitoring of the oxidation process. Attached Figure Description

[0076] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a flowchart of the steps of a real-time monitoring method for the state of Ginkgo biloba extract based on image processing proposed in this invention;

[0078] Figure 2 This is a step hierarchy diagram of obtaining the extract block map in the real-time monitoring method of Ginkgo biloba leaf extract state based on image processing proposed in this invention;

[0079] Figure 3 This is a step hierarchy diagram of obtaining the oxidation risk index in a real-time monitoring method for the state of Ginkgo biloba leaf extract based on image processing proposed in this invention. Detailed Implementation

[0080] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0081] Please see Figures 1-3 The present invention provides a technical solution: a method for real-time monitoring of the state of Ginkgo biloba extract based on image processing.

[0082] Step S1: Construct a multi-light source imaging environment, place the ginkgo leaf extract in the multi-light source imaging environment for imaging, and obtain an image of the ginkgo leaf extract.

[0083] A full-spectrum adjustable light source is arranged at key monitoring points (such as around transparent observation windows or circulation pipes) on the production line of Ginkgo biloba extract. At least two independent light sources at different angles are set up, such as a main light source at the top and an auxiliary light source at the bottom. The spectral bands (emphasizing the 500–600 nm yellow-brown gradient region) and light intensity of each light source are precisely adjusted to form a uniform and stable composite lighting environment. A high-resolution industrial camera is installed at the key monitoring point, and the camera lens is calibrated to face the imaging area, with focus and white balance adjusted. After starting the light source and camera, keeping the light source parameters constant, the Ginkgo biloba extract is placed in the imaging area for imaging. The final output is a high-resolution image of the Ginkgo biloba extract generated under fixed composite light source conditions.

[0084] Step S2: Construct a U-net segmentation model. Input the image of the ginkgo leaf extract into the U-net segmentation model to obtain the extract mask. By combining the image of the ginkgo leaf extract and the extract mask, obtain the foreground image of the extract.

[0085] To construct the U-net segmentation model, a labeled dataset containing diverse industrial scenarios was first built. This dataset includes sample images of Ginkgo biloba extract under different oxidation states (displayed as light yellow, brownish-yellow, and dark brown) and typical interference scenarios (pipe reflection, bubble occlusion, and liquid surface fluctuation). Each sample image underwent pixel-level foreground annotation to generate a ground truth mask. The labeled dataset was then divided into training, validation, and test sets in a 7:2:1 ratio. An adaptive momentum optimizer with a cross-entropy loss function was used to train the U-net segmentation model. The initial learning rate was set to 0.001 and dynamically adjusted using a cosine annealing strategy. During training, the segmentation accuracy was monitored by real-time calculation of the Intersection over Union (IoU) and Dice coefficients. An early stopping mechanism was triggered when the validation set IoU did not improve for five consecutive epochs. Finally, the model parameters achieving an IoU ≥ 0.95 and a recall ≥ 98% on the test set were selected as the final model parameters for the U-net segmentation model, resulting in the trained U-net segmentation model.

[0086] The image of the ginkgo leaf extract is input into a trained U-net segmentation model, which includes an input layer, an encoder, a bottleneck layer, a decoder, and an output layer.

[0087] The input layer is used to receive images of ginkgo leaf extract and normalize the pixel value range of the ginkgo leaf extract images from [0,255] to [0,1].

[0088] The encoder includes: downsampling stage 1, downsampling stage 2, downsampling stage 3, and downsampling stage 4; wherein downsampling stage 1 includes: two consecutive 3×3 convolutions + same padding + ReLU activation function, and the output size is... Recorded as Followed by 2×2 max pooling with a stride of 2, the output size is... The downsampling stage 2 includes: two consecutive 3×3 convolutions + same padding + ReLU activation function, with an output size of... Recorded as Followed by 2×2 max pooling with a stride of 2, the output size is... The downsampling stage 3 includes: two consecutive 3×3 convolutions + same padding + ReLU activation function, with an output size of... Recorded as Followed by 2×2 max pooling with a stride of 2, the output size is... The downsampling stage 4 includes: two consecutive 3×3 convolutions + same padding + ReLU activation function, with an output size of... Recorded as Followed by 2×2 max pooling with a step size of 2, the final output is... To the bottleneck layer.

[0089] The bottleneck layer consists of two consecutive 3×3 convolutions, a same padding method, and a ReLU activation function.

[0090] The decoder includes: upsampling stage 1, upsampling stage 2, upsampling stage 3, and upsampling stage 4.

[0091] The upsampling stage 1 includes: scaling the input size using a 2×2 transposed convolution. Upsampling to The step size is 2, with no padding, and then it is combined with the feature map saved in encoder downsampling stage 4. The concatenation is performed along the channel dimension, followed by two consecutive 3×3 convolutions with same padding and ReLU activation. Upsampling stage 2 includes: transposing the output of upsampling stage 1 through a 2×2 transpose convolution. Upsampling to The step size is 2, with no padding, and then it is combined with the feature map saved in the encoder downsampling stage 3. The concatenation is performed along the channel dimension, followed by two consecutive 3×3 convolutions with same padding and ReLU activation. Upsampling stage 3 includes: transposing the output of upsampling stage 2 using a 2×2 transpose convolution. Upsampling to The step size is 2, with no padding, and then it is combined with the feature map saved in the encoder downsampling stage 2. The concatenation is performed along the channel dimension, followed by two consecutive 3×3 convolutions with same padding and ReLU activation. Upsampling stage 4 includes: transposing the output of upsampling stage 3 using a 2×2 transpose convolution. Upsampling to The step size is 2, with no padding, and then it is combined with the feature map saved in encoder downsampling stage 1. The concatenation is performed along the channel dimension, followed by two consecutive 3×3 convolutions with the same padding method and the ReLU activation function.

[0092] Output layer: 1×1 convolution with stride of 1, same padding, followed by a sigmoid activation function, and finally outputs an extract mask.

[0093] It should be noted that the extract mask is the output result obtained by performing pixel-level binary segmentation on the ginkgo leaf extract image using the U-net segmentation model. Its size is exactly the same as the input ginkgo leaf extract image. The extract mask is essentially a single-channel grayscale image, where each pixel corresponds to only two values: 0 or 1. The area with a pixel value of 1 represents the foreground area of ​​the ginkgo leaf extract image as determined by the U-net segmentation model, accurately covering the part of the ginkgo leaf extract image that actually contains extract. The area with a pixel value of 0 represents the background area of ​​the ginkgo leaf extract image, corresponding to the transparent observation window, circulation pipe wall, and other parts of the original image that do not contain extract.

[0094] By combining the ginkgo leaf extract image and the extract mask, a foreground image of the extract is obtained: ensuring that the spatial pixels of the ginkgo leaf extract image and the extract mask are perfectly aligned, and expanding the single-channel extract mask into a three-channel extract mask (the expansion method is that the R, G, and B channels all copy the binary data of the single-channel extract mask), so that the number of channels of the extract mask is the same as the number of channels of the ginkgo leaf extract image. Subsequently, the foreground image of the extract is generated through mask condition assignment, where the original RGB values ​​of the foreground region are retained, and the background region is marked with the special value NaN.

[0095]

[0096] in, The pixel value of pixel (h,w) in channel c of the foreground image of the extractant. This represents the pixel value (h,w) of the ginkgo leaf extract image in channel c. This represents the pixel value of pixel (h, w) in the extraction liquid mask on channel c, where c is the channel index, h is the height index, w is the width index, and NaN is a null value.

[0097] It should be noted that this extract mask is a precise separation of the extract area and the background area in the ginkgo leaf extract image. Only the pixels with a pixel value of 1 in the extract mask that correspond to the ginkgo leaf extract image are retained, while the pixels with a pixel value of 0 in the extract mask that correspond to the ginkgo leaf extract image are removed. This results in a pure foreground image of the extract, effectively eliminating background interference.

[0098] It should be noted that the null value NaN is used to achieve physical isolation and computational protection between the foreground and background. By uniformly marking non-extraction liquid areas (such as pipe walls and observation windows) as NaN, all subsequent image processing steps can automatically ignore these invalid pixels and only analyze the real extraction liquid areas with a mask value of 1 to provide a pure and uncontaminated data basis for oxidation state analysis.

[0099] Step S3: Perform threshold segmentation on the foreground image of the extract using an adaptive thresholding method to obtain an extract block map; perform boundary optimization on the extract block map using a watershed boundary optimization method to obtain an optimized extract block map.

[0100] The foreground image of the extract is converted from the RGB color space to the HSV color space, and the saturation channel S in the HSV color space of the foreground image of the extract is extracted.

[0101] It should be noted that the core basis for choosing the HSV color space lies in its channel decoupling characteristics and strong correlation with the oxidation characteristics of ginkgo leaves: the three-channel coupling of the RGB space causes light fluctuations to directly interfere with color information, while the HSV space achieves decoupling by separating hue, saturation, and lightness. Among them, the saturation channel S is highly sensitive to the oxidation process of ginkgo leaf extract. The degradation of flavonoids caused by oxidation will cause the liquid color to gradually change from light yellow to dark brown. This process is shown as a non-linear increasing response in the S channel (the mean difference in saturation between the unoxidized and fully oxidized states measured in experiments is 0.67). Moreover, the S channel has inherent robustness to interference such as changes in light intensity and pipe reflections, thus providing a stable and reliable feature input for subsequent adaptive threshold segmentation.

[0102] The saturation channel S in the foreground image of the extract is segmented using an adaptive thresholding method: First, the saturation histogram is calculated.

[0103]

[0104] in, This represents the statistical value of pixel value k in the saturation channel. The saturation of pixel (h, w) in the extract block image, k is a statistical variable value, round() is the rounding function, h is the height index, w is the width index, H is the maximum height, and W is the maximum width.

[0105] Calculate the between-class variance:

[0106]

[0107] in, Here, AQ represents the inter-class variance, AQ represents the proportion of pixels in the stable region, and LQ represents the proportion of pixels in the oxidized region. The average saturation in the stable region, denoted as the average saturation of the oxidation zone, and T is the candidate threshold.

[0108] It should be noted that the pixel ratio of the stable region is , =1- , .

[0109] It should be noted that inter-class variance, as a core evaluation metric for image segmentation quality, characterizes the degree of separation between stable and oxidized regions. Mathematically, this metric is defined as the average saturation of the two types of regions (stable and oxidized) after segmentation (denoted as σ², ... and The weighted sum of squared deviations relative to the average saturation of the foreground image of the extract is used, with the weights being the proportion of pixels in the corresponding region out of the total pixels (AQ for stable regions and LQ for oxidized regions). The magnitude of the inter-class variance directly reflects, at a specific threshold T, the uniformity of saturation features within each of the two regions (intra-class similarity) and the difference in saturation features between the two regions (inter-class separability). Maximizing the inter-class variance involves iterating through all possible thresholds T to find the optimal threshold that most significantly distinguishes the stable and oxidized regions statistically, resulting in the clearest and most defined segmentation boundary. .

[0110] By iterating through the values ​​of T from 1 to 255, the inter-class variance corresponding to each T is calculated. Solve for the optimal threshold The optimal threshold is the value of T that maximizes the inter-class variance.

[0111] Through the optimal threshold The foreground image of the extract is segmented to obtain a block image of the extract:

[0112]

[0113] in, The label for the pixel (h,w) in the extract block image. The saturation of pixel (h,w) in the extract block image. This is the optimal threshold.

[0114] After obtaining the extract block map, the initial binary segmentation results are... Perform morphological erosion to remove pixels contaminated by noise at the edges of the two types of regions (such as liquid surface reflections, microbubbles, etc.) and interference points in the transition zone.

[0115] The boundary optimization of the extract block map is performed using the watershed boundary optimization method: First, the extract block map is... Performing a Euclidean distance transformation yields the distance matrix D:

[0116]

[0117] Where D(h,w) represents the minimum Euclidean distance from pixel (h,w) to pixels in the stable region, h is the height index, and w is the width index. For the pixel height index of the stable region, This is the pixel width index for the stable region.

[0118] It should be noted that the stable region pixels are... =0 pixels.

[0119] Based on the distance matrix D, seed points for the oxidation region are defined as follows:

[0120]

[0121] in, Let D(h,w) represent the seed point in the oxidation region, and let D(h,w) represent the minimum Euclidean distance from pixel (h,w) to pixels in the stable region. This represents the maximum value of the distance matrix D across all pixels.

[0122] Morphological erosion operations are used to remove noise and transition zone pixels at the edges of stable regions, retaining only pixels with an erosion result of 0 (using 3). 3 rectangular kernels, when pixels 3 Other neighboring pixels within a 3-neighbor range include the pixel itself. When all pixels are in the stable region, this pixel The pixels in the stable region whose erosion result is 0 are taken as seed points of the stable region. : .

[0123] Construct a marker matrix based on the seed points of the oxidation region and the seed points of the stable region. :

[0124]

[0125] in, Let h be the value of pixel (h, w) in the marker matrix, where h is the height index and w is the width index.

[0126] It should be noted that the tag matrix The pixels in the stable region retained after erosion are marked as -1, the core pixels in the oxidized region retained after erosion are marked as 1, and all edge pixels that were eroded and non-core pixels in the oxidized region are marked as 1. Then, the unknown regions to be optimized are uniformly assigned (marked as 0), thus ensuring that the watershed algorithm is strictly limited to reconstructing the physical boundary within the transition zone with significant gradients.

[0127] Based on the tag matrix Watershed segmentation is performed using distance gradient:

[0128]

[0129] in, The watershed label for pixel (h,w) () represents the standard watershed algorithm function. This refers to the value corresponding to height h and width w in the marker matrix. This represents the gradient of the distance matrix.

[0130] The labels for the optimized binary image are:

[0131]

[0132] in, To optimize the label of pixel (h,w) in the binary image, The watershed label for pixel (h,w).

[0133] It should be noted that the watershed label ,when When the value is greater than or equal to 1, the label of the pixel (h,w) is an oxide region. When the value is less than 1, the label of the pixel (h,w) is a stable region.

[0134] It should be noted that the watershed boundary optimization method performs boundary optimization on the extract block map generated by adaptive threshold segmentation, and finally outputs the optimized extract block map (i.e., the optimized binary map) as the segmentation label of the oxidized area and the stable area. At the same time, the original saturation channel S is independently retained for subsequent boundary gradient analysis: the optimized binary map is essentially a single-channel binary matrix (where 0 represents the stable area and 1 represents the oxidized area), and its function is only to identify the physical area category to which the pixel belongs; while the saturation channel S (value range [0,1]) in the HSV color space is completely retained in memory as the original color feature, does not participate in the watershed operation and its value remains unchanged.

[0135] It should be noted that the watershed boundary optimization method is used for boundary optimization because inter-class variance, as a global threshold segmentation method, essentially performs hard segmentation of pixels based on a single saturation threshold. This makes it difficult to accurately characterize the continuous gradient changes in the oxidation transition region of the extract image, easily leading to blurred boundary positioning or jagged artifacts. The watershed optimization method transforms the segmentation problem into the analysis of the image gradient topography, simulating the process of water inundation. It can effectively identify and fit the inherent watershed lines in the image, which are naturally formed by saturation or color gradients. This can significantly improve the continuity of the region boundary, thus laying a more reliable foundation for subsequent accurate quantification of the oxidation area.

[0136] Step S4: Calculate the boundary probability of the optimized extract block map using a boundary probability model based on dual-distance variation to obtain the block boundary probability value; perform edge detection on the optimized extract block map using the Sobel operator to obtain the block gradient magnitude; calculate the block boundary confidence by combining the block boundary probability value and the block gradient magnitude; and determine the block boundary of the optimized extract block map based on the block boundary confidence value to obtain the block boundary pixels.

[0137] The boundary probability of the optimized extract block map is calculated using a boundary probability model based on dual-distance variation, yielding the block boundary probability values:

[0138]

[0139] in, Let (h,w) be the block boundary probability value. Let (h,w) be the distance from pixel (h,w) to the nearest oxidation core region. Let (h,w) be the distance from pixel (h,w) to the nearest stable region. This is the scale parameter, which defaults to 2.

[0140] It should be noted that the boundary probability model based on dual-distance variation identifies boundary uncertainty regions by quantifying the spatial distance difference between pixels and the oxidation core region and the stable region: First, the Euclidean distance from each foreground pixel (h,w) in the optimized binary segmentation image to the nearest oxidation core region (defined by the center region of the foreground connected region after watershed optimization), and the distance from that point to the nearest stable region are calculated. Then, an exponential mapping function is used... The absolute value of the distance difference is converted into a block boundary confidence probability value. The closer this value is to 1, the higher the probability that the pixel is located at the boundary between the oxidation region and the stable region.

[0141] Edge detection within a 3×3 neighborhood is performed on the saturation channel of the pixel (h,w) in the optimized extract block image using the Sobel operator to obtain the block gradient magnitude:

[0142] ;

[0143] ;

[0144] ;

[0145] in, Let (h,w) be the horizontal gradient component of pixel (h,w). Let (h,w) be the vertical gradient component of pixel (h,w). Let be the block gradient magnitude of pixel (h, w), where i is the horizontal offset and j is the vertical offset. For the horizontal direction operator of the Sobel operator, For the Sobel operator in the vertical direction, Let be the saturation value of pixel (h,w) with a horizontal offset of i and a vertical offset of j.

[0146] After normalizing the standard deviation of the block gradient magnitude of pixel (h,w) within a 3×3 neighborhood, the block boundary confidence score is calculated by combining it with the block boundary probability value.

[0147]

[0148] in, For the confidence level of the block boundary, Let (h,w) be the block boundary probability value. This represents the block gradient magnitude of pixel (h,w), where k is an adjustment coefficient, with a default value of 5.

[0149] It should be noted that the block boundary confidence is calculated by combining the block boundary probability value and the block gradient magnitude. The block boundary probability value is based on the dual distance transform model, which quantifies the spatial distance difference between the pixel and the oxidation and stable regions (the smaller the distance difference, the higher the probability), reflecting the geometric position probability of the pixel on the boundary. The block gradient magnitude captures the local change intensity of the saturation channel through the Sobel operator, characterizing the visual gradient features of the boundary region. The fusion of the two can complement each other to utilize spatial location information and image gradient information, effectively filtering out noisy pixels with low probability or low gradient, and ensuring that only pixels with high confidence are judged as true boundary pixels.

[0150] The block boundary confidence is used to determine the boundary of the optimized extract block map: a pixel is included in the block boundary pixel set Ω if and only if the pixel is located in the oxidation region of the optimized extract block map and the block boundary confidence is higher than the preset boundary pixel confidence threshold (e.g., 0.7).

[0151] Step S5: By performing pixel gradient change analysis on the boundary pixels of the block, the boundary gradient feature vector is obtained; the boundary gradient feature vectors within a preset time period are collected to obtain the boundary gradient feature vector sequence.

[0152] For each boundary pixel (h, w) The boundary pixel set Ω is used to calculate the boundary normal direction angle of a pixel based on its horizontal and vertical gradient components.

[0153]

[0154] in, Let (h,w) be the horizontal gradient component of pixel (h,w). Let (h,w) be the vertical gradient component of pixel (h,w). The angle of the boundary normal direction of pixel (h,w).

[0155] It should be noted that the angle of the boundary normal direction Pointing to the side with high saturation, this is the unit vector pointing from the stable region to the oxidation region in the extract block diagram to optimize the extraction solution.

[0156] For pixel (h,w) along the normal direction Sample a path on the saturation channel S, with a path length of m (m is greater than or equal to 3 and is an odd number) as the number of sampling points:

[0157]

[0158]

[0159] in, The coordinates of the path points are the row coordinates. Here are the coordinates of the path points, and L is the path sampling index. The angle of the boundary normal direction of pixel (h,w).

[0160] The saturation value of each sampling point is obtained by bilinear interpolation. Calculate the average of the absolute differences in saturation between adjacent sampling points along the path:

[0161]

[0162] in, Let (h,w) be the gradient smoothness of pixel (h,w). Let be the saturation of the (L+1)th sampling point of pixel (h,w). Let be the saturation of the Lth sampling point of pixel (h,w).

[0163] The global gradient smoothness is then the mean of the smoothness at all boundary points:

[0164]

[0165] Where SM represents the global gradient smoothness. The total number of pixels representing the pixel geometry of the block boundary. Let (h,w) be the gradient smoothness of pixel (h,w).

[0166] For each boundary pixel (h, w) Calculate the standard deviation of the saturation values ​​within a 5×5 neighborhood to obtain the local ambiguity:

[0167]

[0168] in, Let (h,w) be the local blur of pixel (h,w). This represents the saturation value at the offset position (i,j) within the neighborhood. The average saturation value for a 5×5 neighborhood.

[0169] The global edge ambiguity is then the average of the ambiguities at all boundary points:

[0170]

[0171] Where BD represents the global edge ambiguity. The total number of pixels representing the pixel geometry of the block boundary. Let (h,w) be the local blur of pixel (h,w).

[0172] For each boundary pixel (h, w) By finding the maximum and minimum saturation values ​​on the path from the saturation values ​​of m sampled path points, the saturation transition speed of pixel (h,w) is calculated:

[0173]

[0174] in, Let (h,w) be the saturation transition speed of pixel (h,w). This represents the maximum saturation value among the sampling path points of pixel (h,w). It represents the minimum saturation value among the sampling path points of pixel (h,w).

[0175] The global saturation transition speed is calculated by using the saturation transition speed of each pixel:

[0176]

[0177] Where TS is the global saturation transition rate. For the set of pixels at the block boundary, The saturation transition rate is the saturation rate at a single boundary point.

[0178] The global edge ambiguity, global gradient smoothness, and global saturation transition speed are used to obtain the boundary gradient feature vector [BD,SM,TS].

[0179] It should be noted that the global edge blur (BD), global gradient smoothness (SM), and global saturation transition speed (TS) are integrated into a boundary gradient feature vector. This aims to construct a comprehensive and complementary feature representation from different key dimensions of the oxidation process. BD is the mean of the local blur of the block boundary pixel set. Local blur is the standard deviation of the saturation values ​​within a 5×5 neighborhood of the boundary pixel. A larger local blur indicates more drastic saturation fluctuations within the boundary pixel's neighborhood (e.g., brightness jumps caused by local reflections, color unevenness caused by microbubbles). A larger BD value indicates more significant environmental interference during the oxidation process, potentially leading to local over-oxidation or uneven oxidation, which is an indirect signal of increased oxidation risk. SM is the mean of the gradient smoothness of the block boundary pixel set, while the local pixel... The gradient smoothness of a point is the average of the saturation differences of the sampling points along the normal direction. The smaller the gradient smoothness of a local pixel, the gentler the change in saturation along the normal direction from the stable region to the oxidation region. A smaller SM value indicates a more uniform transition of the overall oxidation boundary without abrupt changes. TS is the average saturation transition speed in the pixel set of the block boundary. The saturation transition speed of a local pixel is the ratio of the difference between the maximum and minimum saturation on the sampling path to the number of sampling intervals. The larger the saturation transition speed of a local pixel, the faster the change from the "lowest saturation in the stable region" to the "highest saturation in the oxidation region." A larger TS value indicates a faster advancement of the overall oxidation boundary, a more intense oxidation reaction, and a higher risk of the extract exceeding the safe oxidation threshold in a short period of time. Combining these three factors into a boundary gradient feature vector [BD, SM, TS] can capture whether oxidation is disturbed (BD), determine whether the oxidation process is uniform (SM), and quantify the oxidation rate (TS), forming a comprehensive characterization of the oxidation state of the extract.

[0180] Collect boundary gradient feature vectors within a preset time period to obtain a boundary gradient feature vector sequence: During the continuous operation of the industrial production line, images are synchronously acquired and steps S1-S5 are executed with a fixed sampling period to obtain the boundary gradient feature vector [BD,SM,TS] at each sampling time, and finally obtain the boundary gradient feature vector sequence.

[0181] Step S6: Construct an evolution trend model for the extract. Input the boundary gradient feature vector sequence into the evolution trend model for the extract and output an oxidation risk index. Compare the oxidation risk index with a preset threshold. If the index is greater than the preset threshold, issue an early warning, thereby realizing the status monitoring of the Ginkgo biloba extract.

[0182] An evolution trend model of the extract is constructed. This model adopts an encoder-decoder architecture, taking as input a sequence of boundary gradient feature vectors for 10 consecutive time steps (each time step includes three-dimensional features: global edge ambiguity, global gradient smoothness, and global saturation transition speed). The temporal pattern features are extracted by the feature encoder, and then the sequence prediction decoder outputs the three-dimensional feature prediction values ​​for the next 5 time steps.

[0183] The extraction solution evolution trend model includes a time series input layer, an LSTM feature encoder, a GRU sequence prediction decoder, and a risk index calculation module.

[0184] The time-series input layer is configured with a fixed-length feature sequence input, receiving boundary gradient feature vectors from 10 consecutive time steps. Each feature vector contains three dimensions: edge blurring, gradient smoothness, and saturation transition speed. The input layer maintains the time dimension unchanged and outputs a 10×3-dimensional feature tensor, providing a standardized time-series data format for subsequent processing.

[0185] LSTM Feature Encoder: The encoder employs a two-layer stacked LSTM structure. The first layer has 64 LSTM units, processing the 10×3 dimensional input sequence and outputting a 10×64 dimensional sequence state. The second layer has 32 LSTM units, receiving the output of the first layer and generating the 32-dimensional hidden state for the final time step. Both LSTM layers use the tanh activation function and sigmoid gating mechanism, with hidden state dimensions of 64 and 32, respectively. This design effectively extracts long-term dependent features of the oxidation process through two-layer information compression, while controlling the feature dimensions within a reasonable range.

[0186] GRU Sequence Prediction Decoder: The decoder constructs a recursive prediction structure based on gated recurrent units, with the initial state inheriting the 32-dimensional hidden state output by the encoder. The core unit is configured with 32 GRU units, using reset and update gate mechanisms to control the information flow. After each prediction step, the 32-dimensional hidden state is projected into a 3D feature output (weight matrix 32×3, no bias term) through a fully connected layer. The recursive process executes 5 loops, generating the predicted features for time steps t+1 to t+5 sequentially. The ReLU activation function ensures that all predicted values ​​are non-negative, conforming to physical feature constraints.

[0187] Risk Index Calculation Module: This module receives the 5×3D prediction matrix output from the decoder and calculates the oxidation risk index step-by-step.

[0188]

[0189] in, This is the oxidation risk index at time step t+k. Let X be the eigenvalue at the (t+k)th time step. This is the reference value for the Xth feature. denoted as the weight coefficient of the Xth feature, BD as the global edge ambiguity, SM as the global gradient smoothness, and TS as the global saturation transition speed.

[0190] It should be noted that the edge ambiguity weighting coefficient Gradient smoothness weight coefficient Saturation transition speed weighting coefficient The Analytic Hierarchy Process (AHP) can be used to invite five experts on Ginkgo biloba extract to score the importance of the three risk dimensions BD, SM, and TS, thereby determining the weight coefficients of BD, SM, and TS.

[0191] It should be noted that the reference value of the Xth feature... , where BD is the critical value for the safe oxidation of Ginkgo biloba extract corresponding to each feature in the boundary feature vector. For example, when BD > 0.6, the oxidation rate of Ginkgo biloba extract increases sharply, raising the risk. When SM is 0.6; when SM > 0.3, the oxidation process changes from gradual to rapid. When the concentration is 0.3 and TS > 0.15, the degradation rate of the active ingredient is ≥ 2% / h, which exceeds the safety range. It is 0.15.

[0192] It should be noted that the oxidation risk index integrates three features: global edge ambiguity (BD), global gradient smoothness (SM), and global saturation transition speed (TS), based on the actual value of each feature at time step t+k. and corresponding reference value The relationship between the characteristic value and the reference value is analyzed to quantify their contribution to oxidation risk under different circumstances: when the characteristic value does not exceed the reference value, the risk contribution is linearly reflected by the ratio of the actual value to the reference value; when the characteristic value exceeds the reference value, the risk contribution is amplified by the square of the ratio (in a non-linear form), reflecting the characteristic of increased risk after exceeding the limit; and then the weighting coefficients corresponding to each characteristic are used to further quantify the risk contribution. By integrating and summing the contributions of the three factors, the oxidation risk index at time step t+k is finally obtained. This allows for a precise and comprehensive quantification of the oxidation risk level of Ginkgo biloba extract at that time step, providing a scientific quantitative basis for monitoring the oxidation state of the extract and for risk warning.

[0193] First, set the preset threshold. Real-time prediction of risk index for each future time step and The risk levels are determined by cyclical comparison and classification. The lower wind power level corresponds to the oxidation risk index at time step t+k. A value ≤ a preset safety threshold (e.g., 0.3) indicates that the extract is in a stable oxidation state, and it is only necessary to maintain the regular real-time monitoring frequency under the original multi-source imaging environment and continuously collect boundary gradient feature vector sequences; the medium-risk level corresponds to... The value is between the preset safety threshold and the warning threshold (e.g., 0.3 < 0.3). ≤0.7) indicates the beginning of an accumulation of oxidation risk. Immediately check the stability of the full-spectrum adjustable light source parameters, camera white balance, and focus calibration status at the monitoring point. Simultaneously, shorten the monitoring cycle to half of the original cycle and strengthen the tracking of the trend of boundary gradient feature vector sequence changes; high-risk levels correspond to... A warning threshold (e.g., 0.7) indicates that the oxidation reaction in the extract is progressing rapidly and may lead to localized over-oxidation. This necessitates immediately triggering an audible and visual warning and sending a warning message containing exceedance characteristics and risk index deviation values ​​to the operator's terminal. Automatic production control measures should be initiated (e.g., lowering the storage temperature, adjusting the solution pH, and adding antioxidants at a preset dosage if necessary). Simultaneously, subsequent production processes involving the extract should be suspended until... It falls back below the safety threshold and remains stable for one monitoring cycle.

[0194] This technical solution proposes a real-time monitoring method for the state of Ginkgo biloba leaf extract based on image processing. The method constructs a multi-source imaging environment containing specific spectral and angular light sources, and uses a high-resolution industrial camera to capture images of the Ginkgo biloba leaf extract. Subsequently, a well-trained U-net segmentation model is applied to perform pixel-level segmentation of the image, generating an extract mask, and combining it with the original image to obtain a clean foreground image of the extract. An optimized binary segmentation map of the extract is obtained by adaptive thresholding and watershed boundary optimization of the saturation channel of the foreground image. Further, the confidence level of boundary pixels in the binary map is calculated based on a dual-distance transform model, and high-confidence block boundary pixels are selected accordingly. The gradient features (edge ​​blurring, gradient smoothness, and saturation transition speed) of these boundary pixels are calculated to form a time-series feature vector. Finally, these feature sequences are input into an encoder-decoder architecture extract evolution trend model for prediction, outputting an oxidation risk index for future time steps, and achieving state warning by comparing it with a preset threshold.

[0195] The core significance of obtaining a foreground image of the extract by combining the image of the ginkgo leaf extract with the extract mask lies in achieving precise separation between the foreground and background, completely eliminating interference. The mask generated by U-Net segmentation accurately identifies the foreground region (pixel value 1) and background regions such as pipe walls and observation windows (pixel value 0) in the image, which actually contain the extract. After expanding the mask to three channels and aligning it with the original image, a conditional assignment operation is performed (the foreground retains its original RGB values, and the background is marked as NaN). The final extract foreground image contains only the visual information of the target extract region. This step is crucial, as it removes interference signals from non-target regions in the imaging environment, providing a clean, noise-free data foundation for subsequent state analysis based on the color and texture features of the extract region (such as adaptive thresholding segmentation), significantly improving the accuracy and robustness of subsequent analysis.

[0196] By combining the block boundary probability value and the block gradient magnitude, the block boundary confidence is calculated. The block boundary probability value is based on a dual-distance transform model, which quantifies the spatial distance difference between the pixel and the oxidation and stable regions (the smaller the distance difference, the higher the probability), reflecting the geometric position probability of the pixel on the boundary. The block gradient magnitude captures the local change intensity of the saturation channel through the Sobel operator, characterizing the visual gradient features of the boundary region. The fusion of the two can complement each other to utilize spatial location information and image gradient information, effectively filtering out noisy pixels with low probability or low gradient, and ensuring that only pixels with high confidence are judged as true boundary pixels.

[0197] By using the block boundary confidence score to determine the block boundaries of the optimized extract binary segmentation map, the most representative oxidation-stabilization interface pixels can be accurately located for subsequent extraction of key gradient change features. Strict screening is performed to identify the pixels carrying the most significant information about oxidation state changes (color and saturation gradient changes), effectively filtering out low-confidence pixels that are close to the stable region but not at the boundary after watershed optimization.

[0198] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, the phrase "comprising an element defined as..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0199] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for real-time monitoring of the state of Ginkgo biloba leaf extract based on image processing, characterized in that: Includes the following steps: Step S1: Construct a multi-light source imaging environment, place the ginkgo leaf extract in the multi-light source imaging environment for imaging, and obtain an image of the ginkgo leaf extract; Step S2: Construct a U-net segmentation model. Input the image of the ginkgo leaf extract into the U-net segmentation model and output the extract mask. By combining the image of the ginkgo leaf extract and the extract mask, obtain the foreground image of the extract. Step S3: Perform threshold segmentation on the foreground image of the extract using an adaptive thresholding method to obtain an extract block map; perform boundary optimization on the extract block map using a watershed boundary optimization method to obtain an optimized extract block map; Step S4: Calculate the boundary probability of the optimized extract block map using a boundary probability model based on dual-distance variation to obtain the block boundary probability value; perform edge detection on the optimized extract block map using the Sobel operator to obtain the block gradient magnitude; calculate the block boundary confidence by combining the block boundary probability value and the block gradient magnitude; and determine the block boundary of the optimized extract block map based on the block boundary confidence value to obtain the block boundary pixels. Specifically, the boundary probability of the optimized extract block map is calculated using a boundary probability model based on dual-distance variation, resulting in the block boundary probability values: ； in, Let (h,w) be the block boundary probability value. Let (h,w) be the distance from pixel (h,w) to the nearest oxidation core region. Let (h,w) be the distance from pixel (h,w) to the nearest stable region. For scale parameters; Edge detection within a 3×3 neighborhood is performed on the saturation channel of the pixel (h,w) in the optimized extract block image using the Sobel operator to obtain the block gradient magnitude: ； ； ； in, Let (h,w) be the horizontal gradient component of pixel (h,w). Let (h,w) be the vertical gradient component of pixel (h,w). Let be the block gradient magnitude of pixel (h, w), where i is the horizontal offset and j is the vertical offset. For the horizontal direction operator of the Sobel operator, For the Sobel operator in the vertical direction, Let i be the saturation value of pixel (h,w) with a horizontal offset of i and a vertical offset of j. After normalizing the standard deviation of the block gradient magnitude of pixel (h,w) within a 3×3 neighborhood, the block boundary confidence score is calculated by combining it with the block boundary probability value. ； in, For the confidence level of the block boundary, Let (h,w) be the block boundary probability value. Let be the block gradient magnitude of pixel (h,w), and k be the adjustment coefficient; Step S5: By performing pixel gradient change analysis on the boundary pixels of the block, the boundary gradient feature vector is obtained; the boundary gradient feature vectors within a preset time period are collected to obtain a boundary gradient feature vector sequence. Step S6: Construct an evolution trend model for the extract. Input the boundary gradient feature vector sequence into the evolution trend model for the extract and output an oxidation risk index. Compare the oxidation risk index with a preset threshold. If the index is greater than the preset threshold, issue an early warning, thereby realizing the status monitoring of the Ginkgo biloba extract.

2. The method for real-time monitoring of the state of Ginkgo biloba extract based on image processing according to claim 1, characterized in that: The construction of a multi-light source imaging environment, in which the ginkgo leaf extract is placed and photographed to obtain an image of the ginkgo leaf extract, includes the following specific steps: A full-spectrum adjustable light source was set up at the monitoring point of the Ginkgo biloba extract production line to form a uniform and stable composite lighting environment. A camera was installed at the monitoring point, and the camera lens was calibrated so that it was directly facing the imaging area of ​​the monitoring point and the focus and white balance were adjusted. After the light source and camera were turned on, the light source parameters were kept unchanged, and the Ginkgo biloba extract was placed in the imaging area of ​​the monitoring point for shooting, and finally the image of the Ginkgo biloba extract was obtained.

3. The method for real-time monitoring of the state of Ginkgo biloba extract based on image processing according to claim 2, characterized in that: The process of constructing the U-net segmentation model and returning the cloud server parameter set to the communication board includes the following steps: The U-net segmentation model consists of: an input layer, an encoder, a bottleneck layer, a decoder, and an output layer. The encoder includes: downsampling stage 1, downsampling stage 2, downsampling stage 3 and downsampling stage 4; The bottleneck layer consists of two consecutive 3×3 convolutions with the same padding method and a ReLU activation function. The decoder includes: upsampling stage 1, upsampling stage 2, upsampling stage 3, and upsampling stage 4; Output layer: 1×1 convolution with stride of 1, same padding, followed by a sigmoid activation function, and finally outputs an extract mask.

4. The method for real-time monitoring of the state of Ginkgo biloba extract based on image processing according to claim 3, characterized in that: The process of obtaining the foreground image of the extract by combining the image of the ginkgo leaf extract with the extract mask includes the following steps: The foreground image of the extract is obtained by combining the image of the ginkgo leaf extract with the extract mask: ； in, The pixel value of pixel (h,w) in channel c of the foreground image of the extractant. This represents the pixel value (h,w) of the ginkgo leaf extract image in channel c. Let be the pixel value of pixel (h, w) in the extraction liquid mask on channel c, where c is the channel index, h is the height index, and w is the width index. It is an empty value.

5. The method for real-time monitoring of the state of Ginkgo biloba extract based on image processing according to claim 4, characterized in that: The step of performing threshold segmentation on the foreground image of the extract using an adaptive thresholding method to obtain an extract block map includes the following steps: The saturation channel S in the foreground image of the extract is segmented using an adaptive thresholding method: First, the saturation histogram is calculated. ； in, This represents the statistical value of pixel value k in the saturation channel. The saturation of pixel (h, w) in the extract block image, k is a statistical variable value, round() is the rounding function, h is the height index, w is the width index, H is the maximum height, and W is the maximum width. Calculate the between-class variance: ； in, Here, AQ represents the inter-class variance, AQ represents the proportion of pixels in the stable region, and LQ represents the proportion of pixels in the oxidized region. The average saturation in the stable region, The average saturation of the oxidation zone is denoted as T, and the candidate threshold is denoted as T. By iterating through the values ​​of T from 1 to 255, the inter-class variance corresponding to each T is calculated. Solve for the optimal threshold The optimal threshold is the value of T that maximizes the inter-class variance. Through the optimal threshold The foreground image of the extract is segmented to obtain a block image of the extract: ； in, The label for the pixel (h,w) in the extract block image. The saturation of pixel (h,w) in the extract block image. This is the optimal threshold.

6. The method for real-time monitoring of the state of Ginkgo biloba extract based on image processing according to claim 5, characterized in that: The step of optimizing the extract block map using the watershed boundary optimization method to obtain an optimized extract block map includes the following specific steps: The boundary optimization of the extract block map is performed using the watershed boundary optimization method: First, the extract block map is... Performing a Euclidean distance transformation yields the distance matrix D: ； Where D(h,w) represents the minimum Euclidean distance from pixel (h,w) to pixels in the stable region, h is the height index, and w is the width index. For the pixel height index of the stable region, Index of the pixel width of the stable region; Based on the distance matrix D, seed points for the oxidation region are defined as follows: ； in, Let D(h,w) represent the seed point in the oxidation region, and let D(h,w) represent the minimum Euclidean distance from pixel (h,w) to pixels in the stable region. This represents the maximum value of the distance matrix D across all pixels; Define the seed point of the stable region , ; Construct a marker matrix based on the seed points of the oxidation region and the seed points of the stable region. : ； in, Let h be the value of pixel (h, w) in the marker matrix, where h is the height index and w is the width index; Based on the tag matrix Watershed segmentation is performed using distance gradient: ； in, The watershed label for pixel (h,w) () represents the standard watershed algorithm function. This refers to the value corresponding to height h and width w in the marker matrix. Represents the gradient of the distance matrix; The labels for the optimized binary image are: ； in, To optimize the label of pixel (h,w) in the binary image, The watershed label for pixel (h,w).

7. The method for real-time monitoring of the state of Ginkgo biloba extract based on image processing according to claim 6, characterized in that: The construction of the extract evolution trend model, which involves inputting the boundary gradient feature vector sequence into the extract evolution trend model and outputting an oxidation risk index, includes the following specific steps: The extract evolution trend model includes a time series input layer, an LSTM feature encoder, a GRU sequence prediction decoder, and a risk index calculation module; Risk Index Calculation Module: This module receives the 5×3D prediction matrix output from the decoder and calculates the oxidation risk index step-by-step. ； in, This is the oxidation risk index at time step t+k. This is the edge ambiguity weighting coefficient. Let X be the eigenvalue at the (t+k)th time step. This is the reference value for the Xth feature. denoted as the weight coefficient of the Xth feature, BD as the global edge ambiguity, SM as the global gradient smoothness, and TS as the global saturation transition speed.

Images