A method for defect identification of traditional Chinese medicine decoction pieces based on image feature fusion

Abstract

This invention provides a method for identifying defects in traditional Chinese medicine (TCM) decoction pieces based on image feature fusion, belonging to the fields of TCM quality control and machine vision technology. It employs multi-angle image acquisition followed by dual-scale random delay preprocessing; image segmentation is optimized based on texture entropy and gradient variance weighted pre-screening, with single-path aggregation and regularization of defect pixels; microscopic features are extracted using multi-scale convolution; a baseline coordinate system is constructed from the remaining images, generating multiple sets of heterogeneous frames and partitioning and analyzing the true boundaries to construct a dual-distribution map and a panoramic base map; multi-dimensional features are dynamically weighted and fused based on confidence levels, and the input is used to obtain defect results from an improved MobileNetV3 model supporting multi-label output. This invention features adaptive thresholding, single-spectral defect differentiation judgment, and background clutter filtering strategies, significantly reducing missed detections of minor defects and false detections of natural textures, making it suitable for automated quality inspection in multi-category decoction piece production lines.

Description

Technical Field

[0001] This invention relates to the fields of traditional Chinese medicine quality control and machine vision technology, and in particular to a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion. Background Technology

[0002] Traditional Chinese medicine (TCM) decoction pieces are the core material basis for the clinical application of TCM, and their quality directly affects medication safety and efficacy. Defect detection before shipment is a crucial step in quality control. Common defects include five main categories: insect infestation (micropores, insect tunnels), mold (spots, mycelium), edge defects, localized damage, and residue adhering to the herbs. Currently, the industry's quality inspection methods are mainly divided into two categories: manual visual screening and automated machine vision recognition. Manual quality inspection relies on the experience and visual discrimination ability of quality inspectors, which has problems such as low inspection efficiency (only 300-500 pieces can be inspected per person per hour), high missed inspection rate (the missed inspection rate of minor defects reaches 15%-20%), strong subjectivity, and susceptibility to fatigue. It cannot meet the continuous production needs of thousands of pieces per hour in modern production lines.

[0003] Current machine vision inspection generally employs a technical architecture of single-view image acquisition, fixed-weight feature fusion, and convolutional neural network classification, which has three major drawbacks: Feature mining dimension is limited: panoramic images are only captured by a single shooting mode, which can only extract global appearance features and cannot capture micron-level defect information such as micropores, insect infestation, and minor edge defects distributed across frames at the edge of the slices. Fixed weights for feature fusion: Multi-feature fusion is performed using pre-set fixed weights. The weak features corresponding to minor defects are easily covered by the background features of the main body of the medicinal slices, resulting in a high rate of missed detection. False defects cause serious interference: it is impossible to effectively distinguish between the natural texture of medicinal slices (such as epidermal wrinkles and cross-sectional textures) and real defects, with a false detection rate of 8%-12%.

[0004] To address the aforementioned issues, this invention proposes a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion. By employing techniques such as multi-scale feature extraction through splitting and routing, dynamic allocation of fusion weights based on defect confidence, and verification of boundary authenticity using a multi-image combination framework, this method systematically solves the industry pain points of missed detection of minor defects and false detection of defects. Summary of the Invention

[0005] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion. The method extracts deep features from a single image by splitting and decomposing the image, constructs a dual distribution map based on the real defect boundary partitioning of the combined framework to extract two-level boundary defect features in layers, and then dynamically weights and fuses multi-dimensional features based on defect confidence. The method retains defect details at different scales in layers and adaptively increases the weight of micro-defect features, effectively solving the technical problem of missing subtle defects.

[0006] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, comprising: Step 1: Acquire multiple initial images of the same target medicinal slice using various acquisition methods. Preprocess the initial images by randomly delaying and superimposing them with the corresponding reference duration to obtain a standardized image set. Perform rapid defect pre-detection on each standardized image in the standardized image set. Calculate the weighted sum of texture entropy and gradient variance of each image as a defect probability score. Select the single standardized image with the highest defect probability and send it to Step 2. Send the remaining standardized images to Step 3. Step 2: Receive the single standardized image output from Step 1, extract the neighborhood pixel matrix of the high-defect area, and after grouping and normalizing the defect-sensitive pixels within the neighborhood pixel matrix, extract the deep basic feature vector through the defect convolution operator. And then transport it to step 4; Step 3: Select the single image with the highest image quality score from all remaining standardized images to construct the smallest bounding rectangle of the herbal slice outline as the basic pixel frame and reference coordinate system. After registering the remaining images in all remaining standardized images, stack them in layers to generate multiple sets of heterogeneous combined frames. Analyze the real defect boundaries of the combined frames and divide the first and second regions to be analyzed based on the boundary width. Map and label the defect pixels in the two regions with the reference coordinate system to generate the first distribution map and the second distribution map respectively. At the same time, combine the pre-stored boundary pixels of qualified herbal slices in the same batch to obtain a panoramic base map. The first defect features are extracted from the first distribution map and the second distribution map, respectively. Second defect feature Simultaneously, panoramic appearance features are extracted from the panoramic base map. Send to step 4: Step 4: Adaptively generate dynamic weights based on the defect confidence scores corresponding to each feature, and then... , , , Weighted fusion yields the fused feature vector. And send it to step 5; Step 5: Input the pre-trained defect recognition model and simultaneously output the defect classification label and pixel-level defect coordinates of the medicinal slices.

[0007] Preferably, the reference duration includes the reference duration of a single module. Full module baseline duration ; Set two sets of independent random variables with the same value range. , Local random delay and global random delay are generated respectively, and the parameters satisfy the calculation formula: ,in, The image frame rate of the image acquisition device. The single-frame imaging interval; , These are dimensionless calibration coefficients; Preprocessing includes three levels of steps: distortion correction, spatial noise reduction, and ROI cropping of the medicinal slices. The spatial noise reduction module is delayed. The ROI cropping module is delayed when reading the distortion-corrected image. Read the image after spatial domain noise reduction.

[0008] Preferably, the defect-sensitive pixels within the matrix are grouped and normalized, including: All defect-sensitive pixels within the neighborhood pixel matrix are uniformly grouped into the same column of the matrix, and the remaining empty spaces in the neighborhood pixel matrix are filled with irrelevant background pixels. The neighborhood pixel matrix is ​​a square matrix.

[0009] Preferably, a first distribution map and a second distribution map are generated, including: Three images are randomly selected from the remaining standardized images each time, registered, and stacked in a random order to generate eight heterogeneous combined frames. Along the normal direction of the actual defect boundary, measure the distance between the pixel grayscale abrupt changes on both sides, and take the average of 5 consecutive points as the width of this boundary segment, thus constructing a boundary width sequence. Concentrated width Find the mode of sequence W and its maximum width. ; Using the center line of the actual defect boundary as the central axis, extend to both sides of the axis. Define the first region to be analyzed, and expand it on both sides. Define the second region to be analyzed; Calculate the panorama base image stitching scaling factor ,in, The total pixel area of ​​a single preprocessed image. The target canvas pixel area for the panoramic base image; All defect pixels to be labeled are uniformly mapped to the reference coordinate system corresponding to the basic pixel frame. Defect labeling information of all combined frames is collected at the same coordinate position, and valid defect pixels are labeled. Among them, for defect pixels detected only in a single spectral image, a spectral specific marker is added during labeling. A first distribution map is generated by filling the effective defect pixels in the first region to be analyzed, and a second distribution map is generated by filling the effective defect pixels in the second region to be analyzed. The size of the first distribution map and the second distribution map is the same as the size of the basic pixel frame. According to the splicing scaling factor After scaling all the remaining registered images, they are stitched together with pre-stored boundary pixels to obtain a panoramic base map with the same size as the target canvas. During stitching, an individual shape deviation correction algorithm is used to compensate for the normal difference between individual slices and the average boundary.

[0010] Preferred, for , , , Weighted fusion yields the fused feature vector. ,include: ,in, , , , The defect confidence levels are, in order, deep basic features, first defect features, second defect features, and panoramic appearance features; , , , This represents the number of candidate pixels for defects within the corresponding feature domain. , , , This represents the total number of pixels across the entire domain corresponding to the feature. , , , Adaptive dynamic weights for corresponding features; It is the minimum value; For defective pixels with spectrally specific markers, the confidence level of their corresponding features is multiplied by a correction factor of 1.5.

[0011] Preferably, the defect identification model is an improved MobileNetV3 lightweight neural network; The model pre-training dataset contains five categories of manually labeled Chinese herbal medicine defect samples: broken, insect-infested, moldy, missing edges, and dregs attached. The model output simultaneously encapsulates the defect category label and the corresponding defect pixel coordinates. For medicinal slices with multiple defects, the model simultaneously outputs all defect category labels and corresponding pixel-level defect coordinates; When pixel regions of different defects overlap, the contour coordinates are marked and the overlapping areas are labeled according to the defect type. The pixel-level defect coordinates are a list of the minimum bounding rectangle coordinates and contour coordinates of each defect region.

[0012] Preferably, the true defect boundary of the analytical composite frame includes: The combined frame is pre-divided into two independent sub-regions: the effective imaging area of ​​the medicinal slices and the blank background area around the frame, using Otsu threshold segmentation. Initial candidate boundaries are extracted for each of the two independent sub-regions. For the initial candidate boundaries detected in the blank background area, the shortest distance between them and the effective imaging area of ​​the medicinal slices is calculated. Only the boundaries with a distance less than or equal to a preset number of pixels are included in the real boundary candidate set, while background interference boundaries with a distance greater than a preset number of pixels are removed. At the same time, invalid boundaries caused by background noise are screened out from all the initial candidate boundaries extracted from the medicinal slices area. All initial candidate boundaries remaining in the physical region of the medicinal slices are regarded as the first boundary. Along the extension direction of each first boundary, two consecutive closed local neighborhood blocks are intercepted at equal intervals around the boundary line, and the grayscale difference of the pixels inside the local neighborhood blocks is compared. The first boundary, whose grayscale difference between neighboring pixels on both sides is less than the difference threshold, is identified as a false boundary generated by the skin folds and natural texture of the cross-section of the medicinal slices and is directly discarded. The first boundary, whose grayscale difference between neighboring pixels on both sides is greater than or equal to the difference threshold, is marked as the second boundary. Retrieve all remaining heterogeneous combined frames generated after imaging the same batch of medicinal slices, and use a unified reference pixel coordinate system as the mapping reference to map the coordinate points of all the current second boundaries to the corresponding positions of the other combined frames one by one. The frequency of boundary recurrence at the same coordinate point in different combined frames was statistically analyzed. For ordinary visible light defects, the second boundary that is detected simultaneously in 3 or more different combined frames and does not coincide with the average boundary of the pre-stored qualified medicinal slices of the same batch is defined as the true defect boundary. For single-spectral defects with spectral specific markers, the second boundary that is detected simultaneously in 2 or more different combined frames is defined as the true defect boundary. The second boundary that appears sporadically only in a single combined frame is judged as a false boundary caused by registration misalignment or transient imaging disturbance and is removed.

[0013] Preferably, the first region to be analyzed and the second region to be analyzed are divided based on the boundary width, including: The continuous boundary segments are classified into different states along the overall extension of the real defect boundary. The misalignment abrupt change segment with a boundary curvature greater than the preset curvature and the smooth transition segment with a boundary curvature less than or equal to the preset curvature are distinguished. The pixel misalignment diffusion characteristics of different segments are matched to pre-adapt the basic expansion scale. The basic expansion scale of the misalignment abrupt change segment is M times that of the smooth transition segment. For each layer boundary segment, the preset concentrated width and maximum width are dynamically corrected in intervals, taking into account the dense distribution characteristics of local pixel misalignment in the segment. Global coupling constraint calibration is performed on all initial regions formed by segmented expansion. Gaussian smoothing kernel is used to smooth abrupt changes in region edges and unify the global region boundary extension trend, thus forming the first and second regions to be analyzed.

[0014] Compared with the prior art, the beneficial effects of this application are: The rate of missed detection of minute defects is significantly reduced: By extracting deep features from a single image and boundary features from multiple images at multiple scales, and combining dynamic weight fusion, the rate of missed detection of micropores with a diameter ≤0.1mm and minute defects with a width ≤0.2mm that are visible in at least two images is reduced from 15%-20% in the prior art to below 1.2%. The false defect false detection rate has been significantly reduced: through multi-frame boundary verification and natural texture removal algorithms, the false detection rate has been reduced from 8%-12% to below 2.5%; The detection speed meets the requirements of the pipeline: adopting the improved lightweight MobileNetV3 model, the overall detection speed can reach 2500-3000 pieces / hour in the pipeline parallel processing mode, which can be adapted to modern high-speed pipelines. High robustness: random delay preprocessing can effectively resist pipeline vibration interference, multispectral imaging and Otsu adaptive threshold segmentation can effectively resist illumination changes, and multi-group heterogeneous combination frame design can effectively resist registration error and imaging disturbance, thus effectively improving detection stability. Good generalization: Through the threshold adaptive calibration module, it can be adapted to Chinese herbal medicine pieces of different colors, textures and shapes, which solves the problem of poor generalization of fixed threshold.

[0015] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, as described in an embodiment of the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0019] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, such as... Figure 1 As shown, it includes: Step 1: Acquire multiple initial images of the same target medicinal slice using various acquisition methods. Preprocess the initial images by randomly delaying and superimposing them with the corresponding reference duration to obtain a standardized image set. Perform rapid defect pre-detection on each standardized image in the standardized image set. Calculate the weighted sum of texture entropy and gradient variance of each image as a defect probability score. Select the single standardized image with the highest defect probability and send it to Step 2. Send the remaining standardized images to Step 3. In this embodiment, the specific method for rapid defect pre-detection is as follows: For each standardized image, calculate the texture entropy E of its gray-level co-occurrence matrix and the variance D of its Sobel gradient magnitude. To ensure dimensional consistency and weight effectiveness, the maximum-minimum normalized weighted method is used to calculate the defect probability score. The score P ranges from 0 to 1. A higher score indicates a greater likelihood of defects in the image. This pre-detection step only involves arithmetic operations and takes ≤1ms per image, which does not affect the overall detection speed. It should be noted that the texture entropy ranges from 0 to 8, and the variance of the Sobel gradient magnitude ranges from 0 to 65025.

[0020] Image quality scoring criteria: Scoring is conducted from three dimensions: sharpness (Laplacian variance), illumination uniformity (grayscale standard deviation), and contour integrity (contour closure). The image with the highest weighted sum is used as the benchmark image, where sharpness has a weight of 0.5, illumination uniformity has a weight of 0.3, and contour integrity has a weight of 0.2.

[0021] Step 2: Receive the single standardized image output from Step 1, extract the neighborhood pixel matrix of the high-defect area, and after grouping and normalizing the defect-sensitive pixels within the neighborhood pixel matrix, extract the deep basic feature vector through the defect convolution operator. And then transport it to step 4; Step 3: Select the single image with the highest image quality score from all remaining standardized images to construct the smallest bounding rectangle of the herbal slice outline as the basic pixel frame and reference coordinate system. After registering the remaining images in all remaining standardized images, stack them in layers to generate multiple sets of heterogeneous combined frames. Analyze the real defect boundaries of the combined frames and divide the first and second regions to be analyzed based on the boundary width. Map and label the defect pixels in the two regions with the reference coordinate system to generate the first distribution map and the second distribution map respectively. At the same time, combine the pre-stored boundary pixels of qualified herbal slices in the same batch to obtain a panoramic base map. The first defect features are extracted from the first distribution map and the second distribution map, respectively. Second defect feature Simultaneously, panoramic appearance features are extracted from the panoramic base map. Send to step 4: Step 4: Adaptively generate dynamic weights based on the defect confidence scores corresponding to each feature, and then... , , , Weighted fusion yields the fused feature vector. And send it to step 5; Step 5: Input the pre-trained defect recognition model and simultaneously output the defect classification label and pixel-level defect coordinates of the medicinal slices.

[0022] Multiple acquisition methods: A three-dimensional acquisition scheme combining multi-angle, multi-spectral, and multi-focal lengths is employed, specifically: Angle dimensions: three fixed angles: front (0°), 45° side, and 90° cross-section, covering all visible surfaces of the medicinal slices; Spectral dimensions: Visible light (400-760nm) is used to detect appearance defects, and near-infrared light (760-1000nm) is used to detect shallow mold and insect infestation; Focal length dimension: Standard focal length (16mm) was used to acquire panoramic images, and macro focal length (50mm) was used to acquire magnified images of areas with high defect incidence. A total of 6 initial images (3 angles × 2 spectra) were acquired for the same target slice, with an acquisition frame rate of [missing information]. .

[0023] In this embodiment, the specific steps of threshold adaptive calibration are as follows: Before testing each batch of medicinal slices, 10 standardized images of qualified medicinal slices from the same batch are randomly collected, and the average gray value of the batch is calculated. Average texture entropy Maximum grayscale difference in natural texture Based on this, all judgment thresholds are dynamically adjusted: Defect-sensitive pixel grayscale difference threshold: ; Defect-sensitive pixel gradient threshold: ; False boundary gray-level difference threshold: .

[0024] In this embodiment, the high-incidence area of ​​defects is determined based on the statistical distribution of defects in traditional Chinese medicine decoction pieces. The high-incidence area of ​​defects is as follows: Edge region: A ring-shaped area 5-10 pixels away from the outline of the medicinal slice; Cross-sectional area: the middle 1 / 3 region in the thickness direction of the medicinal slices; Texture-dense regions: texture entropy greater than a threshold The region is defined as follows: A 5×5 square neighborhood pixel matrix of the region is selected as the analysis unit.

[0025] In this embodiment, the defect-sensitive pixel determination criterion is: pixels that simultaneously meet the following three conditions: The difference between the gray value and the average gray value of the neighborhood is greater than (8-bit grayscale image); Gradient magnitude greater than ; Local texture entropy greater than .

[0026] The clustering and normalization operation gathers all defect-sensitive pixels in the 5×5 neighborhood matrix into the 3rd column (middle column) of the matrix in row-major order. The remaining empty spaces are filled by the average background pixels in the matrix. This concentrates the scattered small defect pixels, which can significantly enhance the response of subsequent convolution operators to weak defect features and improve the extraction accuracy of deep features.

[0027] In this embodiment, the defect convolution operator includes composite convolution kernels of three sizes: 3×3, 5×5, and 7×7, corresponding to defect scales of 0.03-0.1mm, 0.1-0.5mm, and 0.5-2mm, respectively. The 3×3 convolution kernel employs a custom composite defect convolution operator, combining the advantages of Sobel edge detection and Gabor texture detection. Specific values ​​are as follows: This operator can simultaneously enhance edge mutations and texture anomalies. Its response sensitivity to minute defects such as micropores, insect damage, and minor edge defects is 2.3 times that of ordinary convolution operators. The 5×5 and 7×7 convolution kernels adopt the same design principle, with center weights of 28 and 52, respectively, to adapt to larger-scale defects. For sub-pixel level defects with a diameter <0.03mm, the sub-pixel edge detection algorithm (Zernike moment method) is used to extract the sub-pixel coordinates of the defects. The confidence of the defect candidate pixels is multiplied by a correction factor of 2 to ensure that they obtain sufficient weight in dynamic weight fusion.

[0028] Sensitivity comparison experiment of defective convolution operators: Comparison objects: ordinary 3×3 Sobel convolution operator, ordinary 3×3 Laplacian convolution operator, and the 3×3 composite defective convolution operator of the present invention; Test set: Contains 1000 microporous insect-bored samples with diameters of 0.05-0.1 mm; Evaluation metric: Defect characteristic response intensity (maximum value of operator output); Experimental results: The average response intensity of the operator of this invention is 128.5, which is 2.3 times that of the Sobel operator (55.9) and 1.8 times that of the Laplacian operator (72.1), proving that its response sensitivity to small defects is significantly improved.

[0029] Test samples: 100 micro-hole insect-bored specimens each with diameters of 0.05 mm, 0.08 mm, and 0.1 mm. The experimental results are shown in Table 1. Table 1 Experimental Results Experiments show that even for microporous insect infestations with a diameter of 0.05 mm, the dynamic weight can reach over 0.58, ensuring that the defect features dominate after fusion and effectively avoiding missed detection.

[0030] In this embodiment, the basic pixel framework and reference coordinate system are constructed as follows: Select the image with the highest quality score from the remaining 5 standardized images as the baseline image; Extract the minimum bounding rectangle of the herbal slice outline in the reference image as the basic pixel frame, and the frame size is consistent with the size of the herbal slice area in the reference image; Establish a pixel reference coordinate system with the top left corner of the rectangle as the origin, the horizontal x-axis pointing to the right, and the vertical y-axis pointing downwards.

[0031] In this embodiment, multiple heterogeneous combined frameworks are generated: Three images are randomly selected from the remaining four images each time, and the improved ORB algorithm is used for image registration (feature point extraction → brute force matching → RANSAC to remove mismatches → homography matrix calculation → image transformation). The improvements to the ORB algorithm are as follows: Based on the conventional ORB algorithm, grayscale gradient features and edge features are added as auxiliary matching features. For low-texture medicinal slices (such as Poria cocos and Dioscorea opposita), the matching weight of edge features is increased to 0.7 to ensure that the registration error is ≤1 pixel.

[0032] The registered images are stacked in a random order to generate 8 heterogeneous combined frames. It should be noted that multiple random combinations can reduce the impact of a single registration error on defect detection and improve the robustness of boundary analysis.

[0033] The first defect feature is the multi-dimensional edge-texture fusion feature of the core region of the true defect boundary, specifically including: Edge gradient features: The horizontal gradient of each pixel in the first region to be analyzed is calculated using the Sobel operator. and vertical gradient Generate gradient magnitude map and gradient direction pattern The gradient magnitude histogram (divided into 8 directions and 16 magnitude intervals) yields 128-dimensional edge gradient features; Local Binary Pattern (LBP) Texture Features: The texture features of the first region to be analyzed are extracted using a uniform LBP operator with a 3×3 neighborhood, generating a 256-dimensional LBP histogram feature. Gray-scale statistical features: Six statistical measures—mean, variance, skewness, kurtosis, energy, and entropy—are calculated for pixels within the first region to be analyzed, yielding six-dimensional gray-scale statistical features. These features are then concatenated to obtain a 390-dimensional first defect feature vector. .

[0034] The second defect feature is the global shape-distribution feature of the actual defect boundary extension region, specifically including: Defect contour shape features: Extract the minimum bounding rectangle, minimum bounding circle, and convex hull contour of all valid defect pixels in the second region to be analyzed, and calculate seven shape descriptors of the contour: perimeter, area, aspect ratio, roundness, rectangularity, and convexity, to obtain seven-dimensional shape features. Defect spatial distribution characteristics: The second region to be analyzed is divided into a 4×4 grid, and the proportion of defect pixels in each grid is counted to generate 16-dimensional spatial distribution characteristics; Gray-level co-occurrence matrix (GLCM) features: Calculate the gray-level co-occurrence matrix in the four directions (0°, 45°, 90°, and 135°) within the second region to be analyzed. Extract four statistics: contrast, correlation, energy, and homogeneity, to obtain 16-dimensional GLCM features. Concatenate these features to obtain a 39-dimensional second defect feature vector. .

[0035] The overall appearance characteristics refer to the macroscopic morphology and color characteristics of the medicinal slices, specifically including: Overall shape features: Seven shape descriptors, namely area, perimeter, aspect ratio, roundness, rectangularity, convexity, and Euler number, are extracted from the outline of the medicinal slices in the panoramic base image to obtain seven-dimensional shape features; Color features: Convert the panoramic base image to the HSV color space, and calculate the histograms of the H, S, and V channels (each divided into 16 intervals) to obtain 48-dimensional color histogram features; Overall texture features: Four texture features—contrast, roughness, orientation, and linearity—are extracted from the panoramic base image using the gray-level difference statistical method, resulting in a 4-dimensional overall texture feature. These features are then concatenated to obtain a 59-dimensional panoramic appearance feature vector. .

[0036] In this embodiment, the individual shape deviation correction algorithm is as follows: when stitching the panoramic base image, the deviation value between the outline of a single slice and the pre-stored average boundary is calculated. For areas with a deviation ≤ 10 pixels, they are determined to be normal individual differences and are not marked as defects; for areas with a deviation > 10 pixels, they are determined to be possible edge defects and are included in subsequent analysis.

[0037] In this embodiment, the process of analyzing the true defect boundary of the composite frame includes: extracting initial candidate boundaries by region, eliminating false boundaries generated by natural textures, and verifying the true defect boundary by the recurrence frequency of multi-frame boundaries.

[0038] In this embodiment, the true defect boundary refers to the non-natural boundary caused by defects such as breakage, insect infestation, mold, and missing edges of the medicinal slices, which is different from the inherent contour and natural boundaries such as skin wrinkles and cross-sectional textures of the medicinal slices.

[0039] The beneficial effects of the above technical solution are as follows: by replacing random image selection with defect probability pre-screening, the problem of micro-defects being missed due to not being sent to the deep feature channel is avoided. At the same time, a multi-scale defect convolution operator is used to cover defects of the full scale from 0.03 to 2 mm, and the reference image with the best image quality is selected to eliminate the accumulation of registration error. Combined with a multi-dimensional feature diversion extraction and dynamic weight fusion architecture, the synchronous and accurate detection of micron-level defects and macro-level defects is achieved. The overall detection robustness and accuracy are effectively improved compared with the existing technology.

[0040] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, wherein the reference time includes the reference time for a single module. Full module baseline duration ; Set two sets of independent random variables with the same value range. , Local random delay and global random delay are generated respectively, and the parameters satisfy the calculation formula: ,in, The image frame rate of the image acquisition device. The single-frame imaging interval; , These are dimensionless calibration coefficients; Preprocessing includes three levels of steps: distortion correction, spatial noise reduction, and ROI cropping of the medicinal slices. The spatial noise reduction module is delayed. The ROI cropping module is delayed when reading the distortion-corrected image. Read the image after spatial domain noise reduction.

[0041] In this embodiment, The value is 33.3ms. The value is 0.5. The value is 1.5.

[0042] Pre-treatment three-stage process: Distortion correction: The Zhang Zhengyou calibration method was used to correct camera lens distortion; Spatial domain noise reduction: Delay the reading of the corrected image by Tran_glo and use 3×3 median filtering to eliminate salt and pepper noise; ROI cropping: Delay the reading of the denoised image by Tran_loc, extract the effective area of ​​the medicinal slices based on Otsu threshold segmentation, and crop out the background pixels.

[0043] Random delays can break the periodic synchronization noise caused by the vibration of the conveyor belt in the production line. Different modules use different delay scales. Global delay ensures that the noise reduction module can obtain noise information from multiple frames, while local delay reduces the calculation delay of ROI pruning.

[0044] In this embodiment, the time base for delayed reading in this invention is the completion time of the previous module, that is: The spatial noise reduction module delays the output of the corrected image and triggers the completion signal from the distortion correction module. The image is then read again for noise reduction; the ROI cropping module delays after the spatial noise reduction module outputs the denoised image and triggers the completion signal. The image is then read and cropped. This time base design avoids the impact of pipeline transmission delays and module processing speed fluctuations on delay accuracy, ensuring the effectiveness of random delays.

[0045] In this embodiment, This is the local delay coefficient, used to control the base delay of the ROI trimming module; This is the global delay coefficient, used to control the basic delay of the spatial noise reduction module. Both are calibrated using a combination of orthogonal experimental method and vibration spectrum analysis. The specific steps are as follows: The vibration acceleration signal during normal operation of the production line is collected, and the vibration spectrum is obtained by performing Fourier transform to determine the main vibration frequency fv and its period Tv=1 / fv; Design a three-factor, three-level orthogonal experiment. The factor levels are shown in Table 2. Table 2 Orthogonal Experiment Table Nine sets of experiments were conducted, using the signal-to-noise ratio (SNR) and ROI cropping accuracy of the preprocessed images as evaluation metrics. Experimental combinations with SNR≥35dB and ROI clipping accuracy≥99.5% were selected. If multiple combinations exist, the combination with the smallest computational delay was selected as the final calibration value.

[0046] In this embodiment, the main vibration frequency of the production line is 5Hz, and the period Tv = 200ms. The optimal value is obtained through the above calibration method. =0.5, =1.5, random delay interval is [0.2] 0.5 ].

[0047] It should be noted that before testing each batch of medicinal slices, an adaptive threshold calibration is performed, dynamically adjusting all judgment thresholds based on the statistical characteristics of 10 qualified medicinal slices from the same batch.

[0048] The beneficial effects of the above technical solution are: by using global and local dual-scale random delay preprocessing, the periodic synchronous noise of pipeline vibration and imaging is effectively broken, which improves the signal-to-noise ratio of the preprocessed image. At the same time, the newly added batch threshold adaptive calibration module can automatically adjust all judgment thresholds based on the statistical characteristics of qualified medicinal slices in the same batch, without the need for manual parameter adjustment for each product category, thus solving the problem of generalization that fixed thresholds cannot adapt to medicinal slices of different colors and textures.

[0049] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, which involves grouping and regularizing defect-sensitive pixels within a matrix, including: All defect-sensitive pixels within the neighborhood pixel matrix are uniformly grouped into the same column of the matrix, and the remaining empty spaces in the neighborhood pixel matrix are filled with irrelevant background pixels. The neighborhood pixel matrix is ​​a square matrix.

[0050] The beneficial effects of the above scheme are: it significantly enhances the detectability of subpixel-level micro-defects and reduces the false negative rate of micro-defects from the feature extraction level.

[0051] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, which generates a first distribution map and a second distribution map, including: Three images are randomly selected from the remaining standardized images each time, registered, and stacked in a random order to generate eight heterogeneous combined frames. Along the normal direction of the actual defect boundary, measure the distance between the pixel grayscale abrupt changes on both sides, and take the average of 5 consecutive points as the width of this boundary segment, thus constructing a boundary width sequence. Concentrated width Find the mode of sequence W and its maximum width. ; Using the center line of the actual defect boundary as the central axis, extend to both sides of the axis. Define the first region to be analyzed, and expand it on both sides. Define the second region to be analyzed; Calculate the panorama base image stitching scaling factor ,in, The total pixel area of ​​a single preprocessed image. The target canvas pixel area for the panoramic base image; All defect pixels to be labeled are uniformly mapped to the reference coordinate system corresponding to the basic pixel frame. The defect labeling information of all combined frames is collected at the same coordinate position, and the valid defect pixels are labeled. Among them, for defect pixels detected only in a single spectral image, a spectral specific marker is added during labeling. A first distribution map is generated by filling the effective defect pixels in the first region to be analyzed, and a second distribution map is generated by filling the effective defect pixels in the second region to be analyzed. The size of the first distribution map and the second distribution map is the same as the size of the basic pixel frame. According to the splicing scaling factor After scaling all the remaining registered images, they are stitched together with pre-stored boundary pixels to obtain a panoramic base image with the same size as the target canvas. During the stitching process, an individual shape deviation correction algorithm is used to compensate for the normal difference between individual slices and the average boundary.

[0052] In this invention, three images are randomly selected, registered, and then stacked in a random order to generate eight sets of heterogeneous composite frames. The design basis for this is as follows: Reasonableness of the number of combinations: In step 1, 6 initial images are acquired. One image is randomly selected and sent to step 2, leaving 5 images. One of these is then randomly selected as the baseline image to construct the basic pixel framework. Finally, 4 images remain to generate the combined framework. The number of combinations of selecting 3 images from these 4 is... There are types, and each combination has There are 24 possible heterogeneous combinations with different stacking orders. Selecting 8 groups is the optimal balance between detection accuracy and computational complexity: when the number of combinations is ≥8, the boundary detection accuracy is improved by less than 1%, but the computation time increases by more than 30%; when the number of combinations is <8, the boundary detection accuracy decreases by more than 5%.

[0053] Heterogeneity refers to the two-dimensional differences between different combined frames: first, the difference in image composition (the three selected images are different); and second, the difference in stacking order (the layering order of the same three images is different). This heterogeneity can effectively reduce the impact of single registration errors and imaging noise on boundary analysis, thereby improving robustness.

[0054] In this embodiment, the specific steps for measuring the boundary width along the normal direction of the actual defect boundary are as follows: The actual defect boundary is refined at the sub-pixel level to obtain the boundary center line with a single pixel width; Select n sampling points at equal intervals along the boundary centerline (sampling interval is 1 pixel), and draw the boundary normal at each sampling point; Traverse the pixels from the sampling point outwards along the normal direction, recording the location of the first abrupt change in grayscale value (points where the grayscale value change is ≥20), and denote them as the left abrupt change points. and the mutation point on the right ; calculate and The Euclidean distance between them is used as the boundary width at that sampling point. ; The average width of the boundary at 5 consecutive sampling points is taken as the width of the boundary segment, and a boundary width sequence is constructed. .

[0055] In this embodiment, the panoramic base map is generated using a feature point-based multi-image fusion and stitching algorithm. The specific steps are as follows: According to the splicing scaling factor The remaining registered images are scaled to make their resolution match that of the target canvas. SIFT feature points are extracted from each scaled image. Brute-force matching is used to match feature points. False matching points are removed using the RANSAC algorithm. The homography matrix between images is then calculated. Transform all images to the reference coordinate system based on the homography matrix to generate the initial stitched image; A weighted average fusion method is used to process overlapping areas of the image, and the fusion weight is inversely proportional to the distance of the pixel from the image center. The average boundary pixels of qualified medicinal slices from the same batch are superimposed onto the corresponding positions of the initial stitched image to fill in the incomplete boundary areas caused by the lack of imaging angle, and finally generate a panoramic base map.

[0056] In this embodiment, a spectrally specific marker is added to a defect pixel during the defect pixel mapping and labeling stage when the defect pixel is detected only in a single visible or near-infrared spectral image and has no corresponding defect signal in all other spectral images. This primarily targets: near-infrared visible defects such as shallow mold, internal insect infestation, and internal cracks in medicinal slices; and visible defects such as minor surface scratches and light-colored impurities attached to medicinal residue. For example, in Astragalus membranaceus slices, where only near-infrared images show insect holes and visible light shows no abnormalities, the pixel is marked with a spectrally specific marker. A true defect is determined if the boundary meets the detection criteria of two sets of frames, with a confidence level of ×1.5. The beneficial effects of the above technical solution are: by adding spectral-specific markers to distinguish between ordinary defects and shallow defects that are only visible in a single spectrum, and by using an individual shape deviation correction algorithm to compensate for the normal differences between individual slices and the average boundary, it not only solves the problem of missed detection of shallow mold and internal insect infestation that can only be detected by near-infrared spectroscopy, but also avoids false detection of missing edges caused by individual shape differences in irregular slices such as angelica and astragalus, thus improving the accuracy of panoramic base map stitching.

[0057] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion. , , , Weighted fusion yields the fused feature vector. ,include: ,in, , , , The defect confidence levels are, in order, deep basic features, first defect features, second defect features, and panoramic appearance features; , , , This represents the number of candidate pixels for defects within the corresponding feature domain. , , , This represents the total number of pixels across the entire domain corresponding to the feature. , , , Adaptive dynamic weights for corresponding features; It is a local minimum.

[0058] For defective pixels with spectrally specific markers, the confidence level of their corresponding features is multiplied by a correction factor of 1.5.

[0059] In this embodiment, the dynamic weights are positively correlated with the defect confidence level. When a minor defect exists within a certain feature domain, its confidence level automatically increases, and the corresponding weight also increases, thereby preventing minor defect features from being submerged by global background features. For example, the confidence level of micropore insect infestation in deep basic features can reach 0.8, and its weight will automatically increase to above 0.6, ensuring that defect features dominate after fusion.

[0060] In this embodiment, a defect candidate pixel refers to a pixel that satisfies any of the following conditions: Defect-sensitive pixels identified in step 2 (grayscale difference ≥ 20, gradient magnitude ≥ 15, local texture entropy ≥ 4.5); In step 3, pixels marked as defects in 3 or more combined frames; Pixels in the panoramic base image whose boundary deviation from the average boundary of qualified medicinal slices is ≥3 pixels.

[0061] In this embodiment, This represents the total number of pixels in the neighborhood pixel matrix of all high-defect areas captured. This represents the total number of pixels in the first region to be analyzed. This represents the total number of pixels in the second region to be analyzed. This represents the total number of pixels in the actual area of ​​the medicinal slices in the panoramic base image.

[0062] The beneficial effects of the above technical solution are: applying a 1.5 times confidence correction coefficient to defect pixels with spectrally specific markers can automatically increase the fusion weight of weak single-spectrum defects, ensuring that they are not submerged by global background features, and reducing the missed detection rate of 0.1mm internal insect infestation visible only in near-infrared images from 15% to below 2%.

[0063] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, wherein the defect identification model is an improved MobileNetV3 lightweight neural network; The model pre-training dataset contains five categories of manually labeled Chinese herbal medicine defect samples: broken, insect-infested, moldy, missing edges, and dregs attached. For herbs with multiple defects, all defect category labels and corresponding pixel-level defect coordinates are output simultaneously. When pixel regions of different defects overlap, the contour coordinates are labeled according to the defect type and the overlapping area is marked. The pixel-level defect coordinates are a list of the minimum bounding rectangle coordinates and contour coordinates of each defect region.

[0064] In this embodiment, the present invention employs an improved MobileNetV3-Small lightweight neural network, as follows: CBAM attention mechanism insertion: A CBAM attention module is inserted after the depthwise convolution (DWConv) and before the pointwise convolution (PWConv) in each bottleneck layer. The CBAM module contains a channel attention submodule and a spatial attention submodule. The channel attention submodule adopts a parallel structure of average pooling + max pooling, and the spatial attention submodule adopts a 7×7 convolution kernel.

[0065] Dual output head design: Classification Header: Two fully connected layers are connected after the global average pooling layer. The first layer has 128 neurons and the activation function is ReLU; the second layer has 6 neurons (corresponding to defects of classes 0-5) and the activation function is Softmax, which outputs the defect classification probability.

[0066] Segmentation Head: The output feature map of the last bottleneck layer is upsampled to 1 / 4 of the input image size, and then gradually upsampled to the input image size through 3 transposed convolutional layers. Finally, a 1×1 convolutional layer outputs a 6-channel pixel-level defect mask with the activation function Sigmoid.

[0067] Loss function optimization: A combined loss function of cross-entropy loss (classification) and Dice loss (segmentation) is adopted, with a total loss of... Among them, the Dice loss is specifically used to solve the class imbalance problem in small object segmentation, and its calculation formula is: ,in, This represents the pixel defect probability predicted by the model. For pixel-based true labels, , This represents the total number of pixels.

[0068] It should be noted that when the model detects multiple defect categories, it outputs the classification probability and pixel mask for each category respectively; for pixel overlapping regions, it retains the labels of all defect categories and marks the coordinate range of the overlapping region and the defect type involved in the output results.

[0069] The model pre-training dataset of this invention contains 100,000 images of Chinese herbal medicine slices, each 256×256 pixels, covering 20 common types of slices such as ginseng, astragalus, angelica, licorice, and isatis root. The labeling standards for various defects are as follows: Damage: Cracks, breaks, dents, and other damage appearing on the surface of the medicinal slices. The minimum size for marking this is 2×2 pixels. Insect infestation: Insect holes or tunnels on or inside the surface of medicinal slices, with a minimum dimension of 1×1 pixel (single micropore). Mold: Mold spots and mycelia on the surface of medicinal slices, with a minimum labeling size of 3×3 pixels; Missing edge: The missing part of the edge of the medicinal slice, with a minimum dimension of width ≥ 1 pixel and length ≥ 5 pixels; Medicinal dregs: Other medicinal dregs or impurities adhering to the surface of the medicinal slices, with a minimum labeling size of 2×2 pixels.

[0070] The annotation employed pixel-level semantic segmentation using the LabelMe annotation tool, and was cross-annotated by three engineers with over five years of experience in traditional Chinese medicine quality inspection. For controversial annotations, a fourth senior quality inspector made the final decision, ensuring annotation consistency ≥95%. The dataset was divided into training, validation, and test sets in a 7:2:1 ratio.

[0071] The model inference speed test of this invention was conducted on the following hardware platform: CPU: Intel Corei7-12700K@3.6GHz; GPU: NVIDIA RTX 3060 12GB; Memory: 32GB DDR4 @ 3200MHz; Deployment framework: TensorRT 8.4.1.5 (INT8 quantization); A three-stage pipelined parallel processing approach is adopted. The first stage is responsible for image acquisition and preprocessing, the second stage is responsible for multi-scale feature extraction and fusion, and the third stage is responsible for model inference and result output. Data is transferred between stages through a circular buffer to achieve continuous detection. Test results show that, in pipelined parallel processing mode, the overall detection speed can reach 2800 pieces / hour, which is suitable for the detection needs of modern high-speed pipelines.

[0072] The beneficial effects of the above technical solution are: by improving the multi-label classification output head and overlapping defect annotation mechanism of the MobileNetV3 model, multiple mixed defects existing on the same piece of medicine can be identified at the same time, and the contour coordinates of the overlapping areas of different defects can be marked separately, which solves the pain point that the existing technology can only output a single defect category and cannot handle mixed defects.

[0073] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, which analyzes the true defect boundaries of a combined framework, including: The combined frame is pre-divided into two independent sub-regions: the effective imaging area of ​​the medicinal slices and the blank background area around the frame, using Otsu threshold segmentation. Initial candidate boundaries are extracted for each of the two independent sub-regions. For the initial candidate boundaries detected in the blank background area, the shortest distance between them and the effective imaging area of ​​the medicinal slices is calculated. Only the boundaries with a distance less than or equal to a preset number of pixels are included in the real boundary candidate set, while background interference boundaries with a distance greater than a preset number of pixels are removed. At the same time, invalid boundaries caused by background noise are screened out from all the initial candidate boundaries extracted from the medicinal slices area. All initial candidate boundaries remaining in the physical region of the medicinal slices are regarded as the first boundary. Along the extension direction of each first boundary, two consecutive closed local neighborhood blocks are intercepted at equal intervals around the boundary line, and the grayscale difference of the pixels inside the local neighborhood blocks is compared. The first boundary, whose grayscale difference between neighboring pixels on both sides is less than the difference threshold, is identified as a false boundary generated by the skin folds and natural texture of the cross-section of the medicinal slices and is directly discarded. The first boundary, whose grayscale difference between neighboring pixels on both sides is greater than or equal to the difference threshold, is marked as the second boundary. Retrieve all remaining heterogeneous combined frames generated after imaging the same batch of medicinal slices, and use a unified reference pixel coordinate system as the mapping reference to map the coordinate points of all the current second boundaries to the corresponding positions of the other combined frames one by one. The frequency of boundary recurrence at the same coordinate point in different combined frames was statistically analyzed. For ordinary visible light defects, the second boundary that is detected simultaneously in 3 or more different combined frames and does not coincide with the average boundary of the pre-stored qualified medicinal slices of the same batch is defined as the true defect boundary. For single-spectral defects with spectral specific markers, the second boundary that is detected simultaneously in 2 or more different combined frames is defined as the true defect boundary. The second boundary that appears sporadically only in a single combined frame is judged as a false boundary caused by registration misalignment or transient imaging disturbance and is removed.

[0074] In this embodiment, the true defect boundary is analyzed as follows: Region segmentation and initial boundary extraction: Otsu threshold segmentation is used to divide the combined frame into the herbal slice entity region and the background region; the boundaries detected in the background region are verified by distance, and only the boundaries connected to the herbal slice entity are retained. The boundaries extracted from the herbal slice entity region need to be further filtered.

[0075] False boundary removal: Along the extension direction of each initial boundary, extract two 3×3 local neighborhood blocks at equal intervals around the boundary line; if the gray-level difference of the neighboring pixels on both sides of the boundary is less than 1, the false boundary removal method is used. (Smooth gradient) is judged as a false boundary caused by the natural texture (wrinkles, cross-sectional texture) of the medicinal slices and discarded; if the grayscale difference is greater than or equal to (Fault-like abrupt change), marked as the second boundary, where, The difference threshold is 30 pixels.

[0076] Multi-frame verification of true defect boundaries: Map all second boundaries to the reference coordinate system of the remaining 7 sets of combined frames; count the frequency of boundary recurrence at the same coordinate point. Ordinary defects need to be reproduced in 3 or more frames, and single-spectral defects need to be reproduced in 2 or more frames. Second boundaries that do not coincide with the pre-stored qualified slice boundaries are defined as true defect boundaries; boundaries that appear only in a single combined frame are judged as false boundaries caused by registration misalignment or transient imaging disturbance and are removed.

[0077] The pre-stored boundary pixels are the average boundary pixels of 1000 qualified slices of medicinal herbs from the same batch, which are collected in advance and stored in the local database to distinguish between the inherent boundaries and defective boundaries of the medicinal herbs.

[0078] In this invention, the grayscale difference threshold used to distinguish between false boundaries and real defect boundaries is calibrated using the ROC curve method. The specific steps are as follows: Construct a boundary verification set containing 1000 samples, of which 500 are false boundaries generated by natural textures and 500 are real defect boundaries; Using integers between 10 and 50 as gray-scale difference thresholds, the true positive rate (TPR) and false positive rate (FPR) at each threshold were calculated. Plot the ROC curve, calculate the area under the curve (AUC), and select the threshold with the largest Youden Index as the optimal difference threshold. The Youden Index is calculated as: J = TPR - FPR.

[0079] In this embodiment, the optimal difference threshold for the 8-bit grayscale image obtained by the above method is 30, corresponding to a TPR of 98.2% and an FPR of 1.5%.

[0080] In this invention, the threshold criteria for determining the boundaries detected simultaneously in three or more different combined frames as real defect edges are as follows: Statistical analysis shows that the probability of recurrence of false boundaries caused by registration misalignment and transient imaging perturbation in different combined frames is ≤10%, and the probability of recurrence in 3 or more combined frames is ≤0.1%. The probability of recurrence of a true defect boundary in different combined frames is ≥90%, and the probability of recurrence in 3 or more combined frames is ≥99.9%. Therefore, selecting 3 sets as the recurrence frequency threshold can control the false detection rate of the false boundary to below 0.1% while ensuring an extremely low false detection rate. For single-spectral defects, since they only appear in specific spectral images, adjusting the recurrence frequency threshold to 2 sets can reduce the false detection rate from 15% to below 2%.

[0081] The average boundary of the pre-stored qualified medicinal slices in the same batch is calculated using the contour alignment averaging method based on shape context. The specific steps are as follows: Collect standardized images of 1000 qualified slices of medicinal herbs from the same batch, extract the outline of each slice, and obtain 1000 initial outlines; Select one of the contours as the reference contour, use shape context description and calculate the matching cost between other contours and the reference contour; The optimal contour matching point pairs are solved by dynamic programming algorithm, and all contours are aligned to the coordinate system of the reference contour. The average contour is obtained by averaging the corresponding pixel coordinates of all aligned contours. The average profile was smoothed (using a 5×5 Gaussian smoothing kernel) to obtain the average boundary of qualified medicinal slices in the same batch.

[0082] The beneficial effects of the above technical solution are as follows: by verifying the background boundary distance, background interference boundaries unrelated to the actual medicinal slices, such as conveyor belt scratches, stains, and lens dust, are eliminated, reducing the false detection rate caused by the background from 8% to below 0.5%. At the same time, two sets of different recurrence frequency thresholds are set for single-spectral defects, which ensures the effect of false boundary elimination while avoiding the misjudgment of defects that only appear in a single-spectral image as false boundaries due to insufficient recurrence frequency.

[0083] This invention provides a method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion, which divides a first region to be analyzed and a second region to be analyzed based on boundary width, including: The continuous boundary segments are classified into different states along the overall extension of the real defect boundary. The misalignment abrupt change segment with a boundary curvature greater than the preset curvature and the smooth transition segment with a boundary curvature less than or equal to the preset curvature are distinguished. The pixel misalignment diffusion characteristics of different segments are matched to pre-adapt the basic expansion scale. The basic expansion scale of the misalignment abrupt change segment is M times that of the smooth transition segment. For each layer boundary segment, the preset concentrated width and maximum width are dynamically corrected in intervals, taking into account the dense distribution characteristics of local pixel misalignment in the segment. Global coupling constraint calibration is performed on all initial regions formed by segmented expansion. Gaussian smoothing kernel is used to smooth abrupt changes in region edges and unify the global region boundary extension trend, thus forming the first and second regions to be analyzed.

[0084] In this embodiment, the preset curvature is calibrated as follows: the boundary curvature is used to distinguish between abrupt dislocation sections and smooth transition sections. The preset curvature is calibrated using statistical analysis, which involves statistically analyzing the curvature distribution of 1000 real defect boundaries and calculating the average curvature. and standard deviation Take the preset curvature as In this embodiment, the average curvature was 0.03 and the standard deviation was 0.035, so the preset curvature was 0.1.

[0085] In this embodiment, M is the basic expansion scale multiple of the misalignment abrupt change section. Experiments show that the pixel misalignment diffusion range of the misalignment abrupt change section (such as crack tip, missing corner point) is 1.2-1.8 times that of the smooth transition section. Taking M=1.5 can minimize the introduction of background noise while ensuring coverage of all defect areas.

[0086] In this embodiment, the concentration width and maximum width The interval dynamic correction formula is as follows: ,in, , The corrected center width and maximum width, This refers to the local pixel misalignment density of a segment, which is the ratio of the number of defect candidate pixels to the total number of pixels in that segment. The correction factor has a value range of 0.5-1.0, and in this embodiment, k0=0.8 is used; When the pixel misalignment density within a segment is high, it indicates a large defect range, and the region expansion width is automatically increased; when the pixel misalignment density is low, the region expansion width is automatically decreased, thereby achieving adaptive adjustment of region division.

[0087] In this embodiment, the boundary curvature is calculated using the curvature formula for adjacent three points: ,in, To quantify the degree of local curvature at the boundary of defects in traditional Chinese medicine decoction pieces, it is the sole criterion for dividing boundary segments. When identified as a dislocation-abrupt section (such as crack tips, missing corners, or edges of worm-eaten holes), the defect spread in this area is larger, with a basic expansion scale 1.5 times that of the smooth section; when When the area is identified as a smooth transition zone (such as a straight edge defect or the middle section of a long crack), the defect diffusion range of this type of area is relatively small. It represents the tangent direction of a pixel on the defect boundary, which is used to calculate the normal direction of the boundary (perpendicular to the tangent), and then measure the boundary width and divide the region to be analyzed; It directly reflects the degree of bending at the defect boundary, for example, the crack tip. The absolute value is maximized, corresponding to the curvature k being maximized; for linear boundaries The corresponding curvature k=0; To ensure that boundaries with the same degree of curvature have consistent calculated curvature values ​​regardless of their overall tilt angle, for example, a horizontal arc and an arc with the same radius tilted at 45° will have exactly the same curvature value, thus avoiding segmentation errors caused by different angles at which the medicinal slices are placed.

[0088] The beneficial effects of the above technical solution are: it clarifies the calculation formula of three adjacent points of the boundary curvature, making the segmentation of the defect boundary more accurate. At the same time, it combines the local pixel misalignment density to dynamically adjust the expansion width of the area to be analyzed, which not only ensures that the defect diffusion range of abrupt regions such as crack tips and missing corners is completely covered, but also avoids excessive expansion of smooth areas and the introduction of too much background noise, thus improving the extraction quality of boundary defect features.

[0089] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A traditional Chinese medicine decoction piece defect identification method based on image feature fusion, characterized by, include: Step 1: Acquire multiple initial images of the same target medicinal slice using various acquisition methods. Preprocess the initial images by randomly delaying and superimposing them with the corresponding reference duration to obtain a standardized image set. Perform rapid defect pre-detection on each standardized image in the standardized image set. Calculate the weighted sum of texture entropy and gradient variance of each image as a defect probability score. Select the single standardized image with the highest defect probability and send it to Step 2. Send the remaining standardized images to Step 3. Step 2: receiving the single normalized image output from step 1, intercepting the neighborhood pixel matrix of the high-defect area, and collecting the defect-sensitive pixels in the neighborhood pixel matrix, and then performing regularization and convolution operator operation to extract deep basic feature vectors and transported to step 4; Step 3: Select the single image with the highest image quality score from all remaining standardized images to construct the smallest bounding rectangle of the herbal slice outline as the basic pixel frame and reference coordinate system. After registering the remaining images in all remaining standardized images, stack them in layers to generate multiple sets of heterogeneous combined frames. Analyze the real defect boundaries of the combined frames and divide the first and second regions to be analyzed based on the boundary width. Map and label the defect pixels in the two regions with the reference coordinate system to generate the first distribution map and the second distribution map respectively. At the same time, combine the pre-stored boundary pixels of qualified herbal slices in the same batch to obtain a panoramic base map. The first defect features are extracted from the first distribution map and the second distribution map, respectively. Second defect feature Simultaneously, panoramic appearance features are extracted from the panoramic base map. Send to step 4: Step 4: Adaptively generate dynamic weights based on the defect confidence scores corresponding to each feature, and then... , , , Weighted fusion yields the fused feature vector. And send it to step 5; Step 5: The The pre-trained defect recognition model is inputted, and the medicinal material defect classification label and pixel-level defect coordinates are outputted synchronously.

2. The method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion according to claim 1, characterized in that, The reference duration includes a single-module reference duration , a full-module reference duration ; Set two sets of independent random variables with the same value range. , Local random delay and global random delay are generated respectively, and the parameters satisfy the calculation formula: wherein, is an image capture device imaging frame rate, is a single frame imaging interval; , is a dimensionless calibration coefficient; The preprocessing includes three processes of distortion correction, spatial domain noise reduction, and effective area ROI (Region of Interest) clipping of the decoction piece, and the spatial domain noise reduction module is delayed The image after distortion correction is read, and the ROI clipping module is delayed The image after spatial domain noise reduction is read.

3. The image feature fusion-based traditional Chinese medicine decoction piece defect recognition method according to claim 1, characterized in that, The defect-sensitive pixels within the matrix are grouped and normalized, including: All defect-sensitive pixels within the neighborhood pixel matrix are uniformly grouped into the same column of the matrix, and the remaining empty spaces in the neighborhood pixel matrix are filled with irrelevant background pixels. The neighborhood pixel matrix is ​​a square matrix.

4. The image feature fusion-based traditional Chinese medicine decoction piece defect recognition method according to claim 1, characterized in that, Generate the first distribution map and the second distribution map respectively, including: Three images are randomly selected from the remaining standardized images each time, registered, and stacked in a random order to generate eight heterogeneous combined frames. Along the normal direction of the actual defect boundary, measure the distance between the pixel grayscale abrupt changes on both sides, and take the average of 5 consecutive points as the width of this boundary segment, thus constructing a boundary width sequence. Concentrated width Find the mode of sequence W and its maximum width. ; Using the center line of the actual defect boundary as the central axis, extend to both sides of the axis. Define the first region to be analyzed, and expand it on both sides. Define the second region to be analyzed; Calculate the panorama base image stitching scaling factor ,in, The total pixel area of ​​a single preprocessed image. The target canvas pixel area for the panoramic base image; All defect pixels to be labeled are uniformly mapped to the reference coordinate system corresponding to the basic pixel frame. The defect labeling information of all combined frames is collected at the same coordinate position, and the valid defect pixels are labeled. Among them, for defect pixels detected only in a single spectral image, a spectral specific marker is added during labeling. A first distribution map is generated by filling the effective defect pixels in the first region to be analyzed, and a second distribution map is generated by filling the effective defect pixels in the second region to be analyzed. The size of the first distribution map and the second distribution map is the same as the size of the basic pixel frame. According to the splicing scaling factor After scaling all the remaining registered images, they are stitched together with pre-stored boundary pixels to obtain a panoramic base map with the same size as the target canvas. During stitching, an individual shape deviation correction algorithm is used to compensate for the normal difference between individual slices and the average boundary.

5. The method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion according to claim 1, characterized in that, right , , , Weighted fusion yields the fused feature vector. ,include: ,in, , , , The defect confidence levels are, in order, deep basic features, first defect features, second defect features, and panoramic appearance features; , , , This represents the number of candidate pixels for defects within the corresponding feature domain. , , , This represents the total number of pixels across the entire domain corresponding to the feature. , , , Adaptive dynamic weights for corresponding features; It is the minimum value; For defective pixels with spectrally specific markers, the confidence level of their corresponding features is multiplied by a correction factor of 1.

5.

6. The method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion according to claim 1, characterized in that, The defect identification model is an improved lightweight MobileNetV3 neural network; The model pre-training dataset contains five categories of manually labeled Chinese herbal medicine defect samples: broken, insect-infested, moldy, missing edges, and dregs attached. The model output simultaneously encapsulates the defect category label and the corresponding defect pixel coordinates. For medicinal slices with multiple defects, the model simultaneously outputs all defect category labels and corresponding pixel-level defect coordinates; When pixel regions of different defects overlap, the contour coordinates are marked and the overlapping areas are labeled according to the defect type. The pixel-level defect coordinates are a list of the minimum bounding rectangle coordinates and contour coordinates of each defect region.

7. The method for identifying defects in traditional Chinese medicine decoction pieces based on image feature fusion according to claim 1, characterized in that, Analyzing the true defect boundaries of the composite framework includes: The combined frame is pre-divided into two independent sub-regions: the effective imaging area of ​​the medicinal slices and the blank background area around the frame, using Otsu threshold segmentation. Initial candidate boundaries are extracted for each of the two independent sub-regions. For the initial candidate boundaries detected in the blank background area, the shortest distance between them and the effective imaging area of ​​the medicinal slices is calculated. Only the boundaries with a distance less than or equal to a preset number of pixels are included in the real boundary candidate set, while background interference boundaries with a distance greater than a preset number of pixels are removed. At the same time, invalid boundaries caused by background noise are screened out from all the initial candidate boundaries extracted from the medicinal slices area. All initial candidate boundaries remaining in the physical region of the medicinal slices are regarded as the first boundary. Along the extension direction of each first boundary, two consecutive closed local neighborhood blocks are intercepted at equal intervals around the boundary line, and the grayscale difference of the pixels inside the local neighborhood blocks is compared. The first boundary, whose grayscale difference between neighboring pixels on both sides is less than the difference threshold, is identified as a false boundary generated by the skin folds and natural texture of the cross-section of the medicinal slices and is directly discarded. The first boundary, whose grayscale difference between neighboring pixels on both sides is greater than or equal to the difference threshold, is marked as the second boundary. Retrieve all remaining heterogeneous combined frames generated after imaging the same batch of medicinal slices, and use a unified reference pixel coordinate system as the mapping reference to map the coordinate points of all the current second boundaries to the corresponding positions of the other combined frames one by one. The frequency of boundary recurrence at the same coordinate point in different combined frames was statistically analyzed. For ordinary visible light defects, the second boundary that is detected simultaneously in 3 or more different combined frames and does not coincide with the average boundary of the pre-stored qualified medicinal slices of the same batch is defined as the true defect boundary. For single-spectral defects with spectral specific markers, the second boundary that is detected simultaneously in 2 or more different combined frames is defined as the true defect boundary. The second boundary that appears sporadically only in a single combined frame is judged as a false boundary caused by registration misalignment or transient imaging disturbance and is removed.

8. The method for defect identification of traditional Chinese medicine decoction pieces based on image feature fusion according to claim 7, characterized in that, The first and second regions to be analyzed are divided based on the boundary width, including: The continuous boundary segments are classified into different states along the overall extension of the real defect boundary. The misalignment abrupt change segment with a boundary curvature greater than the preset curvature and the smooth transition segment with a boundary curvature less than or equal to the preset curvature are distinguished. The pixel misalignment diffusion characteristics of different segments are matched to pre-adapt the basic expansion scale. The basic expansion scale of the misalignment abrupt change segment is M times that of the smooth transition segment. For each layer boundary segment, the preset concentrated width and maximum width are dynamically corrected in intervals, taking into account the dense distribution characteristics of local pixel misalignment in the segment. Global coupling constraint calibration is performed on all initial regions formed by segmented expansion. Gaussian smoothing kernel is used to smooth abrupt changes in region edges and unify the global region boundary extension trend, thus forming the first and second regions to be analyzed.

Images