A medicinal material image identification method based on superpixel multi-path fusion

Abstract

The application discloses a medicinal material image identification method based on superpixel multi-path fusion, comprising the following steps: designing an image medicinal material region intelligent segmentation module, adopting a superpixel segmentation method to process medicinal material barrel images, and preliminarily acquiring region images; designing an image effective region intelligent screening module, screening out background regions and dividing single medicinal material regions and multi-medicinal material regions by using a deep network and a threshold screening strategy; and designing a medicinal material intelligent identification module, further processing the multi-medicinal material regions to acquire single medicinal material regions and single medicinal material images, and designing a double-path network to identify the single medicinal material regions and the single medicinal material, respectively, collecting identification results, and determining all medicinal material categories contained in the medicinal material barrel. The application adopts superpixel segmentation, target detection and image classification technologies, realizes accurate identification of all medicinal material categories in the medicinal material barrel, and provides technical support for realizing intelligent real-time review of an automatic dispensing system.

Description

Technical Field

[0001] This invention belongs to the field of image processing and classification. It utilizes computer vision technologies such as superpixel segmentation, target detection, and image classification to process images of medicinal herb barrels, thereby achieving accurate identification of multiple medicinal herbs. Specifically, it relates to a method for identifying medicinal herb images based on superpixel multi-path fusion. Background Technology

[0002] Traditional manual dispensing processes are complex, cumbersome, inefficient, and prone to errors, leading to a growing demand for automated dispensing systems. With the rapid development of artificial intelligence, intelligent processing of traditional Chinese medicine (TCM) has become highly feasible. However, most current automated TCM dispensing systems still require manual review to ensure accuracy and reliability, making it difficult to meet the need for real-time verification.

[0003] Therefore, designing an intelligent identification method based on the images of medicinal material barrels collected by the automated dispensing system to achieve real-time identification of medicinal materials is of great significance for realizing real-time verification and intelligent processing of the dispensing system. Summary of the Invention

[0004] Purpose of the invention: To overcome the shortcomings of existing recognition technologies in meeting the practical needs of multi-medicinal herb identification tasks, this invention provides a medicinal herb image identification method based on superpixel multi-path fusion. It intelligently processes images of medicinal herb containers to obtain effective medicinal herb region images; and utilizes these medicinal herb region images to achieve accurate identification of multiple medicinal herb categories within the container.

[0005] Technical Solution: To achieve the above objectives, this invention provides a method for identifying medicinal material images based on superpixel multi-path fusion, comprising the following steps:

[0006] S1: The image of the medicinal herb barrel is processed using the intelligent segmentation module for the medicinal herb region, so as to divide the image of the medicinal herb barrel into multiple effective region images.

[0007] S2: The image region is processed using an intelligent filtering module for effective image regions. First, a background discrimination model based on a deep network is used to perform preliminary filtering of the image region, removing background regions and retaining the medicinal material regions. Then, a threshold filtering strategy is used to divide the medicinal material regions into single-medicinal-material regions and multi-medicinal-material regions.

[0008] S3: Perform secondary processing on the multi-herb region, use the superpixel segmentation algorithm to perform secondary segmentation, and use the image effective region intelligent screening module to further obtain single herb region and multi-herb region; use the herb detection model to detect the finally determined multi-herb region and obtain single herb image.

[0009] S4: Use a dual-path network to identify individual medicinal material regions and individual medicinal materials, summarize the dual-path identification results, and determine all categories of medicinal materials contained in the medicinal material container.

[0010] Furthermore, in step S1, the intelligent segmentation module for the medicinal herb region mainly utilizes the SLIC superpixel segmentation algorithm to segment the image of the medicinal herb container. The SLIC algorithm uses the color and spatial features of pixels to cluster and generate superpixels. Here, the color feature is the L, a, b value of the pixel in the CIELAB color space, and the spatial feature is the coordinate value of the pixel. The specific process of the SLIC algorithm is as follows: based on the expected superpixel size... Uniform generation A superpixel center point; to avoid the center point falling at the target edge, at the initial center point Within the neighborhood, find the pixel with the smallest gradient as the final initial center point. ; then, at each center point Within a local area, using each pixel L, a, b values , , With the center point L, a, b values , , Calculate color feature distance The specific calculation formula is as follows: , And using pixels coordinates , With the center point coordinates , Calculate spatial feature distance The specific calculation formula is as follows: ,

[0011] Calculate the distance between color and spatial features for each pixel. With the center point The comprehensive distance is calculated using the following formula: , Using the above method, the combined distance between each pixel and different center points is obtained. The pixel with the smallest distance is then assigned the corresponding center point category. Finally, the average distance of all pixels within each superpixel is calculated to obtain the new center point position. This process is repeated iteratively until convergence to generate the final superpixel region.

[0012] Considering the high resolution of the processed medicinal herb barrel images and the large-area clustering of similar medicinal herbs in the images, the expected superpixel size in the SLIC algorithm is set accordingly. The resolution is set to 100 pixels to ensure complete segmentation of the medicinal herb area. Additionally, there are significant color differences between different types of medicinal herbs in the herb barrel image; therefore, the weights in the SLIC algorithm are adjusted accordingly. The value is set to 20, making the generation of superpixels more dependent on color to ensure accurate segmentation of local areas of a single medicinal herb.

[0013] Furthermore, the specific process of the intelligent segmentation module for medicinal material regions in step S1 includes the following steps: A1: The SLIC algorithm is used to process the image of the medicinal herb barrel, and local clustering is performed in the image of the medicinal herb barrel. Multiple superpixel regions are generated based on the clustering results.

[0014] A2: Dilate and expand each superpixel region to generate the smallest bounding rectangle containing each region; use the top-left corner coordinates, width, and height of each rectangle to crop the corresponding rectangular image from the original herb barrel image as the final segmentation result.

[0015] Further, in step S2, preliminary screening is performed by training a ResNet50 network on image data of the background and medicinal herb regions to obtain a background discrimination model. The segmented regions are then classified, and based on the classification results, background regions are removed, while medicinal herb regions are retained.

[0016] Further, the threshold screening strategy in step S2 calculates the histogram distance index for each medicinal herb region image by measuring the distance between the gradient and the gray-level probability histogram of a local region. Since the texture and gray-level features of a single medicinal herb region image are uniformly distributed, the distance between the gradient and the gray-level histogram of a local region in this type of image is generally small, resulting in a smaller calculated histogram distance index. Conversely, images with multiple medicinal herb regions exhibit differences in texture and structure, leading to a larger distance between the gradient and the gray-level histogram of a local region in this type of image, resulting in a larger calculated histogram distance index. Therefore, a histogram distance threshold is used to classify images with an index greater than the threshold as multi-medicinal herb region images, and vice versa.

[0017] Furthermore, the secondary processing of the multi-herb region in step S3 includes the following steps: A1: The SLIC algorithm is used to perform secondary superpixel segmentation on the multi-herb region image. Since the resolution of the multi-herb region image is relatively small, the expected superpixel size in the SLIC algorithm is adjusted. Adjusted to 70 pixels for more precise segmentation.

[0018] A2: Using the region filtering module in step S2, filter out the background region and divide it into single-herb region and multi-herb region.

[0019] A3: Using a medicinal herb detection model, the finalized multi-herb region is detected to obtain the specific region of each individual herb. The obtained specific region information includes the coordinates of its center point. and the width of the area and height To ensure the integrity of medicinal materials within the target area, the regional expansion coefficient is designed. For width and height The process involves expanding the target area; using the expanded width and height, along with the center point coordinates, the specific coordinates of the top-left and bottom-right corners of the area are calculated. and Finally, a complete image of a single medicinal herb is extracted based on the coordinate information; the formula for calculating the specific coordinates is as follows: , Further, in step S4, the Swin Transformer network is trained on single-herb region images and single-herb images respectively to obtain single-herb region identification models and single-herb identification models. The models are then used to identify the corresponding types of images, and a threshold filtering strategy is used to filter the identification results. The specific process of the threshold filtering strategy includes the following steps: A1: Calculate the number of images that are classified into a certain category. Percentage of total images proportion The specific calculation formula is as follows: , A2: The calculated proportion With the set threshold The comparison is performed. If the value is less than a set threshold, the category is removed, indicating that the image of the medicinal herb container does not contain that category of medicinal herbs. If the value is greater than the set threshold, the category is retained, indicating that the image of the medicinal herb container contains that category of medicinal herbs. The specific calculation formula is as follows: , in, This indicates the reserved label for this category of medicinal materials. A value of 0 indicates discarding, while a value of 1 indicates retaining.

[0020] Furthermore, the specific calculation process of the histogram distance index of the medicinal material region image in the step includes the following steps: A1: Obtain grayscale values ​​and gradient magnitudes. First, extract the image of the medicinal herb region. Scaling to uniform Then the region image is converted to grayscale. The corresponding grayscale value is obtained; then, the Sobel operator's x-direction convolution kernel is used. Calculate the gradient of a grayscale image in the x-direction. The specific calculation formula is as follows: , Then, using the Sobel operator's y-direction convolution kernel Calculate the gradient of a grayscale image in the y-direction. The specific calculation formula is as follows: , Finally, the gradient magnitude is calculated using the gradients of the grayscale image in the x and y directions. The specific calculation formula is as follows: , A2: Obtain the local region probability histogram. First, normalize the gradient magnitude and grayscale values ​​of the medicinal herb region image. Then, use... A window of a certain size is used to divide the image into local regions; within each local region... Within, the gradient magnitudes are calculated separately. and grayscale value The probability histogram is used to represent the texture and brightness distribution characteristics of a local region; the histogram range is [0, 1], and the number of intervals is 20. A3: Calculate the histogram distance index First, utilize the values ​​of each interval in the gradient histogram of the local region. The Euclidean distance between the gradient histograms of local regions can be calculated pairwise using the following formula. , , And use the values ​​of each interval in the grayscale histogram The Euclidean distance between the gray-level histograms of local regions is calculated pairwise using the following formula. This allows for the quantification of differences in gradient and grayscale distribution among different local regions. , Next, all gradient histogram distances and grayscale histogram distances were sorted separately, and the top [number] were selected. Minimum distance and ;in, The specific value is determined by the total number of distances in the corresponding histogram. The decision is made, and the specific calculation formula is as follows: , Next, the mean values ​​are calculated as feature values ​​of the medicinal herb region image in terms of gradient and grayscale, respectively. Finally, the two feature values ​​are multiplied together using the following formula to obtain the histogram distance index of the medicinal herb region image. ; .

[0021] Furthermore, the medicinal herb detection model is trained on labeled medicinal herb image data using the YOLOv5s target detection network, and can accurately detect the specific location of a single medicinal herb within a multi-herb region.

[0022] Furthermore, the identification of individual medicinal materials employs two stitching strategies to process the images of individual medicinal materials. The two stitching strategies are: 1) Self-stitching of individual medicinal material images: A single image is reused four times to form a large image, addressing the problem of small individual medicinal material images that differ significantly from the input size required by the network, and mitigating additional distortion caused by directly scaling the image. 2) Random stitching of individual medicinal material images: Four images of the same category of individual medicinal materials are stitched together to form a large image, enhancing the feature information contained in the large image and enabling the model to learn diverse features among medicinal materials of the same category and differences between different categories of medicinal materials. Simultaneously, a two-step identification strategy is designed based on these two stitching strategies, the specific process of which includes the following steps:

[0023] A1: The image of a single medicinal herb is processed using a self-stitching method to obtain a large self-stitched image. Then, an identification model trained based on the large self-stitched image is used to identify the herb, obtaining preliminary identification results for the single medicinal herb image.

[0024] A2: Based on the preliminary identification results, four images are randomly selected from those classified as belonging to the same category and stitched together to form a large, randomly stitched image. To improve the accuracy of identification, multiple rounds of random stitching are used, generating multiple large, randomly stitched images from each individual herb image. Then, the identification model trained based on these large, randomly stitched images is used to identify the herbs and obtain the corresponding category determination results. After obtaining the category identification results corresponding to the large, randomly stitched images, a voting method is used for the smaller images. The category of all large images generated from each smaller image is statistically analyzed, and the category with the highest number of votes is taken as the final category of the individual herb image.

[0025] Beneficial effects: Compared with the prior art, the present invention has the following advantages: 1. This invention employs a superpixel segmentation-based intelligent segmentation algorithm for medicinal material regions. By utilizing multiple rounds of superpixel segmentation and image filtering algorithms, it obtains effective regions in the image of medicinal material barrels, including single-medicinal-material region images and multi-medicinal-material region images, providing data support for subsequent medicinal material target detection and identification.

[0026] 2. This invention employs a target detection and cropping algorithm for single medicinal materials. It uses a medicinal material detection model to detect images of multiple medicinal material regions and determine the region of a single medicinal material. Then, it uses a dilation cropping algorithm to crop out the complete image of a single medicinal material.

[0027] 3. This invention employs a medicinal herb identification algorithm oriented towards single medicinal herb regions and single medicinal herbs. It designs a dual-path network to intelligently identify single medicinal herb regions and single medicinal herbs respectively; and summarizes the multi-path identification results to determine all medicinal herb categories contained in the medicinal herb barrel image. Attached Figure Description

[0028] Figure 1 This is an overall framework diagram of the present invention; Figure 2 This is a diagram illustrating the superpixel segmentation process in this embodiment; Figure 3 This is a diagram illustrating the calculation process of the histogram distance index for the medicinal herb region image in this embodiment; Figure 4 This is a diagram illustrating the effect of the medicinal material detection model in this embodiment; Figure 5 These are illustrations showing the effects of the two splicing strategies in this embodiment; Figure 6 This is a flowchart illustrating the two-step classification strategy in this embodiment; Figure 7 This is an example image showing the identification results of the medicinal herb barrel in this embodiment. Detailed Implementation

[0029] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0030] like Figure 1 As shown in the figure, this embodiment provides a method for identifying medicinal material images based on superpixel multi-path fusion, including the following steps: S1: The image of the medicinal herb barrel is processed using a smart segmentation module to divide it into multiple valid region images. For example... Figure 2 As shown, the processing steps of the intelligent segmentation module for medicinal herb areas include the following: A1: Process the image of the medicinal herb barrel using the SLIC algorithm, and perform the following steps within the image of the medicinal herb barrel: Figure 2 (a) shows the local clustering. Multiple clusters are generated based on the clustering results, such as... Figure 2 (b) shows the superpixel region.

[0031] A2: Dilate and expand each superpixel region to generate the smallest bounding rectangle containing each region; using the top-left corner coordinates, width, and height of each rectangle, crop out the region from the original herb barrel image as follows: Figure 2 The rectangular image shown in (c) is the final segmentation result.

[0032] Considering the high resolution of the processed medicinal herb barrel images and the large-area clustering of similar medicinal herbs in the images, the expected superpixel size in the SLIC algorithm is set accordingly. The resolution is set to 100 pixels to ensure complete segmentation of the medicinal herb area. Additionally, there are significant color differences between different types of medicinal herbs in the herb barrel image; therefore, the weights in the SLIC algorithm are adjusted accordingly. The value is set to 20, making the generation of superpixels more dependent on color to ensure accurate segmentation of local areas of a single medicinal herb.

[0033] S2: The image region intelligent filtering module processes the region images. First, preliminary filtering is performed using a background discrimination model trained on background and medicinal herb region image data based on a ResNet50 network. The segmented region images are then categorized, and background regions are removed, retaining only the medicinal herb regions. Next, a threshold filtering strategy is used to further divide the medicinal herb region images. A histogram distance index is calculated for each medicinal herb region. Using a histogram distance threshold, images with an index greater than the threshold are classified as multi-medicinal herb region images, while those with an index less than the threshold are classified as single-medicinal herb region images. For example... Figure 3 As shown, the process of calculating the histogram distance index of the medicinal material region image includes the following steps: A1: Image of the medicinal herb region Scaling to uniform Then the region image is converted to grayscale. This yields the corresponding grayscale value. Next, the Sobel operator's x-direction convolution kernel is used... Calculate the gradient of a grayscale image in the x-direction. The specific calculation formula is as follows: , Then, using the Sobel operator's y-direction convolution kernel Calculate the gradient of a grayscale image in the y-direction. The specific calculation formula is as follows: , Finally, the gradient magnitude is calculated using the gradients of the grayscale image in the x and y directions. The specific calculation formula is as follows: .

[0034] A2: Normalize the gradient magnitude and grayscale values ​​of the medicinal herb region image. Using... A window of a specific size is used to divide the original image into local regions, resulting in multiple local regions. Within each local region... Within, the gradient magnitudes are calculated separately. and grayscale value A probability histogram is used to represent the texture and brightness distribution characteristics of a local region. The histogram ranges from [0, 1] and has 20 intervals.

[0035] A3: Utilize the values ​​of each interval in the gradient histogram of the local region. The Euclidean distance between the gradient histograms of local regions can be calculated pairwise using the following formula. , , And use the values ​​of each interval in the grayscale histogram The Euclidean distance between the gray-level histograms of local regions is calculated pairwise using the following formula. This allows for the quantification of differences in gradient and grayscale distribution across different local regions.

[0036] , Next, all gradient histogram distances and grayscale histogram distances were sorted separately, and the top [number] were selected. Minimum distance and .in, The specific value is determined by the total number of distances in the corresponding histogram. The decision is made, and the specific calculation formula is as follows:

[0037] Next, the mean values ​​are calculated as feature values ​​for the medicinal herb region image in terms of gradient and grayscale, respectively. Finally, the two feature values ​​are multiplied together using the following formula to obtain the histogram distance index for the medicinal herb region image. .

[0038] .

[0039] S3: Perform secondary processing on the multi-herb region to further obtain images of single herb regions and single herbs. The specific processing includes the following steps: A1: The SLIC algorithm is used to perform secondary superpixel segmentation on the multi-herb region image. Since the resolution of the multi-herb region image is relatively small, the expected superpixel size in the SLIC algorithm is adjusted. Adjusted to 70 pixels for more precise segmentation.

[0040] A2: Using the region filtering module in step S2, filter out the background region and divide it into single-herb region and multi-herb region.

[0041] A3: A medicinal herb detection model trained on labeled medicinal herb image data using a YOLOv5s-based object detection network is used to detect the finally determined multi-herb regions and obtain the specific region of each individual herb. The obtained specific region information includes the center point coordinates. and the width of the area and height To ensure the integrity of the medicinal materials within the target area, the regional expansion coefficient is designed. For width and height The process involves expanding the target area. Using the expanded width and height, along with the center point coordinates, the specific coordinates of the top-left and bottom-right corners of the area are calculated. and Finally, a complete image of a single medicinal herb is extracted based on the coordinate information. The formula for calculating the specific coordinates is as follows: .

[0042] The effect of the medicinal material testing model is as follows Figure 4 As shown in the image, the detection results demonstrate that the medicinal herb detection model can accurately detect the specific locations of most individual medicinal herbs in a multi-herb region image. The cropping results show that, using the region cropping method, based on the detected region information, a complete image of a single medicinal herb can be extracted.

[0043] S4: The Swin Transformer network was trained on images of single medicinal herb regions and single medicinal herbs, respectively, to obtain single medicinal herb region identification models and single medicinal herb identification models. Specific data for the single medicinal herb region images and single medicinal herb images used are shown in Table 1.

[0044] Table 1 Training Data Details

[0045] The accuracy of the trained model in identifying single medicinal regions and single medicinal materials is shown in Table 2. As can be seen from the table, the model's identification accuracy is higher than 0.9 on both types of images, demonstrating good performance.

[0046] Table 2 Model accuracy performance

[0047] When identifying individual medicinal herbs, two stitching strategies are employed to process their images. These strategies are: 1) Self-stitching of individual herb images: A single herb image is reused four times to create a larger image. This addresses the issue of individual herb images being too small, significantly differing from the input size required by the network, and reduces additional distortion caused by direct image scaling. 2) Random stitching of individual herb images: Four images of the same category of herbs are stitched together to create a larger image. This enhances the feature information contained in the larger image, enabling the model to learn diverse features among herbs of the same category and differences between herbs of different categories. The large images formed by processing various types of herbs using these two stitching strategies are shown below. Figure 5 As shown.

[0048] At the same time, based on the two splicing strategies, the following were designed: Figure 6 The two-step identification strategy shown includes the following steps: A1: The image of a single medicinal herb is processed using a self-stitching method to obtain a large self-stitched image. Then, an identification model trained based on the large self-stitched image is used to identify the herb, obtaining preliminary identification results for the single medicinal herb image.

[0049] A2: Based on the preliminary identification results, four images are randomly selected from those classified as belonging to the same category and stitched together to form a large, randomly stitched image. To improve the accuracy of identification, multiple rounds of random stitching are used, generating multiple large, randomly stitched images from each individual herb image. Then, the identification model trained based on these large, randomly stitched images is used to identify the herbs and obtain the corresponding category determination results. After obtaining the category identification results corresponding to the large, randomly stitched images, a voting method is used for the smaller images. The category of all large images generated from each smaller image is statistically analyzed, and the category with the highest number of votes is taken as the final category of the individual herb image.

[0050] After identifying the corresponding image type using the model, a threshold filtering strategy is used to filter and select the identification results. The specific process of the threshold filtering strategy includes the following steps: A1: Calculate the number of images that are classified into a certain category. Percentage of total images proportion The specific calculation formula is as follows: , A2: The calculated proportion With the set threshold The comparison is performed. If the value is less than a set threshold, the category is removed, indicating that the image of the medicinal herb container does not contain that category of herbs. If the value is greater than the set threshold, the category is retained, indicating that the image of the medicinal herb container contains that category of herbs. The specific calculation formula is as follows: , in, This indicates the reserved label for this category of medicinal materials. A value of 0 indicates discarding, while a value of 1 indicates retaining.

[0051] The method proposed in this invention provides intelligent identification results for images of medicinal herb barrels, as follows: Figure 7 As shown. Figure 7 In the diagram, the orange and green dashed boxes represent the single herb region and single herb region obtained through the designed method, respectively. As can be seen from the figure, the final classification results obtained from the two herb barrel images using the method proposed in this invention are consistent with the actual herb categories contained within, demonstrating that the method proposed in this invention can accurately identify all herb categories contained in the herb barrel images.

Claims

1. A method for identifying medicinal materials based on superpixel multi-path fusion, characterized in that, Includes the following steps: S1: Use the intelligent segmentation module for medicinal herb regions to process the image of the medicinal herb barrel and divide the image of the medicinal herb barrel into multiple effective region images; S2: The image effective area intelligent filtering module is used to process the regional image. First, a background discrimination model based on deep network is used to perform preliminary filtering of the regional image, filtering out the background area and retaining the medicinal material area. Then, using a threshold screening strategy, the medicinal material area was divided into single-medicinal-material areas and multi-medicinal-material areas. S3: Perform secondary processing on the multi-herb region, use the superpixel segmentation algorithm to perform secondary segmentation, and use the image effective region intelligent filtering module to further obtain single herb region and multi-herb region; use the herb detection model to detect the finally determined multi-herb region and obtain single herb image; S4: Use a dual-path network to identify individual medicinal material regions and individual medicinal materials, summarize the dual-path identification results, and determine all categories of medicinal materials contained in the medicinal material container.

2. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 1, characterized in that, The specific process of the intelligent segmentation module for medicinal material regions in step S1 includes the following steps: A1: Use the SLIC algorithm to process the image of the medicinal herb barrel, perform local clustering in the image of the medicinal herb barrel, and generate multiple superpixel regions based on the clustering results; A2: Dilate and expand each superpixel region to generate the smallest bounding rectangle containing each region; use the top-left corner coordinates, width, and height of each rectangle to crop the corresponding rectangular image from the original herb barrel image as the final segmentation result.

3. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 2, characterized in that, In step S1, the intelligent segmentation module for the medicinal herb region uses the SLIC superpixel segmentation algorithm to segment the image of the medicinal herb container. The SLIC algorithm uses the color and spatial features of pixels to cluster and generate superpixels; where the color features are the L, a, b values ​​of the pixel in the CIELAB color space, where L represents brightness, a represents red-green value, and b represents yellow-blue value, and the spatial features are the coordinate values ​​of the pixel; the specific process of the SLIC algorithm is as follows: based on the expected superpixel size... Uniform generation A superpixel center point; to avoid the center point falling at the target edge, at the initial center point Within the neighborhood, find the pixel with the smallest gradient as the final initial center point. ; then, at each center point Within a local area, using each pixel L, a, b values , , With the center point L, a, b values , , Calculate color feature distance The specific calculation formula is as follows: ， And using pixels coordinates , With the center point coordinates , Calculate spatial feature distance The specific calculation formula is as follows: ， Calculate the distance between color and spatial features for each pixel. With the center point The comprehensive distance is calculated using the following formula: ， By using the above method, the comprehensive distance between each pixel and different center points is obtained. The minimum distance is selected and the corresponding center point category is assigned to the pixel. Finally, the average of all pixels inside each superpixel is calculated to obtain the new center point position. The above process is repeated iteratively until convergence to generate the final superpixel region. Considering the high resolution of the processed medicinal herb barrel images and the large-area clustering of similar medicinal herbs in the images, the expected superpixel size in the SLIC algorithm is set accordingly. The resolution is set to 100 pixels to ensure complete segmentation of the medicinal herb area. Additionally, there are significant color differences between different types of medicinal herbs in the herb barrel image; therefore, the weights in the SLIC algorithm are adjusted accordingly. The value is set to 20, making the generation of superpixels more dependent on color to ensure accurate segmentation of local areas of a single medicinal herb.

4. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 1, characterized in that, In step S2, preliminary screening is performed by training a ResNet50 network on image data of the background region and the medicinal material region to obtain a background discrimination model; the segmented region images are classified, and based on the classification results, the background region is removed and the medicinal material region is retained.

5. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 4, characterized in that, The threshold screening strategy in step S2 calculates the histogram distance index for each medicinal herb region image by measuring the distance between the gradient and the gray-level probability histogram of a local region. Since the texture and gray-level features of a single medicinal herb region image are uniformly distributed, the distance between the gradient and the gray-level histogram of a local region in this type of image is generally small, resulting in a smaller calculated histogram distance index. Conversely, images with multiple medicinal herb regions exhibit differences in texture and structure, leading to a larger distance between the gradient and the gray-level histogram of a local region in this type of image, resulting in a larger calculated histogram distance index. Therefore, a histogram distance threshold is used to classify images with an index greater than the threshold as multi-medicinal herb region images, and vice versa.

6. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 1, characterized in that, The secondary processing of the multi-herb area in step S3 includes the following steps: A1: The SLIC algorithm is used to perform secondary superpixel segmentation on the multi-herb region image. Since the resolution of the multi-herb region image is relatively small, the expected superpixel size in the SLIC algorithm is adjusted. Adjusted to 70 pixels for more precise segmentation; A2: Using the region filtering module in step S2, filter out the background region and divide it into single-herb region and multi-herb region; A3: Using a medicinal herb detection model, the final determined multi-herb region is detected to obtain the specific region of a single herb. The obtained specific region information includes the center point coordinates. and the width of the area and height To ensure the integrity of medicinal materials within the target area, the regional expansion coefficient is designed. For width and height The process involves expanding the target area; using the expanded width and height, along with the center point coordinates, the specific coordinates of the top-left and bottom-right corners of the area are calculated. and Finally, a complete image of a single medicinal herb is extracted based on the coordinate information; the formula for calculating the specific coordinates is as follows: 。 7. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 1, characterized in that, In step S4, the Swin Transformer network is trained on single medicinal material region images and single medicinal material images to obtain single medicinal material region identification models and single medicinal material identification models; the models are used to identify the corresponding types of images, and the identification results are filtered and screened using a threshold filtering strategy. The specific process of the threshold filtering strategy includes the following steps: A1: Calculate the number of images that are classified into a certain category. Percentage of total images proportion The specific calculation formula is as follows: ， A2: The calculated proportion With the set threshold The comparison is performed. If the value is less than a set threshold, the category is removed, indicating that the image of the medicinal herb container does not contain that category of medicinal herbs. If the value is greater than the set threshold, the category is retained, indicating that the image of the medicinal herb container contains that category of medicinal herbs. The specific calculation formula is as follows: ， in, This indicates the reserved label for this category of medicinal materials. A value of 0 indicates discarding, while a value of 1 indicates retaining.

8. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 5, characterized in that, The specific calculation process of the histogram distance index of the medicinal material area image in the aforementioned steps includes the following steps: A1: Obtain grayscale value and gradient magnitude First, the image of the medicinal herb area. Scaling to uniform Then the region image is converted to grayscale. The corresponding grayscale value is obtained; then, the Sobel operator's x-direction convolution kernel is used. Calculate the gradient of a grayscale image in the x-direction. The specific calculation formula is as follows: ， Then, using the Sobel operator's y-direction convolution kernel Calculate the gradient of a grayscale image in the y-direction. The specific calculation formula is as follows: ， Finally, the gradient magnitude is calculated using the gradients of the grayscale image in the x and y directions. The specific calculation formula is as follows: ， A2: Obtain the probability histogram of the local region First, the gradient magnitude and grayscale values ​​of the medicinal herb region image are normalized; then, using... A window of a certain size is used to divide the image into local regions; within each local region... Within, the gradient magnitudes are calculated separately. and grayscale value The probability histogram is used to represent the texture and brightness distribution characteristics of a local region; the histogram range is [0, 1], and the number of intervals is 20. A3: Calculate the histogram distance index First, utilize the values ​​of each interval in the gradient histogram of the local region. The Euclidean distance between the gradient histograms of local regions can be calculated pairwise using the following formula. , ， And use the values ​​of each interval in the grayscale histogram The Euclidean distance between the gray-level histograms of local regions is calculated pairwise using the following formula. This allows for the quantification of differences in gradient and grayscale distribution among different local regions. ， Next, all gradient histogram distances and grayscale histogram distances were sorted separately, and the top [number] were selected. Minimum distance and ;in, The specific value is determined by the total number of distances in the corresponding histogram. The decision is made, and the specific calculation formula is as follows: ， Next, the mean values ​​are calculated as feature values ​​of the medicinal herb region image in terms of gradient and grayscale, respectively. Finally, the two feature values ​​are multiplied together using the following formula to obtain the histogram distance index of the medicinal herb region image. ; 。 9. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 6, characterized in that, The medicinal herb detection model is trained on labeled medicinal herb image data using the YOLOv5s target detection network, and can accurately detect the specific location of a single medicinal herb within a multi-herb region.

10. The method for identifying medicinal material images based on superpixel multi-path fusion according to claim 7, characterized in that, The identification of individual medicinal materials employs two stitching strategies to process the images of individual medicinal materials. The two stitching strategies are: 1) stitching the images of individual medicinal materials themselves, reusing an image of an individual medicinal material four times to stitch it into a large image, in order to solve the problem that the individual medicinal material images are too small and have a large gap with the input size required by the network, and to reduce the additional distortion caused by directly scaling the image; 2) randomly stitching the images of individual medicinal materials, stitching four images of individual medicinal materials of the same category into a large image, in order to enhance the feature information contained in the large image, so that the model can learn the diverse features between medicinal materials of the same category and the differences between medicinal materials of different categories. Meanwhile, a two-step identification strategy is designed based on the two splicing strategies. The specific process includes the following steps: A1: Process the image of a single medicinal herb using a self-stitching method to obtain a large self-stitched image. Then, use a recognition model trained based on the large self-stitched image to identify it, obtaining preliminary recognition results for the single medicinal herb image; A2: Based on the preliminary identification results, four images are randomly selected from those classified as belonging to the same category and stitched together to form a large, randomly stitched image. To improve the accuracy of identification, multiple rounds of random stitching are used, generating multiple large, randomly stitched images from each individual herb image. Then, the identification model trained based on these large, randomly stitched images is used to identify the herbs and obtain the corresponding category determination results. After obtaining the category identification results corresponding to the large, randomly stitched images, a voting method is used for the smaller images. The category of all large images generated from each smaller image is statistically analyzed, and the category with the highest number of votes is taken as the final category of the individual herb image.

Images