A traditional Chinese medicinal material sorting and grading method based on multi-dimensional visual feature fusion

Abstract

This invention relates to the technical field of medicinal herb image recognition, specifically to a method for sorting and grading Chinese medicinal herbs based on multi-dimensional visual feature fusion. The invention acquires color images of Chinese medicinal herbs on a conveyor belt; calculates the motion blur span based on the conveyor belt speed, camera exposure time, physical field of view width, and pixel width in the direction of motion; combines the effective feature distribution radius and pixel coordinates of the medicinal herbs, and calculates the influence coefficient of each pixel using an attenuation function; calculates the disorder coefficient and absolute gray difference coefficient on the red, green, and blue color channels respectively, thereby obtaining the grayscale evaluation coefficient and weight of each channel; calculates the multi-dimensional similarity between the image to be tested and a standard medicinal herb image, and completes the grading based on this similarity. This invention incorporates industrial site hardware conditions into the algorithm considerations, suppresses edge interference through differentiated influence coefficients, and achieves multi-dimensional feature fusion through dynamic allocation of channel weights, effectively improving the accuracy and robustness of sorting and grading Chinese medicinal herbs.

Description

Technical Field

[0001] This invention relates to the technical field of medicinal material image recognition, specifically to a sorting and grading method for Chinese medicinal materials based on multidimensional visual feature fusion. Background Technology

[0002] The traditional Chinese medicine (TCM) processing industry is currently at a critical stage of transitioning from manual sensory identification to automated visual sorting. Modern TCM sorting lines require systems capable of high-precision identification of fine-grained visual features such as bark width, surface texture, color saturation, and surface defects to achieve industrial-grade classification of specific medicinal materials. While existing machine vision technologies have incorporated deep learning mechanisms such as convolutional neural networks and residual networks, the accuracy of automated identification remains highly unstable in complex and dynamic scenarios during actual industrial deployments. This is primarily due to the vast differences in the natural morphology of TCM materials, and the physical interference from high-speed conveyor belts severely compromises the reliability of visual features, thus hindering the standardization and industrial-scale application of TCM materials.

[0003] Current visual sorting solutions generally assume ideal static lighting conditions. However, in actual high-speed transport processes, the movement of medicinal materials combined with camera exposure inevitably produces motion blur. Simultaneously, lens distortion leads to low resolution and large aberrations at the edges of the field of view. The combined effect of these two factors renders the acquired image features unreliable, directly reducing the accuracy of subsequent grading. Furthermore, industrial images contain a large amount of conveyor belt background and edge distortion areas; motion blur has a more significant impact on edge regions. If all regions in the image are treated equally without distinction, low-quality areas will severely interfere with overall feature judgment, making it difficult for the visual sorting system to meet actual production requirements. Summary of the Invention

[0004] To address the technical problem, the present invention aims to provide a sorting and grading method for traditional Chinese medicinal materials based on multi-dimensional visual feature fusion. The specific technical solution adopted is as follows:

[0005] A sorting and grading method for traditional Chinese medicinal materials based on multidimensional visual feature fusion, the method comprising:

[0006] Images of Chinese medicinal herbs to be sorted on a conveyor belt are acquired to obtain color images; the motion blur span is calculated based on the conveyor belt speed, the exposure time of the color camera, the physical field of view of the color image, and the pixel width in the direction of motion when the color image is acquired.

[0007] Based on the motion blur span, the effective feature distribution radius of Chinese medicinal materials in the color image, and the coordinate position of the pixel, the influence coefficient of all pixels is calculated using the attenuation function; on the red, green, and blue color channels, the disorder coefficient of the color channel is calculated based on the influence coefficient and gray value of each pixel, and the absolute gray difference coefficient of the color channel is calculated based on the influence coefficient and gray value of each pixel.

[0008] Based on the disorder coefficient and absolute gray difference coefficient of each color channel, the grayscale evaluation coefficient of each color channel is obtained; based on the grayscale evaluation coefficient of each color channel, the weight of each color channel is obtained; for each color channel, the multi-dimensional similarity is obtained based on the product of the influence coefficient and grayscale value of each pixel in the color image, the product of the influence coefficient and grayscale value of the corresponding pixel in the pre-entered standard medicinal material image, and the weight of each color channel.

[0009] The grades of Chinese medicinal materials are distinguished based on multi-dimensional similarity.

[0010] Furthermore, methods for obtaining the distribution radius of effective characteristics of Chinese medicinal materials include:

[0011] A pre-trained semantic segmentation network is used to perform binary classification segmentation of the Chinese medicinal herb region and the background region in the image, and a binary mask of the Chinese medicinal herb region is obtained.

[0012] Calculate the root mean square value of the Euclidean distance from all foreground pixels in the binary mask to the centroid pixel of the mask, and use it as the effective feature distribution radius.

[0013] Furthermore, methods for obtaining the motion blur span include:

[0014] The motion fuzz span is obtained according to the motion fuzz span calculation formula, which is shown below:

[0015]

[0016] In the formula, This indicates the range of motion blur in the direction of motion of a color image; Indicates the speed of the conveyor belt; Indicates the exposure time of a color camera; Represents the pixel width of a color image in the direction of motion; This represents the physical field of view width of a color image.

[0017] Furthermore, methods for obtaining the influence coefficient include:

[0018] Establish a Cartesian coordinate system with the bottom left corner of the color image as the origin, and obtain the coordinate position of each pixel in the color image;

[0019] The influence coefficient is obtained according to the formula for calculating the influence coefficient, which is shown below:

[0020]

[0021] In the formula, Indicates the location in the color image The influence coefficient of the pixel; Represents the center pixel of a color image The x-coordinate; Represents the center pixel of a color image The ordinate; Indicates the radius of distribution of effective features of Chinese medicinal materials within a color image; This indicates the range of motion blur in the direction of motion of a color image.

[0022] Furthermore, the methods for obtaining the disorder level coefficient include:

[0023] The level of chaos coefficient is obtained according to the formula for calculating the level of chaos coefficient, which is shown below:

[0024]

[0025] In the formula, A coefficient representing the degree of disorder for each color channel; Indicates the location in the color image The influence coefficient of the pixel; Indicates the location in the color image The grayscale value of the pixel; This represents the total number of pixels in a color image; This represents summing all pixels in a color image. Represents the maximum value function; This represents the absolute value function.

[0026] Furthermore, methods for obtaining the absolute gray difference coefficient include:

[0027] The absolute gray difference coefficient is obtained according to the formula for calculating the absolute gray difference coefficient, which is shown below:

[0028]

[0029] In the formula, This represents the absolute grayscale coefficient for each color channel; Indicates the location in the color image The influence coefficient of the pixel; Indicates the location in the color image The grayscale value of the pixel; This represents the total number of pixels in a color image; Represents the maximum value function; Describes the minimum value function; This represents the arctangent function.

[0030] Furthermore, methods for obtaining grayscale evaluation coefficients include:

[0031] The grayscale evaluation coefficient for each color channel is obtained by averaging the disorder coefficient and the absolute gray difference coefficient.

[0032] Furthermore, the methods for obtaining the weights of each color channel include:

[0033] The grayscale evaluation coefficients of each color channel are normalized to obtain the weight of each color channel in the fusion of multidimensional visual features.

[0034] Furthermore, methods for obtaining multi-dimensional similarity include:

[0035]

[0036] In the formula, Indicates the multidimensional similarity of Chinese medicinal materials within a color image; Indicates the first The weights of each color channel in the fusion of multidimensional visual features; Indicates the location in the color image The influence coefficient of the pixel; Indicates the color image in the first... In the color channels located at The grayscale value of the pixel; This indicates that the pre-recorded standard medicinal material images are in the first... In the color channels located at The influence coefficient of the pixel; This indicates that the pre-recorded standard medicinal material images are in the first... In the color channels located at The grayscale value of the pixel; This indicates that the summation is performed on each color channel.

[0037] Furthermore, the grades of Chinese medicinal materials are differentiated based on multi-dimensional similarity, including:

[0038] Standard medicinal material samples were placed in the shooting area at multiple equally divided angles to collect images of the standard medicinal materials from multiple perspectives.

[0039] The color image is compared with the standard medicinal material images from all directions one by one in a multi-dimensional similarity calculation. The maximum value is selected as the final evaluation score. The grade is divided according to the preset scoring range. The closer the evaluation score is to 1, the higher the grade of the medicinal material. When the value is negative, it will not participate in the subsequent sorting and grading.

[0040] The present invention has the following beneficial effects:

[0041] This invention precisely converts hardware parameters such as conveyor belt speed, camera exposure time, and physical field of view into motion blur span at the digital image level, incorporating physical interference from the industrial environment into the algorithm's considerations. This avoids the limitations of purely software optimization divorced from real-world conditions. By utilizing a disorder coefficient and an absolute gray difference coefficient weighted by an influence coefficient, the regularity of grayscale distribution and the contrast between medicinal materials and the background are quantified, enabling differentiated processing of different regions in the image. This preserves the central high signal-to-noise ratio region and suppresses the interference of high-risk edge regions on the overall judgment. By dynamically allocating weights according to the grayscale evaluation results of each color channel, medicinal materials of different colors can obtain a more reasonable similarity measurement in their main color feature dimensions. Through multi-directional standard sample input combined with a maximum similarity selection strategy, the problem of posture differences caused by random placement of medicinal materials is effectively solved, ensuring that medicinal materials of the same grade can still stably match the correct template at different angles, significantly reducing misclassification. Attached Figure Description

[0042] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of a sorting and grading method for Chinese medicinal materials based on multidimensional visual feature fusion, provided as an embodiment of the present invention. Detailed Implementation

[0044] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] The following description, in conjunction with the accompanying drawings, details a specific scheme for a sorting and grading method for Chinese medicinal materials based on multidimensional visual feature fusion provided by this invention.

[0047] Please see Figure 1This illustrates an embodiment of the present invention providing a sorting and grading method for traditional Chinese medicinal materials based on multidimensional visual feature fusion, the method comprising:

[0048] Step S1: Acquire images of the Chinese medicinal materials to be sorted on the conveyor belt to obtain color images; calculate the motion blur span based on the conveyor belt speed, the exposure time of the color camera, the physical field of view width of the color image and the pixel width in the direction of motion when acquiring the color image.

[0049] In industrial sorting operations, medicinal herbs move at a certain speed along a conveyor belt past a photographing area. Because the camera has a specific exposure time during shooting, and the herbs continue to move within this window, the image recorded on the sensor exhibits motion blur, or trailing shadows along the direction of motion. Physically, this manifests as the distance the object has moved during the exposure time. In this embodiment of the invention, to incorporate motion blur interference from the industrial environment into subsequent steps and avoid purely software optimization outside the implementation scenario, images of the medicinal herbs to be sorted on the conveyor belt are acquired to obtain color images. Based on the conveyor belt speed, the color camera's exposure time, the physical field of view of the color image, and the pixel width along the direction of motion, the motion blur span is calculated.

[0050] In one embodiment of the present invention, the medicinal herbs to be sorted on the conveyor belt are first image-captured to obtain color images. The specific process includes:

[0051] On a high-speed sorting production line in the initial processing stage of Chinese medicinal herbs, a high-resolution industrial-grade global shutter color camera with a fixed lens focal length is installed, taking orthogonal overhead shots perpendicular to the conveyor belt surface. A high-precision incremental photoelectric encoder is installed at the main shaft end of the conveyor belt to provide real-time feedback on the conveyor belt's operating speed. The camera records the actual exposure time of the underlying layer and determines the physical field of view width of the captured image in reality, i.e., the width range that the camera can capture in the direction of movement. The pixel width of the image captured by the camera in the direction of movement is also recorded, i.e., the number of pixels in the image in the direction of movement. When the Chinese medicinal herbs move with the conveyor belt to the center of the imaging area, the camera is triggered to automatically capture a color image of the Chinese medicinal herbs to be sorted.

[0052] Preferably, in one embodiment of the present invention, the method for obtaining the motion blur span includes:

[0053] The motion fuzz span is obtained according to the motion fuzz span calculation formula, which is shown below:

[0054]

[0055] In the formula, This indicates the range of motion blur in the direction of motion of a color image; Indicates the speed of the conveyor belt; Indicates the exposure time of a color camera; It represents the pixel width of a color image in the direction of motion, that is, the number of pixels that each feature point of the Chinese medicinal herb region in the color image spreads across the photosensitive element along the direction of motion during the exposure time. This represents the physical field of view width of a color image.

[0056] In the formula for calculating the span of motion fuzziness This represents the actual displacement distance of the Chinese medicinal materials along the direction of movement of the conveyor belt in physical space within the exposure time window, divided by the width of the physical field of view. This process obtains the translation ratio of the medicinal herbs in the image, reflecting the relative size of the herbs' physical displacement within the entire image; this is a dimensionless value. Finally, it is multiplied by the pixel width of the image in the direction of motion. This obtains the motion blur span of a color image in the direction of motion, in pixels.

[0057] Step S2: Based on the motion blur span, the effective feature distribution radius of Chinese medicinal materials in the color image, and the coordinate position of the pixels, calculate the influence coefficient of all pixels using the attenuation function; calculate the disorder coefficient of the color channel based on the influence coefficient and gray value of each pixel in the red, green, and blue color channels respectively, and calculate the absolute gray difference coefficient of the color channel based on the influence coefficient and gray value of each pixel.

[0058] The inherent distortion characteristics of optical lenses determine that the central region of the field of view has high resolution and good image quality, while the edge region has large aberrations and low resolution. Furthermore, during image acquisition, apart from the effective feature area occupied by the medicinal herbs, the remaining areas are essentially background areas of the conveyor belt with no discriminative value. Moreover, the texture features of the medicinal herbs differ significantly from the image features of the conveyor belt. Motion blur diffusion into the edge region of the effective feature area will significantly intrude into the background region of the conveyor belt, thus greatly interfering with subsequent analysis. Comparatively, the peripheral regions of the image are more sensitive to motion blur. Treating all pixels in the entire image equally will lead to information from low-quality areas contaminating the feature representation of high-quality areas. Therefore, in this embodiment of the invention, based on the motion blur span, the effective feature distribution radius of the medicinal herbs within the color image, and the coordinate position of the pixels, an attenuation function is used to calculate the influence coefficient of all pixels.

[0059] Preferably, in one embodiment of the present invention, the method for obtaining the effective characteristic distribution radius of Chinese medicinal materials includes:

[0060] A pre-trained semantic segmentation network is used to perform binary classification segmentation of the image into medicinal herb regions and background regions, resulting in a binary mask for the medicinal herb regions. In one embodiment of the present invention, semantic segmentation models suitable for binary classification segmentation, such as U-Net and DeepLabV3+, are used to perform binary classification segmentation of the color image into medicinal herb regions and background regions. It should be noted that the above steps are technical means well known to those skilled in the art, and are not limited or elaborated here.

[0061] The root mean square value of the Euclidean distance from all foreground pixels in the binary mask to the centroid pixel of the mask is calculated as the effective feature distribution radius. The larger the effective feature distribution radius, the wider the distribution range of Chinese medicinal materials in the color image and the richer the effective features.

[0062] Preferably, in one embodiment of the present invention, the method for obtaining the influence coefficient includes:

[0063] Establish a Cartesian coordinate system with the bottom left corner of the color image as the origin, and obtain the coordinate position of each pixel in the color image;

[0064] The influence coefficient is obtained according to the formula for calculating the influence coefficient, which is shown below:

[0065]

[0066] In the formula, Indicates the location in the color image The influence coefficient of the pixel; Represents the center pixel of a color image The x-coordinate; Represents the center pixel of a color image The ordinate; This represents the effective feature distribution radius of Chinese medicinal materials within a color image; This indicates the range of motion blur in the direction of motion of a color image.

[0067] The Gaussian decay function is used in the formula for calculating the influence coefficient. Indicates that it is located at From the pixel to the center pixel of the color image The square of the Euclidean distance reflects the location of The degree to which a pixel is spatially offset from the image center; the farther it is offset, the more significant the difference. The larger the value, the more the independent variable of the exponential function tends to negative infinity, and the closer the influence coefficient approaches 0. The larger the value, the wider the effective distribution of Chinese medicinal materials in the image. With the same distance from the center, the slower the decay of the influence coefficient, and the higher the overall importance of the pixels. Motion blur span of a color image in the direction of motion. The larger the value, the more severe the effect of motion blur on the image. In this case, the less important the pixels in the color image are, that is, the smaller the influence coefficient.

[0068] For high-quality Chinese medicinal herbs, their grayscale features exhibit a certain regularity and uniformity in spatial distribution. Low-quality herbs, however, may suffer from defects such as mold, insect infestation, or damage. These defects disrupt the continuity of the herb's surface texture, resulting in a more chaotic grayscale distribution and the generation of more high-frequency components. Therefore, the spatial regularity of grayscale value distribution reflects the quality level of the herbs to some extent. Furthermore, there is usually a significant difference in grayscale values ​​between the effective feature region and the background region in a Chinese medicinal herb image. This difference reflects the salience of the herb's color features relative to the background; the greater the difference, the more prominent the herb's color features. Since different Chinese medicinal herbs possess their own unique color characteristics, in this embodiment of the invention, the color image is decomposed into three independent color channels: red, green, and blue. For each of the three color channels (red, green, and blue), the disorder coefficient is calculated based on the influence coefficient and grayscale value of each pixel, and the absolute gray difference coefficient is also calculated based on the influence coefficient and grayscale value of each pixel.

[0069] Preferably, in one embodiment of the present invention, the method for obtaining the disorder level coefficient includes:

[0070] The level of chaos coefficient is obtained according to the formula for calculating the level of chaos coefficient, which is shown below:

[0071]

[0072] In the formula, A coefficient representing the degree of disorder for each color channel; Indicates the location in the color image The influence coefficient of the pixel; Indicates the location in the color image The grayscale value of the pixel; This represents the total number of pixels in a color image; This represents summing all pixels in a color image. Represents the maximum value function; This represents the absolute value function.

[0073] In the formula for calculating the disorder level coefficient, the weighted average gray value is calculated first. , representing the arithmetic mean of the gray values ​​of all pixels weighted by influence coefficients. Pixels in the central region with larger influence coefficients contribute more to the average, while pixels in the edge region with smaller influence coefficients contribute less, making the average more faithful to the region of highest quality in the image. Then, for each pixel, its weighted gray value is calculated. Compared with weighted average gray value The absolute deviation between them. The average absolute deviation is obtained by averaging the absolute deviations of all pixels, and then using... The mean absolute deviation is normalized so that the range of the disorder coefficient falls within a certain range. Within the range. The closer a value is to 1, the more chaotic and uneven the spatial distribution of gray values ​​in the color image, which may indicate that the medicinal material is defective or of poor quality. The closer the value is to 0, the more uniform and regular the grayscale distribution in the color image, and the better the quality of the medicinal material may be.

[0074] Preferably, in one embodiment of the present invention, the method for obtaining the absolute gray difference coefficient includes:

[0075] The absolute gray difference coefficient is obtained according to the formula for calculating the absolute gray difference coefficient, which is shown below:

[0076]

[0077] In the formula, This represents the absolute grayscale coefficient for each color channel; Indicates the location in the color image The influence coefficient of the pixel; Indicates the location in the color image The grayscale value of the pixel; This represents the total number of pixels in a color image; Represents the maximum value function; Describes the minimum value function; This represents the arctangent function.

[0078] In the formula for calculating the absolute gray difference coefficient, the difference between the weighted gray value and the minimum weighted gray value of each pixel is first calculated. The mean value is calculated to obtain the average deviation of all pixels from the minimum value. This average deviation reflects the increase in the overall gray level of the effective area of ​​the medicinal material in the image relative to the background area; that is, the greater the average deviation, the greater the gray level difference between the medicinal material area and the background, and the more significant the color features. The average deviation is then divided by... Finally, nonlinear compression is performed using the arctangent function, and then multiplied by... Normalization is performed to adjust the absolute gray difference coefficient of each color channel. The range of values ​​falls within Within the range. When the ratio of the numerator to the denominator approaches 0, meaning there is almost no distinction between the medicinal material area and the background, The ratio approaches 0; when the ratio is large, that is, when the contrast between the medicinal material area and the background is strong, Approaching 1.

[0079] Step S3: Obtain the grayscale evaluation coefficient of each color channel based on the disorder coefficient and absolute grayscale difference coefficient of each color channel; obtain the weight of each color channel based on the grayscale evaluation coefficient of each color channel; for each color channel, obtain the multi-dimensional similarity based on the product of the influence coefficient and grayscale value of each pixel in the color image, the product of the influence coefficient and grayscale value of the corresponding pixel in the pre-entered standard medicinal material image, and the weight of each color channel.

[0080] The disorder coefficient and absolute gray difference coefficient obtained in step S2 can measure the sorting reference value of a color image in a single color channel. Therefore, merging the two into a unified gray level evaluation coefficient can comprehensively reflect the quality of the feature information contained in the color channel.

[0081] Preferably, in one embodiment of the present invention, the method for obtaining the grayscale evaluation coefficient includes:

[0082] The average of the disorder coefficient and the absolute gray difference coefficient of each color channel is used to obtain the grayscale evaluation coefficient of each color channel. The larger the grayscale evaluation coefficient, the greater the reference value of the color features contained in that color channel for the sorting of medicinal materials.

[0083] Based on this, different types of Chinese medicinal herbs have their own distinct primary color characteristics. For example, when a medicinal herb is predominantly red, its grayscale value in the red channel is high, indicating rich feature information, while its grayscale value in the blue and green channels is low, indicating relatively poor feature information. If a fixed and uniform three-channel weight is used for similarity calculation, the characteristic advantages of the medicinal herb in its primary color dimension cannot be highlighted. Therefore, it is necessary to dynamically assign weights to each channel in subsequent similarity calculations based on the grayscale evaluation coefficient of each color channel.

[0084] Preferably, in one embodiment of the present invention, the method for obtaining the weight of each color channel includes:

[0085] The grayscale evaluation coefficients of each color channel are normalized to obtain the weight of each color channel in the multi-dimensional visual feature fusion. The higher the grayscale evaluation coefficient of a channel, the richer the color features it carries and the greater its discriminative value, and therefore it should be assigned a higher weight.

[0086] By calculating the weighted cosine similarity between the image under test and the standard medicinal material image on each color channel, and then summing the results with the weights of each channel, a multi-dimensional similarity is obtained, thus achieving the organic fusion of multi-dimensional visual features.

[0087] Before calculating multi-dimensional similarity, standard medicinal material samples need to be pre-entered, i.e., multi-angle images of reference medicinal materials with clearly defined grades. The standard medicinal material samples are placed in the shooting area at multiple equally divided angles. In one embodiment of the present invention, 12 equally divided angles are spaced at 30° intervals for a total of 12 angles, and static images of the standard medicinal materials are generated from multiple angles.

[0088] To ensure the accuracy of pixel-by-pixel similarity calculation, spatial alignment between the color image and the standard medicinal herb image is required. Specifically: First, the aforementioned semantic segmentation network is used to extract binary masks of the medicinal herb regions in both the test color image and the standard medicinal herb image; the centroid coordinates of the two masks are calculated, and the centroids of the medicinal herb regions in the test image and the standard image are aligned through translation transformation; then, based on the size of the bounding rectangle of the mask, the medicinal herb regions in the test image are scaled to the same size as those in the standard image and filled into an image canvas of the same size. After completing the above alignment operation, multi-dimensional similarity is calculated.

[0089] Preferably, in one embodiment of the present invention, the method for obtaining multi-dimensional similarity includes:

[0090]

[0091] In the formula, Indicates the multidimensional similarity of Chinese medicinal materials within a color image; Indicates the first The weights of each color channel in the fusion of multidimensional visual features; Indicates the location in the color image The influence coefficient of the pixel; Indicates the color image in the first... In the color channels located at The grayscale value of the pixel; This indicates that the pre-recorded standard medicinal material images are in the first... In the color channels located at The influence coefficient of the pixel; This indicates that the pre-recorded standard medicinal material images are in the first... In the color channels located at The grayscale value of the pixel; This indicates that the summation is performed on each color channel.

[0092] In the formula, for a single color channel, the dot product term in the numerator is first calculated, which is the weighted gray vector of the color image. Weighted grayscale vector of standard medicinal material image The inner product of the vectors indicates that the two vectors are more aligned in direction and therefore more similar. The denominator is the product of the magnitudes of the two vectors. This is used to normalize the inner product so that the range of values ​​for the fraction is . In practical applications, it is usually This is because the grayscale value is non-negative and the influence coefficient is positive. After calculating the cosine similarity on each of the three color channels, it is multiplied by the weight of each channel. The results are summed to obtain the final multi-dimensional similarity. Channels with richer color features have larger weights and contribute more to the similarity calculation. The closer the multi-dimensional similarity is to 1, the more similar the feature distribution of the tested medicinal material and the standard medicinal material is in each color dimension, that is, the closer the quality of the medicinal material is to the standard grade.

[0093] Among them, the influence coefficient of pre-recorded standard medicinal material images Using the same method as the aforementioned influence coefficient, the effective feature distribution radius of the pre-entered standard medicinal material image is calculated based on the binary mask of the standard medicinal material image according to the aforementioned method for obtaining the effective feature distribution radius.

[0094] In one embodiment of the present invention, the pre-recorded standard medicinal material image is a static image acquired when the conveyor belt is stationary. In this case, the preset motion blur span used to calculate its influence coefficient is set to 1 pixel. It should be noted that the preset motion blur span can be set by the user and is not limited here.

[0095] Step S4: Differentiate the grades of Chinese medicinal materials based on multi-dimensional similarity.

[0096] Because the placement of Chinese medicinal herbs on the conveyor belt is random and multi-directional, a standard image of the herb from a single angle cannot cover all possible orientations. If the herb to be tested is compared only with a standard image from a single angle, herbs with significantly different orientations from the standard image, even if they are of acceptable quality, may be misclassified as low-grade due to these orientation differences. Therefore, it is necessary to pre-enter standard images of the herb sample at multiple equally divided angles, and then calculate the multi-dimensional similarity between the herb to be tested and the standard images from all orientations, using the maximum similarity score as the final evaluation criterion.

[0097] Preferably, in one embodiment of the present invention, the grade of Chinese medicinal materials is distinguished based on multi-dimensional similarity, including:

[0098] The color image is compared with standard medicinal herb images from all directions to calculate multi-dimensional similarity, and the maximum value is selected as the final evaluation score.

[0099] The grade is determined based on a preset scoring range. The closer the score is to 1, the higher the grade of the medicinal material. If the score is negative, the material will not be included in the subsequent sorting and grading.

[0100] In one embodiment of the present invention, the following grading rule can be set: when the final evaluation score is in When the final evaluation score is within the specified range, it is judged as a first-grade medicinal material (high-quality product); when ... When the final evaluation score is within the specified range, it is judged as a second-grade medicinal material (qualified product); when the final evaluation score is within the specified range, it is judged as a second-grade medicinal material (qualified product). At that time, it was judged as a third-grade medicinal material (substandard). It should be noted that since all gray values ​​and influence coefficients involved in the calculation are non-negative numbers, the calculation result of multi-dimensional similarity must be greater than or equal to 0.

[0101] In one embodiment of the present invention, the preset scoring interval includes: a first interval is... The first interval corresponds to Grade 1 medicinal materials (high-quality); the second interval is... The third interval corresponds to Grade II medicinal materials (qualified products); This corresponds to Grade 3 medicinal materials (substandard products).

[0102] In summary, image acquisition was performed on the Chinese medicinal herbs to be sorted on the conveyor belt to obtain color images. Based on the conveyor belt speed, color camera exposure time, physical field of view width of the color image, and pixel width in the direction of motion, the motion blur span was calculated. Based on the motion blur span, the effective feature distribution radius of the Chinese medicinal herbs within the color image, and the coordinate positions of the pixels, the influence coefficient of all pixels was calculated using a decay function. For each of the red, green, and blue color channels, the disorder coefficient of that color channel was calculated based on the influence coefficient and grayscale value of each pixel. The influence coefficient of a pixel and its grayscale value are used to calculate the absolute gray difference coefficient of the color channel. Based on the disorder coefficient and absolute gray difference coefficient of each color channel, the grayscale evaluation coefficient of each color channel is obtained. Based on the grayscale evaluation coefficient of each color channel, the weight of each color channel is obtained. For each color channel, a multi-dimensional similarity is obtained based on the product of the influence coefficient and grayscale value of each pixel in the color image, the product of the influence coefficient and grayscale value of the corresponding pixel in the pre-recorded standard medicinal material image, and the weight of each color channel. The grades of Chinese medicinal materials are distinguished based on the multi-dimensional similarity.

[0103] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion, characterized in that, The method includes: Images of Chinese medicinal herbs to be sorted on a conveyor belt are acquired to obtain color images; the motion blur span is calculated based on the conveyor belt speed, the exposure time of the color camera, the physical field of view of the color image, and the pixel width in the direction of motion when the color image is acquired. Based on the motion blur span, the effective feature distribution radius of Chinese medicinal materials in the color image, and the coordinate position of the pixels, the influence coefficient of all pixels is calculated using the attenuation function; on the red, green, and blue color channels, the disorder coefficient of the color channel is calculated based on the influence coefficient and gray value of each pixel, and the absolute gray difference coefficient of the color channel is calculated based on the influence coefficient and gray value of each pixel. Based on the disorder coefficient and the absolute gray difference coefficient of each color channel, a grayscale evaluation coefficient for each color channel is obtained; based on the grayscale evaluation coefficient of each color channel, a weight for each color channel is obtained; for each color channel, a multi-dimensional similarity is obtained based on the product of the influence coefficient and grayscale value of each pixel in the color image, the product of the influence coefficient and grayscale value of the corresponding pixel in the pre-recorded standard medicinal material image, and the weight of each color channel. The grades of Chinese medicinal materials are distinguished based on the aforementioned multi-dimensional similarity.

2. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the effective characteristic distribution radius of the Chinese medicinal materials includes: A pre-trained semantic segmentation network is used to perform binary classification segmentation of the Chinese medicinal herb region and the background region in the image, and a binary mask of the Chinese medicinal herb region is obtained. The root mean square value of the Euclidean distance from all foreground pixels in the binary mask to the centroid pixel of the mask is calculated and used as the effective feature distribution radius.

3. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the motion blur span includes: The motion fuzz span is obtained according to the motion fuzz span calculation formula, which is as follows: ； In the formula, This indicates the range of motion blur in the direction of motion of a color image; Indicates the speed of the conveyor belt; Indicates the exposure time of a color camera; Represents the pixel width of a color image in the direction of motion; This represents the physical field of view width of a color image.

4. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the influence coefficient includes: A Cartesian coordinate system is established with the lower left corner of the color image as the origin, and the coordinate position of each pixel in the color image is obtained. The influence coefficient is obtained according to the influence coefficient calculation formula, which is shown below: ； In the formula, Indicates the location in the color image The influence coefficient of the pixel; Represents the center pixel of a color image The x-coordinate; Represents the center pixel of a color image The ordinate; Indicates the radius of distribution of effective features of Chinese medicinal materials within a color image; This indicates the range of motion blur in the direction of motion of a color image.

5. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the disorder level coefficient includes: The disorder level coefficient is obtained according to the disorder level coefficient calculation formula, which is shown below: ； In the formula, A coefficient representing the degree of disorder for each color channel; Indicates the location in the color image The influence coefficient of the pixel; Indicates the location in the color image The grayscale value of the pixel; This represents the total number of pixels in a color image; This represents summing all pixels in a color image. Represents the maximum value function; This represents the absolute value function.

6. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the absolute gray difference coefficient includes: The absolute gray difference coefficient is obtained according to the formula for calculating the absolute gray difference coefficient, which is as follows: ； In the formula, This represents the absolute grayscale coefficient for each color channel; Indicates the location in the color image The influence coefficient of the pixel; Indicates the location in the color image The grayscale value of the pixel; This represents the total number of pixels in a color image; Represents the maximum value function; Describes the minimum value function; This represents the arctangent function.

7. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the grayscale evaluation coefficient includes: The grayscale evaluation coefficient for each color channel is obtained by averaging the disorder coefficient and the absolute gray difference coefficient.

8. The method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the weight of each color channel includes: The grayscale evaluation coefficients of each color channel are normalized to obtain the weight of each color channel in the fusion of multidimensional visual features.

9. A method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The method for obtaining the multi-dimensional similarity includes: ； In the formula, Indicates the multidimensional similarity of Chinese medicinal materials within a color image; Indicates the first The weights of each color channel in the fusion of multidimensional visual features; Indicates the location in the color image The influence coefficient of the pixel; Indicates the color image in the first... In the color channels located at The grayscale value of the pixel; This indicates that the pre-recorded standard medicinal material images are in the first... In the color channels located at The influence coefficient of the pixel; This indicates that the pre-recorded standard medicinal material images are in the first... In the color channels located at The grayscale value of the pixel; This indicates that the summation is performed on each color channel.

10. A method for sorting and grading Chinese medicinal materials based on multidimensional visual feature fusion according to claim 1, characterized in that, The grades of Chinese medicinal materials are distinguished based on the aforementioned multi-dimensional similarity, including: Standard medicinal material samples were placed in the shooting area at multiple equally divided angles to collect images of the standard medicinal materials from multiple perspectives. The multi-dimensional similarity between the color image and the standard medicinal material images from all directions is calculated one by one, and the maximum value is selected as the final evaluation score. The grade is divided according to the preset scoring range, and the evaluation score that is closer to 1 corresponds to the higher grade of the medicinal material.

Images