Traditional Chinese medicine recognition system based on fusion image recognition

Abstract

The present invention discloses a Chinese medicine identification system based on fusion image recognition, which belongs to the field of Chinese medicine identification technology and includes: an image acquisition module for acquiring real-time images of Chinese medicine based on computer vision technology; an image processing module for denoising, contrast adjustment, feature extraction and fusion of the real-time images of Chinese medicine to determine the characteristic images of Chinese medicine; a Chinese medicine identification module for performing image recognition on the characteristic images of Chinese medicine according to a Chinese medicine identification model based on fusion image recognition, judging the name of Chinese medicine, and determining the Chinese medicine identification results; a result output module for visually displaying the Chinese medicine identification results and providing an interactive interface for users to perform interactive operations. The present invention solves the problem that existing Chinese medicine identification methods mostly require users to flip through books for search and identification, which is time-consuming and labor-intensive, and cannot quickly and accurately identify Chinese medicine, resulting in low Chinese medicine identification efficiency. The present invention can quickly and accurately identify Chinese medicine, save time and labor, and improve Chinese medicine identification efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of traditional Chinese medicine identification, and in particular to a traditional Chinese medicine identification system based on fusion image recognition. Background Art

[0002] Traditional Chinese medicine is mainly composed of plant medicines, animal medicines and mineral medicines. Since plant medicines account for the majority of traditional Chinese medicines, traditional Chinese medicine is also called Chinese herbal medicine. There are about 5,000 kinds of traditional Chinese medicines used in various places, and the prescriptions formed by combining various medicinal materials are countless. Therefore, it is extremely difficult to quickly identify traditional Chinese medicines.

[0003] Existing methods for identifying traditional Chinese medicine mostly require users to search and identify by flipping through books, which is time-consuming and labor-intensive, and cannot identify traditional Chinese medicine quickly and accurately, resulting in low efficiency in traditional Chinese medicine identification. Summary of the Invention

[0004] The purpose of the present invention is to provide a Chinese medicine identification system based on fusion image recognition, which can quickly and accurately identify Chinese medicine, save time and effort, improve the efficiency of Chinese medicine identification, and solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The Chinese medicine identification system based on fusion image recognition includes:

[0007] An image acquisition module is used to acquire images of traditional Chinese medicine based on computer vision technology and determine real-time images of traditional Chinese medicine based on computer vision;

[0008] Image processing module, used to perform denoising, contrast adjustment, feature extraction and fusion on real-time images of traditional Chinese medicine based on computer vision, and determine the characteristic images of traditional Chinese medicine;

[0009] The Chinese medicine recognition module is used to perform image recognition on the Chinese medicine feature image based on the Chinese medicine recognition model based on fusion image recognition, determine the name of the Chinese medicine, and determine the Chinese medicine recognition result;

[0010] The result output module is used to visualize the results of Chinese medicine identification and provide an interactive interface for users to perform interactive operations.

[0011] Preferably, the image acquisition module includes:

[0012] A Chinese medicine collection unit is used to collect Chinese medicine placed on the collection platform in real time;

[0013] Among them, based on computer vision technology, a camera is used to continuously shoot the traditional Chinese medicine placed on the collection platform at different angles, obtaining continuous pictures of the traditional Chinese medicine at different angles, and then determining the real-time image of the traditional Chinese medicine based on computer vision;

[0014] An image transmission unit, used to transmit real-time images of traditional Chinese medicine based on computer vision;

[0015] Among them, an image transmission link is established between the image acquisition module and the image processing module;

[0016] After the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link, so that the image processing module processes the real-time image of traditional Chinese medicine based on computer vision.

[0017] Preferably, the image processing module includes:

[0018] Image denoising unit, used to denoise real-time images of traditional Chinese medicine based on computer vision;

[0019] Based on the median filter, the median value of the pixels around each pixel in the real-time image of traditional Chinese medicine based on computer vision is calculated to replace the value of the pixel, thereby removing the noise in the real-time image of traditional Chinese medicine based on computer vision and eliminating isolated noise points;

[0020] An image adjustment unit, used for adjusting and processing real-time images of traditional Chinese medicine based on computer vision;

[0021] The grayscale value distribution of the real-time image of traditional Chinese medicine based on computer vision is evened out based on the histogram equalization method, and the contrast of the real-time image of traditional Chinese medicine based on computer vision is adjusted by adjusting the grayscale range of the real-time image of traditional Chinese medicine based on computer vision.

[0022] Preferably, the image processing module further includes:

[0023] A feature extraction unit is used to extract features from the real-time image of traditional Chinese medicine based on computer vision, and to extract useful features from the real-time image of traditional Chinese medicine based on computer vision, including color features, texture features and shape features;

[0024] Based on the color feature description method, the color features of traditional Chinese medicine are described by extracting the color histogram of the real-time image of traditional Chinese medicine based on computer vision;

[0025] Based on the texture feature description method, the gray level co-occurrence matrix is used to describe the texture characteristics of traditional Chinese medicine by statistically analyzing the gray value relationship between pixels at different positions in the real-time image of traditional Chinese medicine based on computer vision.

[0026] Based on the contour feature extraction method, the boundary shape of the Chinese medicine is described by calculating the length, area and perimeter of the Chinese medicine contour to determine the shape characteristics of the Chinese medicine;

[0027] A feature fusion unit is used to fuse the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of the traditional Chinese medicine;

[0028] Based on the weighted fusion algorithm, weights are assigned to the color features, texture features and shape features of the real-time image of traditional Chinese medicine. The weighted color features, texture features and shape features are spliced to form the characteristic image of traditional Chinese medicine.

[0029] Preferably, the traditional Chinese medicine identification module includes:

[0030] Model training unit, used to train the traditional Chinese medicine recognition model based on fusion image recognition;

[0031] According to the needs of traditional Chinese medicine identification based on fusion image recognition, a camera is used to shoot traditional Chinese medicine samples, collect traditional Chinese medicine sample images, and explain the efficacy of the collected traditional Chinese medicine sample images;

[0032] The collected traditional Chinese medicine sample images are divided into training set and test set in a ratio of 7:3;

[0033] Based on deep learning technology, a training set is used to train the deep learning model, so that the deep learning model can autonomously learn to identify Chinese medicine and recognize the name and efficacy of Chinese medicine, and determine the Chinese medicine recognition model based on fusion image recognition;

[0034] Based on the test set, the performance of the traditional Chinese medicine recognition model based on fusion image recognition is tested to determine whether the traditional Chinese medicine recognition model based on fusion image recognition can achieve the expected effect, which is used to verify the generalization ability of the traditional Chinese medicine recognition model based on fusion image recognition.

[0035] Preferably, the traditional Chinese medicine identification module further includes:

[0036] A traditional Chinese medicine identification unit is used to identify traditional Chinese medicine and determine the identification result of traditional Chinese medicine;

[0037] Obtain a traditional Chinese medicine recognition model based on fusion image recognition, and deploy the traditional Chinese medicine recognition model based on fusion image recognition in actual traditional Chinese medicine recognition applications for users to use to identify traditional Chinese medicine;

[0038] The characteristic image of traditional Chinese medicine is used as input and input into the traditional Chinese medicine recognition model based on fusion image recognition. The characteristic image of traditional Chinese medicine is recognized according to the traditional Chinese medicine recognition model based on fusion image recognition, the name of the traditional Chinese medicine is judged, and the traditional Chinese medicine recognition result is determined.

[0039] Preferably, the result output module includes:

[0040] The identification and display unit is used to visually display the TCM identification results, wherein the results are output in the form of text, images or sounds for users to view and use. When the TCM identification results are visually displayed, the names and effects of the TCMs are output;

[0041] When output in text form, the name and efficacy of Chinese medicine are displayed visually;

[0042] When outputting in image form, the name and efficacy of the identified Chinese medicine are marked on the real-time image of the Chinese medicine;

[0043] When output in voice form, the name and efficacy of Chinese medicine are read out.

[0044] Preferably, the result output module further includes:

[0045] The user interaction unit is used for user interactive operations and provides an interactive interface for users to query, save, download and share the results of traditional Chinese medicine identification. The output method of traditional Chinese medicine identification results is adjusted according to user feedback.

[0046] Preferably, the image processing module further includes: the image processing unit, which is used to perform angle analysis and processing on the real-time image of traditional Chinese medicine based on computer vision, including:

[0047] Perform preliminary analysis on the real-time image of traditional Chinese medicine using computer vision to determine whether the real-time image of traditional Chinese medicine has completed the processing of the image denoising unit and the image adjustment unit, and obtain preliminary analysis results;

[0048] When the preliminary analysis result shows that the real-time image of the traditional Chinese medicine has completed the processing of the image denoising unit and the processing of the image adjustment unit, the real-time image of the traditional Chinese medicine is preliminarily identified as a whole, the shooting angle is determined, and the angle information of the real-time image of the traditional Chinese medicine is obtained;

[0049] Acquiring image information of the real-time image of traditional Chinese medicine to obtain image information of the real-time image of traditional Chinese medicine, and filtering the image information of the real-time image of traditional Chinese medicine to filter out image information related to the image of traditional Chinese medicine to obtain target image information of the real-time image of traditional Chinese medicine;

[0050] Matching the real-time image of traditional Chinese medicine with the real-time image angle information of the traditional Chinese medicine, determining the area block corresponding to the image of the traditional Chinese medicine in the real-time image of the traditional Chinese medicine, and obtaining the image area block matching result, including:

[0051] Dividing the image area corresponding to the target image information in the real-time image of traditional Chinese medicine to obtain multiple target image area blocks;

[0052] Determine a main Chinese medicine real-time image according to the number of target image area blocks, obtain Chinese medicine real-time image angle information of the main Chinese medicine real-time image, and determine an angle influencing factor in combination with Chinese medicine real-time image angle information of other Chinese medicine real-time images;

[0053] Get the image features of the target image area block and perform screening and matching using the following formula:

[0054]

[0055] In the above technical solution, Indicates the first The matching results of target image area blocks, Indicates the first target image area blocks eigenvector, Indicates the angle impact factor of the main TCM real-time image, Indicates the Angle impact factor of real-time images of traditional Chinese medicine, Indicates the Zhang real-time image of Chinese medicine target image area blocks eigenvector, Indicates the Zhang real-time image of Chinese medicine target image region blocks, represents the number of eigenvectors, Represents the screening judgment factor;

[0056] Based on the image area block matching results, image quality comparison and analysis are performed according to the target image information to determine the optimal value of the image area block;

[0057] Performing integrity analysis on real-time images of traditional Chinese medicine to determine the existence of complete real-time images of traditional Chinese medicine;

[0058] Perform image quality evaluation on real-time images of traditional Chinese medicine with complete images of traditional Chinese medicine, and determine the best real-time image of traditional Chinese medicine based on the image quality evaluation results;

[0059] According to the optimal value of the image area block, the optimal real-time image of traditional Chinese medicine is corrected according to the area block to obtain the optimal real-time image of traditional Chinese medicine.

[0060] Preferably, when the feature extraction unit performs feature extraction based on the real-time image of traditional Chinese medicine using computer vision, the feature extraction is performed on the best real-time image of traditional Chinese medicine, including:

[0061] Using the first recognition model to perform edge recognition of the Chinese medicine image on the best real-time image of the Chinese medicine, determine the image edge of the Chinese medicine image, and obtain the contour feature information of the Chinese medicine;

[0062] Preliminary analysis of the size of the TCM image based on the image edge of the TCM image is performed to obtain size estimation data of the TCM image;

[0063] An adjustment factor is determined based on the estimated size data of the traditional Chinese medicine image to obtain a first adjustment factor;

[0064] The first adjustment factor is reviewed in combination with the adjustment factor standard. If the first adjustment factor passes the review, the first adjustment factor becomes the target adjustment factor. If the first adjustment factor fails the review, the first adjustment factor is calibrated using the adjustment factor limit value, and the corresponding adjustment factor limit value becomes the target adjustment factor.

[0065] The target area is determined based on the image edge of the traditional Chinese medicine image combined with the target adjustment factor, and the target area is screened out in the best real-time image of the traditional Chinese medicine to obtain the feature extraction target area;

[0066] The second recognition model is used to perform color recognition on the feature extraction target area to obtain the color feature information of traditional Chinese medicine;

[0067] The third recognition model is used to perform texture recognition on the feature extraction target area to obtain the texture feature information of traditional Chinese medicine.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] The present invention collects Chinese medicine images in real time based on computer vision technology, determines the real-time image of Chinese medicine based on computer vision, determines the characteristic image of Chinese medicine by denoising, contrast adjustment, feature extraction and fusion of the real-time image of Chinese medicine based on computer vision, performs image recognition on the characteristic image of Chinese medicine according to a Chinese medicine recognition model based on fusion image recognition, judges the name of Chinese medicine, determines the Chinese medicine recognition result, and visualizes the Chinese medicine recognition result, and provides an interactive interface for users to perform interactive operations, which can quickly and accurately identify Chinese medicine, save time and effort, and improve the efficiency of Chinese medicine recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a module block diagram of the traditional Chinese medicine identification system based on fusion image recognition of the present invention. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] In order to solve the problem that the existing method of identifying Chinese medicine is that users usually search for and identify Chinese medicine by flipping through books, which is time-consuming and labor-intensive, and cannot identify Chinese medicine quickly and accurately, resulting in low efficiency of Chinese medicine identification, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0073] The Chinese medicine identification system based on fusion image recognition includes: an image acquisition module, an image processing module, a Chinese medicine identification module and a result output module.

[0074] It should be noted that, through the interactive communication between the image acquisition module, the image processing module, the traditional Chinese medicine recognition module and the result output module, the traditional Chinese medicine images can be collected based on computer vision technology, and the real-time images of traditional Chinese medicine based on computer vision can be determined. The characteristic images of traditional Chinese medicine can be determined by denoising, contrast adjustment, feature extraction and fusion of the real-time images of traditional Chinese medicine based on computer vision. The characteristic images of traditional Chinese medicine are recognized by the traditional Chinese medicine recognition model based on fusion image recognition, the names of traditional Chinese medicines are judged, the recognition results of traditional Chinese medicines are determined, and finally the recognition results of traditional Chinese medicines are visualized, and an interactive interface is provided for users to perform interactive operations, which can quickly and accurately identify traditional Chinese medicines, save time and effort, and improve the efficiency of traditional Chinese medicine recognition.

[0075] In this embodiment, as a preferred technical solution of the present invention, the image acquisition module includes:

[0076] A Chinese medicine collection unit is used to collect Chinese medicine placed on the collection platform in real time;

[0077] Among them, based on computer vision technology, a camera is used to continuously shoot the traditional Chinese medicine placed on the collection platform at different angles, obtaining continuous pictures of the traditional Chinese medicine at different angles, and then determining the real-time image of the traditional Chinese medicine based on computer vision;

[0078] It should be noted that when using a camera to photograph the traditional Chinese medicine placed on the collection platform, the position of the camera is adjusted to continuously photograph the traditional Chinese medicine placed on the collection platform at different angles, so as to obtain continuous pictures of the traditional Chinese medicine at different angles, facilitate all-round identification of the traditional Chinese medicine, and make the traditional Chinese medicine identification results more accurate.

[0079] An image transmission unit, used to transmit real-time images of traditional Chinese medicine based on computer vision;

[0080] Among them, an image transmission link is established between the image acquisition module and the image processing module;

[0081] After the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link, so that the image processing module processes the real-time image of traditional Chinese medicine based on computer vision.

[0082] It should be noted that after the image acquisition module collects the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits an instruction requesting to establish an image transmission link to the image processing module. After the image processing module receives the instruction requesting to establish an image transmission link transmitted by the image acquisition module, the image processing module transmits an instruction agreeing to establish the image transmission link to the image acquisition module. After the image acquisition module receives the instruction agreeing to establish the image transmission link transmitted by the image processing module, the image acquisition module and the image processing module establish an image transmission link, and the image acquisition module transmits the collected real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link.

[0083] In this embodiment, as a preferred technical solution of the present invention, the image processing module includes:

[0084] Image denoising unit, used to denoise real-time images of traditional Chinese medicine based on computer vision;

[0085] Based on the median filter, the median value of the pixels around each pixel in the real-time image of traditional Chinese medicine based on computer vision is calculated to replace the value of the pixel, thereby removing the noise in the real-time image of traditional Chinese medicine based on computer vision and eliminating isolated noise points;

[0086] It should be noted that, by performing denoising processing on the real-time images of traditional Chinese medicine based on computer vision, the noise in the real-time images of traditional Chinese medicine based on computer vision can be effectively reduced, and the quality of the real-time images of traditional Chinese medicine based on computer vision can be improved.

[0087] An image adjustment unit, used for adjusting and processing real-time images of traditional Chinese medicine based on computer vision;

[0088] The grayscale value distribution of the real-time image of traditional Chinese medicine based on computer vision is evened out based on the histogram equalization method, and the contrast of the real-time image of traditional Chinese medicine based on computer vision is adjusted by adjusting the grayscale range of the real-time image of traditional Chinese medicine based on computer vision.

[0089] It should be noted that by adjusting the contrast of the real-time image of traditional Chinese medicine based on computer vision, the light and dark contrast in the real-time image of traditional Chinese medicine based on computer vision can be made more obvious, and the visualization effect of the real-time image of traditional Chinese medicine based on computer vision can be improved.

[0090] In this embodiment, as a preferred technical solution of the present invention, the image processing module further includes:

[0091] A feature extraction unit is used to extract features from the real-time image of traditional Chinese medicine based on computer vision, and to extract useful features from the real-time image of traditional Chinese medicine based on computer vision, including color features, texture features and shape features;

[0092] Based on the color feature description method, the color features of traditional Chinese medicine are described by extracting the color histogram of the real-time image of traditional Chinese medicine based on computer vision;

[0093] Based on the texture feature description method, the gray level co-occurrence matrix is used to describe the texture characteristics of traditional Chinese medicine by statistically analyzing the gray value relationship between pixels at different positions in the real-time image of traditional Chinese medicine based on computer vision.

[0094] Based on the contour feature extraction method, the boundary shape of the Chinese medicine is described by calculating the length, area and perimeter of the Chinese medicine contour to determine the shape characteristics of the Chinese medicine;

[0095] It should be noted that by performing feature extraction on real-time images of traditional Chinese medicine based on computer vision, color features, texture features and shape features can be extracted to facilitate subsequent identification of traditional Chinese medicine.

[0096] A feature fusion unit is used to fuse the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of the traditional Chinese medicine;

[0097] Based on the weighted fusion algorithm, weights are assigned to the color features, texture features and shape features of the real-time image of traditional Chinese medicine. The weighted color features, texture features and shape features are spliced to form the characteristic image of traditional Chinese medicine.

[0098] It should be noted that by fusing the color features, texture features and shape features of the real-time images of traditional Chinese medicine to determine the characteristic images of traditional Chinese medicine, redundant information can be reduced and the effectiveness of the feature vector can be improved. This can not only retain the information of the original features, but also increase the dimension of the feature vector.

[0099] In this embodiment, as a preferred technical solution of the present invention, the traditional Chinese medicine identification module includes:

[0100] Model training unit, used to train the traditional Chinese medicine recognition model based on fusion image recognition;

[0101] According to the needs of traditional Chinese medicine identification based on fusion image recognition, a camera is used to shoot traditional Chinese medicine samples, collect traditional Chinese medicine sample images, and explain the efficacy of the collected traditional Chinese medicine sample images;

[0102] The collected traditional Chinese medicine sample images are divided into training set and test set in a ratio of 7:3;

[0103] Based on deep learning technology, a training set is used to train the deep learning model, so that the deep learning model can autonomously learn to identify Chinese medicine and recognize the name and efficacy of Chinese medicine, and determine the Chinese medicine recognition model based on fusion image recognition;

[0104] Based on the test set, the performance of the traditional Chinese medicine recognition model based on fusion image recognition is tested to determine whether the traditional Chinese medicine recognition model based on fusion image recognition can achieve the expected effect, which is used to verify the generalization ability of the traditional Chinese medicine recognition model based on fusion image recognition.

[0105] It should be noted that based on deep learning technology, a traditional Chinese medicine recognition model based on fusion image recognition is trained to facilitate the subsequent accurate identification of traditional Chinese medicine.

[0106] In this embodiment, as a preferred technical solution of the present invention, the traditional Chinese medicine identification module further includes:

[0107] A traditional Chinese medicine identification unit is used to identify traditional Chinese medicine and determine the identification result of traditional Chinese medicine;

[0108] Obtain a traditional Chinese medicine recognition model based on fusion image recognition, and deploy the traditional Chinese medicine recognition model based on fusion image recognition in actual traditional Chinese medicine recognition applications for users to use to identify traditional Chinese medicine;

[0109] The characteristic image of traditional Chinese medicine is used as input and input into the traditional Chinese medicine recognition model based on fusion image recognition. The characteristic image of traditional Chinese medicine is recognized according to the traditional Chinese medicine recognition model based on fusion image recognition, the name of the traditional Chinese medicine is judged, and the traditional Chinese medicine recognition result is determined.

[0110] It should be noted that by performing image recognition on the characteristic images of traditional Chinese medicines based on the traditional Chinese medicine recognition model based on fusion image recognition, judging the names of traditional Chinese medicines and determining the identification results of traditional Chinese medicines, traditional Chinese medicines can be identified quickly and accurately, saving time and effort and improving the efficiency of traditional Chinese medicine recognition.

[0111] In this embodiment, as a preferred technical solution of the present invention, the result output module includes:

[0112] The identification and display unit is used to visually display the TCM identification results, wherein the results are output in the form of text, images or sounds for users to view and use. When the TCM identification results are visually displayed, the names and effects of the TCMs are output;

[0113] When output in text form, the name and efficacy of Chinese medicine are displayed visually;

[0114] When outputting in image form, the name and efficacy of the identified Chinese medicine are marked on the real-time image of the Chinese medicine;

[0115] When output in voice form, the name and efficacy of Chinese medicine are read out.

[0116] In this embodiment, as a preferred technical solution of the present invention, the result output module further includes:

[0117] The user interaction unit is used for user interactive operations and provides an interactive interface for users to query, save, download and share the results of traditional Chinese medicine identification. The output method of traditional Chinese medicine identification results is adjusted according to user feedback.

[0118] In this embodiment, as a preferred technical solution of the present invention, the image processing module further includes: the image processing unit is used to perform angle analysis and processing on the real-time image of traditional Chinese medicine based on computer vision, including:

[0119] Perform preliminary analysis on the real-time image of traditional Chinese medicine using computer vision to determine whether the real-time image of traditional Chinese medicine has completed the processing of the image denoising unit and the image adjustment unit, and obtain preliminary analysis results;

[0120] When the preliminary analysis result shows that the real-time image of the traditional Chinese medicine has completed the processing of the image denoising unit and the processing of the image adjustment unit, the real-time image of the traditional Chinese medicine is preliminarily identified as a whole, the shooting angle is determined, and the angle information of the real-time image of the traditional Chinese medicine is obtained;

[0121] Acquiring image information of the real-time image of traditional Chinese medicine to obtain image information of the real-time image of traditional Chinese medicine, and filtering the image information of the real-time image of traditional Chinese medicine to filter out image information related to the image of traditional Chinese medicine to obtain target image information of the real-time image of traditional Chinese medicine;

[0122] Matching the real-time image of traditional Chinese medicine with the real-time image angle information of the traditional Chinese medicine, determining the area block corresponding to the image of the traditional Chinese medicine in the real-time image of the traditional Chinese medicine, and obtaining the image area block matching result, including:

[0123] Dividing the image area corresponding to the target image information in the real-time image of traditional Chinese medicine to obtain multiple target image area blocks;

[0124] Determine a main Chinese medicine real-time image according to the number of target image area blocks, obtain Chinese medicine real-time image angle information of the main Chinese medicine real-time image, and determine an angle influencing factor in combination with Chinese medicine real-time image angle information of other Chinese medicine real-time images;

[0125] Get the image features of the target image area block and perform screening and matching using the following formula:

[0126]

[0127] In the above technical solution, Indicates the first The matching results of target image area blocks, Indicates the first target image area blocks eigenvector, Indicates the angle impact factor of the main TCM real-time image, Indicates the Angle impact factor of real-time images of traditional Chinese medicine, Indicates the Zhang real-time image of Chinese medicine target image area blocks eigenvector, Indicates the Zhang real-time image of Chinese medicine target image region blocks, represents the number of eigenvectors, Represents the screening judgment factor;

[0128] Based on the image area block matching results, image quality comparison and analysis are performed according to the target image information to determine the optimal value of the image area block;

[0129] Performing integrity analysis on real-time images of traditional Chinese medicine to determine the existence of complete real-time images of traditional Chinese medicine;

[0130] Perform image quality evaluation on real-time images of traditional Chinese medicine with complete images of traditional Chinese medicine, and determine the best real-time image of traditional Chinese medicine based on the image quality evaluation results;

[0131] According to the optimal value of the image area block, the optimal real-time image of traditional Chinese medicine is corrected according to the area block to obtain the optimal real-time image of traditional Chinese medicine.

[0132] Among them, image features refer to the image features of traditional Chinese medicine in the image, including: line curvature information, combination structure information, etc. The characteristic vectors include line curvature information vectors, combination construction information vectors, and the like. It is composed of multiple The eigenvectors constitute the Zhang real-time image of Chinese medicine target image area blocks. and The numbering is based on the target image region blocks of the real-time Chinese medicine image, and the value is a positive integer. The number of target image region blocks in different real-time Chinese medicine images can be the same or different. It is a number assigned to the real-time image of traditional Chinese medicine, and its value is a positive integer. The value of is a non-negative number close to 0. The information includes: image clarity, brightness and other parameter information.

[0133] It should be noted that, by conducting preliminary analysis on the real-time images of traditional Chinese medicine for computer vision, it is ensured that the real-time images of traditional Chinese medicine for overall preliminary identification of traditional Chinese medicine images have completed denoising and adjustment processing, and by screening the image information of the real-time images of traditional Chinese medicine to remove the interference of irrelevant information, only the images of traditional Chinese medicine are analyzed during image processing, which reduces the degree of confusion in information analysis and the probability of error in information analysis. The real-time images of traditional Chinese medicine are matched with the angle information of the real-time images of traditional Chinese medicine, and the influence of the image angle is fully considered to avoid deviations in matching caused by different image angles, thereby affecting the comparative analysis of image quality, and ensuring the accuracy and comprehensiveness of the best real-time images of traditional Chinese medicine, thereby providing guarantees for subsequent feature extraction and judgment of the names of traditional Chinese medicine, and improving the accuracy of the traditional Chinese medicine recognition system. In addition, by performing integrity analysis and image quality evaluation on real-time images of traditional Chinese medicine, real-time images of traditional Chinese medicine with high image quality and complete images of traditional Chinese medicine are corrected, ensuring that the best real-time images of traditional Chinese medicine have complete images of traditional Chinese medicine, effectively improving the image quality of the best real-time images of traditional Chinese medicine, so that when performing feature extraction, only feature acquisition needs to be performed on the best real-time images of traditional Chinese medicine, reducing time consumption, reducing the response time of the image processing module of the real-time images of traditional Chinese medicine, realizing real-time processing, and facilitating subsequent accurate identification of traditional Chinese medicine.

[0134] In this embodiment, as a preferred technical solution of the present invention, when the feature extraction unit performs feature extraction based on the real-time image of traditional Chinese medicine using computer vision, feature extraction is performed on the best real-time image of traditional Chinese medicine, including:

[0135] Using the first recognition model to perform edge recognition of the Chinese medicine image on the best real-time image of the Chinese medicine, determine the image edge of the Chinese medicine image, and obtain the contour feature information of the Chinese medicine;

[0136] Preliminary analysis of the size of the TCM image based on the image edge of the TCM image is performed to obtain size estimation data of the TCM image;

[0137] An adjustment factor is determined based on the estimated size data of the traditional Chinese medicine image to obtain a first adjustment factor;

[0138] The first adjustment factor is reviewed in combination with the adjustment factor standard. If the first adjustment factor passes the review, the first adjustment factor becomes the target adjustment factor. If the first adjustment factor fails the review, the first adjustment factor is calibrated using the adjustment factor limit value, and the corresponding adjustment factor limit value becomes the target adjustment factor.

[0139] The target area is determined based on the image edge of the traditional Chinese medicine image combined with the target adjustment factor, and the target area is screened out in the best real-time image of the traditional Chinese medicine to obtain the feature extraction target area;

[0140] The second recognition model is used to perform color recognition on the feature extraction target area to obtain the color feature information of traditional Chinese medicine;

[0141] The third recognition model is used to perform texture recognition on the feature extraction target area to obtain the texture feature information of traditional Chinese medicine.

[0142] Among them, the first recognition model is a contour recognition module, which performs edge recognition on the Chinese medicine image in the best real-time image of Chinese medicine. The second recognition model is a color recognition model, which performs color recognition on the feature extraction target area. The third recognition model is a texture recognition model, which performs texture recognition on the feature extraction target area. In addition, the first recognition model, the second recognition model, and the third recognition model are trained and optimized before use. The larger the size estimate data of the Chinese medicine image, the smaller the adjustment factor. The adjustment factor is a number greater than 0 and less than 1. The adjustment factor standard is an interval number in the range of 0-1, with maximum and minimum values.

[0143] It should be noted that by extracting features from the optimal real-time image of a traditional Chinese medicine, useful features can be obtained from a single optimal real-time image of the medicine, reducing the workload of feature extraction and enabling the characteristics of the medicine to be obtained in a relatively short time. Furthermore, through the first, second, and third recognition models, artificial intelligence is used to acquire the contour, color, and texture feature information of the traditional Chinese medicine. This not only accurately and comprehensively obtains the corresponding feature information, but also reduces time and effort, thereby improving the efficiency of traditional Chinese medicine recognition. Furthermore, when determining the target region for feature extraction, a target adjustment factor is considered to reduce the redundancy of irrelevant images in the optimal real-time image of the traditional Chinese medicine and the proportion of invalid regions identified by the second and third recognition models. This ensures the comprehensiveness of the traditional Chinese medicine image while effectively reducing the interference of irrelevant information. Furthermore, the target adjustment factor can be dynamically adjusted based on the estimated size of the traditional Chinese medicine image, preventing larger traditional Chinese medicines from having smaller margins or smaller traditional Chinese medicines from having larger margins, which affects the traditional Chinese medicine image itself and reduces recognition errors. Furthermore, the adjustment factor standard limits the adjustment factor from unlimited fluctuations based on the actual situation of the traditional Chinese medicine, ensuring the feasibility of the adjustment factor.

[0144] In summary, the real-time collection of traditional Chinese medicine images is carried out based on computer vision technology, and the real-time images of traditional Chinese medicine based on computer vision are determined. The real-time images of traditional Chinese medicine based on computer vision are denoised, contrast adjusted, feature extracted and fused to determine the characteristic images of traditional Chinese medicine. The characteristic images of traditional Chinese medicine are recognized according to the traditional Chinese medicine recognition model based on fusion image recognition, the names of traditional Chinese medicines are judged, the recognition results of traditional Chinese medicines are determined, and the recognition results of traditional Chinese medicines are visualized. An interactive interface is provided for users to perform interactive operations, which can quickly and accurately identify traditional Chinese medicines, save time and effort, and improve the efficiency of traditional Chinese medicine recognition.

[0145] Among them, the traditional Chinese medicine identification system has broad application prospects, and can provide strong technical support for the modernization, standardization and internationalization of traditional Chinese medicine. It can be applied to multiple links such as traditional Chinese medicine planting, processing, storage, and sales. For example, in the process of traditional Chinese medicine planting, it can be used to identify the name and efficacy of traditional Chinese medicine; in the process of traditional Chinese medicine processing, it can be used to identify the type of traditional Chinese medicine; in the process of traditional Chinese medicine storage, it can be used to identify the name of traditional Chinese medicine; in the process of traditional Chinese medicine sales, it can be used to identify the authenticity of traditional Chinese medicine.

[0146] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0147] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A Chinese medicine identification system based on fusion image recognition is characterized by: include: Image acquisition module, collects images of traditional Chinese medicine and determines the real-time images of traditional Chinese medicine; The image processing module performs denoising, contrast adjustment, feature extraction and fusion on the real-time image of traditional Chinese medicine to determine the characteristic image of traditional Chinese medicine; The Chinese medicine recognition module recognizes the characteristic images of Chinese medicine according to the Chinese medicine recognition model, determines the name of the Chinese medicine, and determines the Chinese medicine recognition result; The result output module visualizes the TCM identification results and provides an interactive interface for users to interact; The image processing module further includes: a feature extraction unit for extracting useful features from the real-time image of traditional Chinese medicine; The image processing module further includes: an image processing unit for performing angle analysis and processing on the real-time image of traditional Chinese medicine, including: Conduct preliminary analysis on the real-time images of traditional Chinese medicine to determine whether the real-time images of traditional Chinese medicine have completed the processing of denoising and contrast adjustment, and obtain preliminary analysis results; When the preliminary analysis result shows that the real-time image of the traditional Chinese medicine has completed the processing of denoising and contrast adjustment, the real-time image of the traditional Chinese medicine is preliminarily identified as a whole, the shooting angle is determined, and the angle information of the real-time image of the traditional Chinese medicine is obtained; Acquiring image information of the real-time image of traditional Chinese medicine to obtain image information of the real-time image of traditional Chinese medicine, and filtering the image information of the real-time image of traditional Chinese medicine to filter out image information related to the image of traditional Chinese medicine to obtain target image information of the real-time image of traditional Chinese medicine; Matching the real-time image of traditional Chinese medicine with the real-time image angle information of the traditional Chinese medicine, determining the area block corresponding to the image of the traditional Chinese medicine in the real-time image of the traditional Chinese medicine, and obtaining the image area block matching result, including: Dividing the image area corresponding to the target image information in the real-time image of traditional Chinese medicine to obtain multiple target image area blocks; Determine the main Chinese medicine real-time image according to the number of target image area blocks, obtain angle information of the main Chinese medicine real-time image, and determine the angle influencing factor in combination with the angle information of other Chinese medicine real-time images; Get the image features of the target image area block and perform screening and matching using the following formula: in, Indicates the first The matching results of target image area blocks, Indicates the first target image area blocks eigenvector, Indicates the angle impact factor of the main TCM real-time image, Indicates the Angle impact factor of real-time images of traditional Chinese medicine, Indicates the Zhang real-time image of Chinese medicine target image area blocks eigenvector, Indicates the Zhang real-time image of Chinese medicine target image region blocks, represents the number of eigenvectors, Represents the screening judgment factor; Based on the image area block matching results, image quality comparison and analysis are performed according to the target image information to determine the optimal value of the image area block; Performing a Chinese medicine image integrity analysis on real-time images of Chinese medicine to determine whether there are real-time images of Chinese medicine with complete images; Perform image quality evaluation on real-time images of traditional Chinese medicine that have complete images of traditional Chinese medicine, and determine the best real-time image of traditional Chinese medicine based on the image quality evaluation results; Correcting the best real-time image of traditional Chinese medicine according to the best value of the image area block to obtain the best real-time image of traditional Chinese medicine; The feature extraction unit extracts features from the best real-time image of traditional Chinese medicine.

2. The Chinese medicine identification system based on fusion image recognition according to claim 1, characterized in that: The image acquisition module includes: A Chinese medicine collection unit is used to collect Chinese medicine placed on the collection platform in real time; Among them, based on computer vision technology, a camera is used to continuously shoot the traditional Chinese medicine placed on the collection platform at different angles, obtain continuous pictures of the traditional Chinese medicine at different angles, and determine the real-time image of the traditional Chinese medicine based on computer vision; An image transmission unit, used to transmit real-time images of traditional Chinese medicine based on computer vision; Among them, an image transmission link is established between the image acquisition module and the image processing module; After the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link, so that the image processing module processes the real-time image of traditional Chinese medicine based on computer vision.

3. The Chinese medicine identification system based on fusion image recognition according to claim 2, characterized in that: The image processing module includes: Image denoising unit, used to denoise real-time images of traditional Chinese medicine based on computer vision; Based on the median filter, the median value of the pixels around each pixel in the real-time image of traditional Chinese medicine based on computer vision is calculated to replace the value of the pixel, thereby removing the noise in the real-time image of traditional Chinese medicine based on computer vision and eliminating isolated noise points; An image adjustment unit, used for adjusting and processing real-time images of traditional Chinese medicine based on computer vision; The grayscale value distribution of the real-time image of traditional Chinese medicine based on computer vision is evened out based on the histogram equalization method, and the contrast of the real-time image of traditional Chinese medicine based on computer vision is adjusted by adjusting the grayscale range of the real-time image of traditional Chinese medicine based on computer vision.

4. The Chinese medicine identification system based on fusion image recognition according to claim 3, characterized in that: The image processing module further includes: A feature extraction unit is used to extract features from the real-time image of traditional Chinese medicine based on computer vision, and to extract useful features from the real-time image of traditional Chinese medicine based on computer vision, including color features, texture features and shape features; Based on the color feature description method, the color features of traditional Chinese medicine are described by extracting the color histogram of the real-time image of traditional Chinese medicine based on computer vision; Based on the texture feature description method, the gray level co-occurrence matrix is used to describe the texture characteristics of traditional Chinese medicine by statistically analyzing the gray value relationship between pixels at different positions in the real-time image of traditional Chinese medicine based on computer vision. Based on the contour feature extraction method, the boundary shape of the Chinese medicine is described by calculating the length, area and perimeter of the Chinese medicine contour, and the shape characteristics of the Chinese medicine are determined; A feature fusion unit is used to fuse the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of the traditional Chinese medicine; Based on the weighted fusion algorithm, weights are assigned to the color features, texture features and shape features of the real-time image of traditional Chinese medicine. The weighted color features, texture features and shape features are spliced to form the characteristic image of traditional Chinese medicine.

5. The Chinese medicine identification system based on fusion image recognition according to claim 4, characterized in that: The traditional Chinese medicine identification module includes: Model training unit, used to train the traditional Chinese medicine recognition model based on fusion image recognition; According to the needs of traditional Chinese medicine identification based on fusion image recognition, a camera is used to shoot traditional Chinese medicine samples, collect traditional Chinese medicine sample images, and explain the efficacy of the collected traditional Chinese medicine sample images; The collected traditional Chinese medicine sample images are divided into training set and test set in a ratio of 7:3; Based on deep learning technology, a training set is used to train the deep learning model, so that the deep learning model can autonomously learn to identify Chinese medicine and recognize the name and efficacy of Chinese medicine, and determine the Chinese medicine recognition model based on fusion image recognition; Based on the test set, the performance of the traditional Chinese medicine recognition model based on fusion image recognition is tested to determine whether the traditional Chinese medicine recognition model based on fusion image recognition can achieve the expected effect, which is used to verify the generalization ability of the traditional Chinese medicine recognition model based on fusion image recognition.

6. The Chinese medicine identification system based on fusion image recognition according to claim 5, characterized in that: The traditional Chinese medicine identification module further includes: A traditional Chinese medicine identification unit is used to identify traditional Chinese medicine and determine the identification result of traditional Chinese medicine; Obtain a traditional Chinese medicine recognition model based on fusion image recognition, and deploy the traditional Chinese medicine recognition model based on fusion image recognition in actual traditional Chinese medicine recognition applications for users to use to identify traditional Chinese medicine; The characteristic image of traditional Chinese medicine is used as input and input into the traditional Chinese medicine recognition model based on fusion image recognition. The characteristic image of traditional Chinese medicine is recognized according to the traditional Chinese medicine recognition model based on fusion image recognition, the name of the traditional Chinese medicine is judged, and the traditional Chinese medicine recognition result is determined.

7. The Chinese medicine identification system based on fusion image recognition according to claim 6, characterized in that: The result output module includes: The identification and display unit is used to visually display the TCM identification results, wherein the results are output in the form of text, images or sounds for users to view and use. When the TCM identification results are visually displayed, the names and effects of the TCMs are output; When output in text form, the name and efficacy of Chinese medicine are displayed visually; When outputting in image form, the name and efficacy of the identified Chinese medicine are marked on the real-time image of the Chinese medicine; When output in voice form, the name and efficacy of Chinese medicine are read out.

8. The Chinese medicine identification system based on fusion image recognition according to claim 7, characterized in that: The result output module further includes: The user interaction unit is used for user interactive operations and provides an interactive interface for users to query, save, download and share the results of traditional Chinese medicine identification. The output method of traditional Chinese medicine identification results is adjusted according to user feedback.

9. The Chinese medicine identification system based on fusion image recognition according to claim 8, characterized in that: When the feature extraction unit performs feature extraction based on the real-time image of the traditional Chinese medicine using computer vision, the feature extraction is performed on the best real-time image of the traditional Chinese medicine, including: Using the first recognition model to perform edge recognition of the Chinese medicine image on the best real-time image of the Chinese medicine, determine the image edge of the Chinese medicine image, and obtain the contour feature information of the Chinese medicine; Preliminary analysis of the size of the TCM image based on the image edge of the TCM image is performed to obtain size estimation data of the TCM image; An adjustment factor is determined based on the estimated size data of the traditional Chinese medicine image to obtain a first adjustment factor; The first adjustment factor is reviewed in combination with the adjustment factor standard. If the first adjustment factor passes the review, the first adjustment factor becomes the target adjustment factor. If the first adjustment factor fails the review, the first adjustment factor is calibrated using the adjustment factor limit value, and the corresponding adjustment factor limit value becomes the target adjustment factor. The target area is determined based on the image edge of the traditional Chinese medicine image combined with the target adjustment factor, and the target area is screened out in the best real-time image of the traditional Chinese medicine to obtain the feature extraction target area; The second recognition model is used to perform color recognition on the feature extraction target area to obtain the color feature information of traditional Chinese medicine; The third recognition model is used to perform texture recognition on the feature extraction target area to obtain the texture feature information of traditional Chinese medicine.

Images