Traditional Chinese medicine recognition system based on fusion image recognition

Abstract

The invention discloses a traditional Chinese medicine recognition system based on fusion image recognition, and belongs to the technical field of traditional Chinese medicine recognition. The image processing module is used for performing denoising, contrast adjustment, feature extraction and fusion on the traditional Chinese medicine real-time image to determine a traditional Chinese medicine feature image; the traditional Chinese medicine recognition module is used for performing image recognition on the traditional Chinese medicine feature images according to a traditional Chinese medicine recognition model based on fusion image recognition, judging traditional Chinese medicine names and determining traditional Chinese medicine recognition results; and the result output module is used for visually displaying the traditional Chinese medicine recognition result, providing an interactive interface and carrying out interactive operation by a user. The traditional Chinese medicine recognition method and device solve the problem that the traditional Chinese medicine recognition efficiency is low due to the fact that time and labor are consumed and traditional Chinese medicine cannot be quickly and accurately recognized when most traditional Chinese medicine recognition modes are used for searching and recognizing books browsed by users. The traditional Chinese medicine can be quickly and accurately recognized, time and labor are saved, and the traditional Chinese medicine recognition efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of traditional Chinese medicine identification, and in particular to a traditional Chinese medicine identification system based on fusion image recognition. Background Art

[0002] Traditional Chinese medicine is mainly composed of plant medicine, animal medicine and mineral medicine. Because plant medicine accounts for the majority of traditional Chinese medicine, it is also called Chinese herbal medicine. There are about 5,000 kinds of traditional Chinese medicine used in various places, and the prescriptions formed by combining various medicinal materials are countless. Therefore, it is extremely difficult to quickly identify traditional Chinese medicine.

[0003] The existing method of identifying traditional Chinese medicine mostly requires users to search and identify by flipping through books, which is time-consuming and labor-intensive, and cannot identify traditional Chinese medicine quickly and accurately, resulting in low efficiency in traditional Chinese medicine identification. Summary of the invention

[0004] The purpose of the present invention is to provide a Chinese medicine identification system based on fusion image recognition, which can quickly and accurately identify Chinese medicine, save time and effort, improve the efficiency of Chinese medicine identification, and solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The Chinese medicine recognition system based on fusion image recognition includes:

[0007] An image acquisition module is used to acquire images of traditional Chinese medicine based on computer vision technology and determine real-time images of traditional Chinese medicine based on computer vision;

[0008] The image processing module is used to perform denoising, contrast adjustment, feature extraction and fusion on the real-time image of traditional Chinese medicine based on computer vision, and determine the characteristic image of traditional Chinese medicine;

[0009] The Chinese medicine recognition module is used to perform image recognition on the Chinese medicine feature image according to the Chinese medicine recognition model based on fusion image recognition, judge the name of the Chinese medicine, and determine the Chinese medicine recognition result;

[0010] The result output module is used to visualize the results of Chinese medicine identification and provide an interactive interface for users to perform interactive operations.

[0011] Preferably, the image acquisition module comprises:

[0012] A Chinese medicine collection unit is used to collect the Chinese medicine placed on the collection platform in real time;

[0013] Among them, based on computer vision technology, a camera is used to continuously shoot the traditional Chinese medicine placed on the collection platform at different angles, and continuous pictures of the traditional Chinese medicine at different angles are obtained, and then the real-time image of the traditional Chinese medicine based on computer vision is determined;

[0014] An image transmission unit, used to transmit real-time images of traditional Chinese medicine based on computer vision;

[0015] Among them, an image transmission link is established between the image acquisition module and the image processing module;

[0016] After the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link, so that the image processing module processes the real-time image of traditional Chinese medicine based on computer vision.

[0017] Preferably, the image processing module includes:

[0018] An image denoising unit is used to denoise real-time images of traditional Chinese medicine based on computer vision;

[0019] Based on the median filter, the median of the pixels around each pixel in the real-time image of traditional Chinese medicine based on computer vision is calculated to replace the value of the pixel, thereby removing the noise in the real-time image of traditional Chinese medicine based on computer vision and eliminating isolated noise points;

[0020] An image adjustment unit, used for adjusting and processing the real-time image of traditional Chinese medicine based on computer vision;

[0021] The gray value distribution of the real-time image of traditional Chinese medicine based on computer vision is evened out based on the histogram equalization method, and the contrast of the real-time image of traditional Chinese medicine based on computer vision is adjusted by adjusting the gray range of the real-time image of traditional Chinese medicine based on computer vision.

[0022] Preferably, the image processing module further includes:

[0023] A feature extraction unit is used to extract features from the real-time image of traditional Chinese medicine based on computer vision, and to extract useful features from the real-time image of traditional Chinese medicine based on computer vision, including color features, texture features and shape features;

[0024] Based on the color feature description method, the color features of Chinese medicine are described by extracting the color histogram of the real-time image of Chinese medicine based on computer vision;

[0025] Based on the texture feature description method, the gray-level co-occurrence matrix is ​​used to describe the texture features of traditional Chinese medicine by statistically analyzing the gray-level value relationship between pixels at different positions in the real-time image of traditional Chinese medicine based on computer vision.

[0026] Based on the contour feature extraction method, the boundary shape of the Chinese medicine is described by calculating the length, area and perimeter of the Chinese medicine contour to determine the shape characteristics of the Chinese medicine;

[0027] A feature fusion unit is used to fuse the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of traditional Chinese medicine;

[0028] Based on the weighted fusion algorithm, weights are assigned to the color features, texture features and shape features of the real-time image of traditional Chinese medicine, and the weighted color features, texture features and shape features are spliced ​​to form a characteristic image of traditional Chinese medicine.

[0029] Preferably, the traditional Chinese medicine identification module includes:

[0030] Model training unit, used to train the traditional Chinese medicine recognition model based on fusion image recognition;

[0031] According to the needs of Chinese medicine identification based on fusion image recognition, a camera is used to shoot Chinese medicine samples, collect Chinese medicine sample images, and explain the efficacy of the collected Chinese medicine sample images;

[0032] The collected Chinese medicine sample images are divided into training set and test set in a ratio of 7:3;

[0033] Based on deep learning technology, the deep learning model is trained with a training set, so that the deep learning model can autonomously learn Chinese medicine recognition and identify the name and efficacy of Chinese medicine, and determine the Chinese medicine recognition model based on fusion image recognition;

[0034] Based on the test set, the performance of the Chinese medicine recognition model based on fusion image recognition is tested to determine whether the Chinese medicine recognition model based on fusion image recognition can achieve the expected effect, which is used to verify the generalization ability of the Chinese medicine recognition model based on fusion image recognition.

[0035] Preferably, the traditional Chinese medicine identification module further includes:

[0036] A Chinese medicine identification unit is used to identify the Chinese medicine and determine the identification result of the Chinese medicine;

[0037] Obtain a traditional Chinese medicine recognition model based on fusion image recognition, and deploy the traditional Chinese medicine recognition model based on fusion image recognition in actual traditional Chinese medicine recognition applications for users to use to identify traditional Chinese medicine;

[0038] The characteristic image of traditional Chinese medicine is taken as input and input into the traditional Chinese medicine recognition model based on fusion image recognition. The characteristic image of traditional Chinese medicine is subjected to image recognition according to the traditional Chinese medicine recognition model based on fusion image recognition, the name of traditional Chinese medicine is judged, and the recognition result of traditional Chinese medicine is determined.

[0039] Preferably, the result output module includes:

[0040] The identification and display unit is used to visually display the TCM identification results, wherein the results are output in the form of text, images or sounds for users to view and use. When the TCM identification results are visually displayed, the names and effects of the TCM are output;

[0041] When output in text form, the name and efficacy of Chinese medicine are displayed visually;

[0042] When outputting in image form, the name and efficacy of the identified Chinese medicine are marked on the real-time image of the Chinese medicine;

[0043] When output in voice form, the name and efficacy of Chinese medicine are read out.

[0044] Preferably, the result output module further includes:

[0045] The user interaction unit is used for user interactive operations and provides an interactive interface for users to query, save, download and share the results of Chinese medicine identification. The output method of Chinese medicine identification results is adjusted according to user feedback.

[0046] Preferably, the image processing module further includes: the image processing unit is used to perform angle analysis and processing on the real-time image of traditional Chinese medicine based on computer vision, including:

[0047] Conduct a preliminary analysis on the real-time image of traditional Chinese medicine using computer vision to determine whether the real-time image of traditional Chinese medicine has completed the processing of the image denoising unit and the image adjustment unit, and obtain a preliminary analysis result;

[0048] When the preliminary analysis result shows that the real-time image of the traditional Chinese medicine has completed the processing of the image denoising unit and the processing of the image adjustment unit, the real-time image of the traditional Chinese medicine is subjected to preliminary recognition of the image of the traditional Chinese medicine as a whole, the shooting angle is determined, and the angle information of the real-time image of the traditional Chinese medicine is obtained;

[0049] Acquiring image information of the real-time image of traditional Chinese medicine to obtain image information of the real-time image of traditional Chinese medicine, and screening the image information of the real-time image of traditional Chinese medicine to screen out image information about the image of traditional Chinese medicine to obtain target image information of the real-time image of traditional Chinese medicine;

[0050] The real-time image of the traditional Chinese medicine is matched with the angle information of the real-time image of the traditional Chinese medicine to determine the area block corresponding to the image of the traditional Chinese medicine in the real-time image of the traditional Chinese medicine, and the image area block matching result is obtained, including:

[0051] Dividing the image area corresponding to the target image information in the real-time image of traditional Chinese medicine to obtain a plurality of target image area blocks;

[0052] Determine the main Chinese medicine real-time image according to the number of target image area blocks, obtain the Chinese medicine real-time image angle information of the main Chinese medicine real-time image, and determine the angle influencing factor in combination with the Chinese medicine real-time image angle information of other Chinese medicine real-time images;

[0053] Get the image features of the target image area block and perform screening and matching using the following formula:

[0054]

[0055] In the above technical solution, Indicates the main Chinese medicine real-time image The matching results of target image area blocks, Indicates the main Chinese medicine real-time image The target image area block Eigenvector, Indicates the angle influence factor of the main TCM real-time image, Indicates Angle influence factor of real-time image of traditional Chinese medicine, Indicates Zhang real-time image of traditional Chinese medicine The target image area block Eigenvector, Indicates Zhang real-time image of traditional Chinese medicine target image region blocks, represents the number of eigenvectors, Represents the screening judgment factor;

[0056] Based on the image area block matching results, image quality comparison and analysis are performed according to the target image information to determine the optimal value of the image area block;

[0057] Performing integrity analysis on real-time images of traditional Chinese medicines to determine the existence of real-time images of traditional Chinese medicines with complete images;

[0058] Perform image quality evaluation on real-time images of traditional Chinese medicine with complete images of traditional Chinese medicine, and determine the best real-time image of traditional Chinese medicine according to the image quality evaluation results;

[0059] According to the optimal value of the image area block, the optimal real-time image of traditional Chinese medicine is corrected according to the area block to obtain the optimal real-time image of traditional Chinese medicine.

[0060] Preferably, when the feature extraction unit performs feature extraction based on the real-time image of traditional Chinese medicine using computer vision, feature extraction is performed on the best real-time image of traditional Chinese medicine, including:

[0061] Using the first recognition model to perform edge recognition of the Chinese medicine image on the best real-time image of the Chinese medicine, determine the image edge of the Chinese medicine image, and obtain the contour feature information of the Chinese medicine;

[0062] Preliminary analysis of the size of the Chinese medicine image based on the image edge of the Chinese medicine image is performed to obtain size estimation data of the Chinese medicine image;

[0063] An adjustment factor is determined according to the estimated size data of the traditional Chinese medicine image to obtain a first adjustment factor;

[0064] The first adjustment factor is reviewed in combination with the adjustment factor standard. When the first adjustment factor passes the review, the first adjustment factor is the target adjustment factor. When the first adjustment factor fails the review, the first adjustment factor is corrected using the adjustment factor limit value, and the corresponding adjustment factor limit value is used as the target adjustment factor.

[0065] The target area is determined based on the image edge of the traditional Chinese medicine image combined with the target adjustment factor, and the target area is screened out in the best real-time image of the traditional Chinese medicine to obtain the feature extraction target area;

[0066] The second recognition model is used to perform color recognition on the feature extraction target area to obtain color feature information of traditional Chinese medicine;

[0067] The third recognition model is used to perform texture recognition on the feature extraction target area to obtain the texture feature information of traditional Chinese medicine.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] The present invention collects Chinese medicine images in real time based on computer vision technology, determines real-time images of Chinese medicine based on computer vision, determines characteristic images of Chinese medicine by denoising, contrast adjustment, feature extraction and fusion of real-time images of Chinese medicine based on computer vision, performs image recognition on the characteristic images of Chinese medicine according to a Chinese medicine recognition model based on fusion image recognition, judges the names of Chinese medicines, determines the recognition results of Chinese medicines, and visualizes the recognition results of Chinese medicines, and provides an interactive interface for users to perform interactive operations, so as to quickly and accurately identify Chinese medicines, save time and effort, and improve the recognition efficiency of Chinese medicines. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a module block diagram of the traditional Chinese medicine identification system based on fusion image recognition of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In order to solve the problem that the existing method of identifying Chinese medicine is mostly for users to search and identify by flipping through books, which is time-consuming and labor-intensive, and cannot quickly and accurately identify Chinese medicine, resulting in low efficiency of Chinese medicine identification, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0073] The Chinese medicine identification system based on fusion image recognition includes: an image acquisition module, an image processing module, a Chinese medicine identification module and a result output module.

[0074] It should be noted that, through the interactive communication between the image acquisition module, the image processing module, the Chinese medicine recognition module and the result output module, the Chinese medicine image can be collected based on computer vision technology, and the real-time image of the Chinese medicine based on computer vision can be determined. The real-time image of the Chinese medicine based on computer vision can be denoised, contrast adjusted, feature extracted and fused to determine the characteristic image of the Chinese medicine. The characteristic image of the Chinese medicine can be recognized by the Chinese medicine recognition model based on fusion image recognition, the name of the Chinese medicine can be judged, the recognition result of the Chinese medicine can be determined, and finally the recognition result of the Chinese medicine can be visualized and an interactive interface can be provided for users to perform interactive operations. The Chinese medicine can be identified quickly and accurately, saving time and effort, and improving the efficiency of Chinese medicine recognition.

[0075] In this embodiment, as a preferred technical solution of the present invention, the image acquisition module includes:

[0076] A Chinese medicine collection unit is used to collect the Chinese medicine placed on the collection platform in real time;

[0077] Among them, based on computer vision technology, a camera is used to continuously shoot the traditional Chinese medicine placed on the collection platform at different angles, and continuous pictures of the traditional Chinese medicine at different angles are obtained, and then the real-time image of the traditional Chinese medicine based on computer vision is determined;

[0078] It should be noted that when using a camera to photograph the traditional Chinese medicine placed on the collection platform, the position of the camera is adjusted to continuously photograph the traditional Chinese medicine placed on the collection platform at different angles, so as to obtain continuous pictures of the traditional Chinese medicine at different angles, facilitate all-round identification of the traditional Chinese medicine, and make the identification results of the traditional Chinese medicine more accurate.

[0079] An image transmission unit, used to transmit real-time images of traditional Chinese medicine based on computer vision;

[0080] Among them, an image transmission link is established between the image acquisition module and the image processing module;

[0081] After the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link, so that the image processing module processes the real-time image of traditional Chinese medicine based on computer vision.

[0082] It should be noted that after the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits an instruction requesting to establish an image transmission link to the image processing module. After the image processing module receives the instruction requesting to establish an image transmission link transmitted by the image acquisition module, the image processing module transmits an instruction agreeing to establish the image transmission link to the image acquisition module. After the image acquisition module receives the instruction agreeing to establish the image transmission link transmitted by the image processing module, the image acquisition module establishes an image transmission link with the image processing module, and the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link.

[0083] In this embodiment, as a preferred technical solution of the present invention, the image processing module includes:

[0084] An image denoising unit is used to denoise real-time images of traditional Chinese medicine based on computer vision;

[0085] Based on the median filter, the median of the pixels around each pixel in the real-time image of traditional Chinese medicine based on computer vision is calculated to replace the value of the pixel, thereby removing the noise in the real-time image of traditional Chinese medicine based on computer vision and eliminating isolated noise points;

[0086] It should be noted that by denoising the real-time images of traditional Chinese medicine based on computer vision, the noise in the real-time images of traditional Chinese medicine based on computer vision can be effectively reduced and the quality of the real-time images of traditional Chinese medicine based on computer vision can be improved.

[0087] An image adjustment unit, used for adjusting and processing the real-time image of traditional Chinese medicine based on computer vision;

[0088] The gray value distribution of the real-time image of traditional Chinese medicine based on computer vision is evened out based on the histogram equalization method, and the contrast of the real-time image of traditional Chinese medicine based on computer vision is adjusted by adjusting the gray range of the real-time image of traditional Chinese medicine based on computer vision.

[0089] It should be noted that by adjusting the contrast of the real-time image of traditional Chinese medicine based on computer vision, the light and dark contrast in the real-time image of traditional Chinese medicine based on computer vision can be made more obvious, and the visualization effect of the real-time image of traditional Chinese medicine based on computer vision can be improved.

[0090] In this embodiment, as a preferred technical solution of the present invention, the image processing module further includes:

[0091] A feature extraction unit is used to extract features from the real-time image of traditional Chinese medicine based on computer vision, and to extract useful features from the real-time image of traditional Chinese medicine based on computer vision, including color features, texture features and shape features;

[0092] Based on the color feature description method, the color features of Chinese medicine are described by extracting the color histogram of the real-time image of Chinese medicine based on computer vision;

[0093] Based on the texture feature description method, the gray-level co-occurrence matrix is ​​used to describe the texture features of traditional Chinese medicine by statistically analyzing the gray-level value relationship between pixels at different positions in the real-time image of traditional Chinese medicine based on computer vision.

[0094] Based on the contour feature extraction method, the boundary shape of the Chinese medicine is described by calculating the length, area and perimeter of the Chinese medicine contour to determine the shape characteristics of the Chinese medicine;

[0095] It should be noted that by performing feature extraction on real-time images of traditional Chinese medicine based on computer vision, color features, texture features and shape features can be extracted to facilitate subsequent identification of traditional Chinese medicine.

[0096] A feature fusion unit is used to fuse the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of traditional Chinese medicine;

[0097] Based on the weighted fusion algorithm, weights are assigned to the color features, texture features and shape features of the real-time image of traditional Chinese medicine, and the weighted color features, texture features and shape features are spliced ​​to form a characteristic image of traditional Chinese medicine.

[0098] It should be noted that by fusing the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of traditional Chinese medicine, redundant information can be reduced and the effectiveness of the feature vector can be improved. This can both retain the information of the original features and increase the dimension of the feature vector.

[0099] In this embodiment, as a preferred technical solution of the present invention, the traditional Chinese medicine identification module includes:

[0100] Model training unit, used to train the traditional Chinese medicine recognition model based on fusion image recognition;

[0101] According to the needs of Chinese medicine identification based on fusion image recognition, a camera is used to shoot Chinese medicine samples, collect Chinese medicine sample images, and explain the efficacy of the collected Chinese medicine sample images;

[0102] The collected Chinese medicine sample images are divided into training set and test set in a ratio of 7:3;

[0103] Based on deep learning technology, the deep learning model is trained with a training set, so that the deep learning model can autonomously learn Chinese medicine recognition and identify the name and efficacy of Chinese medicine, and determine the Chinese medicine recognition model based on fusion image recognition;

[0104] Based on the test set, the performance of the Chinese medicine recognition model based on fusion image recognition is tested to determine whether the Chinese medicine recognition model based on fusion image recognition can achieve the expected effect, which is used to verify the generalization ability of the Chinese medicine recognition model based on fusion image recognition.

[0105] It should be noted that based on deep learning technology, a traditional Chinese medicine recognition model based on fusion image recognition is trained to facilitate the subsequent accurate identification of traditional Chinese medicine.

[0106] In this embodiment, as a preferred technical solution of the present invention, the traditional Chinese medicine identification module further includes:

[0107] A Chinese medicine identification unit is used to identify the Chinese medicine and determine the identification result of the Chinese medicine;

[0108] Obtain a traditional Chinese medicine recognition model based on fusion image recognition, and deploy the traditional Chinese medicine recognition model based on fusion image recognition in actual traditional Chinese medicine recognition applications for users to use to identify traditional Chinese medicine;

[0109] The characteristic image of traditional Chinese medicine is taken as input and input into the traditional Chinese medicine recognition model based on fusion image recognition. The characteristic image of traditional Chinese medicine is subjected to image recognition according to the traditional Chinese medicine recognition model based on fusion image recognition, the name of traditional Chinese medicine is judged, and the recognition result of traditional Chinese medicine is determined.

[0110] It should be noted that by performing image recognition on the characteristic images of traditional Chinese medicines based on the traditional Chinese medicine recognition model based on fusion image recognition, judging the names of traditional Chinese medicines and determining the identification results of traditional Chinese medicines, traditional Chinese medicines can be identified quickly and accurately, saving time and effort and improving the efficiency of traditional Chinese medicine recognition.

[0111] In this embodiment, as a preferred technical solution of the present invention, the result output module includes:

[0112] The identification and display unit is used to visually display the TCM identification results, wherein the results are output in the form of text, images or sounds for users to view and use. When the TCM identification results are visually displayed, the names and effects of the TCM are output;

[0113] When output in text form, the name and efficacy of Chinese medicine are displayed visually;

[0114] When outputting in image form, the name and efficacy of the identified Chinese medicine are marked on the real-time image of the Chinese medicine;

[0115] When output in voice form, the name and efficacy of Chinese medicine are read out.

[0116] In this embodiment, as a preferred technical solution of the present invention, the result output module further includes:

[0117] The user interaction unit is used for user interactive operations and provides an interactive interface for users to query, save, download and share the results of Chinese medicine identification. The output method of Chinese medicine identification results is adjusted according to user feedback.

[0118] In this embodiment, as a preferred technical solution of the present invention, the image processing module further includes: the image processing unit is used to perform angle analysis and processing on the real-time image of traditional Chinese medicine based on computer vision, including:

[0119] Conduct a preliminary analysis on the real-time image of traditional Chinese medicine using computer vision to determine whether the real-time image of traditional Chinese medicine has completed the processing of the image denoising unit and the image adjustment unit, and obtain a preliminary analysis result;

[0120] When the preliminary analysis result shows that the real-time image of the traditional Chinese medicine has completed the processing of the image denoising unit and the processing of the image adjustment unit, the real-time image of the traditional Chinese medicine is subjected to preliminary recognition of the image of the traditional Chinese medicine as a whole, the shooting angle is determined, and the angle information of the real-time image of the traditional Chinese medicine is obtained;

[0121] Acquiring image information of the real-time image of traditional Chinese medicine to obtain image information of the real-time image of traditional Chinese medicine, and screening the image information of the real-time image of traditional Chinese medicine to screen out image information about the image of traditional Chinese medicine to obtain target image information of the real-time image of traditional Chinese medicine;

[0122] The real-time image of the traditional Chinese medicine is matched with the angle information of the real-time image of the traditional Chinese medicine to determine the area block corresponding to the image of the traditional Chinese medicine in the real-time image of the traditional Chinese medicine, and the image area block matching result is obtained, including:

[0123] Dividing the image area corresponding to the target image information in the real-time image of traditional Chinese medicine to obtain a plurality of target image area blocks;

[0124] Determine the main Chinese medicine real-time image according to the number of target image area blocks, obtain the Chinese medicine real-time image angle information of the main Chinese medicine real-time image, and determine the angle influencing factor in combination with the Chinese medicine real-time image angle information of other Chinese medicine real-time images;

[0125] Get the image features of the target image area block and perform screening and matching using the following formula:

[0126]

[0127] In the above technical solution, Indicates the main Chinese medicine real-time image The matching results of target image area blocks, Indicates the main Chinese medicine real-time image The target image area block Eigenvector, Indicates the angle influence factor of the main TCM real-time image, Indicates Angle influence factor of real-time image of traditional Chinese medicine, Indicates Zhang real-time image of traditional Chinese medicine The target image area block Eigenvector, Indicates Zhang real-time image of traditional Chinese medicine target image region blocks, represents the number of eigenvectors, Represents the screening judgment factor;

[0128] Based on the image area block matching results, image quality comparison and analysis are performed according to the target image information to determine the optimal value of the image area block;

[0129] Performing integrity analysis on real-time images of traditional Chinese medicines to determine the existence of real-time images of traditional Chinese medicines with complete images;

[0130] Perform image quality evaluation on real-time images of traditional Chinese medicine with complete images of traditional Chinese medicine, and determine the best real-time image of traditional Chinese medicine according to the image quality evaluation results;

[0131] According to the optimal value of the image area block, the optimal real-time image of traditional Chinese medicine is corrected according to the area block to obtain the optimal real-time image of traditional Chinese medicine.

[0132] Among them, image features refer to the image features of traditional Chinese medicine in the image, including: line curvature information, combination structure information, etc. The characteristic vectors include line bending information vectors, combination construction information vectors, and the like. It is composed of multiple The feature vectors constitute Zhang real-time image of traditional Chinese medicine target image area blocks. and It is numbered based on the target image region blocks of the real-time Chinese medicine image, and its value is a positive integer. The number of target image region blocks in different real-time Chinese medicine images may be the same or different. It is a number assigned to the real-time image of traditional Chinese medicine, and its value is a positive integer. Screening judgment factor The value of is a non-negative number close to 0. The information includes: image clarity, brightness and other parameter information.

[0133] It should be noted that, by conducting preliminary analysis on the real-time images of traditional Chinese medicine using computer vision, it is ensured that the real-time images of traditional Chinese medicine used for the overall preliminary identification of traditional Chinese medicine images have been denoised and adjusted, and by screening the image information of the real-time images of traditional Chinese medicine to interfere with irrelevant information, only the images of traditional Chinese medicine are analyzed during image processing, which reduces the degree of confusion in information analysis and the probability of errors in information analysis. The real-time images of traditional Chinese medicine are matched with the angle information of the real-time images of traditional Chinese medicine, fully considering the influence of the image angle to avoid matching deviations caused by different image angles, thereby affecting the comparative analysis of image quality, and ensuring the accuracy and comprehensiveness of the best real-time images of traditional Chinese medicine, thereby providing guarantees for subsequent feature extraction and judgment of the names of traditional Chinese medicines, and improving the accuracy of the traditional Chinese medicine recognition system. In addition, by performing integrity analysis and image quality evaluation on real-time images of traditional Chinese medicine, real-time images of traditional Chinese medicine with high image quality and complete images are corrected, ensuring that the best real-time images of traditional Chinese medicine have complete images of traditional Chinese medicine, effectively improving the image quality of the best real-time images of traditional Chinese medicine, so that when extracting features, only feature acquisition needs to be performed on the best real-time images of traditional Chinese medicine, reducing time consumption, reducing the response time of the image processing module of the real-time images of traditional Chinese medicine, realizing real-time processing, and facilitating subsequent accurate identification of traditional Chinese medicine.

[0134] In this embodiment, as a preferred technical solution of the present invention, when the feature extraction unit performs feature extraction based on the real-time image of traditional Chinese medicine using computer vision, feature extraction is performed on the best real-time image of traditional Chinese medicine, including:

[0135] Using the first recognition model to perform edge recognition of the Chinese medicine image on the best real-time image of the Chinese medicine, determine the image edge of the Chinese medicine image, and obtain the contour feature information of the Chinese medicine;

[0136] Preliminary analysis of the size of the Chinese medicine image based on the image edge of the Chinese medicine image is performed to obtain size estimation data of the Chinese medicine image;

[0137] An adjustment factor is determined according to the estimated size data of the traditional Chinese medicine image to obtain a first adjustment factor;

[0138] The first adjustment factor is reviewed in combination with the adjustment factor standard. When the first adjustment factor passes the review, the first adjustment factor is the target adjustment factor. When the first adjustment factor fails the review, the first adjustment factor is corrected using the adjustment factor limit value, and the corresponding adjustment factor limit value is used as the target adjustment factor.

[0139] The target area is determined based on the image edge of the traditional Chinese medicine image combined with the target adjustment factor, and the target area is screened out in the best real-time image of the traditional Chinese medicine to obtain the feature extraction target area;

[0140] The second recognition model is used to perform color recognition on the feature extraction target area to obtain color feature information of traditional Chinese medicine;

[0141] The third recognition model is used to perform texture recognition on the feature extraction target area to obtain the texture feature information of traditional Chinese medicine.

[0142] Among them, the first recognition model is a contour recognition module, which performs edge recognition on the image of traditional Chinese medicine in the best real-time image of traditional Chinese medicine. The second recognition model is a color recognition model, which performs color recognition on the target area for feature extraction. The third recognition model is a texture recognition model, which performs texture recognition on the target area for feature extraction. Moreover, the first recognition model, the second recognition model and the third recognition model are trained and optimized before use. The larger the size estimation data of the traditional Chinese medicine image, the smaller the adjustment factor, which is a number greater than 0 and less than 1. The adjustment factor standard is an interval number within the range of 0-1, with a maximum value and a minimum value.

[0143] It should be noted that by extracting features from the best real-time image of Chinese medicine, useful features can be obtained in one best real-time image of Chinese medicine, reducing the workload of feature extraction, so that the features of Chinese medicine can be obtained in a relatively short time. Moreover, the artificial intelligence acquisition of Chinese medicine contour feature information, Chinese medicine color feature information and Chinese medicine texture feature information is realized through the first recognition model, the second recognition model and the third recognition model, which can not only accurately and comprehensively obtain the corresponding feature information, but also reduce time waste, save time and effort, and thus improve the efficiency of Chinese medicine recognition. In addition, when determining the target area for feature extraction, the target adjustment factor is considered to reduce the redundancy of irrelevant images in the best real-time image of Chinese medicine, reduce the proportion of the second recognition model and the third recognition model for invalid areas, ensure the comprehensiveness of Chinese medicine images, and effectively reduce the interference of irrelevant information, and the target adjustment factor can be dynamically adjusted according to the size estimation data of the Chinese medicine image, avoiding the occurrence of small edge blanks in larger Chinese medicines or large edge blanks in smaller Chinese medicines that affect the Chinese medicine image itself, reducing the recognition error, and also limiting the adjustment factor to fluctuate infinitely with the actual situation of Chinese medicine through the adjustment factor standard, ensuring the feasibility of the adjustment factor.

[0144] In summary, computer vision technology is used to collect images of traditional Chinese medicine in real time, and real-time images of traditional Chinese medicine based on computer vision are determined. Denoising, contrast adjustment, feature extraction and fusion are performed on real-time images of traditional Chinese medicine based on computer vision to determine characteristic images of traditional Chinese medicine. Image recognition is performed on characteristic images of traditional Chinese medicine according to a traditional Chinese medicine recognition model based on fusion image recognition, the name of the traditional Chinese medicine is judged, the recognition result of the traditional Chinese medicine is determined, and the recognition result of the traditional Chinese medicine is visualized. An interactive interface is provided for users to perform interactive operations, which can quickly and accurately identify traditional Chinese medicine, save time and effort, and improve the efficiency of traditional Chinese medicine recognition.

[0145] Among them, the Chinese medicine identification system has broad application prospects, and can provide strong technical support for the modernization, standardization and internationalization of Chinese medicine. It can be applied to multiple links such as Chinese medicine planting, processing, storage and sales. For example, in the process of Chinese medicine planting, it can be used to identify the name and efficacy of Chinese medicinal materials; in the process of Chinese medicine processing, it can be used to identify the types of Chinese medicinal materials; in the storage process of Chinese medicine, it can be used to identify the name of Chinese medicinal materials; in the sales process of Chinese medicine, it can be used to identify the authenticity of Chinese medicinal materials.

[0146] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0147] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A Chinese medicine identification system based on fusion image recognition, characterized in that: include: An image acquisition module is used to acquire images of traditional Chinese medicine based on computer vision technology and determine real-time images of traditional Chinese medicine based on computer vision; The image processing module is used to perform denoising, contrast adjustment, feature extraction and fusion on the real-time image of traditional Chinese medicine based on computer vision, and determine the characteristic image of traditional Chinese medicine; The Chinese medicine recognition module is used to perform image recognition on the Chinese medicine feature image according to the Chinese medicine recognition model based on fusion image recognition, judge the name of the Chinese medicine, and determine the Chinese medicine recognition result; The result output module is used to visualize the results of Chinese medicine identification and provide an interactive interface for users to perform interactive operations.

2. The Chinese medicine identification system based on fusion image recognition as claimed in claim 1, characterized in that: The image acquisition module comprises: A Chinese medicine collection unit is used to collect the Chinese medicine placed on the collection platform in real time; Among them, based on computer vision technology, a camera is used to continuously shoot the traditional Chinese medicine placed on the collection platform at different angles, obtain continuous shooting pictures of the traditional Chinese medicine at different angles, and determine the real-time image of the traditional Chinese medicine based on computer vision; An image transmission unit, used to transmit real-time images of traditional Chinese medicine based on computer vision; Among them, an image transmission link is established between the image acquisition module and the image processing module; After the image acquisition module acquires the real-time image of traditional Chinese medicine based on computer vision, the image acquisition module transmits the acquired real-time image of traditional Chinese medicine based on computer vision to the image processing module according to the established image transmission link, so that the image processing module processes the real-time image of traditional Chinese medicine based on computer vision.

3. The Chinese medicine identification system based on fusion image recognition as claimed in claim 2, characterized in that: The image processing module comprises: An image denoising unit is used to denoise real-time images of traditional Chinese medicine based on computer vision; Based on the median filter, the median of the pixels around each pixel in the real-time image of traditional Chinese medicine based on computer vision is calculated to replace the value of the pixel, thereby removing the noise in the real-time image of traditional Chinese medicine based on computer vision and eliminating isolated noise points; An image adjustment unit, used for adjusting and processing the real-time image of traditional Chinese medicine based on computer vision; The gray value distribution of the real-time image of traditional Chinese medicine based on computer vision is evened out based on the histogram equalization method, and the contrast of the real-time image of traditional Chinese medicine based on computer vision is adjusted by adjusting the gray range of the real-time image of traditional Chinese medicine based on computer vision.

4. The Chinese medicine identification system based on fusion image recognition as claimed in claim 3, characterized in that: The image processing module further includes: A feature extraction unit is used to extract features from the real-time image of traditional Chinese medicine based on computer vision, and to extract useful features from the real-time image of traditional Chinese medicine based on computer vision, including color features, texture features and shape features; Based on the color feature description method, the color features of Chinese medicine are described by extracting the color histogram of the real-time image of Chinese medicine based on computer vision; Based on the texture feature description method, the gray-level co-occurrence matrix is ​​used to describe the texture features of traditional Chinese medicine by statistically analyzing the gray-level value relationship between pixels at different positions in the real-time image of traditional Chinese medicine based on computer vision. Based on the contour feature extraction method, the boundary shape of the Chinese medicine is described by calculating the length, area and perimeter of the Chinese medicine contour to determine the shape characteristics of the Chinese medicine; A feature fusion unit is used to fuse the color features, texture features and shape features of the real-time image of traditional Chinese medicine to determine the characteristic image of traditional Chinese medicine; Based on the weighted fusion algorithm, weights are assigned to the color features, texture features and shape features of the real-time image of traditional Chinese medicine, and the weighted color features, texture features and shape features are spliced ​​to form a characteristic image of traditional Chinese medicine.

5. The Chinese medicine identification system based on fusion image recognition as claimed in claim 4, characterized in that: The traditional Chinese medicine identification module comprises: Model training unit, used to train the traditional Chinese medicine recognition model based on fusion image recognition; According to the needs of Chinese medicine identification based on fusion image recognition, a camera is used to shoot Chinese medicine samples, collect Chinese medicine sample images, and explain the efficacy of the collected Chinese medicine sample images; The collected Chinese medicine sample images are divided into training set and test set in a ratio of 7:3; Based on deep learning technology, the deep learning model is trained with a training set, so that the deep learning model can autonomously learn Chinese medicine recognition and identify the name and efficacy of Chinese medicine, and determine the Chinese medicine recognition model based on fusion image recognition; Based on the test set, the performance of the Chinese medicine recognition model based on fusion image recognition is tested to determine whether the Chinese medicine recognition model based on fusion image recognition can achieve the expected effect, which is used to verify the generalization ability of the Chinese medicine recognition model based on fusion image recognition.

6. The Chinese medicine identification system based on fusion image recognition as claimed in claim 5, characterized in that: The traditional Chinese medicine identification module also includes: A Chinese medicine identification unit is used to identify the Chinese medicine and determine the identification result of the Chinese medicine; Obtain a traditional Chinese medicine recognition model based on fusion image recognition, and deploy the traditional Chinese medicine recognition model based on fusion image recognition in actual traditional Chinese medicine recognition applications for users to use to identify traditional Chinese medicine; The characteristic image of traditional Chinese medicine is taken as input and input into the traditional Chinese medicine recognition model based on fusion image recognition. The characteristic image of traditional Chinese medicine is subjected to image recognition according to the traditional Chinese medicine recognition model based on fusion image recognition, the name of traditional Chinese medicine is judged, and the recognition result of traditional Chinese medicine is determined.

7. The Chinese medicine identification system based on fusion image recognition as claimed in claim 6, characterized in that: The result output module comprises: The identification and display unit is used to visually display the TCM identification results, wherein the results are output in the form of text, images or sounds for users to view and use. When the TCM identification results are visually displayed, the names and effects of the TCM are output; When output in text form, the name and efficacy of Chinese medicine are displayed visually; When outputting in image form, the name and efficacy of the identified Chinese medicine are marked on the real-time image of the Chinese medicine; When output in voice form, the name and efficacy of Chinese medicine are read out.

8. The Chinese medicine identification system based on fusion image recognition as claimed in claim 7, characterized in that: The result output module further includes: The user interaction unit is used for user interactive operations and provides an interactive interface for users to query, save, download and share the results of Chinese medicine identification. The output method of Chinese medicine identification results is adjusted according to user feedback.

9. The Chinese medicine identification system based on fusion image recognition as claimed in claim 4, characterized in that: The image processing module further includes: the image processing unit is used to perform angle analysis and processing on the real-time image of traditional Chinese medicine based on computer vision, including: Conduct a preliminary analysis on the real-time image of traditional Chinese medicine using computer vision to determine whether the real-time image of traditional Chinese medicine has completed the processing of the image denoising unit and the image adjustment unit, and obtain a preliminary analysis result; When the preliminary analysis result shows that the real-time image of the traditional Chinese medicine has completed the processing of the image denoising unit and the processing of the image adjustment unit, the real-time image of the traditional Chinese medicine is subjected to preliminary recognition of the image of the traditional Chinese medicine as a whole, the shooting angle is determined, and the angle information of the real-time image of the traditional Chinese medicine is obtained; Acquiring image information of the real-time image of traditional Chinese medicine to obtain image information of the real-time image of traditional Chinese medicine, and screening the image information of the real-time image of traditional Chinese medicine to screen out image information about the image of traditional Chinese medicine to obtain target image information of the real-time image of traditional Chinese medicine; The real-time image of the traditional Chinese medicine is matched with the angle information of the real-time image of the traditional Chinese medicine to determine the area block corresponding to the image of the traditional Chinese medicine in the real-time image of the traditional Chinese medicine, and the image area block matching result is obtained, including: Dividing the image area corresponding to the target image information in the real-time image of traditional Chinese medicine to obtain a plurality of target image area blocks; Determine the main Chinese medicine real-time image according to the number of target image area blocks, obtain Chinese medicine real-time image angle information of the main Chinese medicine real-time image, and determine the angle influencing factor in combination with the Chinese medicine real-time image angle information of other Chinese medicine real-time images; Get the image features of the target image area block and perform screening and matching using the following formula: In the above technical solution, Indicates the main Chinese medicine real-time image The matching results of target image area blocks, Indicates the main Chinese medicine real-time image The target image area block Eigenvector, Indicates the angle influence factor of the main TCM real-time image, Indicates Angle influence factor of real-time image of traditional Chinese medicine, Indicates Zhang real-time image of traditional Chinese medicine The target image area block Eigenvector, Indicates Zhang real-time image of traditional Chinese medicine target image region blocks, represents the number of eigenvectors, Represents the screening judgment factor; Based on the image area block matching results, image quality comparison and analysis are performed according to the target image information to determine the optimal value of the image area block; Performing integrity analysis on real-time images of traditional Chinese medicines to determine the existence of real-time images of traditional Chinese medicines with complete images; Perform image quality evaluation on real-time images of traditional Chinese medicine with complete images of traditional Chinese medicine, and determine the best real-time image of traditional Chinese medicine according to the image quality evaluation results; According to the optimal value of the image area block, the optimal real-time image of traditional Chinese medicine is corrected according to the area block to obtain the optimal real-time image of traditional Chinese medicine.

10. The Chinese medicine identification system based on fusion image recognition as claimed in claim 9, characterized in that: When the feature extraction unit performs feature extraction based on the real-time image of traditional Chinese medicine using computer vision, feature extraction is performed on the best real-time image of traditional Chinese medicine, including: Using the first recognition model to perform edge recognition of the Chinese medicine image on the best real-time image of the Chinese medicine, determine the image edge of the Chinese medicine image, and obtain the contour feature information of the Chinese medicine; Preliminary analysis of the size of the Chinese medicine image based on the image edge of the Chinese medicine image is performed to obtain size estimation data of the Chinese medicine image; An adjustment factor is determined according to the estimated size data of the traditional Chinese medicine image to obtain a first adjustment factor; The first adjustment factor is reviewed in combination with the adjustment factor standard. When the first adjustment factor passes the review, the first adjustment factor is the target adjustment factor. When the first adjustment factor fails the review, the first adjustment factor is corrected using the adjustment factor limit value, and the corresponding adjustment factor limit value is used as the target adjustment factor. The target area is determined based on the image edge of the traditional Chinese medicine image combined with the target adjustment factor, and the target area is screened out in the best real-time image of the traditional Chinese medicine to obtain the feature extraction target area; The second recognition model is used to perform color recognition on the feature extraction target area to obtain color feature information of traditional Chinese medicine; The third recognition model is used to perform texture recognition on the feature extraction target area to obtain the texture feature information of traditional Chinese medicine.

Images