A visual recognition system for Chinese herbal medicine categories

Abstract

The present invention provides a visual recognition system for Chinese herbal medicine categories, which relates to the technical field of Chinese herbal medicine recognition. The system comprises: performing preliminary recognition on the Chinese herbal medicine to be recognized based on the appearance features of the Chinese herbal medicine to be recognized. If the recognition result is a mixed category, obtaining a sub-classification model corresponding to the mixed category, and inputting a feature vector group set corresponding to the Chinese herbal medicine to be recognized into the sub-classification model, obtaining the confidence of the Chinese herbal medicine to be recognized output by the sub-classification model corresponding to each preset category of Chinese herbal medicine, and if there are at least two confidences greater than a preset confidence threshold, obtaining a non-image feature vector of the Chinese herbal medicine to be recognized; filling the non-image feature vector into an empty vector in a feature vector group set corresponding to the Chinese herbal medicine to be recognized to obtain a target vector group set XA' corresponding to the Chinese herbal medicine to be recognized; inputting XA' into ZA to obtain the category corresponding to the Chinese herbal medicine to be recognized; the present invention can accurately determine the category corresponding to the Chinese herbal medicine to be recognized.

Description

Technical Field

[0001] The present invention relates to the technical field of Chinese herbal medicine identification, and in particular to a visual identification system for Chinese herbal medicine categories. Background Art

[0002] In the field of Chinese herbal medicine, there are many kinds of Chinese herbal medicines, and the medicinal properties of different kinds of Chinese herbal medicines vary greatly. Therefore, accurately identifying different kinds of Chinese herbal medicines is crucial for configuring Chinese herbal medicine prescriptions. Usually, Chinese herbal medicines are identified manually. Due to the large number of types of Chinese herbal medicines and limited human energy, the manual identification of Chinese herbal medicines will lead to inaccurate identification results. Based on this, the prior art uses a large model to identify Chinese herbal medicine images to identify Chinese herbal medicines. Due to the large number of categories of Chinese herbal medicines, if all Chinese herbal medicines are identified by one model, a large amount of training is required for the large model. The training process is relatively complicated and the training is also relatively generalized. When the large model outputs the result, it usually outputs the result with the highest confidence. However, some Chinese herbal medicines of different categories are similar in appearance, for example: Stephania tetrandra and Cnidium monnieri, raspberry and March bubble, Codonopsis pilosula and Saposhnikovia divaricata are all similar. When identifying Chinese herbal medicines of similar categories in the above manner, there will be multiple recognition results with high confidence. If only the result with the highest confidence is output, the recognition accuracy of the Chinese herbal medicine category will be low. Summary of the invention

[0003] In view of the above technical problems, the technical solution adopted by the present invention is:

[0004] According to a Chinese herbal medicine category visual identification system provided in this application, the system includes:

[0005] A processor and a storage medium, wherein the storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the following steps:

[0006] Q100, obtain the appearance feature TA corresponding to the Chinese herbal medicine to be identified; wherein TA is obtained through the image corresponding to the Chinese herbal medicine to be identified.

[0007] Q200, input TA into the preset appearance classification model to obtain the initial category LA corresponding to the Chinese herbal medicine to be identified.

[0008] Q300, if LA is a preset mixed category, obtain the sub-classification model ZA corresponding to LA; wherein the mixed category includes at least two Chinese herbal medicine categories; and each mixed category corresponds to a preset sub-classification model.

[0009] Q400, obtain the feature vector set XA corresponding to the Chinese herbal medicine to be identified = (XA 1 , XA2 , …, XA u , …, XA v ), u=1, 2,...,v; among them, XA u is the uth feature vector group corresponding to the Chinese herbal medicine to be identified, v is the number of feature vector groups in the feature vector group set corresponding to the Chinese herbal medicine to be identified; XA u Includes several feature vectors of different dimensions; the last k feature vector groups of XA are empty.

[0010] Q500, input XA into ZA to obtain the confidence of the Chinese herbal medicine to be identified corresponding to each preset category of Chinese herbal medicine, so as to obtain a confidence list θ=(θ 1 ,θ 2 ,…,θ f ,…,θ g ), f = 1, 2, ..., g; where θ f is the confidence that the Chinese herbal medicine to be identified is the fth preset category of Chinese herbal medicine, and g is the number of preset Chinese herbal medicine categories.

[0011] Q600, traverse θ, if there are at least two confidences in θ that are greater than a preset confidence threshold, obtain the non-image feature vector JE of the Chinese herbal medicine to be identified.

[0012] Q700, fill JE into the empty vector in XA to obtain the target vector set XA' corresponding to the Chinese herbal medicine to be identified.

[0013] Q800, input XA' into ZA to obtain the corresponding category of the Chinese herbal medicine to be identified.

[0014] The Chinese herbal medicine category visual recognition system of the present invention first performs preliminary recognition of the Chinese herbal medicine to be recognized based on the appearance characteristics of the Chinese herbal medicine to be recognized. If the recognition result is a mixed category, a sub-classification model corresponding to the mixed category is obtained, and the feature vector set corresponding to the Chinese herbal medicine to be recognized is input into the sub-classification model, and the confidence of the Chinese herbal medicine to be recognized corresponding to each preset category of Chinese herbal medicine output by the sub-classification model is obtained. If there are at least two confidences greater than a preset confidence threshold, a non-image feature vector of the Chinese herbal medicine to be recognized is obtained; the non-image feature vector is filled into the empty vector in the feature vector set corresponding to the Chinese herbal medicine to be recognized to obtain the target vector set XA' corresponding to the Chinese herbal medicine to be recognized; XA' is input into ZA to obtain the category corresponding to the Chinese herbal medicine to be recognized; since the non-image features corresponding to the Chinese herbal medicine to be recognized are further added, the sub-classification model can accurately determine the category corresponding to the Chinese herbal medicine to be recognized.

[0015] Furthermore, the present invention incorporates large model technology and integrates the capabilities of large models to provide users with more accurate and efficient solutions in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of the steps executed by a processor of a visual identification system for Chinese herbal medicine categories provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0019] It should be noted that, based on the present disclosure, those skilled in the art should understand that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement a device and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0020] Embodiment 1:

[0021] The following will refer to Figure 1 The flowchart of the steps executed by the processor of the Chinese herbal medicine category visual identification system is shown to introduce a Chinese herbal medicine category visual identification system.

[0022] The Chinese herbal medicine category visual identification system comprises: a processor and a storage medium, wherein the storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the following steps:

[0023] Q100, obtain the appearance feature TA corresponding to the Chinese herbal medicine to be identified; wherein TA is obtained through the image corresponding to the Chinese herbal medicine to be identified.

[0024] In this embodiment, the Chinese herbal medicine to be identified can be photographed by a depth camera to generate a three-dimensional image corresponding to the Chinese herbal medicine to be identified. There may be multiple Chinese herbal medicines in the image corresponding to the Chinese herbal medicine to be identified. Each Chinese herbal medicine can be first identified using a large model, and then the size information of one of the Chinese herbal medicines can be obtained to obtain the appearance feature TA corresponding to the Chinese herbal medicine to be identified; for example, the length, width, height, outline and shape and other parameters of a Chinese herbal medicine to be identified are obtained, and then the corresponding appearance features are generated; it should be noted that those skilled in the art can use the existing appearance feature generation method to generate the appearance features corresponding to the Chinese herbal medicine to be identified, which will not be elaborated here.

[0025] Q200, input TA into the preset appearance classification model to obtain the initial category LA corresponding to the Chinese herbal medicine to be identified.

[0026] In this embodiment, the appearance classification model is a model obtained after a large amount of training, for example: a large number of Chinese herbal medicines of different categories and shapes are obtained, and then the appearance features corresponding to the Chinese herbal medicines of each category are obtained, and the preset initial classification model is trained to obtain the preset appearance classification model; it should be noted that Chinese herbal medicines of different categories may have similarities in appearance, for example: Stephania tetrandra and Cnidium monnieri, Rubus idaeus and Psoralea corylifolia, Codonopsis pilosula and Saposhnikovia divaricata, Prunus persica and Apricot kernel are relatively similar in appearance; while some Chinese herbal medicines will not be similar to other categories of Chinese herbal medicines, for example: Panax notoginseng, etc., are not similar in appearance to other categories of Chinese herbal medicines; therefore, the initial category obtained according to the appearance characteristics may be a single category or a mixed category.

[0027] Furthermore, after step Q200, the at least one instruction or the at least one program is loaded and executed by the processor, and the following steps are also implemented:

[0028] Q210, if LA is a preset single category, LA is determined as the category corresponding to the Chinese herbal medicine to be identified; wherein a single category only includes one category of Chinese herbal medicine.

[0029] In this embodiment, the results output by the appearance classification model are divided into a single category and a mixed category. The single category is a category that only contains one Chinese herbal medicine. Therefore, if LA is a preset single category, it means that the Chinese herbal medicine to be identified is identified as a single category of Chinese herbal medicine according to its corresponding appearance characteristics. For example, the single category in which Panax notoginseng is located only contains Panax notoginseng. Since a single category only contains one category of Chinese herbal medicine, if LA is a preset single category, LA can be directly determined as the category corresponding to the Chinese herbal medicine to be identified without the need for subsequent judgment, thereby improving the recognition efficiency while ensuring the recognition accuracy.

[0030] Q300, if LA is a preset mixed category, obtain the sub-classification model ZA corresponding to LA; wherein the mixed category includes at least two Chinese herbal medicine categories; and each mixed category corresponds to a preset sub-classification model.

[0031] In this embodiment, the mixed category includes at least two categories of Chinese herbal medicines. For example, a mixed category includes two categories of Chinese herbal medicines, namely peach kernels and almonds. If the Chinese herbal medicine to be identified is identified as almonds, then the initial category of the Chinese herbal medicine to be identified is a mixed category. Since peach kernels and almonds are very similar in appearance, when the result output by the appearance classification model is a mixed category, the result may be wrong. Therefore, when the identification result of the Chinese herbal medicine to be identified is a mixed category, further judgment is required. A corresponding sub-classification model is preset for each mixed category. The sub-classification model corresponding to each mixed category is trained for the Chinese herbal medicine categories included in the mixed category, and can accurately distinguish similar Chinese herbal medicine categories in the mixed category. The sub-classification model is highly targeted, the model size is small, the training process is simple, and the recognition accuracy is high.

[0032] Q400, obtain the feature vector set XA corresponding to the Chinese herbal medicine to be identified = (XA 1 , XA 2 , …, XA u , …, XA v ), u=1, 2,...,v; among them, XA u is the uth feature vector group corresponding to the Chinese herbal medicine to be identified, v is the number of feature vector groups in the feature vector group set corresponding to the Chinese herbal medicine to be identified; XA u Includes several feature vectors of different dimensions; the last k feature vector groups of XA are empty.

[0033] Further, step Q400 may include the following steps:

[0034] Q410, identify each Chinese herbal medicine to be identified in the image corresponding to the Chinese herbal medicine to be identified, so as to obtain a list of Chinese herbal medicines to be identified A=(A 1 , A 2 , …, A i , …, A n ), i=1, 2,...,n; among them, A i is the ith Chinese herbal medicine to be identified in the image corresponding to the acquired Chinese herbal medicine to be identified, and n is the number of Chinese herbal medicines to be identified in the image corresponding to the Chinese herbal medicine to be identified.

[0035] In this embodiment, when photographing the Chinese herbal medicine to be identified, multiple Chinese herbal medicines are usually photographed. Therefore, there are several Chinese herbal medicines to be identified in the image corresponding to the Chinese herbal medicine to be identified. The existing target object recognition method can be used to identify each Chinese herbal medicine to be identified in the image corresponding to the Chinese herbal medicine to be identified, thereby obtaining A.

[0036] Q420, obtain the image parameter vector corresponding to each Chinese herbal medicine to be identified in A, so as to obtain the image parameter vector list B corresponding to A=(B 1 , B 2 , …, B i , …, B n ), where B i A i The corresponding image parameter vector; B i =(B i，1 , B i，2 , …, B i，j , …, B i，m ), j = 1, 2, ..., m; B i，j A i The corresponding j-th image parameter, m is the number of image parameters.

[0037] In this embodiment, the image parameters include clarity, completeness, shooting angle, etc. When photographing several Chinese herbal medicines to be identified that are placed together, some of the Chinese herbal medicines to be identified have good shooting angles, cleanliness and completeness, while some are poorer. The clarity, completeness and shooting angle corresponding to each Chinese herbal medicine to be identified can be obtained; the completeness can be understood as the proportion of the unobstructed part to the whole part; the shooting angle can be understood as the angle between the front of the Chinese herbal medicine to be identified and the axis of the camera; thereby, the image parameter vector corresponding to each Chinese herbal medicine to be identified can be obtained.

[0038] Q430, obtain the similarity between each image parameter vector in B and the standard image parameter vector BA corresponding to LA, so as to obtain the similarity list corresponding to B α=(α 1 , α 2 , …, α i , …, α n ), where α i For B i Similarity with BA.

[0039] In this embodiment, a standard image parameter vector is preset for each mixed category; the standard image parameter vector can be obtained by photographing a large number of Chinese herbal medicines corresponding to a single LA; the standard image parameter vector can accurately characterize the corresponding Chinese herbal medicines; the higher the similarity between the image parameter vector in B and BA, the better the placement posture and angle of the Chinese herbal medicine to be identified corresponding to the image parameter vector, and the more accurately the features of each dimension can be extracted.

[0040] Q440, traverse α, if α i ≥α', then B i Determine the target Chinese herbal medicine to be identified, so as to obtain the target Chinese herbal medicine list C = (C 1 , C2 , …, C p , …, C q ), p = 1, 2, ..., q; where C p is the pth target Chinese herbal medicine determined, q is the number of the target Chinese herbal medicines determined; α' is the preset similarity threshold; C x The similarity is greater than C x+1 similarity; x=1, 2,…, q-1.

[0041] In this embodiment, if α i ≥α', indicating α i The corresponding Chinese herbal medicine to be identified has a better placement posture, and its corresponding features in various dimensions can be better extracted later. Therefore, it is determined as the target Chinese herbal medicine to be identified, thereby determining the Chinese herbal medicine to be identified with a better placement posture.

[0042] Q450, based on C, determine the feature vector XA corresponding to the Chinese herbal medicine to be identified.

[0043] Further, step Q450 may include the following steps:

[0044] Q451, obtain the preset initial feature vector set CA = (CA 1 , C.A. 2 , …, C.A. u , …, C.A. v ), u=1, 2, …, v; where CA u is the uth initial feature vector group in the preset initial feature vector group set; CA u =(CA u，1 , C.A. u，2 , …, C.A. u，a , …, C.A. u，b ), a=1, 2, …, b; CA u，a is the feature vector of the ath dimension in the uth initial feature vector group in the preset initial feature vector group set, b is the number of dimensions of the preset feature vector; CA is empty.

[0045] In this embodiment, an empty initial feature vector set CA may be preset, the length of CA being the same as the length of XA, and being used to store the feature vector of each dimension of each Chinese herbal medicine to be identified.

[0046] Q452, Q452, obtain the feature vector of each dimension corresponding to each target Chinese herbal medicine in C, so as to obtain the feature vector group list XC corresponding to C = (XC 1 , XC 2 , …, XC p , …, XC q ), where XC pC p The corresponding feature vector group; XC p =(XC p，1 , XC p，2 , …, XC p，a , …, XC p，b ); XC p，a C p The corresponding feature vector of the a-th dimension.

[0047] In this embodiment, for a Chinese herbal medicine to be identified, it corresponds to feature vectors of multiple dimensions. For example, the feature vector of the texture of the target Chinese herbal medicine to be identified can be extracted to obtain a feature vector of the texture dimension; the feature vector of the color of the target Chinese herbal medicine to be identified can be extracted to obtain a feature vector of the color dimension of the Chinese herbal medicine to be identified; thereby obtaining a feature vector group corresponding to each target Chinese herbal medicine to be identified.

[0048] Q453, if q=vk, then fill each feature vector group in XC into the first vk empty feature vector groups in XA in turn to obtain XA.

[0049] In this embodiment, the last k feature vector groups in XA are empty and serve as spare empty vector groups for storing the non-image feature vectors of the Chinese herbal medicines to be identified subsequently; if q=vk, it means that the number of the identified target Chinese herbal medicines to be identified is the same as the number of vector groups for storing the image features of the Chinese herbal medicines to be identified in XA, then each feature vector group in XC can be filled in turn into the first vk empty feature vector groups in XA to obtain XA; it can be understood that the last k feature vector groups of XA are still empty.

[0050] Q454, if q>vk, then fill the first vk feature vector groups in XC into CA in sequence to obtain XA.

[0051] In this embodiment, if q>vk, it means that the number of identified target Chinese herbal medicines is greater than the number of vector groups in XA used to store image features of the Chinese herbal medicines to be identified. Then the first vk feature vector groups in XC are filled into CA in sequence to obtain XA.

[0052] Q455, if q<vk, obtain the first preset value N=0 and the second preset value M=1.

[0053] Q456, if N×q+M=vk, then CX M Fill in CA N×q+M If yes, go to Q457; otherwise, go to Q458.

[0054] Q457, if M<q, obtain M=M+1; otherwise, obtain M=1 and N=N+1; enter Q456.

[0055] Q458, the CA filled into the feature vector group is determined to be XA.

[0056] In this embodiment, there is also a situation where q<vk, q<vk, which means that the number of identified target Chinese herbal medicines to be identified is less than the number of vector groups in XA used to store the image features of the Chinese herbal medicines to be identified. At this time, the feature vector groups in XC need to be cyclically filled into CA in sequence to obtain XA.

[0057] Q500, input XA into ZA to obtain the confidence of the Chinese herbal medicine to be identified corresponding to each preset category of Chinese herbal medicine, so as to obtain a confidence list θ=(θ 1 ,θ 2 ,…,θ f ,…,θ g ), f = 1, 2, ..., g; where θ f is the confidence that the Chinese herbal medicine to be identified is the fth preset category of Chinese herbal medicine, and g is the number of preset Chinese herbal medicine categories.

[0058] In this embodiment, XA is input into ZA, and the confidence that the Chinese herbal medicine to be identified output by ZA is the Chinese herbal medicine of each preset category is obtained, thereby obtaining θ.

[0059] Q600, traverse θ, if there are at least two confidences in θ that are greater than a preset confidence threshold, obtain the non-image feature vector JE of the Chinese herbal medicine to be identified.

[0060] In this embodiment, if there are at least two confidence levels in θ that are greater than a preset confidence threshold, it means that the probability that the Chinese herbal medicine to be identified is at least two categories of Chinese herbal medicine is relatively high. At this time, if the result with the highest confidence level is directly output, it is not necessarily correct. Therefore, the non-image feature vector JE of the Chinese herbal medicine to be identified is obtained; the non-image feature can be the smell characteristics, texture characteristics, and taste characteristics of the Chinese herbal medicine to be identified.

[0061] Furthermore, after step Q600, the at least one instruction or the at least one program is loaded and executed by the processor, and the following steps are also implemented:

[0062] Q610, if there is only one confidence level in θ that is greater than a preset confidence threshold, the category of the Chinese herbal medicine corresponding to the confidence level in θ that is greater than the preset confidence threshold is determined as the category corresponding to the Chinese herbal medicine to be identified.

[0063] In this embodiment, if there is only one confidence value in θ that is greater than a preset confidence threshold, it means that the sub-classification model accurately identifies the category corresponding to the Chinese herbal medicine to be identified.

[0064] Q700, fill JE into the empty vector in XA to obtain the target vector set XA' corresponding to the Chinese herbal medicine to be identified.

[0065] In this embodiment, in the above steps, the last k feature vector groups of XA are always empty, and JE can be filled into some of the last k empty feature vector groups of XA to obtain XA'.

[0066] Q800, input XA' into ZA to obtain the corresponding category of the Chinese herbal medicine to be identified.

[0067] In this embodiment, since there are great differences in the smell, taste and texture corresponding to different categories of Chinese herbal medicines, after adding the above-mentioned non-image feature vector to XA, the sub-classification model can accurately identify the category corresponding to the Chinese herbal medicine to be identified, thereby improving the recognition accuracy of Chinese herbal medicines of similar categories.

[0068] In this embodiment, the Chinese herbal medicine to be identified is firstly identified based on its appearance features. If the identification result is a mixed category, a sub-classification model corresponding to the mixed category is obtained, and the feature vector set corresponding to the Chinese herbal medicine to be identified is input into the sub-classification model, and the confidence of the Chinese herbal medicine to be identified corresponding to each preset category of Chinese herbal medicine output by the sub-classification model is obtained. If there are at least two confidences greater than a preset confidence threshold, a non-image feature vector of the Chinese herbal medicine to be identified is obtained; the non-image feature vector is filled into the empty vector in the feature vector set corresponding to the Chinese herbal medicine to be identified to obtain the target vector set XA' corresponding to the Chinese herbal medicine to be identified; XA' is input into ZA to obtain the category corresponding to the Chinese herbal medicine to be identified; since the non-image features corresponding to the Chinese herbal medicine to be identified are further added, the sub-classification model can accurately determine the category corresponding to the Chinese herbal medicine to be identified.

[0069] At the same time, the present embodiment incorporates large model technology and integrates the capabilities of large models to provide users with more accurate and efficient solutions in the future.

[0070] Embodiment 2:

[0071] Based on the above embodiment 1, in order to improve recognition efficiency, the processor may further perform the following steps:

[0072] S100, obtaining the appearance feature TA corresponding to the Chinese herbal medicine to be identified; wherein TA is obtained through the image corresponding to the Chinese herbal medicine to be identified.

[0073] In this embodiment, the Chinese herbal medicine to be identified can be photographed by a depth camera to generate a three-dimensional image corresponding to the Chinese herbal medicine to be identified. There may be multiple Chinese herbal medicines in the image corresponding to the Chinese herbal medicine to be identified. Each Chinese herbal medicine can be first identified using a large model, and then the size information of one of the Chinese herbal medicines can be obtained to obtain the appearance feature TA corresponding to the Chinese herbal medicine to be identified; for example, the length, width, height, outline and shape and other parameters of a Chinese herbal medicine to be identified are obtained, and then the corresponding appearance features are generated; it should be noted that those skilled in the art can use the existing appearance feature generation method to generate the appearance features corresponding to the Chinese herbal medicine to be identified, which will not be elaborated here.

[0074] S200, inputting TA into a preset appearance classification model to obtain an initial category LA corresponding to the Chinese herbal medicine to be identified.

[0075] In this embodiment, the appearance classification model is a model obtained after a large amount of training, for example: a large number of Chinese herbal medicines of different categories and shapes are obtained, and then the appearance features corresponding to the Chinese herbal medicines of each category are obtained, and the preset initial classification model is trained to obtain the preset appearance classification model; it should be noted that Chinese herbal medicines of different categories may have similarities in appearance, for example: Stephania tetrandra and Cnidium monnieri, Rubus idaeus and Psoralea corylifolia, Codonopsis pilosula and Saposhnikovia divaricata, Prunus persica and Apricot kernel are relatively similar in appearance; while some Chinese herbal medicines will not be similar to other categories of Chinese herbal medicines, for example: Panax notoginseng, etc., are not similar in appearance to other categories of Chinese herbal medicines; therefore, the initial category obtained according to the appearance characteristics may be a single category or a mixed category.

[0076] S300, if LA is a preset single category, LA is determined as the category corresponding to the Chinese herbal medicine to be identified; wherein the single category only includes one category of Chinese herbal medicine.

[0077] In this embodiment, the results output by the appearance classification model are divided into a single category and a mixed category. The single category is a category that only contains one Chinese herbal medicine. Therefore, if LA is a preset single category, it means that the Chinese herbal medicine to be identified is identified as a single category of Chinese herbal medicine according to its corresponding appearance characteristics. For example, the single category in which Panax notoginseng is located only contains Panax notoginseng. Since a single category only contains one category of Chinese herbal medicine, if LA is a preset single category, LA can be directly determined as the category corresponding to the Chinese herbal medicine to be identified without the need for subsequent judgment, thereby improving the recognition efficiency while ensuring the recognition accuracy.

[0078] S400, if LA is a preset mixed category, obtain a sub-classification model ZA corresponding to LA; wherein the mixed category includes at least two Chinese herbal medicine categories; and each mixed category corresponds to a preset sub-classification model.

[0079] In this embodiment, the mixed category includes at least two categories of Chinese herbal medicines. For example, a mixed category includes two categories of Chinese herbal medicines, namely peach kernels and almonds. If the Chinese herbal medicine to be identified is identified as almonds, then the initial category of the Chinese herbal medicine to be identified is a mixed category. Since peach kernels and almonds are very similar in appearance, when the result output by the appearance classification model is a mixed category, the result may be wrong. Therefore, when the identification result of the Chinese herbal medicine to be identified is a mixed category, further judgment is required. A corresponding sub-classification model is preset for each mixed category. The sub-classification model corresponding to each mixed category is trained for the Chinese herbal medicine categories included in the mixed category, and can accurately distinguish similar Chinese herbal medicine categories in the mixed category. The sub-classification model is highly targeted, the model size is small, the training process is simple, and the recognition accuracy is high.

[0080] Furthermore, the sub-classification model ZA corresponding to the LA can be obtained by the following steps:

[0081] S410, obtaining a number of Chinese herbal medicine image samples corresponding to LA to obtain a Chinese herbal medicine image sample list D = (D 1 , D 2 , …, D c , …, D e ), c = 1, 2, ..., e; where D c is the cth Chinese herbal medicine image sample obtained, and d is the number of Chinese herbal medicine image samples obtained.

[0082] S420, obtaining the feature vector of each dimension of each Chinese herbal medicine image sample in D, so as to obtain the feature vector list set LD corresponding to D = (LD 1 , LD 2 ,…,LD c ,…,LD e ), where LD c D c The corresponding list of eigenvectors; LD c =(LD c，1 , LD c，2 ,…,LD c，a ,…,LD c，b );LD c，a D c The corresponding feature vector of the a-th dimension.

[0083] S430, respectively identify the Chinese herbal medicine image samples in D according to the feature vectors of each dimension in LD to obtain the recognition accuracy rate corresponding to the feature vectors of each dimension, and then obtain the recognition accuracy rate list β corresponding to LD = (β 1 , β 2 , …, β a , …, β b), where β a is the recognition accuracy corresponding to the feature vector of the ath dimension.

[0084] In this embodiment, when identifying Chinese herbal medicine image samples, it can be based on features of multiple dimensions, for example: it can be identified based on features of texture dimension, it can also be identified based on features of color dimension, and it can also be identified based on features of shape dimension; for features of different dimensions, the corresponding recognition accuracy is also different; the recognition accuracy corresponding to the features of each dimension can be obtained; it can be understood that the higher the recognition accuracy, the more capable the feature is of representing the corresponding Chinese herbal medicine.

[0085] S440, according to β, determine the training weight corresponding to the feature vector of each dimension to obtain a training weight list γ=(γ 1 , γ 2 , …, γ a , …, γ b ), where γ a is the training weight corresponding to the feature vector of the ath dimension; γ a =β a / ∑ b a＝1 β a .

[0086] S450, according to γ, set corresponding training weights for the feature vectors of each dimension to train the initial sub-classification model to obtain the sub-classification model ZA corresponding to LA.

[0087] In this embodiment, for more important features, a higher weight is set for the feature when training the sub-model, thereby improving the training efficiency of the sub-model and the accuracy of subsequent recognition.

[0088] S500, determining the target Chinese herbal medicine that meets the preset conditions according to a number of image parameters corresponding to the sub-image of each Chinese herbal medicine in the image corresponding to the Chinese herbal medicine to be identified; the image parameters include the clarity, completeness and shooting angle of each sub-image of the Chinese herbal medicine.

[0089] Further, step S500 may include the following steps:

[0090] S510, identifying each Chinese herbal medicine to be identified in the image corresponding to the Chinese herbal medicine to be identified, so as to obtain a list of Chinese herbal medicines to be identified A=(A 1 , A 2 , …, A i , …, A n ), i=1, 2,...,n; among them, A iis the ith Chinese herbal medicine to be identified in the image corresponding to the acquired Chinese herbal medicine to be identified, and n is the number of Chinese herbal medicines to be identified in the image corresponding to the Chinese herbal medicine to be identified.

[0091] In this embodiment, when photographing the Chinese herbal medicine to be identified, multiple Chinese herbal medicines are usually photographed. Therefore, there are several Chinese herbal medicines to be identified in the image corresponding to the Chinese herbal medicine to be identified. The existing target object recognition method can be used to identify each Chinese herbal medicine to be identified in the image corresponding to the Chinese herbal medicine to be identified, thereby obtaining A.

[0092] S520, obtaining the image parameter vector corresponding to each Chinese herbal medicine to be identified in A, so as to obtain the image parameter vector list B corresponding to A=(B 1 , B 2 , …, B i , …, B n ), where B i A i The corresponding image parameter vector; B i =(B i，1 , B i，2 , …, B i，j , …, B i，m ), j = 1, 2, ..., m; B i，j A i The corresponding j-th image parameter, m is the number of image parameters.

[0093] In this embodiment, the image parameters include clarity, completeness, shooting angle, etc. When photographing several Chinese herbal medicines to be identified that are placed together, some of the Chinese herbal medicines to be identified have good shooting angles, cleanliness and completeness, while some are poorer. The clarity, completeness and shooting angle corresponding to each Chinese herbal medicine to be identified can be obtained; the completeness can be understood as the proportion of the unobstructed part to the whole part; the shooting angle can be understood as the angle between the front of the Chinese herbal medicine to be identified and the axis of the camera; thereby, the image parameter vector corresponding to each Chinese herbal medicine to be identified can be obtained.

[0094] S530, obtaining the similarity between each image parameter vector in B and the standard image parameter vector BA corresponding to LA, so as to obtain a similarity list corresponding to B α=(α 1 , α 2 , …, α i , …, α n ), where α i For B i Similarity with BA.

[0095] In this embodiment, a standard image parameter vector is preset for each mixed category; the standard image parameter vector can be obtained by photographing a large number of Chinese herbal medicines corresponding to a single LA; the standard image parameter vector can accurately characterize the corresponding Chinese herbal medicines; the higher the similarity between the image parameter vector in B and BA, the better the placement posture and angle of the Chinese herbal medicine to be identified corresponding to the image parameter vector, and the more accurately the features of each dimension can be extracted.

[0096] S540, traverse α, if α i ≥α', then B i Determine the target Chinese herbal medicine to be identified, so as to obtain the target Chinese herbal medicine list C = (C 1 , C 2 , …, C p , …, C q ), p = 1, 2, ..., q; where C p is the pth target Chinese herbal medicine determined, q is the number of the target Chinese herbal medicines determined; α' is the preset similarity threshold; C x The similarity is greater than C x+1 similarity; x=1, 2,…, q-1.

[0097] In this embodiment, if α i ≥α', indicating α i The corresponding Chinese herbal medicine to be identified has a better placement posture, and its corresponding features in various dimensions can be better extracted later. Therefore, it is determined as the target Chinese herbal medicine to be identified, thereby determining the Chinese herbal medicine to be identified with a better placement posture.

[0098] S600, determining a feature vector set XA corresponding to the Chinese herbal medicine to be identified according to feature vectors of different dimensions corresponding to each target Chinese herbal medicine.

[0099] Further, step S600 may include the following steps:

[0100] S610, obtaining a preset initial feature vector set CA = (CA 1 , C.A. 2 , …, C.A. r , …, C.A. s ), r = 1, 2, ..., s; where CA r is the rth initial feature vector group in the preset initial feature vector group set, s is the number of initial feature vector groups in the preset initial feature vector group set; CA r =(CA r，1 , C.A. r，2 , …, C.A. r，a , …, C.A. r，b ), a=1, 2, …, b; CA r，ais the feature vector of the ath dimension in the rth initial feature vector group in the preset initial feature vector group set, b is the number of dimensions of the preset feature vector; CA is empty.

[0101] In this embodiment, an empty initial feature vector set CA may be preset to store the feature vector of each dimension of each Chinese herbal medicine to be identified.

[0102] S620, obtaining the feature vector of each dimension corresponding to each target Chinese herbal medicine in C, so as to obtain a feature vector group list XC corresponding to C=(XC 1 , XC 2 , …, XC p , …, XC q ), where XC p C p The corresponding feature vector group; XC p =(XC p，1 , XC p，2 , …, XC p，a , …, XC p，b ); XC p，a C p The corresponding feature vector of the a-th dimension.

[0103] In this embodiment, for a Chinese herbal medicine to be identified, it corresponds to feature vectors of multiple dimensions. For example, the feature vector of the texture of the target Chinese herbal medicine to be identified can be extracted to obtain a feature vector of the texture dimension; the feature vector of the color of the target Chinese herbal medicine to be identified can be extracted to obtain a feature vector of the color dimension of the Chinese herbal medicine to be identified; thereby obtaining a feature vector group corresponding to each target Chinese herbal medicine to be identified.

[0104] S630, if q=s, fill the feature vector groups in XC into CA in sequence to obtain XA.

[0105] In this embodiment, if q=s, it means that the number of identified target Chinese herbal medicines is the same as the number of feature vector groups in the preset initial feature vector group set, then XC can be directly determined as XA; or each feature vector group in XC can be filled into CA in turn to obtain XA.

[0106] Furthermore, after step S630, the method may further include the following steps:

[0107] S640, if q>s, then fill the first s feature vector groups in XC into CA in sequence to obtain XA.

[0108] In this embodiment, if q>s, it means that the number of target Chinese herbal medicines identified is greater than the number of feature vector groups in the preset initial feature vector group set. At this time, only the feature vector groups of some target Chinese herbal medicines to be identified are needed. Therefore, the first s feature vector groups in XC are filled into CA in sequence to obtain XA.

[0109] S650: If q<s, obtain a first preset value N=0 and a second preset value M=1.

[0110] S660, if N×q+M<s, then XC M Fill in CA N×q+M If yes, execute S670; otherwise, execute S680.

[0111] S670, if M<q, obtain M=M+1; otherwise, obtain M=1 and N=N+1; and enter S660.

[0112] S680, the CA filled into the feature vector group is determined as XA.

[0113] In this embodiment, there is also a situation where q<s, q<s means that the number of target Chinese herbal medicines to be identified is less than the number of feature vector groups in the preset initial feature vector group set. At this time, the feature vector groups in XC need to be filled into CA in sequence to obtain XA.

[0114] Through the above method, the Chinese herbal medicines with better placement postures in the images of the Chinese herbal medicines to be identified can be identified, and feature vectors of different categories can be extracted for them, so that the subsequent model can refer to features of more dimensions when identifying the Chinese herbal medicines to be identified, thereby improving the recognition accuracy.

[0115] S700, input XA into ZA to obtain the category corresponding to the Chinese herbal medicine to be identified.

[0116] In this embodiment, ZA is a trained sub-classification model, which can accurately identify the category corresponding to the Chinese herbal medicine to be identified according to XA.

[0117] In this embodiment, the appearance features corresponding to the Chinese herbal medicine to be identified are obtained through the image corresponding to the Chinese herbal medicine to be identified; the appearance features are input into a preset appearance classification model to obtain the initial category corresponding to the Chinese herbal medicine to be identified; if the initial category is a preset single category, the initial category is determined as the category corresponding to the Chinese herbal medicine to be identified; for Chinese herbal medicines of a single category, they can be identified through the appearance features. Since the appearance features are relatively simple, the category corresponding to the Chinese herbal medicine to be identified can be quickly and accurately determined through the method of the present invention; in addition, if the initial category is a preset mixed category, a sub-classification model corresponding to the initial category is obtained; a feature vector corresponding to the Chinese herbal medicine to be identified is obtained; the feature vector corresponding to the Chinese herbal medicine to be identified is input into the sub-classification model corresponding to the initial category to obtain the category corresponding to the Chinese herbal medicine to be identified; for Chinese herbal medicines with multiple relatively similar mixed categories, corresponding sub-classification models are preset, and Chinese herbal medicines of each category in the mixed category can be identified, thereby improving the recognition accuracy of similar Chinese herbal medicines.

[0118] At the same time, the present embodiment incorporates large model technology and integrates the capabilities of large models to provide users with more accurate and efficient solutions in the future.

[0119] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0120] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.

[0121] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0122] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0123] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0124] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0125] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0126] The electronic device is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0127] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one memory mentioned above, and a bus connecting different system components (including the memory and the processor).

[0128] The memory stores program codes, which can be executed by the processor, so that the processor executes the steps in various embodiments described in this specification.

[0129] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0130] The memory may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0131] The bus may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0132] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0133] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0134] An embodiment of the present invention further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method according to various exemplary embodiments of the present invention described above in this specification.

[0135] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are only for illustration, not for limiting the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention.

Claims

1. A visual identification system for Chinese herbal medicine categories, characterized in that: The system includes: a processor and a storage medium, wherein the storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the following steps: Q100, obtaining the appearance feature TA corresponding to the Chinese herbal medicine to be identified; wherein TA is obtained through the image corresponding to the Chinese herbal medicine to be identified; Q200, inputting TA into a preset appearance classification model to obtain the initial category LA corresponding to the Chinese herbal medicine to be identified; Q300, if LA is a preset mixed category, then obtain a sub-classification model ZA corresponding to LA; wherein the mixed category includes at least two Chinese herbal medicine categories; and each mixed category corresponds to a preset sub-classification model; Q400, obtain the feature vector set XA corresponding to the Chinese herbal medicine to be identified = (XA1, XA2, ..., XA u , …, XA v ), u=1, 2,...,v; among them, XA u is the uth feature vector group corresponding to the Chinese herbal medicine to be identified, v is the number of feature vector groups in the feature vector group set corresponding to the Chinese herbal medicine to be identified; XA u Includes several feature vectors of different dimensions; the last k feature vector groups of XA are empty; Q500, input XA into ZA to obtain the confidence of the Chinese herbal medicine to be identified corresponding to each preset category of Chinese herbal medicine, so as to obtain a confidence list θ=(θ1, θ2, ..., θ f ,…,θ g ), f = 1, 2, ..., g; where θ f is the confidence that the Chinese herbal medicine to be identified is the fth preset category of Chinese herbal medicine, and g is the number of preset Chinese herbal medicine categories; Q600, traverse θ, if there are at least two confidences in θ that are greater than a preset confidence threshold, obtain the non-image feature vector JE of the Chinese herbal medicine to be identified; Q700, fill JE into the empty vector in XA to obtain the target vector set XA' corresponding to the Chinese herbal medicine to be identified; Q800, input XA' into ZA to obtain the corresponding category of the Chinese herbal medicine to be identified.

2. The Chinese herbal medicine category visual identification system according to claim 1, characterized in that: After step Q600, the at least one instruction or the at least one program is loaded and executed by the processor, and the following steps are also implemented: Q610, if there is only one confidence level in θ that is greater than a preset confidence threshold, the category of the Chinese herbal medicine corresponding to the confidence level in θ that is greater than the preset confidence threshold is determined as the category corresponding to the Chinese herbal medicine to be identified.

3. The Chinese herbal medicine category visual identification system according to claim 1, characterized in that: After step Q200, the at least one instruction or the at least one program is loaded and executed by the processor, and the following steps are also implemented: Q210, if LA is a preset single category, LA is determined as the category corresponding to the Chinese herbal medicine to be identified; wherein a single category only includes one category of Chinese herbal medicine.

4. The Chinese herbal medicine category visual identification system according to claim 1, characterized in that: Q400 includes the following steps: Q410, identify each Chinese herbal medicine to be identified in the image corresponding to the Chinese herbal medicine to be identified, so as to obtain a list of Chinese herbal medicines to be identified A=(A1, A2, ..., A i , …, A n ), i=1, 2,...,n; where, A i is the i-th Chinese herbal medicine to be identified in the image corresponding to the obtained Chinese herbal medicine to be identified, and n is the number of Chinese herbal medicines to be identified in the image corresponding to the Chinese herbal medicine to be identified; Q420, obtain the image parameter vector corresponding to each Chinese herbal medicine to be identified in A, so as to obtain the image parameter vector list B corresponding to A = (B1, B2, ..., B i , …, B n ), where B i A i The corresponding image parameter vector; B i =(B i，1 , B i，2 , …, B i，j , …, B i，m ), j = 1, 2, ..., m; B i，j A i The corresponding j-th image parameter, m is the number of image parameters; Q430, obtain the similarity between each image parameter vector in B and the standard image parameter vector BA corresponding to LA, so as to obtain a similarity list α corresponding to B = (α1, α2, ..., α i , …, α n ), where α i For B i Similarity with BA; Q440, traverse α, if α i ≥α', then B i Determine the target Chinese herbal medicine to be identified, so as to obtain a target Chinese herbal medicine list C = (C1, C2, ..., C p , …, C q ), p = 1, 2, ..., q; where C p is the pth target Chinese herbal medicine determined, q is the number of the target Chinese herbal medicines determined; α' is the preset similarity threshold; C x The similarity is greater than C x+1 Similarity; x = 1, 2, ..., q-1; Q450, based on C, determine the feature vector set XA corresponding to the Chinese herbal medicine to be identified.

5. The Chinese herbal medicine category visual identification system according to claim 4, characterized in that: Step Q450 includes the following steps: Q451, obtain a preset initial feature vector set CA = (CA1, CA2, ..., CA u , …, C.A. v ), u=1, 2, …, v; where CA u is the uth initial feature vector group in the preset initial feature vector group set; CA u =(CA u，1 , C.A. u，2 , …, C.A. u，a , …, C.A. u，b ), a=1, 2, …, b; CA u，a is the feature vector of the ath dimension in the uth initial feature vector group in the preset initial feature vector group set, b is the number of dimensions of the preset feature vector; CA is empty; Q452, obtain the feature vector of each dimension corresponding to each target Chinese herbal medicine in C, so as to obtain the feature vector group list XC corresponding to C = (XC1, XC2, ..., XC p , …, XC q ), where XC p C p The corresponding feature vector group; XC p =(XC p，1 , XC p，2 , …, XC p，a , …, XC p，b ); XC p，a C p The corresponding feature vector of the ath dimension; Q453, if q = vk, then fill each feature vector group in XC into the first vk empty feature vector groups in XA in turn to obtain XA; Q454, if q>vk, then fill the first vk feature vector groups in XC into CA in sequence to obtain XA; Q455, if q<vk, obtain the first preset value N=0 and the second preset value M=1; Q456, if N×q+M=vk, then CX M Fill in CA N×q+M In; go to Q457, otherwise, go to Q458; Q457, if M<q, obtain M=M+1; otherwise, obtain M=1 and N=N+1; enter Q456; Q458, the CA filled into the feature vector group is determined to be XA.

6. The Chinese herbal medicine category visual identification system according to claim 1, characterized in that: The non-image feature vector JE is obtained through the smell characteristics, taste characteristics and texture characteristics of the Chinese herbal medicine to be identified.

7. The Chinese herbal medicine category visual identification system according to claim 4, characterized in that: The image parameters include clarity, completeness, and shooting angle.

8. The Chinese herbal medicine category visual identification system according to claim 1, characterized in that: The image corresponding to the Chinese herbal medicine to be identified is a three-dimensional image.

Images