Method and device for identifying traditional Chinese medicine decoction pieces

Abstract

The invention relates to a traditional Chinese medicine decoction piece identification method and device, and the method comprises the steps: obtaining color images, depth images and weight data of traditional Chinese medicine decoction pieces of different types and different masses, carrying out the preprocessing of the weight data, the color images and the depth images, and generating segmented images, weight images, and processed color images and depth images; obtaining a training data set marked with the category and quality of the decoction pieces; inputting the training data set into a decoction piece identification network for training to obtain a trained decoction piece identification model; and obtaining a color image, a depth image and weight data of the to-be-identified decoction piece, preprocessing the color image, the depth image and the weight data of the to-be-identified decoction piece to generate a segmentation image, a weight image, the processed color image and the processed depth image of the to-be-identified decoction piece, inputting the segmentation image, the weight image, the processed color image and the processed depth image into the decoction piece identification model for reasoning, and outputting an identification result. According to the invention, accurate and rapid identification of the type and quality of the traditional Chinese medicine decoction pieces is realized based on the multi-scale feature identification model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for identifying Chinese herbal medicine slices. Background Art

[0002] Chinese herbal medicine slices refer to the products of Chinese herbal medicines after processing, which can be directly used in clinical practice or in the production of preparations. Chinese herbal medicine slices are usually processed on the basis of Chinese herbal medicines, such as slicing, dicing, grinding and other processes. During the processing, Chinese herbal medicines are usually cut into regular shapes, such as round slices, oval slices, etc., for easy identification and measurement. Generally, different types and different qualities of herbal medicines of the same category have differences in appearance and other attributes, such as color, size, texture, density, etc.

[0003] The difficulty in identifying Chinese herbal medicines lies in the fact that many different types of herbal medicines are similar in appearance with subtle differences. The differences between qualified, unqualified and high-quality herbal medicines of the same type are also easily affected by other factors. In addition, the efficiency of classifying herbal medicines by traditional methods (such as classification based on the color or shape of the Chinese medicine) is low, and it requires more reliance on professional medical knowledge, and sometimes it is difficult to distinguish even with professional knowledge.

[0004] At present, the recognition schemes of Chinese herbal medicine pieces based on deep learning are still relatively limited. Moreover, with the increase in the types and similarities of herbal medicine pieces, the recognition accuracy and recognition rate of deep learning models face serious challenges and show a downward trend.

[0005] In the known solutions that use cameras to collect medicinal material image data for intelligent identification, without limiting the collection light, collection camera, and collection environment, the collected images will be relatively different due to factors such as light, vibration, and the camera's own parameters. The image differences caused by image acquisition are greater than the differences between images of different types or different qualities, seriously affecting the identification of categories and quality.

[0006] In addition, information such as the size, weight, and density of Chinese herbal medicines determines their quality and is also key information used to distinguish different types. How to effectively incorporate this information into intelligent identification solutions is also a problem that needs to be further addressed. Summary of the invention

[0007] The embodiments of the present application provide a method and device for identifying Chinese herbal medicines, so as to at least provide a stable data collection environment, realize multi-dimensional data collection of Chinese herbal medicines, and realize accurate and rapid identification of the types and qualities of Chinese herbal medicines based on deep learning technology with multi-scale features, thereby improving the recognition speed and recognition accuracy.

[0008] In a first aspect, the present application provides a method for identifying Chinese herbal medicine pieces, comprising:

[0009] The training data set acquisition step is to acquire color images, depth images and weight data of Chinese herbal medicine pieces of different categories and qualities through a data acquisition module, and pre-process the weight data and the color images and depth images to generate a segmented image, a weight image and a processed color image and depth image to obtain a training data set marked with the category and quality of the herbal medicine pieces;

[0010] A model training step, inputting the training data set into a medicinal piece recognition network for training to obtain a trained medicinal piece recognition model;

[0011] The step of identifying Chinese herbal medicine pieces is to obtain the color image, depth image and weight data of the herbal medicine pieces to be identified through the data acquisition module, and pre-process the color image, depth image and weight data of the herbal medicine pieces to be identified to generate a segmented image, weight image and processed color image and depth image of the herbal medicine pieces to be identified, and input them into the herbal medicine piece identification model for reasoning, and output the identification result.

[0012] Based on the above steps, this application performs model training based on color images, depth images and weight data of Chinese herbal medicines of different categories and qualities, ensuring the richness and diversity of the training data set, which helps to improve the recognition accuracy and generalization ability of the model. Using the model to identify the types and quality of herbal medicines helps to reduce human intervention and errors, improve the efficiency and degree of automation of recognition, and provide strong technical support for the production, circulation, and use of Chinese herbal medicines.

[0013] In some embodiments, the training data set acquisition step includes:

[0014] The data acquisition step includes collecting color images and depth images of each category of Chinese herbal medicine pieces of different qualities to be identified through the image acquisition unit of the data acquisition module, and collecting weight data of each category of Chinese herbal medicine pieces of different qualities to be identified through the weighing unit of the data acquisition module; collecting multiple color images and depth images for each category of pieces helps to reduce accidental errors and improve data accuracy. In addition to the differences in quality and category dimensions, multiple image data can also be different in placement position, number of pieces, source of pieces, etc., thereby increasing data diversity;

[0015] A data preprocessing step, converting the weight data into a weight image, generating the segmented image based on the color image and the depth image, and normalizing the segmented image, the color image and the depth image;

[0016] The data concatenation step concatenates the segmented image, the weight image, the depth image and the color image along the image channel dimension.

[0017] Based on this, the fusion of multi-dimensional features can help the model understand the characteristics of Chinese herbal medicines more comprehensively and improve the accuracy and robustness of recognition.

[0018] In some embodiments, converting the weight data into a weight image comprises:

[0019] A weight reference value is obtained, a quotient of the weight data and the weight reference value is calculated, and the obtained quotient is filled into a two-dimensional matrix and stored as a weight image.

[0020] Through the above steps, the weight data is converted into a weight image, so that the weight information can be connected in series with other image data (such as color images, depth images) in the form of images, which can intuitively reflect the weight characteristics of Chinese herbal medicine pieces and is easy to visualize and explain. After connecting it with other image data, the feature information of the model input can be enriched, which helps the model to understand the characteristics of Chinese herbal medicine pieces more comprehensively, thereby improving the accuracy of recognition. During the model training and application process, the recognition effect of the model can be verified by observing the weight image, which helps to better understand and explain the decision-making process of the model.

[0021] In some embodiments, generating the segmented image based on the color image and the depth image includes:

[0022] The color image and the depth image are subjected to edge detection, and a union operation is performed on the edge detection results, the edge-surrounded area is filled with white, and the area outside the edge is filled with black, so as to form a segmented image.

[0023] In some of the embodiments, the segmented image, color image and depth image are normalized by dividing the pixel values ​​of the segmented image by 255 to generate a segmented image containing only 0 and 1; the color image can be processed using Min-Max normalization.

[0024] In some embodiments, the depth image is normalized, including:

[0025] A maximum value maxv and a minimum value minv are set, wherein the maximum value maxv represents the distance from the binocular camera of the image acquisition unit to the medicine tray of the weighing unit, and the minimum value minv represents the distance from the upper surface of the Chinese herbal medicine piece with the highest height to the binocular camera;

[0026] The depth value v of the depth image is calculated based on the following calculation model to obtain a normalized depth image. The calculation model is expressed as follows:

[0027]

[0028] In some embodiments, the herbal medicine recognition network includes a high-resolution multi-scale feature fusion network, an MHSA network (Multi Head Self Attention), a global average pooling layer (GA) and a fully connected layer (FC) connected in sequence.

[0029] In a second aspect, the present application embodiment provides a Chinese herbal medicine slice identification device, comprising:

[0030] The data acquisition module includes an image acquisition unit and a weighing unit disposed in a closed housing, and is used to obtain color images, depth images and weight data of the Chinese herbal medicine slices, so as to prevent the image acquisition unit from being affected by external light, dust and other factors during the acquisition process through the closed housing;

[0031] A processor, used to implement the Chinese herbal medicine slice identification method as described in the first aspect above when executing a preset computer program;

[0032] The display module is used to display information input by the user or information provided to the user and various menus of the user terminal.

[0033] The data acquisition module based on the above structure can realize multi-dimensional stable data acquisition, ensure the comprehensiveness of data acquisition, and display it in the display module, interact with the operator, and intuitively see the collected data and recognition results.

[0034] In some embodiments, the image acquisition unit includes:

[0035] A binocular camera, disposed on the top of the closed housing and electrically connected to the processor, for collecting two-dimensional high-definition color images and depth images;

[0036] The light source is arranged above or beside the binocular camera to avoid directly irradiating the camera lens and causing reflection or glare.

[0037] In another embodiment, in order to make the light of the light source more dispersed and soft, a soft light device such as soft light paper or soft light cloth is used under the light source to form a uniform and stable light, which is more conducive to image acquisition.

[0038] In some embodiments, the weighing unit is arranged at the bottom of the closed shell through a pull-out structure, and the weighing unit includes: an electronic scale and a medicine tray arranged on the upper surface of the electronic scale, the electronic scale is electrically connected to the processor, and the Chinese herbal medicines to be tested are placed on the medicine tray for weighing. The operator can pull out the weighing unit to place the Chinese herbal medicines to be tested or replace them, which improves the convenience and scalability of the device.

[0039] In some embodiments, the image acquisition unit further includes a brightness calibration unit, which is used to:

[0040] Detecting whether the brightness of the light source is attenuated based on a reference brightness, or taking a plurality of groups of images of an empty medicine tray, and calculating the average brightness of the plurality of groups of images of the medicine tray, and judging whether the deviation of the average brightness from the reference brightness exceeds a preset range based on a reference brightness;

[0041] If so, a brightness adjustment message is sent to the display module or the brightness of the light source is adjusted via a dimmer.

[0042] In some embodiments, the upper surface of the medicine tray is a frosted surface of a single color; the material is a material that does not deform after long-term use, such as stainless steel, engineering plastics, ceramics, etc.

[0043] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 is a flow chart of a method for identifying Chinese herbal medicine pieces according to an embodiment of the present application;

[0046] Figure 2 is a flowchart of the training data set acquisition step according to an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of the principle of the data concatenation step according to an embodiment of the present application;

[0048] Figure 4 It is a structural principle diagram of a decoction piece identification network according to an embodiment of the present application;

[0049] Figure 5 It is a structural schematic diagram of a Chinese herbal medicine slice identification device according to an embodiment of the present application.

[0050] In the figure:

[0051] 1. Data acquisition module; 2. Processor; 3. Display module;

[0052] 101. binocular camera; 102. light source; 103. soft light device;

[0053] 104. Electronic scale; 105. Medicine tray. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0055] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0056] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0057] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0058] This embodiment provides a method for identifying Chinese herbal medicine pieces. Figure 1-Figure 2 is a flow chart of a method for identifying Chinese herbal medicine pieces according to an embodiment of the present application, such as Figure 1-Figure 2 As shown, the process includes the following steps:

[0059] The training data set acquisition step S1 is to acquire color images, depth images and weight data of Chinese herbal medicine pieces of different categories and qualities through a data acquisition module, and pre-process the weight data and the color images and depth images to generate a segmented image, a weight image and a processed color image and depth image, so as to obtain a training data set marked with the category and quality of the herbal medicine pieces;

[0060] Model training step S2, inputting the training data set into a medicinal piece recognition network for training to obtain a trained medicinal piece recognition model;

[0061] In the Chinese herbal medicine slice identification step S3, the color image, depth image and weight data of the herbal medicine slice to be identified are obtained through the data acquisition module, and the color image, depth image and weight data of the herbal medicine slice to be identified are pre-processed to generate a segmented image, a weight image and a processed color image and depth image of the herbal medicine slice to be identified, and the images are input into the herbal medicine slice identification model for reasoning, and the identification result is output.

[0062] Based on the above steps, this application performs model training based on color images, depth images and weight data of Chinese herbal medicines of different categories and qualities, ensuring the richness and diversity of the training data set, which helps to improve the recognition accuracy and generalization ability of the model. Using the model to identify the types and quality of herbal medicines helps to reduce human intervention and errors, improve the efficiency and degree of automation of recognition, and provide strong technical support for the production, circulation, and use of Chinese herbal medicines.

[0063] refer to Figure 2 As shown, in the above embodiment, the training data set acquisition step S1 includes:

[0064] Data acquisition step S101, collecting color images and depth images of each category of Chinese herbal medicine pieces of different qualities to be identified through the image acquisition unit of the data acquisition module, and collecting weight data of each category of Chinese herbal medicine pieces of different qualities to be identified through the weighing unit of the data acquisition module; wherein the quality can at least be divided into qualified, unqualified, and excellent; the initial data set can be stored based on the category, quality and corresponding color image and depth image of the Chinese herbal medicine pieces, and collecting multiple color images and depth images for each category of pieces is helpful to reduce accidental errors and improve the accuracy of data. In addition to the differences in quality and category dimensions, multiple image data can also be different in placement position, number of pieces, source of pieces, etc., thereby increasing the diversity of data;

[0065] Data preprocessing step S102, converting the weight data into a weight image, generating the segmented image based on the color image and the depth image, and normalizing the segmented image, the color image and the depth image;

[0066] In the data concatenation step S103, the segmented image, the weight image, the depth image and the color image are concatenated along the image channel dimension to obtain a six-channel image, wherein the segmented image has one channel, the weight image has one channel, the depth image has one channel, and the color image has three channels. Figure 3 shown.

[0067] The preprocessing process in the Chinese herbal medicine slice identification step S3 is the same as the processing process in the training data set acquisition step S1, refer to the data preprocessing step S102 and the data concatenation step S103, which will not be repeated here.

[0068] Based on this, the fusion of multi-dimensional features can help the model understand the characteristics of Chinese herbal medicines more comprehensively and improve the accuracy and robustness of recognition.

[0069] In some embodiments, converting the weight data into a weight image comprises:

[0070] A weight reference value is obtained, and the quotient of the weight data and the weight reference value is calculated, and the obtained quotient is filled into a one-dimensional matrix and stored as a weight image. The weight reference value is configured as the weight of the medicine tray of the weighing unit covered with the heaviest type of Chinese herbal medicine pieces among the Chinese herbal medicine pieces to be identified. The number of rows and columns of the two-dimensional matrix is ​​the same as the size of the color image and the depth image. As shown in the figure, assuming that the weight data is 15.3 grams and the weight reference value is 100, the calculated quotient is 0.153, and 0.153 is filled into the two-dimensional matrix to generate a weight image with a filling value of 0.153 that meets the required length and width.

[0071] Through the above steps, the weight data is converted into a weight image, so that the weight information can be connected in series with other image data (such as color images, depth images) in the form of images, which can intuitively reflect the weight characteristics of Chinese herbal medicine pieces and is easy to visualize and explain. After connecting it with other image data, the feature information of the model input can be enriched, which helps the model to understand the characteristics of Chinese herbal medicine pieces more comprehensively, thereby improving the accuracy of recognition. During the model training and application process, the recognition effect of the model can be verified by observing the weight image, which helps to better understand and explain the decision-making process of the model.

[0072] In some embodiments, generating the segmented image based on the color image and the depth image includes:

[0073] The color image and the depth image are subjected to edge detection, and the edge detection results are subjected to a union operation, the edge-surrounded area is filled with white, the filling value is 255, indicating the medicinal material part, and the area outside the edge is filled with black, the filling value is 0, which is used to indicate the medicine tray part, to form a segmented image. Optionally, the Canny edge detector is used for edge detection to create a completely black image, and the inside of the found contour is filled with white. If it is necessary to ensure that the background is completely black, a mask operation can be performed again to create a mask and set the area outside the mask to black.

[0074] In some embodiments, the segmented image, color image and depth image are normalized by dividing the pixel value of the segmented image by 255 to generate a segmented image containing only 0 and 1; the color image can be processed by Min-Max normalization, or other normalization methods in deep learning, which are not listed here. The depth image is normalized, including:

[0075] A maximum value maxv and a minimum value minv are set, wherein the maximum value maxv represents the distance from the binocular camera of the image acquisition unit to the medicine tray of the weighing unit, and the minimum value minv represents the distance from the upper surface of the Chinese herbal medicine piece with the highest height to the binocular camera;

[0076] The depth value v of the depth image is calculated based on the following calculation model to obtain a normalized depth image. The calculation model is expressed as follows:

[0077]

[0078] refer to Figure 4 As shown, the slice identification network of this embodiment includes a high-resolution multi-scale feature fusion network, an MHSA network, a global pooling layer and a fully connected layer connected in sequence. Among them, the high-resolution multi-scale feature fusion is used as the backbone network, which is responsible for extracting the high-resolution and multi-scale features of the input data, and HRNet is specifically selected; the MHSA network receives the output features of the backbone network, and captures the feature correlation at different positions in the feature image through multiple attention heads, enhancing the model's ability to understand global and local features. Each attention head will learn a different weight matrix for constructing queries, keys and values, thereby calculating the attention score and updating the feature representation; the global pooling layer performs dimensionality reduction processing on the output features of the MHSA network, and reduces the feature dimension by calculating the average value of the feature map, while retaining global information; the fully connected layer receives the features after global pooling, performs category and quality prediction, and outputs the probability that the current input data belongs to each category and the probability that it belongs to each quality.

[0079] The training loss uses a binary cross entropy loss function to evaluate the model's predictive performance. This loss function is suitable for multi-label classification tasks, where each label is independent and can be true or false at the same time, and can comprehensively evaluate the model's performance in various categories and qualities. By continuously training and optimizing the model, the accuracy and efficiency of recognition can be further improved.

[0080] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0081] This embodiment provides a device for identifying Chinese herbal medicine pieces. Figure 5 Schematic diagram of the structure of the Chinese herbal medicine identification device according to the embodiment of the present application. Those skilled in the art can understand that Figure 5 The structure of the Chinese herbal medicine slice identification device shown in the figure does not constitute a limitation on the Chinese herbal medicine slice identification device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0082] Combine the following Figure 5 The components of the Chinese herbal medicine identification device are introduced in detail:

[0083] The Chinese herbal medicine slice identification device comprises: a data acquisition module 1, a processor 2, a display module 3 and other components. The processor 2 is used to implement the Chinese herbal medicine slice identification method as described in the above embodiment when executing a preset computer program; the display module 3 is used to display information input by the user or information provided to the user and various menus of the user terminal.

[0084] The data acquisition module 1 includes an image acquisition unit and a weighing unit arranged in a closed shell, which are used to obtain color images, depth images and weight data of Chinese herbal medicine slices, so as to avoid the image acquisition unit being affected by external light, dust and other factors during the acquisition process through the closed shell; wherein, the image acquisition unit includes: a binocular camera 101 and a light source 102, the binocular camera 101 is arranged on the top of the closed shell and electrically connected to the processor 2, and collects two-dimensional high-definition color images and collects depth images; optionally, the binocular camera 101 can also be replaced by a TOF camera or other cameras or camera combinations that can collect color images and depth images. The light source 102 is arranged above or beside the binocular camera 101 to avoid direct exposure to the camera lens, causing reflection or glare. Optionally, the light source 102 adopts a surface light source, a parallel light source, a coaxial light source, etc. When a surface light source is adopted, a camera shooting hole can be set on the light source panel to avoid the light source blocking the camera shooting. When multiple light sources 102 are used, the light sources 102 are evenly distributed around the camera to form a uniform lighting environment.

[0085] Among them, the weighing unit is arranged at the bottom of the closed shell through a pull-out structure, and the weighing unit includes: an electronic scale 104 and a medicine tray 105 arranged on the upper surface of the electronic scale 104, and the electronic scale 104 is electrically connected to the processor 2. The Chinese herbal medicines to be tested are placed on the medicine tray 105 for weighing. The operator can pull out the weighing unit to place the Chinese herbal medicines to be tested or replace them, which improves the convenience and scalability of the device.

[0086] The data acquisition module 1 based on the above structure can realize multi-dimensional stable data acquisition, ensure the comprehensiveness of data acquisition, and display it in the display module 3, interact with the operator, and can intuitively see the collected data and recognition results.

[0087] In another embodiment, in order to make the light of the light source 102 more dispersed and soft, a soft light device 103 such as soft light paper or soft light cloth is used under the light source 102 to form uniform and stable light, which is more conducive to image acquisition.

[0088] In some other embodiments, the image acquisition unit further includes a brightness calibration unit, which is used to:

[0089] Detecting whether the brightness of the light source 102 is attenuated based on a reference brightness, or taking multiple sets of images of an empty medicine tray, and calculating the average brightness of the multiple sets of images of the medicine tray, and judging whether the deviation of the average brightness from the reference brightness exceeds a preset range based on a reference brightness;

[0090] If so, a brightness adjustment message is sent to the display module 3 or the brightness of the light source 102 is adjusted through a dimmer. The display module 3 can prompt the operator to replace the light source 102, or the dimmer can be used to change the current or voltage of the light source 102 to control the brightness of the light source 102.

[0091] Based on the brightness calibration unit, the brightness of the light source 102 can be monitored to effectively avoid the impact of the attenuation phenomenon of the light source over a long period of time on the imaging quality. The same medicinal material will have differences in color and texture when photographed at different brightnesses. The present application can ensure the stability of the imaging quality by timely adjusting the brightness of the light source.

[0092] In the above embodiment, the brightness of the light source 102 is detected by a brightness sensor, such as an illuminance meter, and the reference brightness and the preset range can be flexibly adjusted.

[0093] In some of the embodiments, the upper surface of the medicine tray 105 is a frosted surface of a single color to prevent reflection and ensure the clarity of image acquisition; its single color can also be used as a reference color. When color cast occurs during camera acquisition, image color correction is achieved to ensure recognition effect; the material used is a material that does not deform over a long period of use, such as stainless steel, engineering plastics, ceramics, etc.

[0094] When using the Chinese herbal medicine slice identification device of the present application for identification, the operator pulls out the medicine tray 105, puts the medicine slice to be identified in the medicine tray 105 and pushes it in, clicks the identification button (not shown) on the screen of the display module 3 to obtain the image and weight of the Chinese herbal medicine slice, runs the Chinese herbal medicine slice identification method of the above embodiment to load the model file generated after training, obtains the identification result, and outputs the category and quality data of the medicine slice on the screen. In the identification result, the probability of each category is sorted from large to small, and the top three categories with the largest probability values ​​are output, and at the same time, the unqualified, qualified, excellent and their corresponding probability values ​​are output.

[0095] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0096] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying Chinese herbal medicine pieces, characterized in that: include: The training data set acquisition step is to acquire color images, depth images and weight data of Chinese herbal medicine pieces of different categories and qualities through a data acquisition module, and pre-process the weight data and the color images and depth images to generate a segmented image, a weight image and a processed color image and depth image to obtain a training data set marked with the category and quality of the herbal medicine pieces; A model training step, inputting the training data set into a medicinal piece recognition network for training to obtain a trained medicinal piece recognition model; The step of identifying Chinese herbal medicine pieces is to obtain the color image, depth image and weight data of the herbal medicine pieces to be identified through the data acquisition module, and pre-process the color image, depth image and weight data of the herbal medicine pieces to be identified to generate a segmented image, weight image and processed color image and depth image of the herbal medicine pieces to be identified, and input them into the herbal medicine piece identification model for reasoning, and output the identification result.

2. The method for identifying Chinese herbal medicine slices according to claim 1, characterized in that: The training data set acquisition step comprises: A data acquisition step, collecting color images and depth images of each category of Chinese herbal medicine pieces of different qualities to be identified through the image acquisition unit of the data acquisition module, and collecting weight data of each category of Chinese herbal medicine pieces of different qualities to be identified through the weighing unit of the data acquisition module; A data preprocessing step, converting the weight data into a weight image, generating the segmented image based on the color image and the depth image, and normalizing the segmented image, the color image and the depth image; The data concatenation step concatenates the segmented image, the weight image, the depth image and the color image along the image channel dimension.

3. The method for identifying Chinese herbal medicine slices according to claim 2, characterized in that: Converting the weight data into a weight image comprises: A weight reference value is obtained, a quotient of the weight data and the weight reference value is calculated, and the obtained quotient is filled into a two-dimensional matrix and stored as a weight image.

4. The method for identifying Chinese herbal medicine slices according to claim 2 or 3, characterized in that: Generating the segmented image based on the color image and the depth image includes: The color image and the depth image are subjected to edge detection, and a union operation is performed on the edge detection results, the edge-surrounded area is filled with white, and the area outside the edge is filled with black, so as to form a segmented image.

5. The method for identifying Chinese herbal medicine slices according to claim 2, characterized in that: The depth image is normalized, comprising: A maximum value maxv and a minimum value minv are set, wherein the maximum value maxv represents the distance from the binocular camera of the image acquisition unit to the medicine tray of the weighing unit, and the minimum value minv represents the distance from the upper surface of the Chinese herbal medicine piece with the highest height to the binocular camera; The depth value v of the depth image is calculated based on the following calculation model to obtain a normalized depth image. The calculation model is expressed as follows:

6. The method for identifying Chinese herbal medicine slices according to claim 2 or 3, characterized in that: The decoction piece recognition network includes a high-resolution multi-scale feature fusion network, an MHSA network, a global pooling layer and a fully connected layer which are connected in sequence.

7. A Chinese herbal medicine slice identification device, characterized in that: include: The data acquisition module includes an image acquisition unit and a weighing unit disposed in a closed housing, and is used to obtain color images, depth images and weight data of Chinese herbal medicine pieces; A processor, configured to implement the method for identifying Chinese herbal medicine pieces according to any one of claims 1 to 6 when executing a preset computer program; The display module is used to display information input by the user or information provided to the user and various menus of the user terminal.

8. The Chinese herbal medicine slice identification device according to claim 7, characterized in that: The image acquisition unit comprises: A binocular camera, disposed on the top of the closed housing and electrically connected to the processor, for collecting color images and depth images; The light source is arranged above or beside the binocular camera.

9. The Chinese herbal medicine slice identification device according to claim 8, characterized in that: The weighing unit is arranged at the bottom of the closed shell through a pull-out structure. The weighing unit comprises: an electronic scale and a medicine tray arranged on the upper surface of the electronic scale. The electronic scale is electrically connected to the processor.

10. The Chinese herbal medicine slice identification device according to claim 9, characterized in that: The image acquisition unit further includes a brightness calibration unit, which is used to: Detecting whether the brightness of the light source is attenuated based on a reference brightness, or taking a plurality of groups of images of an empty medicine tray, and calculating the average brightness of the plurality of groups of images of the medicine tray, and judging whether the deviation of the average brightness from the reference brightness exceeds a preset range based on a reference brightness; If so, a brightness adjustment message is sent to the display module or the brightness of the light source is adjusted via a dimmer.

Images