Wine-brewing sorghum variety identification method and device based on computer vision

Abstract

The invention relates to the technical field of wine brewing raw material detection, discloses a wine brewing sorghum variety identification method and device based on computer vision, and aims at solving the problem that an existing method is poor in efficiency and accuracy, and the scheme mainly comprises the steps that a multi-modal database of wine brewing sorghum is established; constructing a sorghum grain recognition model, a sorghum physical feature recognition model and a single-grain sorghum variety recognition model according to the multi-modal database; inputting the sorghum overall visual data of the to-be-identified sorghum into the sorghum particle identification model to obtain a sorghum overall labeling result; obtaining single-grain sorghum visual data according to the overall sorghum labeling result; inputting the visual data of each single-grain sorghum into a sorghum physical feature recognition model to obtain a single-grain sorghum physical feature recognition result; and inputting the single-grain sorghum visual data and the corresponding single-grain sorghum physical feature recognition result into a single-grain sorghum variety recognition model to obtain a variety recognition result. The sorghum variety identification efficiency and accuracy are improved, and the method is suitable for batch identification.

Description

Technical Field

[0001] The present invention relates to the technical field of brewing raw material detection, and particularly relates to a method and device for identifying brewing sorghum varieties based on computer vision. Background Art

[0002] Sorghum varieties directly affect the liquor yield, flavor complexity and flavor type adaptability of Baijiu through characteristics such as starch type, tannin content and cooking resistance. Therefore, the variety of brewing sorghum directly affects the flavor and quality of Baijiu. Accurate variety identification is crucial for raw material screening and brewing process control of Baijiu. Traditional sorghum variety identification mainly relies on manual experience and laboratory analysis, which has problems of low efficiency and high cost, and is easily affected by human errors, resulting in poor accuracy.

[0003] Application Publication No. CN116337781A discloses a method for identifying sorghum varieties based on spectral image reconstruction. This method uses the spectral features and image features of sorghum samples to establish a sorghum reconstructed spectral variety identification data set, then uses the sorghum reconstructed spectral variety identification data set to train a BiLSTM_Attention network model to obtain a sorghum variety identification model, and finally conducts sorghum variety identification based on the sorghum variety identification model. The inventor has found through research that this method has at least the following problems: when constructing the sorghum variety identification model, it only relies on single image features, lacks consideration of physical features, and has poor accuracy. In addition, it lacks an effective particle recognition mechanism, resulting in variety identification being limited to single-grain manual selection, and there is still a problem of low efficiency, making it difficult to meet the needs of large-scale industrial applications. Summary of the Invention

[0004] The present invention aims to solve the problems of poor efficiency and accuracy in existing sorghum variety identification methods, and proposes a method and device for identifying brewing sorghum varieties based on computer vision.

[0005] The technical solutions adopted by the present invention to solve the above technical problems are as follows:

[0006] In a first aspect, the present invention provides a method for identifying brewing sorghum varieties based on computer vision, the method comprising:

[0007] Establish a multi-modal database of brewing sorghum, the multi-modal database including sorghum overall visual data, sorghum overall annotation data, single-grain sorghum visual data, single-grain sorghum physical feature data and single-grain sorghum variety data;

[0008] Construct a sorghum particle recognition model according to the sorghum overall visual data and sorghum overall annotation data, and construct a sorghum physical feature recognition model according to the single-grain sorghum visual data and single-grain sorghum physical feature data;

[0009] Construct a single sorghum variety recognition model based on the single sorghum visual data, single sorghum physical characteristic data, and the data of the variety to which the single sorghum belongs;

[0010] Input the overall sorghum visual data of the sorghum to be recognized into the sorghum grain recognition model to obtain the overall sorghum annotation result; according to the overall sorghum annotation result, obtain the single sorghum visual data of each single sorghum from the overall sorghum visual data of the sorghum to be recognized; input the single sorghum visual data of each single sorghum into the sorghum physical characteristic recognition model respectively to obtain the single sorghum physical characteristic recognition result;

[0011] Input the single sorghum visual data of each single sorghum in the overall sorghum visual data of the sorghum to be recognized and its corresponding single sorghum physical characteristic recognition result into the single sorghum variety recognition model respectively to obtain the variety recognition result of each single sorghum in the sorghum to be recognized.

[0012] Further, the overall sorghum visual data is an overall sorghum visual image containing multiple sorghum grains captured by a high-resolution camera, the shooting background of the overall sorghum visual image is a single-color background, and multiple sorghum grains do not stack on each other;

[0013] The overall sorghum annotation data is the rectangular frame coordinate data that marks the position of a single sorghum using a rectangular frame based on the overall sorghum visual image;

[0014] The single sorghum visual data in the multimodal database is a single sorghum visual image captured by a high-resolution camera on a single-color background;

[0015] The single sorghum physical characteristic data includes the area, perimeter, grain length, and grain width of a single sorghum;

[0016] The data of the variety to which the single sorghum belongs is the variety category to which the single sorghum belongs.

[0017] Further, the sorghum grain recognition model is an object detection neural network model constructed based on Faster-RCNN, RetinaNet, YOLO, EfficientDet, or SSD.

[0018] Further, the sorghum physical characteristic evaluation model includes a first grain feature extraction module and a grain physical characteristic recognition module;

[0019] The first grain feature extraction module is used to extract high-dimensional grain features from the single sorghum visual data and reconstruct the high-dimensional grain features into a one-dimensional grain feature tensor, and the grain physical characteristic recognition module is used to obtain the single sorghum physical characteristic recognition result according to the one-dimensional grain feature tensor.

[0020] Further, the first seed grain feature extraction module is a convolutional neural network or an attention mechanism model. The convolutional neural network is ResNet, EfficientNet, or ConvNeXt, the attention mechanism model is Self-Attention or Transformer, and the seed grain physical feature recognition module is an MLP network.

[0021] Further, the single-grain sorghum variety recognition model includes a second seed grain feature extraction module and a seed grain variety recognition module;

[0022] The second seed grain feature extraction module is used to extract high-dimensional seed grain features from the single-grain sorghum visual data, reconstruct the high-dimensional seed grain features into a one-dimensional seed grain feature tensor, and combine the one-dimensional seed grain feature tensor with the single-grain sorghum physical feature data to obtain a combined seed grain feature tensor. The seed grain variety recognition module is used to obtain the variety recognition result of the single-grain sorghum based on the combined seed grain feature tensor.

[0023] Further, the second seed grain feature extraction module is a convolutional neural network or an attention mechanism model. The convolutional neural network is ResNet, EfficientNet, or ConvNeXt, the attention mechanism model is Self-Attention or Transformer, and the seed grain variety recognition module is an MLP network.

[0024] Further, the method further includes:

[0025] Determine the number of target variety sorghum based on the variety recognition results of each single-grain sorghum in the sorghum to be recognized, and calculate the proportion of the target variety sorghum in the sorghum to be recognized according to the number of target sorghum varieties and the total number of sorghum.

[0026] Further, the method further includes:

[0027] Update and expand the multi-modal database according to a preset period, and after updating and expanding the multi-modal database, update the sorghum grain recognition model, the sorghum physical feature recognition model, and the single-grain sorghum variety recognition model.

[0028] In a second aspect, the present invention provides a device for identifying sorghum varieties for brewing based on computer vision. The device includes:

[0029] A database establishment module for establishing a multi-modal database of brewing sorghum, where the multi-modal database includes sorghum overall visual data, sorghum overall annotation data, single-grain sorghum visual data, single-grain sorghum physical feature data, and single-grain sorghum variety data;

[0030] A model establishment module, configured to construct a sorghum grain recognition model based on the overall sorghum visual data and the overall sorghum annotation data, and construct a sorghum physical feature recognition model based on the single sorghum visual data and the single sorghum physical feature data; construct a single sorghum variety recognition model based on the single sorghum visual data, the single sorghum physical feature data, and the single sorghum variety data.

[0031] A variety recognition module, configured to input the overall sorghum visual data of the sorghum to be recognized into the sorghum grain recognition model to obtain an overall sorghum annotation result; according to the overall sorghum annotation result, obtain the single sorghum visual data of each single sorghum from the overall sorghum visual data of the sorghum to be recognized; input the single sorghum visual data of each single sorghum into the sorghum physical feature recognition model respectively to obtain a single sorghum physical feature recognition result; input the single sorghum visual data of each single sorghum and its corresponding single sorghum physical feature recognition result in the overall sorghum visual data of the sorghum to be recognized into the single sorghum variety recognition model respectively to obtain the variety recognition result of each single sorghum in the sorghum to be recognized.

[0032] The beneficial effects of the present invention are as follows: The method and device for identifying the variety of brewing sorghum based on computer vision provided by the present invention combine object detection and deep learning models to realize the automatic recognition of the overall sorghum image to single grains, and can batch recognize the variety of sorghum from the overall image containing multiple sorghum grains. Compared with manually selecting single grains for variety recognition, the efficiency of sorghum variety recognition is improved; at the same time, the present invention combines visual data and physical feature data for variety classification. Visual data can provide texture and morphological features, and physical feature data can provide physical size features. The combination of the two can cover multi-dimensional differences between varieties and reduce the misjudgment risk of a single data source, thereby improving the accuracy of sorghum variety recognition. Description of the Drawings

[0033] Figure 1 It is a schematic flowchart of the method for identifying the variety of brewing sorghum based on computer vision provided in the embodiment;

[0034] Figure 2 It is a schematic diagram of the overall sorghum visual image provided in the embodiment;

[0035] Figure 3 It is a schematic diagram of the overall sorghum annotation image provided in the embodiment;

[0036] Figure 4 It is a schematic diagram of the overall sorghum annotation result of the sorghum to be recognized provided in the embodiment;

[0037] Figure 5 It is a schematic diagram of the sorghum variety recognition result provided in the embodiment;

[0038] Figure 6Schematic structural diagram of the computer vision-based sorghum variety identification device provided for the embodiment. Detailed implementation manners

[0039] In order to enable the personnel in the technical field to better understand the solution of the present invention, the technical solution in this embodiment will be clearly and completely described below in conjunction with the accompanying drawings in this embodiment.

[0040] In some processes described in the specification of the present invention and the above-mentioned accompanying drawings, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations are only used to distinguish each different operation, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0041] In order to improve the efficiency and accuracy of sorghum variety identification, the technical solution of the present invention is proposed. In the present invention, a multi-modal database of brewing sorghum is established. The multi-modal database includes sorghum overall visual data, sorghum overall annotation data, single-grain sorghum visual data, single-grain sorghum physical characteristic data, and single-grain sorghum variety data. A sorghum grain identification model is constructed according to the sorghum overall visual data and the sorghum overall annotation data, and a sorghum physical characteristic identification model is constructed according to the single-grain sorghum visual data and the single-grain sorghum physical characteristic data. A single-grain sorghum variety identification model is constructed according to the single-grain sorghum visual data, the single-grain sorghum physical characteristic data, and the single-grain sorghum variety data. The sorghum overall visual data of the sorghum to be identified is input into the sorghum grain identification model to obtain the sorghum overall annotation result. According to the sorghum overall annotation result, the single-grain sorghum visual data of each single-grain sorghum is obtained from the sorghum overall visual data of the sorghum to be identified. The single-grain sorghum visual data of each single-grain sorghum is respectively input into the sorghum physical characteristic identification model to obtain the single-grain sorghum physical characteristic identification result. The single-grain sorghum visual data of each single-grain sorghum in the sorghum overall visual data of the sorghum to be identified and its corresponding single-grain sorghum physical characteristic identification result are respectively input into the single-grain sorghum variety identification model to obtain the variety identification result of each single-grain sorghum in the sorghum to be identified.

[0042] Specifically, the present invention respectively constructs a sorghum grain recognition model, a sorghum physical feature recognition model, and a single sorghum variety recognition model. The sorghum grain recognition model is used to perform sorghum target detection on the overall visual data of sorghum containing multiple sorghum grain images, so as to crop each sorghum grain visually to obtain single sorghum visual data. The sorghum physical feature recognition model is used to respectively perform physical feature recognition on each single sorghum visual data, and the single sorghum variety recognition model is used to respectively perform sorghum variety recognition on each single sorghum visual data and its corresponding single sorghum physical feature data. Through the above process, batch recognition of sorghum varieties can be carried out from the overall image containing multiple sorghum grains, improving the recognition efficiency; and by combining visual data and physical feature data, multi-dimensional differences between varieties can be covered, reducing the risk of misjudgment of a single data source, thereby improving the recognition accuracy.

[0043] Next, the technical solutions in this embodiment will be clearly and completely described in conjunction with the accompanying drawings in this embodiment. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0044] Figure 1 The flowchart of a method for identifying sorghum varieties for brewing based on computer vision is shown. Please refer to Figure 1 , and the method includes the following steps:

[0045] Step 1: Establish a multi-modal database for sorghum for brewing, where the multi-modal database includes overall sorghum visual data, overall sorghum annotation data, single sorghum visual data, single sorghum physical feature data, and data on the variety to which the single sorghum belongs.

[0046] Among them, the overall sorghum visual data is an overall visual image of sorghum containing multiple sorghum grains taken by a high-resolution camera. The background for taking the overall sorghum visual image is a single-color background, and multiple sorghum grains do not overlap with each other.

[0047] In this embodiment, a Sony ILCE-7M4 camera and a SEL2470GM2 lens are used. In a photo box, with the same intensity of background light, the overall sorghum visual image is taken, and each overall sorghum visual image maintains the same spatial resolution. The background for taking the overall sorghum visual image is a single-color background, and there is a significant difference between the single-color background and the color of sorghum. In this embodiment, a white background is used as the single-color background. As Figure 2 shown, each overall sorghum visual image contains several non-overlapping sorghum grains and is stored at a high resolution of 3504×2336 pixels to capture the subtle differences of sorghum seeds.

[0048] The overall sorghum annotation data is the rectangular box coordinate data obtained by using a rectangular box to mark the position of a single sorghum on the basis of the overall sorghum visual data. AsFigure 3 As shown in Figure 3 , in this embodiment, the LabelStudio tool is used for manual annotation. During the annotation process, according to the outer contour of the sorghum grains, a rectangular box is used for annotation, and the rectangular box is saved in the multi-modal database in the form of coordinate data.

[0049] The visual data of a single sorghum grain in the multi-modal database is a visual image of a single sorghum grain taken by a high-resolution camera on a monochromatic background. In this embodiment, an overall image of sorghum of the same variety can be taken first by a high-resolution camera, and the shooting background is the same as that of the overall visual image of the sorghum. Then, by annotating and cropping the overall image, a visual image of a single sorghum grain is obtained. By shooting sorghum of the same variety simultaneously, the acquisition efficiency and accuracy of the visual image of a single sorghum grain can be improved.

[0050] The physical feature data of the single sorghum grain includes the area, perimeter, grain length, and grain width of the single sorghum grain. In this embodiment, the physical feature data of the single sorghum grain can be calculated by computer vision methods, specifically including: First, through the color difference between the monochromatic background and the sorghum seeds, the pixel area except the monochromatic background in the visual image of the single sorghum grain is extracted; then, through the erosion and dilation operations of computer vision, the tiny areas caused by impurities, dust, etc. are cleaned; finally, an arc is used to fit the contour to obtain the fine contour of a single sorghum grain, so as to calculate the area and perimeter; according to the symmetry of the fine contour of a single sorghum grain, two orthogonal directions are fitted to calculate the grain length and grain width.

[0051] The variety data of the single sorghum grain is the variety category to which the single sorghum grain belongs, which can be represented in integer form. In this embodiment, considering two sorghum varieties, waxy sorghum and non-waxy sorghum, they are represented by integers 1 and 2 respectively. In addition, the variety category to which the single sorghum grain belongs can also be characterized by a one-hot vector. For example, waxy sorghum is represented by [1, 0], and non-waxy sorghum is represented by [0, 1].

[0052] Step 2: Construct a sorghum grain recognition model according to the overall sorghum visual data and the overall sorghum annotation data, and construct a sorghum physical feature recognition model according to the visual data of a single sorghum grain and the physical feature data of a single sorghum grain.

[0053] In actual application, the overall sorghum visual data in the multi-modal database is used as input features, and the corresponding overall sorghum annotation data is used as the true label to train and obtain a sorghum grain recognition model. Among them, the sorghum grain recognition model is a target detection neural network model for target detection, and the target detection neural network model can be Faster-RCNN, RetinaNet, YOLO, EfficientDet or SSD.

[0054] In this embodiment, the sorghum grain recognition model selects the Faster-RCNN neural network. Its loss function mainly includes the ROIHead loss and the RPN (Region Proposal Network) loss. Among them, the ROIHead loss includes the classification loss and the bounding box regression loss, and the RPN (Region Proposal Network) loss includes the objectness loss and the anchor box regression loss. The above loss functions are embedded in the definition of the Faster-RCNN neural network. The Faster-RCNN network is defined using the torchvision.models.detection.fasterrcnn_resnet50_fpn function. The above-mentioned various losses are automatically calculated through the command loss_dict = model(images, targets), and then these loss terms are added together through losses = sum(loss for loss in loss_dict.values()) to obtain the final total loss, which is used for backpropagation and model parameter update. Among them, images are the processed image data input to the neural network, target is a tensor with the shape of [N, 4], N is the number of detected targets, and 4 indicates that 4 values are used to represent the rectangular bounding box. In this embodiment, [xmin, ymin, xmax, ymax] is used to represent the coordinate data corresponding to the range of the rectangular bounding box.

[0055] In actual application, the visual data of single sorghum grains is used as input features, and the corresponding physical feature data of single sorghum grains is used as the ground truth label to train and obtain the sorghum physical feature recognition model.

[0056] In this embodiment, the sorghum physical feature evaluation model includes a first grain feature extraction module and a grain physical feature recognition module; the first grain feature extraction module is used to extract high-dimensional grain features from the visual data of single sorghum grains and reconstruct the high-dimensional grain features into a one-dimensional grain feature tensor, and the grain physical feature recognition module is used to obtain the recognition result of the physical features of single sorghum grains according to the one-dimensional grain feature tensor.

[0057] Among them, the first grain feature extraction module can be a convolutional neural network or an attention mechanism model. The convolutional neural network can be ResNet, EfficientNet or ConvNeXt, and the attention mechanism model can be Self-Attention or Transformer. The grain physical feature recognition module can be an MLP network.

[0058] In this embodiment, the ResNet18 network is used as the first grain feature extraction module. The visual data of a single sorghum grain is input into the ResNet18 network, and the output is high-dimensional grain features. Then, the high-dimensional grain features are reconstructed into a one-dimensional grain feature tensor. After that, the one-dimensional grain feature tensor is input into the grain physical feature recognition module represented by the MLP network, and the output is the physical features corresponding to a single sorghum grain. In this embodiment, the SmoothL1 loss function is used as the loss function of the sorghum physical feature recognition model, which is defined by torch.nn.SmoothL1Loss().

[0059] Step 3: Construct a single sorghum variety recognition model based on the visual data of a single sorghum grain, the physical feature data of a single sorghum grain, and the variety data of the single sorghum grain to which it belongs.

[0060] In practical applications, the visual data of a single sorghum grain and its corresponding physical feature data of a single sorghum grain are used as input features, and the corresponding variety data of the single sorghum grain to which it belongs is used as the true label to train and obtain a single sorghum variety recognition model.

[0061] In this embodiment, the single sorghum variety recognition model includes a second grain feature extraction module and a grain variety recognition module; the second grain feature extraction module is used to extract high-dimensional grain features from the visual data of a single sorghum grain, reconstruct the high-dimensional grain features into a one-dimensional grain feature tensor, and combine the one-dimensional grain feature tensor with the physical feature data of a single sorghum grain to obtain a combined grain feature tensor. The grain variety recognition module is used to obtain the variety recognition result of a single sorghum grain based on the combined grain feature tensor.

[0062] Among them, the second grain feature extraction module can be a convolutional neural network or an attention mechanism model. The convolutional neural network can be ResNet, EfficientNet, or ConvNeXt. The attention mechanism model can be Self-Attention or Transformer. The grain variety recognition module can be an MLP network.

[0063] In this embodiment, the ResNet18 network is used as the second grain feature extraction module. The visual data of a single sorghum grain is input into the ResNet18 network, and the output is high-dimensional grain features. Then, the high-dimensional grain features are reconstructed into a one-dimensional grain feature tensor. The one-dimensional grain feature tensor is concatenated and combined with the physical feature data of the corresponding single sorghum grain represented by a one-dimensional vector to obtain a combined grain feature tensor. After that, the combined grain feature tensor is input into the grain variety recognition module represented by the MLP network, and the output is the variety category corresponding to a single sorghum grain.

[0064] Step 4: Input the overall visual data of the sorghum to be identified into the sorghum grain recognition model to obtain the overall annotation result of the sorghum; according to the overall annotation result of the sorghum, obtain the visual data of each single sorghum grain from the overall visual data of the sorghum to be identified; input the visual data of each single sorghum grain into the sorghum physical feature recognition model respectively to obtain the recognition result of the physical features of single sorghum grains.

[0065] When batch variety identification of sorghum is required, use the same method as the overall visual data of sorghum in the multi-modal database to capture and obtain the overall visual image of the sorghum to be identified, that is, keep the same shooting parameters and monochromatic background, and the sorghum grains do not stack on each other. Then input the corresponding overall visual data of the sorghum into the sorghum grain recognition model. The sorghum grain recognition model performs object detection on the overall visual data of the sorghum to obtain the overall annotation result. For the overall annotation result of the sorghum to be identified, please refer to Figure 4 .

[0066] After obtaining the overall annotation result of the sorghum, crop the sorghum in the overall visual image of the sorghum grain by grain according to the overall annotation result to obtain the visual data of each single sorghum grain in the overall visual image of the sorghum, and input them into the sorghum physical feature recognition model respectively to obtain the recognition result of the physical features of each single sorghum grain.

[0067] Step 5: Input the visual data of each single sorghum grain and its corresponding recognition result of the physical features of single sorghum grains in the overall visual data of the sorghum to be identified into the single sorghum variety recognition model respectively to obtain the variety recognition result of each single sorghum grain in the sorghum to be identified.

[0068] After obtaining the visual data of each single sorghum grain and its corresponding recognition result of the physical features of single sorghum grains in the overall visual data of the sorghum to be identified, input the two into the single sorghum variety recognition model, and the variety recognition result of each single sorghum grain can be obtained. Please refer to Figure 5 , in this variety recognition result, there are 14 waxy sorghums and 2 non-waxy sorghums.

[0069] In this embodiment, the method further includes: determining the number of target variety sorghums according to the variety recognition results of each single sorghum grain in the sorghum to be identified, and calculating the proportion of the target variety sorghums in the sorghum to be identified according to the number of target sorghum varieties and the total number of sorghums.

[0070] For example, in Figure 5 , there are 14 waxy sorghums and 2 non-waxy sorghums, then the proportion of waxy sorghums is 87.5%, and the proportion of non-waxy sorghums is 12.5%. By calculating the proportion of the target variety sorghums, the staff can intuitively understand the variety composition of the sorghum to be identified for the next step of processing.

[0071] In this embodiment, the method further includes: updating and expanding the multi-modal database according to a preset period, and after updating and expanding the multi-modal database, updating the sorghum grain recognition model, the sorghum physical feature recognition model, and the single sorghum variety recognition model.

[0072] Among them, the preset period can be set according to the actual situation, and this embodiment does not make any restrictions. By regularly updating the multi-modal database and the model, the data quality and accuracy can be improved, thereby improving the performance and accuracy of the model, and further improving the accuracy of sorghum variety recognition.

[0073] In summary, the sorghum variety recognition method based on computer vision provided in this embodiment combines object detection and deep learning models to achieve automatic recognition of sorghum from the overall image to single grains, and can perform batch recognition of sorghum varieties from the overall image containing multiple sorghum grains. Compared with manually selecting single grains for variety recognition, the efficiency of sorghum variety recognition is improved. At the same time, the present invention combines visual data and physical feature data for variety classification. Visual data can provide texture and morphological features, and physical feature data can provide physical size data. The combination of the two can cover multi-dimensional differences between varieties and reduce the risk of misjudgment of a single data source, thereby improving the accuracy of sorghum variety recognition.

[0074] Based on the above technical solution, this embodiment further proposes a sorghum variety recognition device based on computer vision. Please refer to Figure 6 , the device includes:

[0075] A database establishment module for establishing a multi-modal database of brewing sorghum, where the multi-modal database includes sorghum overall visual data, sorghum overall annotation data, single sorghum visual data, single sorghum physical feature data, and single sorghum variety data;

[0076] A model establishment module for constructing a sorghum grain recognition model according to the sorghum overall visual data and sorghum overall annotation data, constructing a sorghum physical feature recognition model according to the single sorghum visual data and single sorghum physical feature data; constructing a single sorghum variety recognition model according to the single sorghum visual data, single sorghum physical feature data, and single sorghum variety data;

[0077] The variety recognition module is used to input the overall visual data of the sorghum to be recognized into the sorghum grain recognition model to obtain the overall annotation result of the sorghum; according to the overall annotation result of the sorghum, obtain the visual data of each single sorghum grain from the overall visual data of the sorghum to be recognized; input the visual data of each single sorghum grain into the sorghum physical feature recognition model respectively to obtain the recognition result of the physical features of the single sorghum grain; input the visual data of each single sorghum grain and its corresponding recognition result of the physical features of the single sorghum grain in the overall visual data of the sorghum to be recognized into the single sorghum variety recognition model respectively to obtain the variety recognition result of each single sorghum grain in the sorghum to be recognized.

[0078] It can be understood that since the device for identifying the variety of brewing sorghum based on computer vision described in this embodiment is a device for implementing the method for identifying the variety of brewing sorghum based on computer vision described in the embodiment, for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method, and details will not be repeated here.

Claims

1. A method for identifying brewing sorghum varieties based on computer vision, characterized in that: The method comprises: Establishing a multimodal database of brewing sorghum, the multimodal database includes sorghum overall visual data, sorghum overall annotation data, single grain sorghum visual data, single grain sorghum physical feature data and single grain sorghum variety data; Constructing a sorghum grain recognition model based on the sorghum overall visual data and the sorghum overall labeled data, and constructing a sorghum physical feature recognition model based on the single sorghum grain visual data and the single sorghum grain physical feature data; Constructing a single-grain sorghum variety recognition model according to the single-grain sorghum visual data, the single-grain sorghum physical characteristic data and the single-grain sorghum variety data; Inputting the overall visual data of the sorghum to be identified into the sorghum grain recognition model to obtain the overall labeling result of the sorghum; obtaining the single-grain visual data of each single-grain sorghum from the overall visual data of the sorghum to be identified according to the overall labeling result of the sorghum; inputting the single-grain visual data of each single-grain sorghum into the sorghum physical feature recognition model respectively to obtain the physical feature recognition result of the single-grain sorghum; The single-grain visual data of each single-grain sorghum in the overall visual data of the sorghum to be identified and the corresponding single-grain sorghum physical feature recognition results are respectively input into the single-grain sorghum variety recognition model to obtain the variety recognition results of each single-grain sorghum in the sorghum to be identified.

2. The method for identifying brewing sorghum varieties based on computer vision according to claim 1, characterized in that: The sorghum overall visual data is an overall visual image of sorghum containing a plurality of sorghum grains captured by a high-resolution camera, wherein the shooting background of the overall visual image of sorghum is a monochrome background, and the plurality of sorghum grains are not stacked on each other; The sorghum overall marking data is rectangular frame coordinate data for marking the position of a single grain of sorghum using a rectangular frame based on the overall visual image of the sorghum; The single-grain sorghum visual data in the multimodal database is a single-grain sorghum visual image taken with a high-resolution camera on a monochrome background; The single-grain sorghum physical characteristic data include the area, perimeter, grain length and grain width of the single-grain sorghum; The variety data of the single-grain sorghum is the variety category to which the single-grain sorghum belongs.

3. The method for identifying brewing sorghum varieties based on computer vision according to claim 1, characterized in that: The sorghum grain recognition model is a target detection neural network model constructed based on Faster-RCNN, RetinaNet, YOLO, EfficientDet or SSD.

4. The method for identifying brewing sorghum varieties based on computer vision according to claim 1, characterized in that: The sorghum physical characteristic evaluation model includes a first grain characteristic extraction module and a grain physical characteristic recognition module; The first grain feature extraction module is used to extract high-dimensional grain features from single-grain sorghum visual data and reconstruct the high-dimensional grain features into a one-dimensional grain feature tensor. The grain physical feature recognition module is used to obtain single-grain sorghum physical feature recognition results based on the one-dimensional grain feature tensor.

5. The method for identifying brewing sorghum varieties based on computer vision according to claim 4, characterized in that: The first seed particle feature extraction module is a convolutional neural network or an attention mechanism model, the convolutional neural network is ResNet, EfficientNet or ConvNeXt, the attention mechanism model is Self-Attention or Transformer, and the seed particle physical feature recognition module is an MLP network.

6. The method for identifying brewing sorghum varieties based on computer vision according to claim 1, characterized in that: The single-grain sorghum variety recognition model includes a second grain feature extraction module and a grain variety recognition module; The second grain feature extraction module is used to extract high-dimensional grain features from single-grain sorghum visual data, reconstruct the high-dimensional grain features into a one-dimensional grain feature tensor, and combine the one-dimensional grain feature tensor with the single-grain sorghum physical feature data to obtain a combined grain feature tensor. The grain variety identification module is used to obtain a variety identification result of a single-grain sorghum based on the combined grain feature tensor.

7. The method for identifying brewing sorghum varieties based on computer vision according to claim 6, characterized in that: The second seed grain feature extraction module is a convolutional neural network or an attention mechanism model, the convolutional neural network is ResNet, EfficientNet or ConvNeXt, the attention mechanism model is Self-Attention or Transformer, and the seed grain variety identification module is an MLP network.

8. The method for identifying brewing sorghum varieties based on computer vision according to claim 1, characterized in that: The method further comprises: The number of target sorghum varieties is determined according to the variety identification results of each single grain of sorghum in the sorghum to be identified, and the proportion of the target sorghum varieties in the sorghum to be identified is calculated according to the number of target sorghum varieties and the total number of sorghum.

9. The method for identifying brewing sorghum varieties based on computer vision according to claim 1, characterized in that: The method further comprises: The multimodal database is updated and expanded according to a preset period, and after the multimodal database is updated and expanded, the sorghum grain recognition model, the sorghum physical characteristic recognition model and the single-grain sorghum variety recognition model are updated.

10. A device for identifying brewing sorghum varieties based on computer vision, characterized in that: The device comprises: A database establishment module is used to establish a multimodal database of brewing sorghum, wherein the multimodal database includes overall visual data of sorghum, overall annotation data of sorghum, visual data of single grain sorghum, physical characteristic data of single grain sorghum and variety data of single grain sorghum; A model building module is used to build a sorghum grain recognition model based on the sorghum overall visual data and the sorghum overall labeled data, build a sorghum physical feature recognition model based on the single sorghum grain visual data and the single sorghum grain physical feature data; and build a single sorghum grain variety recognition model based on the single sorghum grain visual data, the single sorghum grain physical feature data and the single sorghum grain variety data; The variety identification module is used to input the overall visual data of sorghum to be identified into a sorghum grain identification model to obtain an overall labeling result of sorghum; according to the overall labeling result of sorghum, obtain the single-grain visual data of each single-grain sorghum from the overall visual data of sorghum to be identified; input the single-grain visual data of each single-grain sorghum into a sorghum physical feature identification model to obtain a single-grain sorghum physical feature identification result; input the single-grain visual data of each single-grain sorghum in the overall visual data of sorghum to be identified and the corresponding single-grain physical feature identification result into a single-grain sorghum variety identification model to obtain a variety identification result of each single-grain sorghum in the sorghum to be identified.

Images