An automated tobacco leaf grading method, system, device, and storage medium

Abstract

This invention discloses an automated tobacco leaf grading method, system, device, and storage medium. The method includes: acquiring images of tobacco leaf samples to be tested; constructing an adaptive attention model based on adaptive sparse convolution; inserting the constructed adaptive attention model into a convolutional neural network to obtain a tobacco leaf grading model; and inputting the acquired images of the tobacco leaf samples into the tobacco leaf grading model to obtain the tobacco leaf grading result. The automated tobacco leaf grading method, system, device, and storage medium of this invention utilize an adaptive attention model that extracts deep features of tobacco leaves through adaptive sparse point convolution and adaptive sparse depth convolution, adaptively calculates the weights between channels, and focuses on important features affecting classification performance, thereby improving the network's classification performance. The adaptive attention model module can be embedded into a classic convolutional neural network, possessing feature extraction capabilities and achieving optimal classification accuracy, providing an effective method for tobacco leaf grading.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an automated tobacco leaf grading method, system, device, and storage medium. Background Technology

[0002] The tobacco industry plays a vital role in the economies of countries worldwide, serving as a pillar of many economies. Tobacco leaves, as the basic raw material for the tobacco industry, directly affect the quality of cigarettes. While the criteria for grading tobacco leaves are largely the same across tobacco-producing countries, the standards vary, resulting in numerous grades. China is a major tobacco producer, ranking first globally in tobacco planting area, tobacco leaf purchase volume, and tobacco tax revenue. Based on the characteristics, properties, and chemical composition of tobacco leaves, they can be classified into multiple grades according to Chinese national standards. Traditional tobacco leaf grading methods rely heavily on manual labor, which is highly subjective, results in low grading quality, and may damage the tobacco leaves during the testing process.

[0003] Therefore, there is an urgent need for an automated tobacco leaf grading method, system, equipment, and storage medium. Summary of the Invention

[0004] The purpose of this invention is to provide an automated tobacco leaf grading method, system, device, and storage medium to solve the problems in the prior art, which can extract deep features of tobacco leaves, adaptively calculate the weights between channels, and focus on important features that affect grading performance.

[0005] In a first aspect, the present invention provides an automated tobacco leaf grading method based on an adaptive attention mechanism, comprising:

[0006] Acquire images of the tobacco leaf samples to be tested;

[0007] An adaptive attention model is constructed based on adaptive sparse convolution.

[0008] The constructed adaptive attention model is inserted into a convolutional neural network to obtain a tobacco leaf grading model;

[0009] The images of the collected tobacco leaf samples are input into the tobacco leaf grading model to obtain the tobacco leaf grading results.

[0010] The automated tobacco leaf grading method based on the adaptive attention mechanism described above, preferably, involves acquiring images of the tobacco leaf samples to be tested, specifically including:

[0011] The tobacco leaf acquisition system uses an imaging module to acquire RGB images of tobacco leaves.

[0012] The automated tobacco leaf grading method based on the adaptive attention mechanism described above, preferably, involves acquiring RGB images of tobacco leaves using an imaging module through a tobacco leaf acquisition system, specifically including:

[0013] A servo motor drives a mechanical conveyor belt to transport tobacco leaf samples.

[0014] A data acquisition box is installed on the mechanical conveyor belt, and a halogen tungsten lamp is arranged inside the data acquisition box;

[0015] A DSLR camera is installed above the data acquisition box, and a through hole is provided at a corresponding position on the data acquisition box. The DSLR camera acquires backlit images of the tobacco leaves inside the data acquisition box, and the acquisition frequency of the DSLR camera is controlled by a communication module.

[0016] The automated tobacco leaf grading method based on the adaptive attention mechanism described above, preferably, involves constructing an adaptive attention model based on adaptive sparse convolution, specifically including:

[0017] An adaptive attention model is constructed based on adaptive sparse point convolution and self-adaptive sparse depth convolution.

[0018] The automated tobacco leaf grading method based on the adaptive attention mechanism described above, preferably, involves constructing an adaptive attention model based on adaptive sparse point convolution and adaptive sparse depth convolution, specifically including:

[0019] The input and output feature maps are set to have the same dimensions. For the input feature map F'∈RH×W×C', H represents the height, W represents the width, and C' represents the number of channels.

[0020] The input feature map F' is extracted in channels by global convolution to obtain channel-based global information F1, which is expressed by formula (1):

[0021] F1 = GConv(F') (1),

[0022] Where GConv represents global convolution;

[0023] Weights are generated for the global information F1 using adaptive sparse point convolution, resulting in F2, which is expressed by formula (2):

[0024] F2 = MFPConv(F1) (2);

[0025] The softmax function is used to assign weights to F2, and the weights W are obtained, which are expressed by formula (3):

[0026] W = Softmax(F2) (3);

[0027] The dimension of the input feature map F' is adjusted by adaptive sparse point convolution to obtain F3, which is expressed by formula (4):

[0028] F3 = MFDConv(F') (4);

[0029] The weight W obtained by formula (3) is multiplied element-wise with F3 obtained by formula (4) to obtain the adjusted feature map U, which is expressed by formula (5):

[0030]

[0031] in, This indicates element-wise multiplication;

[0032] The depth features of the adjusted feature map U are extracted by adaptive sparse depthwise convolution to obtain the output feature map U', which is expressed by formula (6):

[0033] U'=MFDConv(U) (6),

[0034] Where U' represents the output feature map.

[0035] In the automated tobacco grading system based on the adaptive attention mechanism described above, preferably, the calculation process of the adaptive sparse point convolution includes:

[0036] The input and output feature maps are assumed to have the same dimensions. For the input feature map F∈RH×W×C, where H represents the height, W represents the width, and C represents the number of channels.

[0037] The input feature map F is divided into G groups, and point convolution is applied to each group of feature maps to reduce the dimensionality. The channels are compressed by a factor of R to obtain the hidden layer.

[0038] Extracting channel difference information from different channels through channel shuffling;

[0039] Point convolution is applied again to increase the dimensionality of the feature maps in each group;

[0040] By concatenating all feature maps, the deep features of the tobacco leaf at the point level are obtained, where:

[0041] The parameter P of the adaptive sparse point convolution is calculated using formula (7):

[0042]

[0043] The parameter P comes from two point convolutional layers used for dimensionality reduction and dimensionality increase;

[0044] Using formula (7), we obtain the formula (8) for calculating the number of channels C with respect to P:

[0045]

[0046] Due to the output feature map and The hidden layer feature maps are connected, and the hidden layer feature maps are connected to... The input feature maps are connected, therefore the connectivity T between the output feature map and the input feature map can be expressed by formula (9):

[0047]

[0048] Wherein, connectivity T is used to represent the number of input-output connections for each output channel;

[0049] When R is a constant, dividing formula (7) by formula (9) yields formula (10) for calculating T with respect to P:

[0050]

[0051] From formula (10), we know that when G increases, T decreases. Combining this with formula (1), we know that when G increases, C increases. In formula (9), let T = C, and we can obtain formula (11):

[0052]

[0053] When the value of G is as shown in formula (11), each output feature Figure 1 Then it connects to all input feature maps.

[0054] The automated tobacco leaf grading method based on the adaptive attention mechanism described above, preferably, includes the following computation process:

[0055] For C depthwise convolutional kernels of size KK, where K≥2, adaptive sparse depthwise convolution performs a differential decomposition of the depthwise convolutional kernels along the kernel dimension, decomposing the two-dimensional convolution into two one-dimensional convolutions. The depthwise convolution with kernel size KK can be decomposed into two one-dimensional convolutions with kernel sizes K1 and 1K respectively. Adaptive sparse depthwise convolution reduces the number of parameters from K2C to 2KC.

[0056] Secondly, the present invention also provides an automated tobacco leaf grading system based on an adaptive attention mechanism, comprising:

[0057] The image acquisition module is used to acquire images of the tobacco leaf samples to be tested.

[0058] The adaptive attention model building module is used to build adaptive attention models based on adaptive sparse convolution.

[0059] The tobacco leaf grading model construction module is used to insert the constructed adaptive attention model into a convolutional neural network to obtain a tobacco leaf grading model;

[0060] The tobacco leaf grading module is used to input the images of the collected tobacco leaf samples into the tobacco leaf grading model to obtain the tobacco leaf grading results.

[0061] Thirdly, the present invention also provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-mentioned automated tobacco leaf grading method based on an adaptive attention mechanism.

[0062] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for automated tobacco leaf grading based on an adaptive attention mechanism.

[0063] This invention provides an automated tobacco leaf grading method, system, device, and storage medium. The adaptive attention model extracts deep features of tobacco leaves through adaptive sparse point convolution and adaptive sparse depth convolution, adaptively calculates the weights between channels, and focuses on important features that affect classification performance, thereby improving the classification performance of the network. The adaptive attention model module can be embedded into a classic convolutional neural network, has the ability to extract features, and achieves the best classification accuracy, providing an effective method for tobacco leaf grading. Attached Figure Description

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings, wherein:

[0065] Figure 1 A flowchart illustrating an embodiment of the automated tobacco leaf grading method based on an adaptive attention mechanism provided by the present invention;

[0066] Figure 2 This is a schematic diagram of the imaging module used to acquire RGB images of tobacco leaves.

[0067] Figure 3 This is a schematic diagram of the structure of adaptive sparse point convolution;

[0068] Figure 4 This is a schematic diagram of the structure of an adaptive sparse depthwise convolution.

[0069] Figure 5 This is a schematic diagram illustrating the construction process of an adaptive attention model.

[0070] Figure 6 A structural block diagram of an embodiment of the automated tobacco leaf grading system based on an adaptive attention mechanism provided by the present invention;

[0071] Figure 7 This is a structural block diagram of the electronic device provided by the present invention. Detailed Implementation

[0072] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The descriptions of the exemplary embodiments are merely illustrative and are in no way intended to limit the present disclosure or its application or use. The present disclosure may be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that the present disclosure will be thorough and complete, and will fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless specifically stated otherwise, the relative arrangement of components and steps, the composition of materials, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0073] The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Terms such as “including” or “contains” mean that the element preceding the term encompasses the element listed after it, and do not exclude the possibility of encompassing other elements as well. Terms such as “above” and “below” are used only to indicate relative positional relationships; when the absolute position of the described object changes, this relative positional relationship may also change accordingly.

[0074] In this disclosure, when a specific component is described as being located between a first component and a second component, an intermediary component may or may not be present between the specific component and the first or second component. When a specific component is described as connecting to other components, the specific component may be directly connected to the other components without having an intermediary component, or it may not be directly connected to the other components but may have an intermediary component.

[0075] All terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in a general dictionary, such as a dictionary, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0076] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0077] Traditional tobacco leaf grading methods rely heavily on manual labor, which is subjective, results in low grading quality, and can damage the tobacco leaves during the inspection process. Machine vision technology can achieve non-destructive inspection of tobacco leaves. Many current studies use digital image processing techniques to extract features such as size, color, and surface texture of tobacco leaves, followed by machine learning algorithms for classification. However, feature extraction and classification in these methods are independent, leading to poor classification stability and significant human intervention.

[0078] Deep learning-based methods can adaptively extract high-level abstract features of tobacco leaves through hierarchical learning. Convolutional neural networks (CNNs), a popular deep learning method, integrate feature extraction and classification and have been widely applied in fields such as food supervision, fault diagnosis, facial recognition, and medical treatment. Existing neural network models for tobacco leaf grading extract features through convolutional layers, compress features through pooling layers, and achieve classification through fully connected layers, enabling intelligent identification of tobacco leaf quality. However, they neglect the impact of important features on the classification process, resulting in low feature extraction efficiency. How to focus on the important features of tobacco leaves and improve the classification performance and efficiency of the network is a problem that urgently needs to be solved.

[0079] Example 1

[0080] like Figure 1 As shown, the automated tobacco leaf grading method based on adaptive attention mechanism provided in this embodiment includes the following steps in actual execution:

[0081] Step S1: Acquire images of the tobacco leaf samples to be tested.

[0082] In step S1, the tobacco leaf acquisition system uses an imaging module to acquire RGB images of the tobacco leaves. This imaging module can be, for example, a color camera. Figure 2 As shown, in one embodiment of the automated tobacco leaf grading method based on an adaptive attention mechanism of the present invention, step S1 may specifically include:

[0083] Step S11: Use a servo motor to drive a mechanical conveyor belt to transport tobacco leaf samples.

[0084] In some embodiments of this invention, a mechanical conveyor belt transports tobacco leaf samples at a uniform speed of 0.5 m / s to 1.5 m / s, for example, 1 m / s, to ensure effective acquisition of tobacco leaf images. It should be noted that this invention does not specifically limit the conveyor belt speed or the variation pattern of the conveyor speed.

[0085] Step S12: Set up a data acquisition box on the mechanical conveyor belt and arrange halogen tungsten lamps inside the data acquisition box.

[0086] The data acquisition box is used to isolate the interference of external natural light. The halogen tungsten lamps are arranged in two rows and evenly distributed inside the data acquisition box to ensure sufficient lighting.

[0087] Step S13: Set up a DSLR camera above the data acquisition box and open a through hole at the corresponding position on the data acquisition box. Use the DSLR camera to acquire backlit images of the tobacco leaves inside the data acquisition box and control the acquisition frequency of the DSLR camera through the communication module.

[0088] The color camera, for example, can be a DSLR camera, which is controlled by a communication module and captures 14 backlit images of tobacco leaves per minute.

[0089] During operation, the tobacco leaf sample is driven by a mechanical conveyor belt and illuminated by a halogen tungsten lamp in the data acquisition box. Its reflected light is captured by a DSLR camera to obtain its RGB image information, which is then displayed on a computer.

[0090] Step S2: Construct an adaptive attention model (SAAM) based on adaptive sparse convolution.

[0091] Specifically, an adaptive attention model is constructed based on adaptive sparse point convolution (SASPConv) and adaptive sparse depthwise convolution (SASDConv). For example... Figure 5 As shown, in one embodiment of the automated tobacco leaf grading method based on an adaptive attention mechanism of the present invention, step S2 may specifically include:

[0092] Step S21: Set the dimensions of the input feature map and the output feature map to be the same. For the input feature map F'∈RH×W×C', where H represents the height, W represents the width, and C' represents the number of channels.

[0093] Step S22: Extract the features of the input feature map F' in the channels through global convolution to obtain the channel-based global information F1, which is expressed by formula (1):

[0094] F1 = GConv(F') (1),

[0095] Here, GConv represents global convolution.

[0096] Step S23: Generate weights for the global information F1 through adaptive sparse point convolution, obtain F2, and express it through formula (2):

[0097] F2=MFPConv(F1) (2).

[0098] Adaptive Sparse Convolution (SASPConv) ensures end-to-end, redundancy-free learning from input to output. It effectively learns the feature maps of each channel by scaling the hidden layer channels and enhancing information exchange between channels through channel shuffling. Therefore, SASPConv can be used to generate weights for F1 and obtain F2.

[0099] Step S24: Use the softmax function to assign weights to F2, obtain the weights W, and express them using formula (3):

[0100] W = Softmax(F2) (3).

[0101] Formula (3) ensures that the weight range is between 0 and 1.

[0102] Step S25: Adjust the dimensions of the input feature map F' using adaptive sparse point convolution (SASPConv) to obtain F3, which is expressed by formula (4):

[0103] F3 = MFDConv(F') (4).

[0104] Dimensional adjustments are made to facilitate subsequent fusion.

[0105] Step S26: Multiply the weight W obtained by formula (3) and F3 obtained by formula (4) element-wise to obtain the adjusted feature map U, which is expressed by formula (5):

[0106]

[0107] in, This indicates element-wise multiplication.

[0108] Step S27: Extract the depth features of the adjusted feature map U through adaptive sparse depthwise convolution (SASDConv) to obtain the output feature map U', which is expressed by formula (6):

[0109] U'=MFDConv(U) (6),

[0110] Where U' represents the output feature map.

[0111] By employing adaptive sparse depthwise convolution, the depth features of U can be further extracted to ensure sufficient and effective feature extraction. Adaptive sparse depthwise convolution also allows for focusing on important tobacco leaf feature maps and allocating computational resources rationally.

[0112] Furthermore, such as Figure 3 As shown, in some embodiments of the present invention, the calculation process of the adaptive sparse point convolution may specifically include:

[0113] Step SA1: Set the dimensions of the input feature map and the output feature map to be the same. For the input feature map F∈RH×W×C, where H represents the height, W represents the width, and C represents the number of channels.

[0114] Step SA2: Divide the input feature map F into G groups, and apply point convolution to reduce the dimensionality of the feature map in each group. The channels are compressed by a factor of R to obtain the hidden layer.

[0115] Step SA3: Extract channel difference information from different channels through channel shuffling.

[0116] Since the output of a grouped channel can only be obtained from the input channel of that group, and the interaction between channels is weak, this invention extracts channel difference information from different channels through channel shuffle.

[0117] Step SA4: Apply point convolution again to increase the dimensionality of the feature maps of each group.

[0118] Step SA5: Concatenate all feature maps to obtain the deep features of the tobacco leaf at the point level, where:

[0119] Based on the method for calculating the number of parameters, the number of parameters P of the adaptive sparse point convolution is calculated using formula (7):

[0120]

[0121] The parameter P comes from two point convolutional layers used for dimensionality reduction and dimensionality increase.

[0122] Step SA6: Using formula (7), we obtain the formula (8) for calculating the number of channels C with respect to P:

[0123]

[0124] Step SA7, due to the output feature map and The hidden layer feature maps are connected, and the hidden layer feature maps are connected to... The input feature maps are connected, therefore the connectivity T between the output feature map and the input feature map can be expressed by formula (9):

[0125]

[0126] Here, connectivity T is used to represent the number of input-output connections for each output channel.

[0127] Step SA8: When R is a constant, divide formula (7) by formula (9) to obtain the formula (10) for calculating T with respect to P:

[0128]

[0129] Step SA9: According to formula (10), when G increases, T decreases. Combining this with formula (1), when G increases, C increases. In formula (9), let T = C, and we can obtain formula (11):

[0130]

[0131] When the value of G is as shown in formula (11), each output feature Figure 1 This connection then links to all input feature maps, thus ensuring both connectivity between the output and input while preventing redundant connections between them.

[0132] Therefore, adaptive sparse point convolution can extract features of tobacco leaves in a lightweight and non-redundant manner.

[0133] Furthermore, such as Figure 4 As shown, in some embodiments of the present invention, the calculation process of the adaptive sparse depthwise convolution may specifically include:

[0134] For C depthwise convolutional kernels of size KK, where K≥2, adaptive sparse depthwise convolution performs a differential decomposition of the depthwise convolutional kernels along the kernel dimension, decomposing the two-dimensional convolution into two one-dimensional convolutions. The depthwise convolution with kernel size KK can be decomposed into two one-dimensional convolutions with kernel sizes K1 and 1K respectively. Adaptive sparse depthwise convolution reduces the number of parameters from K2C to 2KC.

[0135] Step S3: Insert the constructed adaptive attention model into the convolutional neural network to obtain the tobacco leaf grading model.

[0136] The Adaptive Attention Model (SAAM) obtains channel-based global information of tobacco leaf feature maps through global convolution, learns the weights of the feature maps through Adaptive Sparse Point Convolution (SASPConv), and further allocates the weights using the Softmax function, effectively learning and extracting important features of tobacco leaves. The adaptive attention model constructed in this invention is essentially a channel attention module that can be directly inserted into convolutional neural networks to improve the network's classification performance. Furthermore, after weight allocation, the adaptive attention model further extracts features through Adaptive Sparse Depth Convolution (SASDConv). Therefore, the adaptive attention model has feature extraction capabilities and can replace ordinary convolutional layers. Embedding the constructed adaptive attention model (SAAM) into a classic convolutional neural network can help improve the network's classification performance.

[0137] In some embodiments of the present invention, SAAM can be inserted into a convolutional neural network as a single layer to help the network improve classification performance; it can also directly replace the convolutional layer to extract deep features.

[0138] Table 1 shows examples of embedding the Adaptive Attention Model (SAAM) into AlexNet and ShuffleNetV1 after pre-adjustment. AlexNet and ShuffleNetV1 are two classic neural networks, while NAM, ECA, SE, and SK are four classic attention mechanisms. Conv(C i1 ,K1,P1,C o1 S1) represents the number of input channels as C i1 The kernel size is K1, the padding size is P, the stride is S1, and the number of output channels is C. o1 Ordinary convolution. MaxP(K2,S2) pooling with a viewpoint of K2 and a downsampling stride of S2. GAP is global average pooling. In SAAM(C i2 ,K3,P2,C o2 In the SAAM convolutional model (S2, R), the hidden layers are compressed by a factor of R, and the relationship between R and G conforms to Equation 5. The kernel size in SAAM is K3, the padding size is P2, and the stride is S2. The number of input channels per SAAM layer is C. i2 The number of output channels is C o2 Shuffle(C i3 C o3 S3) indicates that the number of input and output channels is C. i3 C o3 The ShuffleNet unit has a depthwise separable convolution kernel size of 3×3, padding of 1, stride of S3, and number of groups of 2.

[0139] Table 1

[0140]

[0141] In Table 1, Conv(3,11,2,48,4) represents convolution with 3 input channels, a kernel size of 11x11, a padding size of 2, 48 output channels, and a stride of 4; MaxP(3,2) represents max pooling with a pooling field size of 3x3 and a stride of 2; GAP represents global average pooling; FC represents a fully connected layer with the parameter representing the number of taps; Shuffle(200,400,2) represents a ShuffleNet unit with 200 input channels, 400 output channels, and a stride of 2; SAAM(128,5,2,128,1,2) represents an example of the SAAM module proposed in this invention, with 128 input channels, a kernel size of 5x5, a padding size of 2, 128 output channels, a stride of 1, and a hidden layer compression factor of 2.

[0142] The comparison results of inserting several popular attention modules (NAM, SE, ECA, SK) into AlexNet and ShuffleNetV1 with SAAM are shown in Table 2. Among them, the structures of NAM, SE, ECA, and SK embedded in the convolutional neural network are all parallel to SAAM, which proves that the proposed SAAM has the strongest ability to focus on important features and suppress unimportant features. It has good network classification performance and strong classification stability, and can effectively realize the automatic grading of tobacco leaves.

[0143] Table 2

[0144]

[0145] Step S4: Input the image of the collected tobacco leaf sample into the tobacco leaf grading model to obtain the tobacco leaf grading result.

[0146] The automatic tobacco leaf grading method based on adaptive attention mechanism provided in this invention uses an adaptive attention model to extract deep features of tobacco leaves through adaptive sparse point convolution and adaptive sparse depth convolution, adaptively calculates the weights between channels, and focuses on important features that affect classification performance, thereby improving the classification performance of the network. The adaptive attention model module can be embedded into a classic convolutional neural network, has the ability to extract features, and achieves the best classification accuracy, providing an effective method for tobacco leaf grading.

[0147] Example 2

[0148] Accordingly, such as Figure 6 As shown, the present invention also provides an automated tobacco leaf grading system based on an adaptive attention mechanism, comprising:

[0149] Image acquisition module 1 is used to acquire images of the tobacco leaf samples to be tested;

[0150] Adaptive attention model building module 2 is used to build an adaptive attention model based on adaptive sparse convolution.

[0151] The tobacco leaf grading model construction module 3 is used to insert the constructed adaptive attention model into a convolutional neural network to obtain a tobacco leaf grading model;

[0152] The tobacco leaf grading module 4 is used to input the image of the collected tobacco leaf sample into the tobacco leaf grading model to obtain the tobacco leaf grading result.

[0153] The automated tobacco grading system based on an adaptive attention mechanism provided in this invention uses an adaptive attention model to extract deep features of tobacco leaves through adaptive sparse point convolution and adaptive sparse depth convolution, adaptively calculates the weights between channels, and focuses on important features that affect classification performance, thereby improving the classification performance of the network. The adaptive attention model module can be embedded into a classic convolutional neural network, has the ability to extract features, and achieves the best classification accuracy, providing an effective method for tobacco leaf grading.

[0154] Example 3

[0155] like Figure 7 As shown, the present invention also provides an electronic device, which includes a memory 22 and a processor 21. The memory 22 stores a computer program 23. When the processor 21 executes the computer program 23, it implements the automated tobacco leaf grading method based on an adaptive attention mechanism of Embodiment 1.

[0156] The electronic device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 7 This is merely an example of an electronic device and does not constitute a limitation on electronic devices. It may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0157] The processor 21 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0158] In some embodiments, the memory 22 may be an internal storage unit of the electronic device, such as a hard disk or memory. In other embodiments, the memory 22 may be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 22 may include both internal and external storage units of the electronic device. The memory 22 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 22 can also be used to temporarily store data that has been output or will be output.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0160] Example 4

[0161] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the automated tobacco leaf grading method based on an adaptive attention mechanism as described in Embodiment 1.

[0162] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0163] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. An automated tobacco leaf grading method based on an adaptive attention mechanism, characterized in that, include: Acquire images of the tobacco leaf samples to be tested; An adaptive attention model is constructed based on adaptive sparse convolution. The global information of the input feature map is extracted by global convolution, and weights are generated and assigned to the global information by adaptive sparse point convolution. The dimension of the input feature map is adjusted by adaptive sparse point convolution, and the weights are multiplied with the adjusted feature map. Finally, the depth features are extracted by adaptive sparse depth convolution to obtain the adaptive attention model. The constructed adaptive attention model is inserted into a convolutional neural network to obtain a tobacco leaf grading model; The images of the collected tobacco leaf samples are input into the tobacco leaf grading model to obtain the tobacco leaf grading results. The calculation process of the adaptive sparse point convolution includes: The input and output feature maps are set to have the same dimensions. For the input feature map F, its height is H, its width is W, and its number of channels is C. The input feature map F is divided into G groups, and point convolution is applied to each group of feature maps to reduce the dimensionality. The channels are compressed by a factor of R to obtain the hidden layer. Extracting channel difference information from different channels through channel shuffling; Point convolution is applied again to increase the dimensionality of the feature maps in each group; By concatenating all feature maps, the deep features of the tobacco leaf at the point level are obtained, where: The parameter P of the adaptive sparse point convolution is calculated using formula (7): （7）， The parameter P comes from two point convolutional layers used for dimensionality reduction and dimensionality increase; Using formula (7), we obtain the formula (8) for calculating the number of channels C with respect to P: （8）； Due to the output feature map and The hidden layer feature maps are connected, and the hidden layer feature maps are connected to... The input feature maps are connected, so the connectivity T between the output feature map and the input feature map can be expressed by formula (9): （9）， Wherein, connectivity T is used to represent the number of input-output connections for each output channel; When R is a constant, dividing formula (7) by formula (9) yields formula (10) for calculating T with respect to P: （10）； From formula (10), we know that when G increases, T decreases. Combining this with formula (1), we know that when G increases, C increases. In formula (9), let T = C, and we can obtain formula (11): （11）， When the value of G is as shown in Equation (11), each output feature map is connected to all input feature maps at once.

2. The automated tobacco leaf grading method based on adaptive attention mechanism according to claim 1, characterized in that, The acquisition of images of the tobacco leaf samples to be tested specifically includes: The tobacco leaf acquisition system uses an imaging module to acquire RGB images of tobacco leaves.

3. The automated tobacco leaf grading method based on adaptive attention mechanism according to claim 2, characterized in that, The method of using a tobacco leaf acquisition system to acquire RGB images of tobacco leaves through an imaging module specifically includes: A servo motor drives a mechanical conveyor belt to transport tobacco leaf samples. A data acquisition box is installed on the mechanical conveyor belt, and a halogen tungsten lamp is arranged inside the data acquisition box; A DSLR camera is installed above the data acquisition box, and a through hole is provided at a corresponding position on the data acquisition box. The DSLR camera acquires backlit images of the tobacco leaves inside the data acquisition box, and the acquisition frequency of the DSLR camera is controlled by a communication module.

4. The automated tobacco leaf grading method based on adaptive attention mechanism according to claim 1, characterized in that, The adaptive attention model constructed based on adaptive sparse point convolution and adaptive sparse depthwise convolution specifically includes: Assuming the input and output feature maps have the same dimension, for the input feature map F... ’ Its height is H, its width is W, and its number of channels is C'; The input feature map F' is extracted in channels by global convolution to obtain channel-based global information F1, which is expressed by formula (1): （1）， Where GConv represents global convolution; Weights are generated for the global information F1 using adaptive sparse point convolution, resulting in F2, which is expressed by formula (2): （2）； The softmax function is used to assign weights to F2, and the weights W are obtained, which are expressed by formula (3): （3）； The dimension of the input feature map F' is adjusted by adaptive sparse point convolution to obtain F3, which is expressed by formula (4): （4）； The weight W obtained by formula (3) is multiplied element-wise with F3 obtained by formula (4) to obtain the adjusted feature map U, which is expressed by formula (5): （5）， in, This indicates element-wise multiplication; The depth features of the adjusted feature map U are extracted by adaptive sparse depthwise convolution to obtain the output feature map U', which is expressed by formula (6): （6）， Where U' represents the output feature map.

5. The automated tobacco leaf grading method based on adaptive attention mechanism according to claim 1, characterized in that, The calculation process of the adaptive sparse depthwise convolution includes: For C depthwise convolution kernels of size K×K, where K≥2, adaptive sparse depthwise convolution performs a differential decomposition of the depthwise convolution kernels along the kernel dimension, decomposing the two-dimensional convolution into two one-dimensional convolutions. The depthwise convolution with kernel size K×K can be decomposed into two one-dimensional convolutions with kernel sizes of K×1 and 1×K, respectively. Adaptive sparse depthwise convolution reduces the number of parameters from K2C to 2KC.

6. An automated tobacco leaf grading system based on an adaptive attention mechanism, employing the method described in any one of claims 1-5, characterized in that, include: The image acquisition module is used to acquire images of the tobacco leaf samples to be tested. The adaptive attention model construction module is used to build an adaptive attention model based on adaptive sparse convolution. It extracts channel global information of the input feature map through global convolution, generates weights for the global information and assigns them using adaptive sparse point convolution, adjusts the dimension of the input feature map through adaptive sparse point convolution, multiplies the weights with the adjusted feature map, and finally extracts depth features through adaptive sparse depth convolution to obtain the adaptive attention model. The tobacco leaf grading model construction module is used to insert the constructed adaptive attention model into a convolutional neural network to obtain a tobacco leaf grading model; The tobacco leaf grading module is used to input the images of the collected tobacco leaf samples into the tobacco leaf grading model to obtain the tobacco leaf grading results.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the automated tobacco leaf grading method based on an adaptive attention mechanism as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the automated tobacco leaf grading method based on an adaptive attention mechanism as described in any one of claims 1 to 5.

Images