Tobacco leaf grading method and device, electronic device and storage medium

Abstract

The invention relates to a tobacco leaf grading method and device, an electronic device and a storage medium, and the method comprises the steps: obtaining a target image containing target tobacco leaves; inputting the target image into a first feature extraction module and a second feature extraction module at the same time, respectively extracting spatial domain features and frequency domain features of the target tobacco leaves, and generating a first feature map and a second feature map; inputting the first feature map and the second feature map into an information fusion module to generate a fused feature map; and outputting a grading result of the target tobacco based on the fused feature map. Through the tobacco leaf classification method and device, the problems that an existing tobacco leaf classification method depends on a single airspace feature, representation is not comprehensive, and the accuracy is limited are solved, effective complementation and deep fusion of airspace local details and frequency domain global structure information are achieved, and the accuracy and robustness of automatic tobacco leaf classification are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of tobacco leaf grading, and in particular to a method, apparatus, electronic device and storage medium for tobacco leaf grading. Background Technology

[0002] As the core raw material of the tobacco industry, the quality grade of tobacco leaves directly determines the sensory quality, aroma, and processing characteristics of cigarette products. Tobacco leaf grading is the process of classifying tobacco leaves into different grades based on multiple dimensions of appearance and internal indicators, including maturity, leaf structure, oil content, color, and length. Accurate and consistent grading results are the core basis for tobacco leaf procurement pricing, aging formulas, and process adjustments, and are crucial for ensuring the stability of tobacco product quality and the economic benefits of the industry.

[0003] Traditional tobacco leaf grading has long relied on the manual visual inspection and experience of professional graders. This method is highly subjective and easily affected by the experience, fatigue, and ambient lighting, resulting in inconsistent grading results and low efficiency, making it difficult to meet the needs of modern large-scale production. With the development of computer vision and artificial intelligence technologies, automated tobacco leaf grading methods based on deep learning have become the mainstream research direction. These methods typically use convolutional neural networks to automatically learn and extract features for classification directly from the RGB images of tobacco leaves.

[0004] In the feature representation of tobacco leaf images, spatial and frequency domain features are complementary: spatial features can intuitively reflect the surface texture details (such as vein distribution, spots and blemishes), shape contours, and other spatial geometric information of tobacco leaves, forming the basis for distinguishing differences in tobacco leaf appearance; frequency domain features, by revealing the frequency distribution patterns of the image, can effectively capture global structural information such as color gradations and texture periodicity of tobacco leaves, and have a stronger representational ability for features related to intrinsic quality such as tobacco leaf maturity and oil content. However, existing deep learning-based tobacco leaf grading methods focus on extracting single features, relying solely on the spatial information of the image for modeling, ignoring the key quality-related information contained in the frequency domain features, resulting in incomplete feature representation and difficulty in meeting the accurate grading requirements in complex scenarios.

[0005] Therefore, there is an urgent need for a technical solution that can simultaneously mine the frequency domain and spatial domain features of tobacco leaves and achieve effective fusion of the dual-domain features, so as to overcome the limitations of existing methods in feature extraction and improve the accuracy and stability of tobacco leaf grading. Summary of the Invention

[0006] This application provides a method, apparatus, electronic device, and storage medium for grading tobacco leaves, which at least solves the problem in related technologies that rely solely on spatial information from images for modeling, ignoring key quality correlation information contained in frequency domain features, resulting in incomplete feature representation and difficulty in meeting the accurate grading requirements in complex scenarios.

[0007] In a first aspect, embodiments of this application provide a method for grading tobacco leaves, comprising: acquiring a target image containing target tobacco leaves;

[0008] The target image is input into the first feature extraction module to extract spatial features that characterize the local appearance details of the target tobacco leaf, and generate a first feature map.

[0009] The target image is input to the second feature extraction module to extract frequency domain features that characterize the global structural properties of the target tobacco leaf, and a second feature map is generated.

[0010] The first feature map and the second feature map are input into the information fusion module to generate a fused feature map;

[0011] Based on the fused feature map, the grading result of the target tobacco leaf is output.

[0012] In some embodiments, the first feature extraction module is an EfficientNet-b0 network that retains pre-trained weights.

[0013] In some embodiments, the second feature extraction module includes a frequency domain transformation unit and a frequency domain feature extraction unit connected in sequence;

[0014] The step of inputting the target image into the second feature extraction module to extract frequency domain features characterizing the global structural attributes of the target tobacco leaf and generating a second feature map includes:

[0015] The target image is input into the frequency domain transformation unit, and a frequency domain feature map is generated through discrete cosine transformation.

[0016] The frequency domain feature map is input to the frequency domain feature extraction unit, and the deep frequency domain features of the frequency domain feature map are extracted by the deep convolutional neural network in the frequency domain feature extraction unit to generate the second feature map.

[0017] In some embodiments, generating a frequency domain feature map via the discrete cosine transform includes:

[0018] The target image is subjected to discrete cosine transform in units of 8×8 pixel blocks to obtain multiple transform coefficient matrices;

[0019] The coefficients in the same position in each transformation coefficient matrix are extracted and recombined to generate the 64-channel frequency domain feature map.

[0020] In some embodiments, the deep convolutional neural network in the frequency domain feature extraction unit is obtained in the following manner:

[0021] The number of input channels in the first convolutional layer of the pre-trained EfficientNet-b0 network was changed from 3 to 64, and the convolutional weights corresponding to the newly added input channels were initialized.

[0022] In some embodiments, the step of inputting the first feature map and the second feature map into the information fusion module to generate a fused feature map includes:

[0023] The first feature map is used as the query vector Q, and the second feature map is used as both the key vector K and the value vector V.

[0024] A cross-attention mechanism is adopted. Attention weights are obtained by calculating the similarity between the query vector Q and the key vector K. The value vector V is then weighted and summed using the attention weights to generate the fused feature map.

[0025] In some embodiments, the step of outputting the grading result of the target tobacco leaf based on the fused feature map includes:

[0026] The fused feature map is sequentially input into a multilayer perceptron for nonlinear enhancement, channel adjustment, and global average pooling to obtain a one-dimensional feature vector.

[0027] The one-dimensional feature vector is input into a fully connected classification layer for mapping, and the grading result of the target tobacco leaf is output based on the preset tobacco leaf grade category.

[0028] Secondly, embodiments of this application provide a tobacco leaf grading device, including: an acquisition module, a first feature extraction module, a second feature extraction module, a fusion module, and a grading module;

[0029] The acquisition module is used to acquire a target image containing the target tobacco leaf;

[0030] The first feature extraction module is used to extract spatial features of the target image that characterize local appearance details of the target tobacco leaf, and generate a first feature map;

[0031] The second feature extraction module is used to extract frequency domain features of the target image that characterize the global structural properties of the target tobacco leaf, and generate a second feature map;

[0032] The fusion module is used to fuse the first feature map and the second feature map to generate a fused feature map;

[0033] The grading module is used to output the grading result of the target tobacco leaf based on the fused feature map.

[0034] Thirdly, embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the tobacco leaf grading method as described in the first aspect above.

[0035] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the tobacco leaf grading method as described in the first aspect above.

[0036] Compared with related technologies, the tobacco leaf grading method provided in this application solves the technical problems of existing methods having a single feature extraction dimension and insufficient utilization of global structural information reflecting the intrinsic quality of tobacco leaves by constructing a technical framework for parallel extraction and deep fusion of frequency domain and spatial domain features. This method achieves the introduction and enhancement of global structural information while retaining the ability to distinguish local details, thereby significantly improving the accuracy and robustness of tobacco leaf grading.

[0037] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0039] Figure 1 This is a hardware structure block diagram of a terminal for a tobacco leaf grading method according to an embodiment of this application;

[0040] Figure 2 This is a flowchart of a tobacco leaf grading method according to an embodiment of this application;

[0041] Figure 3 This is a flowchart illustrating another tobacco leaf grading method according to an embodiment of this application;

[0042] Figure 4 This is a structural block diagram of a tobacco leaf grading device according to an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0044] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0045] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0046] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of a terminal for a tobacco leaf grading method according to an embodiment of this application. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0047] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the tobacco leaf grading method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0048] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0049] This embodiment provides a method for grading tobacco leaves. Figure 2 This is a flowchart of a tobacco leaf grading method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0050] Step S201: Obtain a target image containing the target tobacco leaf.

[0051] Specifically, first, the original image containing the target tobacco leaf is acquired, which can be obtained by taking pictures with an industrial camera or a high-resolution scanner under standard lighting conditions.

[0052] Then, image segmentation tools can be used to segment the original image, extracting regions containing only the target tobacco leaves and removing background interference. Specifically, this approach uses the Segment Anything Model (SAM) to perform the image segmentation preprocessing step. SAM is a general-purpose visual foundation model trained on massive, diverse datasets, possessing powerful zero-shot generalization capabilities. It adapts to tobacco leaves without any specific training, is flexible in its use, and requires no complex parameter adjustments. More importantly, SAM can achieve pixel-level accurate segmentation with clear boundaries. In this application, the discriminative features relied upon for tobacco leaf grading (such as specific textures, subtle color gradations, and vein morphology) are extremely refined. Any residual background pixels or errors in segmentation edges will act as strong noise, contaminating the spatial feature extraction process and making it difficult for subsequent convolutional neural networks to accurately learn the essential morphological discriminative features of tobacco leaves. More importantly, the subsequent frequency domain analysis (discrete cosine transform) is a mathematical transformation of the entire image, the result of which is determined by the value of each pixel in the image. If the segmentation is inaccurate, the frequency components of the background region will be introduced and mixed into the frequency domain representation of the tobacco leaves, fundamentally distorting the uniformity of color distribution and the periodic structure of texture that are directly related to quality, causing distortion of the subsequently extracted frequency domain features and severely affecting grading accuracy. Therefore, in this scheme, selecting a suitable image segmentation tool is the first and crucial step in ensuring the effective operation of all subsequent advanced feature extraction and analysis modules. The pixel-level precision segmentation provided by SAM ensures that the image input to the subsequent dual-branch feature extraction module is a completely pure tobacco leaf subject. This creates conditions for spatial domain features to focus on ontological details and lays a reliable foundation for the accurate representation of the inherent global pattern by frequency domain features. Therefore, in this application, SAM is selected as the corresponding tobacco leaf image segmentation tool to segment and obtain pure tobacco leaf target regions. SAM's excellent versatility ensures the robustness of this scheme, and its ultra-high precision guarantees the purity of the feature source, making it an indispensable preprocessing core component for achieving high-precision tobacco leaf grading.

[0053] Finally, the segmented tobacco leaf images are subjected to data augmentation processing, including horizontal flipping, vertical flipping, translation, scaling, and rotation, to expand data diversity and improve the model's generalization ability. Normalization processing is then performed to obtain the final target image.

[0054] Through the above steps, a high-quality image preprocessing workflow is systematically constructed by high-fidelity image acquisition, accurate SAM-based segmentation, and standardized data augmentation and normalization, providing clean, consistent, and complete input for subsequent dual-path deep feature extraction and fusion.

[0055] Step S202: Input the target image into the first feature extraction module to extract spatial features used to characterize the local appearance details of the target tobacco leaf and generate a first feature map; input the target image into the second feature extraction module to extract frequency domain features used to characterize the global structural attributes of the target tobacco leaf and generate a second feature map.

[0056] Among them, the aforementioned local appearance details, including but not limited to spatial textures (such as leaf vein grids and cell wrinkles), edge contours, specific shape characteristics (such as leaf tip shape and leaf length-to-width ratio), and local color spots or defects determined by the microscopic morphology of the tobacco leaf surface, are the direct basis for distinguishing the appearance quality of different tobacco leaves; the aforementioned global structural attributes, including but not limited to the macroscopic distribution and gradation patterns of color on the entire leaf reflecting the internal quality of the tobacco leaf, the periodicity or directionality of texture patterns, etc., are highlighted through frequency domain transformation and are closely related to key grading indicators such as the maturity, oil content, and uniformity of the tobacco leaf.

[0057] Specifically, the acquired target image is input into the first feature extraction module and the second feature extraction module, respectively. In the first feature extraction module branch, the first feature extraction module is used to extract the spatial features of the target tobacco leaf from the target image. These spatial features focus on the spatial texture details, outline shape, edge information, and local color spots of tobacco leaves, generating a first feature map that represents these fine local information.

[0058] In the second feature extraction module branch, the second feature extraction module is used to extract frequency domain features that are strongly correlated with the intrinsic quality of tobacco leaves. These frequency domain features mainly characterize the macroscopic gradation and distribution uniformity of tobacco leaf color across the entire leaf, as well as global information such as the periodic grid structure of texture, generating a deeper second feature map.

[0059] Through the above steps and the parallel dual-branch design, the local spatial details of the target tobacco leaf are mined and the global frequency domain structure is analyzed, realizing multi-dimensional and in-depth collaborative perception and feature expression of the appearance and internal quality information of the tobacco leaf.

[0060] Step S203: Input the first feature map and the second feature map into the information fusion module to generate a fused feature map.

[0061] Specifically, the first feature map and the second feature map are input into the information fusion module, and the spatial domain information and frequency domain information are deeply fused by the information fusion module to generate a fused feature map.

[0062] Step S204: Based on the fused feature map, output the grading result of the target tobacco leaf.

[0063] The target tobacco leaf grading result is to classify the tobacco leaf into one of the 15 preset quality grades.

[0064] Specifically, based on the fused feature map and combined with the preset quality level, the target tobacco leaves are graded, and the grading results of the target tobacco leaves are output.

[0065] Through the steps S201 to S204 above, the complete technical chain of "SAM precise segmentation preprocessing → spatial / frequency domain dual-branch deep feature extraction → information fusion → decision output" systematically solves the technical problems of single feature dimension and insufficient utilization of global information in existing automatic tobacco grading methods. It achieves effective unification and mutual enhancement of local detail discrimination power and global structural representation power, thereby significantly improving the accuracy, robustness and reliability of tobacco grading.

[0066] In some embodiments, the first feature extraction module is an EfficientNet-b0 network that retains pre-trained weights.

[0067] Specifically, the first feature extraction module uses the EfficientNet-b0 network, fully retaining its pre-trained weights on large general-purpose image datasets (such as ImageNet). The EfficientNet-b0 network achieves a good balance between model depth, width, and resolution through a composite scaling method, exhibiting high parameter efficiency, relatively low computational resource requirements, and superior performance, making it suitable as a basic feature extractor. "Retaining pre-trained weights" means embedding the pre-trained EfficientNet-b0 network as a fixed feature extractor into the architecture of this scheme. During the end-to-end training of the entire network, all parameters of this module are frozen, do not participate in gradient updates, and only perform forward propagation to extract features. Therefore, its weights maintain their initial pre-trained state throughout the entire process. In this way, this module can provide stable and high-quality spatial features during both the training and inference phases of the overall network. .

[0068] This approach adopts this strategy primarily based on the following considerations: First, the available labeled datasets for tobacco leaf grading tasks are typically limited in size. Training a large number of parameters in a deep network (such as EfficientNet-b0) entirely from scratch can easily lead to overfitting, i.e., excessive memorization of training set noise and loss of generalization ability. Training a dedicated deep learning neural network model entirely from scratch not only fails to address the aforementioned data scarcity problem but also introduces new issues such as training instability, slow convergence, and susceptibility to local optima. Second, models pre-trained on datasets like ImageNet have learned rich general visual features, such as various edges, corners, basic textures, and simple shape patterns. These features have strong transferability for understanding the local spatial information of tobacco leaves (such as vein texture and leaf contour). Therefore, directly applying these features as a fixed, high-performance feature extractor can be seen as introducing a powerful general visual prior. This not only effectively avoids the risk of overfitting due to insufficient data but also saves the enormous training overhead of this branch, improving the overall training efficiency and stability of the approach.

[0069] Through the above steps, by selecting the efficient EfficientNet-b0 network architecture and adopting a transfer learning strategy that retains pre-trained weights, this solution introduces a powerful, plug-and-play general visual feature extractor for tobacco leaf grading tasks without significantly increasing training complexity, providing a fundamental guarantee for reliable extraction of spatial features and subsequent successful dual-domain fusion.

[0070] In some embodiments, the second feature extraction module includes a frequency domain transformation unit and a frequency domain feature extraction unit connected in sequence;

[0071] The target image is input to the second feature extraction module to extract frequency domain features characterizing the global structural properties of the target tobacco leaf, generating a second feature map, including:

[0072] The target image is input into the frequency domain transformation unit, and a frequency domain feature map is generated through discrete cosine transformation.

[0073] The frequency domain feature map is input to the frequency domain feature extraction unit, and the deep convolutional neural network in the frequency domain feature extraction unit extracts the deep frequency domain features of the frequency domain feature map to generate the second feature map.

[0074] In some embodiments, generating a frequency domain feature map via the discrete cosine transform includes:

[0075] The target image is subjected to discrete cosine transform in units of 8×8 pixel blocks to obtain multiple transform coefficient matrices;

[0076] The coefficients that are in the same position in each transformation coefficient matrix are extracted and recombined to generate a 64-channel frequency domain feature map.

[0077] Specifically, the preprocessed target image is input into the frequency domain transform unit. This unit performs a discrete cosine transform on the image in 8×8 pixel blocks, outputting an 8×8 transform coefficient matrix for each block. Unlike the traditional approach of directly using these discrete matrices, this scheme performs a crucial structured recombination operation: extracting coefficients at the same positions in all coefficient matrices (e.g., the top left corner of all blocks, the first row and second column, etc.) and recombinating them into new channels according to their corresponding positions. Since each block generates 64 coefficients, the recombination results in a 64-channel frequency domain feature map. Each channel is composed of the responses of all image blocks at the same specific frequency component, thus transforming the frequency information from "intra-block spatial distribution" to "channel-dimensional representation," forming a structured tensor that can be directly and efficiently processed by deep convolutional networks.

[0078] Specifically, this scheme employs Discrete Cosine Transform (DCT) to obtain the frequency domain features of tobacco leaf images. The core idea of ​​DCT is to transform the image from the spatial domain to the frequency domain, revealing the distribution of its frequency components. This scheme chooses DCT primarily based on its inherent compatibility with the tobacco leaf grading task, specifically its following characteristics:

[0079] First, the Discrete Cosine Transform (DCT) can transform an image from a pixel-based spatial domain to a frequency-based frequency domain, thus decomposing image information into components of different frequencies according to their rate of change. In tobacco leaf images, key features reflecting macroscopic quality, such as uniform color transitions and overall color distribution, are manifested as slow and gradual changes in grayscale or color values, which correspond precisely to low-frequency components in the frequency domain. Therefore, through the DCT, these global visual features crucial for grading will naturally concentrate on a few transform coefficients representing low frequencies, allowing subsequent processing steps to directly and efficiently focus on this information.

[0080] Furthermore, compared to the Fourier transform of complex number operations, the DCT is a real number transform, which has lower computational complexity, faster processing speed, and better meets the strict requirements of real-time performance and computational efficiency in the online detection scenario of the tobacco industry.

[0081] Furthermore, the distribution of DCT coefficients in the frequency space naturally maps to the physical structure of tobacco leaf images. Low-frequency coefficients correspond to the global distribution of color and brightness, mid-frequency coefficients reflect the periodicity of texture and the grid-like structure of leaf veins, while high-frequency coefficients are associated with edges, details, and noise. This clear structural representation provides ideal raw information for subsequent channelization reconstruction and deep feature learning in this scheme.

[0082] More importantly, this scheme transforms the aforementioned physical mapping into an explicit feature representation within a deep learning framework by channelizing and recombining the DCT coefficients:

[0083] The channel, composed of the low-frequency coefficients (such as the upper left corner) of all blocks, becomes a "frequency domain overview" of the overall color and oil distribution of the coded tobacco leaf.

[0084] The channels composed of mid-frequency coefficients form a "characteristic response map" that represents the periodicity and structural regularity of tobacco leaf texture.

[0085] The channels, composed of high-frequency coefficients, provide the raw data layer that depicts detailed contours and potential noise.

[0086] This conversion from "frequency components to independent channels" is a key design feature of this scheme. It not only preserves the complete physical meaning of the DCT coefficients, but also creates an input format that is highly compatible with the processing mechanism of convolutional neural networks. The deep learning convolutional neural network can adaptively learn from these "frequency channels" and integrate the frequency domain patterns that are most important for tobacco grade determination, just like understanding RGB color channels. This lays an irreplaceable foundation of frequency domain information for achieving high-precision and interpretable grading.

[0087] Furthermore, the specific mathematical process and efficient implementation of the DCT transform in this scheme are as follows:

[0088] For an image patch of size N×N, the formula for its two-dimensional discrete cosine transform is defined as: ,in These are the image pixel values ​​in the spatial domain. These are the transform coefficients in the frequency domain. ,and .

[0089] To achieve efficient computation and integration with deep learning frameworks, this scheme initializes a DCT kernel and performs transformations through convolution operations. Specifically, it first generates a kernel of size [size missing]. DCT nuclei (of which To determine the size of the discrete cosine transform kernel, in this scheme... For each discrete cosine transform kernel component, (corresponding to the frequency domain) This generates the corresponding cosine vector, and then the kernel matrix of that component is obtained through outer product operation. and according to and The values ​​of are normalized according to the following rules: when and hour, ;when or But not both of them are 0. ;when and hour, .

[0090] The initialized DCT kernel is applied to the preprocessed tobacco leaf image through convolution operations to efficiently calculate the DCT coefficient matrix of each 8×8 image block.

[0091] Subsequently, the system performs the aforementioned coefficient recombination operation. Coefficients with the same position index are extracted from all the calculated coefficient matrices and recombine into independent channels. Since each block generates 64 coefficients, the recombination produces a frequency domain feature map with 64 channels. Each channel specifically carries the global response of the entire image at a particular frequency component.

[0092] Through the above steps, this scheme achieves the key transformation from spatial domain images to structured frequency domain features. By recombining coefficient channels, the frequency structure information revealed by DCT and intrinsically related to tobacco quality (low frequencies correspond to macroscopic color distribution, mid frequencies to texture periodicity, and high frequencies to details and noise) is transformed into a regular multi-channel tensor input that is naturally compatible with convolutional neural network architecture. This structured representation allows the subsequent frequency domain feature extraction network to directly learn and fuse the most effective deep frequency domain patterns for tobacco grade determination from each frequency channel, without requiring complex frequency domain analysis, using a conventional feature map processing approach. The extracted frequency domain features, with their clear semantic structure and global descriptive capabilities, effectively complement the local detail features extracted by the spatial domain branch, laying a high-quality and robust information foundation for subsequent cross-attention deep fusion, thus jointly supporting the performance improvement of the dual-domain fusion grading method of this invention.

[0093] In some embodiments, the deep convolutional neural network in the frequency domain feature extraction unit is obtained as follows:

[0094] The number of input channels in the first convolutional layer of the pre-trained EfficientNet-b0 network was changed from 3 to 64, and the convolutional weights corresponding to the newly added input channels were initialized.

[0095] Specifically, this scheme also selects the EfficientNet-b0 network, which retains pre-trained weights, as the main structure of the deep convolutional neural network in the frequency domain feature extraction unit. As discussed earlier, completely retraining a new network adapted to this frequency domain input would not only significantly increase training costs and convergence time, but also potentially cause overfitting and training instability on limited tobacco leaf datasets, making it difficult to guarantee the robustness and generalization performance of feature extraction. The EfficientNet-b0 network, through a composite scaling strategy, achieves a good balance in depth, width, and resolution, and features high parameter efficiency, fast inference speed, and powerful feature extraction capabilities, which aligns with the requirements of lightweight, real-time, and high discriminative performance in the tobacco leaf grading task of this scheme. At the same time, using a pre-trained model allows us to leverage its pre-trained model, which has already learned rich visual basic features (such as edges, textures, and shapes) on large general datasets (such as ImageNet), as a starting point, fully inheriting its powerful general visual representation and transfer learning capabilities, reducing dependence on the scale of tobacco leaf-specific data, and effectively improving the stability and convergence efficiency of the model in frequency domain feature learning.

[0096] However, in practical applications, the pre-trained EfficientNet-b0 network is designed and trained for processing 3-channel (RGB) natural images, with its input layer convolutional kernels having a fixed channel dimension of 3. In this scheme, the frequency domain feature map output after processing and reconstruction by the frequency domain transformation unit has 64 channels. Therefore, to enable this pre-trained network to receive and effectively process this multi-channel frequency domain input, its input layer must be adaptively modified.

[0097] This transformation process was implemented in the following ways:

[0098] First, structural adaptation is performed. The number of input channels of the first layer convolution (usually a 3x3 convolution) of the pre-trained EfficientNet-b0 network is expanded from 3 to 64 to match the input dimension of the 64-channel frequency domain feature map.

[0099] Next, weight initialization is performed. For the 61 newly added input channels after expansion, the corresponding convolutional kernel weights need to be initialized appropriately. Common strategies include: copying the average of the weights corresponding to the original 3 channels or using an initialization method suitable for deep networks (such as Kaiming initialization) to fill them in.

[0100] Finally, a training strategy is formulated. During subsequent network fine-tuning, the parameters of the entire network (including the modified first layer) will be used in the training. By utilizing the general visual feature representation capabilities already learned in the pre-trained model and adapting it to further extract deep frequency domain patterns related to tobacco leaf grading from the structured frequency domain feature map, efficient and stable transfer learning is achieved.

[0101] Through the above steps, by specifically modifying the input channels of the pre-trained EfficientNet-b0 network, the dimensionality matching problem between the pre-trained model and the novel multi-channel frequency domain input is cleverly solved. This not only retains the original model's powerful feature extraction architecture and transfer learning potential, but also enables it to seamlessly connect with the channelized frequency domain feature representation in this scheme. This allows for the efficient and accurate completion of the task of mining deep discriminative features from the frequency domain feature map, laying a solid technical foundation for the subsequent deep fusion of spatial and frequency domain features.

[0102] In some embodiments, the first feature map and the second feature map are input to the information fusion module to generate a fused feature map, including:

[0103] The first feature map is used as the query vector Q, and the second feature map is used as both the key vector K and the value vector V.

[0104] A cross-attention mechanism is adopted. Attention weights are obtained by calculating the similarity between the query vector Q and the key vector K. The value vector V is then weighted and summed using the attention weights to generate a fused feature map.

[0105] Specifically, to achieve deep complementarity and synergy between spatial and frequency domain features, this scheme employs a cross-attention mechanism to fuse information from both. In its implementation, spatial features... As a query Frequency domain characteristics Simultaneously serving as a key Sum First, regarding , and Perform a linear projection to map it to a space of the same dimension. Let the projected query, key, and value be respectively... , , The features after fusion for: ,in, It is the dimension of the key, used to scale the attention score to prevent the value from becoming too large.

[0106] Specifically, in this scheme, spatial characteristics are included. Frequency domain features as query vector Q Simultaneously serving as both the key vector K and the value vector V, the aim is to establish an information interaction mechanism with clear physical meaning and directionality, so as to achieve effective complementarity between local details in the spatial domain and the global structure in the frequency domain.

[0107] In the cross-attention mechanism, query Q initiates the retrieval, key K provides the set of indexes to be matched, and value V carries the information content to be extracted and aggregated. The assignment strategy in this scheme is based on the following considerations:

[0108] First, spatial characteristics It contains local details such as texture and edges of tobacco leaf images at various spatial locations. Using these details as Q means that these local details actively initiate the retrieval of relevant contextual information.

[0109] Then, frequency domain features The overall properties of tobacco leaf images in the frequency dimension, such as the global distribution of color and the periodic structure of texture, are encoded. Using these properties simultaneously as K and V means that these global structural patterns serve as the target library (K) for retrieval and the information source (V) ultimately used to supplement local details.

[0110] Therefore, based on the above settings, each from The local eigenvectors (as Q) will be related to The similarity is calculated from the global feature vectors (K) at all locations within the model, resulting in a set of attention weights. These weights quantify the strength of the association between the local detail and various global structural patterns. Subsequently, these weights are used to... (As V) is weighted and summed to generate an enhanced vector that corresponds to the original local details and is rich in global context information, and then fused back to the original spatial location.

[0111] Through the above design, this scheme guides the model to selectively filter, weight, and enhance the local details extracted from the spatial domain features during the fusion process, using the global structural information provided by the frequency domain features as a reference. This fusion method not only facilitates deep interaction between the two domain features at the information level, but also ensures that the final output fused features retain both accurate local details and a coordinated global structure, thereby significantly improving the overall feature representation's discriminative ability and model robustness in tobacco leaf grading tasks.

[0112] In some embodiments, based on the fused feature map, the grading result of the target tobacco leaf is output, including:

[0113] The fused feature maps are sequentially input into a multilayer perceptron for nonlinear enhancement, channel adjustment, and global average pooling to obtain a one-dimensional feature vector.

[0114] The one-dimensional feature vector is input into a fully connected classification layer for mapping, and the grading result of the target tobacco leaf is output based on the preset tobacco leaf grade category.

[0115] Specifically, after obtaining the deep fused feature map through the cross-attention mechanism, a series of refinement and mapping processes are required to transform it into a final discriminative hierarchical signal. This process begins with the multilayer perceptron module, which further enhances the feature representation capability by applying nonlinear transformations and enhancements to the fused features, enabling it to model more complex patterns and provide a more discriminative high-dimensional representation for classification. In practical applications, the multilayer perceptron module consists of linear layers, a GELU activation function, and dropout layers to refine the fused features. Further nonlinear transformations can be performed, and the formula can be expressed as: ; .in, It is the weight matrix of the linear layer. It is the bias vector.

[0116] Subsequently, the system optimizes the features through a channel adjustment step. Specifically, one or more 1×1 convolutional layers can be used to transform the channel dimensions of the feature map (e.g., adjusting from 128 dimensions to 256 dimensions, and then adjusting back to 128 dimensions). The 1×1 convolution achieves cross-channel information integration and recalibration without changing the spatial size of the feature map, adaptively improving the response of important feature channels and suppressing secondary or redundant information, thereby optimizing the quality of information flowing to the classifier.

[0117] Next, global average pooling is performed on the channel-adjusted feature map. This operation averages the values ​​at all spatial locations in each channel of the feature map, thus compressing the two-dimensional feature map into a one-dimensional feature vector. This step has a dual key function: first, it fixes the output dimension of the network, making it match the input of subsequent fully connected layers; second, by aggregating spatial locations, it endows the model with a certain degree of spatial translation invariance and significantly reduces the number of parameters, helping to prevent overfitting and improve the model's generalization ability.

[0118] Finally, the obtained one-dimensional feature vector is input into a fully connected classification layer. This layer acts as a linear mapper, mapping the high-dimensional feature vector to a low-dimensional category score space. In this scheme, the dimension of this space is preset to the number of tobacco leaf grade categories (e.g., 15 categories). The network outputs a score (or a probability after softmax) corresponding to each category. By comparing the scores of each category, the most likely grade of the target tobacco leaf can be determined, thus completing the automatic grading decision and outputting the result.

[0119] Through the above process, including nonlinear enhancement (MLP), information optimization (channel adjustment), dimensionality compression and regularization (global average pooling), and final discrimination (fully connected classification), this scheme successfully transforms the fusion feature map rich in complementary information from two domains into accurate tobacco leaf grade labels in a robust and efficient manner. This not only fully explores and utilizes the full discrimination potential of the fusion features, but also constitutes a reliable and reproducible decision terminal for the entire grading method.

[0120] Preferably, Figure 3 This is a flowchart illustrating another tobacco leaf grading method according to an embodiment of this application, as shown below. Figure 3 The flowchart described above fully illustrates the entire process of this solution: First, the original image containing the target tobacco leaves is acquired and preprocessed. Specifically, the Segment Anything Model (SAM) is used to segment the original image, accurately extracting the foreground region containing only the main tobacco leaves, effectively removing interference from complex backgrounds. Subsequently, the segmented image undergoes standardized data augmentation (including horizontal / vertical flipping, random rotation, and normalization) to expand data diversity and improve the model's generalization ability, ultimately obtaining a preprocessed target image with uniform size and a clean background.

[0121] Next, the target image is input in parallel into the dual-branch feature extraction module. In the spatial branch, the RGB image (i.e., the target image) is directly input into an EfficientNet-b0 network that retains ImageNet pre-trained weights to extract spatial features focusing on the local texture, edges, and shape details of the tobacco leaves. In the frequency domain branch, a discrete cosine transform is first used to convert the image from the spatial domain to the frequency domain, i.e., the transformation is performed in 8×8 pixel blocks. The coefficients at the same frequency positions in all transformed blocks are recombined to generate a 64-channel frequency domain feature map with clear physical meaning. Subsequently, this feature map is input into an EfficientNet-b0 network with its input channels adapted to 64 (here, it is also an EfficientNet-b0 network that has been pre-trained and retains its pre-trained weights) for deep feature extraction, obtaining the frequency domain features that characterize the global color distribution and texture periodic structure of tobacco leaves. .

[0122] Then, the extracted spatial features Frequency domain characteristics The input is fed into the cross-attention fusion module. The former is used as the query vector Q, and the latter serves as both the key vector K and the value vector V. By calculating attention weights, the model is guided to utilize the global structural context in the frequency domain to adaptively enhance and interpret local detailed features in the spatial domain, thereby obtaining deeply fused features. .

[0123] Subsequently, the fused features are nonlinearly enhanced by a multilayer perceptron, then channel-adjusted through 1×1 convolution, and aggregated into a one-dimensional feature vector by global average pooling. Finally, this vector is input into a fully connected classification layer, mapped to a preset tobacco grade category space (e.g., 15 grades), and the final grading result (i.e., the output label) is output.

[0124] Through the complete process described above, this scheme systematically achieves the complementarity and enhancement of local details and global structural information in tobacco leaf images via a collaborative architecture of "SAM precise segmentation—dual-domain parallel extraction—cross-attention fusion". As shown in Table 1, the comparative experimental results demonstrate that this scheme achieves significantly better performance than single-model methods on the tobacco leaf grading task, with an accuracy of 0.9145. Its precision, recall, F1 score, and Kappa coefficient all comprehensively outperform mainstream models such as EfficientNet-b0, ResNet50, and Vision Transformer, fully validating the effectiveness and advancement of the proposed frequency domain and spatial domain feature fusion framework.

[0125] Table 1

[0126]

[0127] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0128] This embodiment also provides a tobacco leaf grading device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0129] Figure 4 This is a structural block diagram of a tobacco leaf grading device according to an embodiment of this application, such as... Figure 4 As shown, the device includes: an acquisition module 10, a first feature extraction module 21, a second feature extraction module 22, a fusion module 30, and a hierarchical module 40;

[0130] The acquisition module 10 is used to acquire a target image containing the target tobacco leaf;

[0131] The first feature extraction module 21 is used to extract spatial features of the target image that characterize the local appearance details of the target tobacco leaf, and generate a first feature map;

[0132] The second feature extraction module 22 is used to extract frequency domain features of the target image that characterize the global structural attributes of the target tobacco leaf, and generate a second feature map;

[0133] The fusion module 30 is used to fuse the first feature map and the second feature map to generate a fused feature map;

[0134] The grading module 40 is used to output the grading results of the target tobacco leaves based on the fused feature map.

[0135] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0136] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0137] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0138] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0139] S1, Obtain the target image containing the target tobacco leaf.

[0140] S2, the target image is input to the first feature extraction module to extract spatial features used to characterize local appearance details and generate a first feature map; the target image is input to the second feature extraction module to extract frequency domain features used to characterize the global structural properties of the target tobacco leaf and generate a second feature map.

[0141] S3, input the first feature map and the second feature map into the information fusion module to generate a fused feature map.

[0142] S4. Based on the fused feature map, output the grading result of the target tobacco leaf.

[0143] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0144] Furthermore, in conjunction with the tobacco leaf grading methods in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the tobacco leaf grading methods in the above embodiments.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0146] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0147] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for grading tobacco leaves, characterized in that, include: Obtain the target image containing the target tobacco leaf; The target image is input into the first feature extraction module to extract spatial features that characterize the local appearance details of the target tobacco leaf, and generate a first feature map. The target image is input to the second feature extraction module to extract frequency domain features that characterize the global structural properties of the target tobacco leaf, and a second feature map is generated. The first feature map and the second feature map are input into the information fusion module to generate a fused feature map; Based on the fused feature map, the grading result of the target tobacco leaf is output.

2. The tobacco leaf grading method according to claim 1, characterized in that, The first feature extraction module is an EfficientNet-b0 network that retains pre-trained weights.

3. The tobacco leaf grading method according to claim 1, characterized in that, The second feature extraction module includes a frequency domain transformation unit and a frequency domain feature extraction unit connected in sequence; The step of inputting the target image into the second feature extraction module to extract frequency domain features characterizing the global structural attributes of the target tobacco leaf and generating a second feature map includes: The target image is input into the frequency domain transformation unit, and a frequency domain feature map is generated through discrete cosine transformation. The frequency domain feature map is input to the frequency domain feature extraction unit, and the deep frequency domain features of the frequency domain feature map are extracted by the deep convolutional neural network in the frequency domain feature extraction unit to generate the second feature map.

4. The tobacco leaf grading method according to claim 3, characterized in that, The generation of the frequency domain feature map via the discrete cosine transform includes: The target image is subjected to discrete cosine transform in units of 8×8 pixel blocks to obtain multiple transform coefficient matrices; The coefficients in the same position in each transformation coefficient matrix are extracted and recombined to generate the 64-channel frequency domain feature map.

5. The tobacco leaf grading method according to claim 3 or 4, characterized in that, The deep convolutional neural network in the frequency domain feature extraction unit is obtained in the following way: The number of input channels in the first convolutional layer of the pre-trained EfficientNet-b0 network was changed from 3 to 64, and the convolutional weights corresponding to the newly added input channels were initialized.

6. The tobacco leaf grading method according to claim 1, characterized in that, The step of inputting the first feature map and the second feature map into the information fusion module to generate a fused feature map includes: The first feature map is used as the query vector Q, and the second feature map is used as both the key vector K and the value vector V. A cross-attention mechanism is adopted. Attention weights are obtained by calculating the similarity between the query vector Q and the key vector K. The value vector V is then weighted and summed using the attention weights to generate the fused feature map.

7. The tobacco leaf grading method according to any one of claims 1 to 6, characterized in that, The step of outputting the grading result of the target tobacco leaf based on the fused feature map includes: The fused feature map is sequentially input into a multilayer perceptron for nonlinear enhancement, channel adjustment, and global average pooling to obtain a one-dimensional feature vector. The one-dimensional feature vector is input into a fully connected classification layer for mapping, and the grading result of the target tobacco leaf is output based on the preset tobacco leaf grade category.

8. A tobacco leaf grading device, characterized in that, include: The module includes an acquisition module, a first feature extraction module, a second feature extraction module, a fusion module, and a hierarchical module. The acquisition module is used to acquire a target image containing the target tobacco leaf; The first feature extraction module is used to extract spatial features of the target image that characterize local appearance details of the target tobacco leaf, and generate a first feature map; The second feature extraction module is used to extract frequency domain features of the target image that characterize the global structural properties of the target tobacco leaf, and generate a second feature map; The fusion module is used to fuse the first feature map and the second feature map to generate a fused feature map; The grading module is used to output the grading result of the target tobacco leaf based on the fused feature map.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the tobacco leaf grading method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the tobacco leaf grading method according to any one of claims 1 to 7 when it is run.

Images