Tea leaf grading method, device and equipment across modalities fusion, and medium

Abstract

The application relates to a tea leaf grading method and device based on cross-modal fusion, equipment and a medium. The method comprises the following steps: acquiring a tea visual detection image and a tea hyperspectral image; inputting the tea visual detection image into an image segmentation model to generate independent leaf image segmentation result information; extracting an independent leaf visual detection image set and an independent leaf hyperspectral image set from the tea visual detection image and the tea hyperspectral image based on the independent leaf image segmentation result information; performing spectral analysis on each independent leaf hyperspectral image in the independent leaf hyperspectral image set to obtain an independent leaf hyperspectral analysis result image; inputting the independent leaf hyperspectral analysis result image and the independent leaf visual detection image corresponding to the same independent leaf into an image analysis model to obtain tea leaf independent leaf grading result information of the independent leaf. The method can reduce the redundancy of hyperspectral data, realize multi-dimensional feature collaborative modeling and improve the grading accuracy.

Description

Technical Field

[0001] This application belongs to the field of tea grading, and in particular relates to a cross-modal fusion method, apparatus, equipment and medium for grading fresh tea leaves. Background Technology

[0002] In tea production, the grading of fresh tea leaves fundamentally determines product quality. Currently, this is mainly achieved through manual sensory evaluation, but the results are easily affected by human subjectivity and environmental factors. With the rapid development of computer technology, image classification technology based on computer vision is being applied in various fields. Currently, some studies both domestically and internationally have achieved good results in classifying plant leaves using computer vision.

[0003] Research and application of computer vision technology for tea classification are limited. Classification primarily relies on algorithms such as color histograms and edge detection to extract shape and color features of buds and leaves. However, depending solely on visual features, it's difficult to comprehensively reflect the intrinsic quality of tea, resulting in a weak correlation between grading results and actual quality. Furthermore, existing tea classification techniques often use only single-modal data, making them prone to overfitting on small datasets and severely impacting classification performance. Hyperspectral images, on the other hand, require excessive data volume, demanding high storage and computing power, making them difficult to deploy in production environments. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, equipment, and medium for grading fresh tea leaves that can reduce the redundancy of hyperspectral data, achieve multi-dimensional feature collaborative modeling, and improve grading accuracy and intelligence through cross-modal fusion, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a cross-modal fusion method for grading fresh tea leaves, including:

[0006] Acquire visual inspection images and hyperspectral images of fresh tea leaves;

[0007] The visual detection image of fresh tea leaves is input into the independent bud and leaf image segmentation model to generate independent bud and leaf image segmentation result information;

[0008] Based on the segmentation results of independent bud and leaf images, a set of visual detection images of independent bud and leaf and a set of hyperspectral images of independent bud and leaf were extracted from visual detection images of fresh tea leaves and hyperspectral images of fresh tea leaves, respectively.

[0009] Spectral analysis was performed on each independent bud and leaf hyperspectral image in the independent bud and leaf hyperspectral image set to obtain the hyperspectral analysis result image of the independent bud and leaf;

[0010] The hyperspectral analysis results image of the independent bud and leaf corresponding to the same independent bud and leaf and the visual detection image of the independent bud and leaf in the independent bud and leaf visual detection image set are input into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0011] In one embodiment, spectral analysis is performed on each individual bud hyperspectral image in the individual bud hyperspectral image set to obtain an image of the individual bud hyperspectral analysis result, including:

[0012] Obtain target frequency band setting information, which is used to characterize the target analysis frequency band for spectral analysis;

[0013] Based on the target frequency band setting information, spectral analysis is performed on the frequency band information corresponding to the target frequency band setting information in the spectral data of each pixel in the hyperspectral image of independent buds and leaves to obtain the spectral analysis result parameters of each pixel in the hyperspectral image of independent buds and leaves.

[0014] Based on the spectral analysis parameters of each pixel in the hyperspectral image of an independent bud and leaf, a hyperspectral analysis result image of an independent bud and leaf is constructed.

[0015] In one embodiment, the target frequency band setting information includes water content target frequency band setting information and chlorophyll content target frequency band setting information; the spectral analysis result parameters include water content spectral analysis result parameters and chlorophyll content spectral analysis result parameters; the independent bud and leaf hyperspectral analysis result image includes an independent bud and leaf hyperspectral water content analysis result channel and an independent bud and leaf hyperspectral chlorophyll content analysis result channel; and based on the spectral analysis result parameters of each pixel in the independent bud and leaf hyperspectral image, an independent bud and leaf hyperspectral analysis result image is constructed, including:

[0016] Based on the water content spectral analysis parameters of each pixel in the hyperspectral image of independent buds and leaves, a channel for the hyperspectral water content analysis results of independent buds and leaves in the hyperspectral analysis result image of independent buds and leaves is constructed.

[0017] Based on the spectral analysis results of chlorophyll content at each pixel in the hyperspectral image of independent buds and leaves, a channel for analyzing the hyperspectral chlorophyll content of independent buds and leaves in the hyperspectral analysis result image of independent buds and leaves is constructed.

[0018] An image of hyperspectral analysis results of independent buds and leaves was constructed by combining the results of hyperspectral water content analysis of independent buds and leaves and the results of hyperspectral chlorophyll content analysis of independent buds and leaves.

[0019] Among them, the parameters of the water content spectral analysis result are obtained based on the target frequency band setting information of water content, and the parameters of the chlorophyll content spectral analysis result are obtained based on the target frequency band setting information of chlorophyll content.

[0020] In one embodiment, based on the target frequency band setting information, spectral analysis is performed on the frequency band information corresponding to the target frequency band setting information in the spectral data of each pixel in the hyperspectral image of an independent bud and leaf to obtain the spectral analysis result parameters of each pixel in the hyperspectral image of an independent bud and leaf, including:

[0021] Based on the target frequency band setting information of water content, the spectral absorption intensity parameter of the frequency band information corresponding to the target frequency band setting information of water content in the spectral data of each pixel in the hyperspectral image of independent buds and leaves is calculated, and the spectral absorption intensity parameter corresponding to the target frequency band setting information of water content is set as the water content spectral analysis result parameter of each pixel in the hyperspectral image of independent buds and leaves.

[0022] Based on the chlorophyll content target frequency band setting information, the spectral absorption intensity parameters of the frequency band information corresponding to the chlorophyll content target frequency band setting information in the spectral data of each pixel in the hyperspectral image of independent buds and leaves are calculated, and the spectral absorption intensity parameters corresponding to the chlorophyll content target frequency band setting information are set as the chlorophyll content spectral analysis result parameters of each pixel in the hyperspectral image of independent buds and leaves.

[0023] The expression for the spectral absorption intensity parameter is as follows:

[0024] ;

[0025] In the formula, For the first The spectral absorption intensity parameters corresponding to the project's standard frequency band settings. For the first The total number of wavelengths in the frequency band information corresponding to the project's target frequency band setting information. For the first The reflectivity of the reference frequency band for the project's frequency band setting information. For the first The frequency band information corresponding to the project's target frequency band setting information is the first one. wavelength, For the first The frequency band information corresponding to the project's target frequency band setting information is the first one. Reflectivity of wavelength.

[0026] In one embodiment, the visual detection image of an individual bud includes a red channel, a blue channel, and a green channel. The input channels of the tea leaf fresh bud and leaf grading image analysis model are at least five. The hyperspectral analysis result image of the individual bud and leaf corresponding to the same individual bud and leaf, and the visual detection images of individual buds and leaves in the set of visual detection images of individual buds and leaves are input into the tea leaf fresh bud and leaf grading image analysis model to obtain the grading result information of the individual buds and leaves of the tea leaf fresh buds and leaves, including:

[0027] The hyperspectral analysis results of the independent buds and leaves corresponding to the same independent bud and leaf, including the hyperspectral analysis results of the independent bud and leaf, the hyperspectral analysis results of the independent bud and leaf, the hyperspectral analysis results of the independent bud and leaf, the red channel of the visual detection image of the independent bud and leaf, the blue channel of the visual detection image of the independent bud and leaf, and the green channel of the visual detection image of the independent bud and leaf, are input into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0028] Among them, the image analysis model for the independent bud and leaf grading of fresh tea leaves is based on the improved YOLO model. The improved YOLO model is obtained by replacing the convolutional module in the cross-stage partial feature fusion module of the original YOLOV8 model with a depthwise separable convolutional module, and adding a channel attention module between the neck network and the head network.

[0029] In one embodiment, the output of the image analysis model for grading independent buds and leaves of fresh tea leaves includes information on the leaf-bud ratio and maturity of independent buds and leaves. The grading result information for independent buds and leaves of fresh tea leaves includes information on the leaf-bud ratio, maturity, and independent bud and leaf grading score. The cross-modal fusion method for grading fresh tea leaves also includes:

[0030] Based on the leaf-bud ratio and maturity information of independent buds, the grading score information of independent buds is calculated.

[0031] Based on the grading and scoring information of individual buds and leaves, a comprehensive grading result information of the tea fresh leaf batch to which the individual buds and leaves belong is generated.

[0032] The expressions for the independent bud and leaf grading score information and the comprehensive grading result information are as follows:

[0033] ;

[0034] In the formula, For the first The grading and scoring information of independent buds and leaves. For the first The number of leaves with independent buds and leaves, For the first The number of buds with independent leaf buds, For the first The score corresponding to the maturity information of individual buds and leaves. For the first Information on the comprehensive grading results of fresh tea leaves batches belonging to individual buds and leaves. For the first The average value of the grading score information of the independent buds and leaves in the batch of fresh tea leaves to which the independent buds and leaves belong.

[0035] In one embodiment, the independent bud image segmentation model is trained based on an improved U-shaped neural network model. The improved U-shaped neural network model has three downsampling levels, and the convolutional modules of the improved U-shaped neural network model are residual convolutional modules. A spatial attention module is added between the two residual convolutional modules with the largest downsampling factor in the improved U-shaped neural network model.

[0036] Secondly, this application also provides a cross-modal fusion grading device for fresh tea leaves, comprising:

[0037] The cross-modal data acquisition module is used to acquire visual inspection images and hyperspectral images of fresh tea leaves;

[0038] The bud and leaf image segmentation module is used to input visual inspection images of fresh tea leaves into the independent bud and leaf image segmentation model to generate independent bud and leaf image segmentation result information.

[0039] The independent bud and leaf image generation module is used to extract the independent bud and leaf visual detection image set and the independent bud and leaf hyperspectral image set from the tea fresh leaf visual detection image and the tea fresh leaf hyperspectral image respectively based on the independent bud and leaf image segmentation result information.

[0040] The hyperspectral analysis module is used to perform spectral analysis on each independent bud and leaf hyperspectral image in the independent bud and leaf hyperspectral image set to obtain the hyperspectral analysis result image of the independent bud and leaf;

[0041] The independent bud and leaf grading module is used to input the hyperspectral analysis result image of the independent bud and leaf corresponding to the same independent bud and leaf and the visual detection image of the independent bud and leaf in the independent bud and leaf visual detection image set into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0042] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method as described in any of the first aspects of this application.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the first aspects of this application.

[0044] The aforementioned cross-modal fusion method, apparatus, equipment, and medium for grading fresh tea leaves, by constructing a visual-hyperspectral cross-modal fusion framework, can fuse the morphological and color features of visual images with the chemical composition features of hyperspectral images, overcoming the limitations of single-modality fusion, enhancing feature discrimination ability, and thus significantly improving the comprehensiveness and accuracy of grading results. By combining the appearance features and chemical composition features of buds and leaves, the tenderness and maturity of fresh tea leaves can be analyzed, making the grading results of independent buds and leaves of fresh tea leaves more reflective of the actual quality of the buds and leaves. By adopting an independent bud and leaf segmentation model to generate bud and leaf mask information, the influence of environmental interference can be reduced, improving the targeting of subsequent image analysis models for independent bud and leaf grading of fresh tea leaves, reducing the parameters required for the image analysis models for independent bud and leaf grading of fresh tea leaves, and improving the performance of the image analysis models for independent bud and leaf grading of fresh tea leaves. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A schematic diagram illustrating the application environment of a cross-modal fusion method for grading fresh tea leaves, provided in one embodiment of this application;

[0047] Figure 2 A flowchart illustrating a cross-modal fusion method for grading fresh tea leaves, provided as an embodiment of this application;

[0048] Figure 3 A flowchart illustrating a hyperspectral analysis method provided in one embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the structure of an image analysis model for grading fresh tea leaves and buds, provided in one embodiment of this application.

[0050] Figure 5 A flowchart illustrating another cross-modal fusion method for grading fresh tea leaves, provided as an embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the structure of an independent bud and leaf image segmentation model provided in one embodiment of this application;

[0052] Figure 7 This is a schematic diagram of a cross-modal fusion grading device for fresh tea leaves provided in one embodiment of this application.

[0053] In the diagram: 101-Computing platform; 102-Data acquisition terminal; 103-Database; 700-Cross-modal fusion tea leaf grading device; 701-Cross-modal data acquisition module; 702-Bud and leaf image segmentation module; 703-Independent bud and leaf image generation module; 704-Hyperspectral analysis module; 705-Independent bud and leaf grading module. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The cross-modal fusion method for grading fresh tea leaves provided in this application can be applied to, for example... Figure 1 In the application environment shown, the computing platform 101 can communicate with the data acquisition terminal 102 via a communication channel. The database 103 can store the data that the computing platform 101 needs to process. The database 103 can be integrated into the computing platform 101 or placed in the cloud or other network storage devices. The data acquisition terminal 102 can acquire raw tea leaf image data for analyzing the grading of fresh tea leaves and send the raw tea leaf image data to the computing platform 101. The computing platform 101 can generate tea leaf grading results based on the raw tea leaf image data acquired by the data acquisition terminal 102. The computing platform 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, industrial computers, industrial controllers, IoT devices, and servers. The server can be implemented using a standalone server or a server cluster composed of multiple servers.

[0056] In one exemplary embodiment, such as Figure 2 As shown, a cross-modal fusion method for grading fresh tea leaves is provided, which is then applied to... Figure 1 Taking the computing platform 101 as an example, the explanation includes the following steps S201 to S205. Wherein:

[0057] Step S201: Obtain visual detection images and hyperspectral images of fresh tea leaves.

[0058] Specifically, the computing platform 101 can acquire visual inspection images and hyperspectral images of fresh tea leaves collected by the data acquisition terminal through the communication channel.

[0059] Optionally, the data acquisition terminal for collecting visual inspection images of fresh tea leaves can be an industrial camera, and the data acquisition terminal for collecting hyperspectral images of fresh tea leaves can be an imaging hyperspectral camera.

[0060] Optionally, the spectral wavelength range of the imaging hyperspectral camera may include, but is not limited to, the visible-near infrared wavelength range (400 nm to 1000 nm) and the short-wave infrared wavelength range (1000 nm to 2000 nm). The visible-near infrared wavelength range can be used to analyze chlorophyll and water content, while the short-wave infrared wavelength range can supplement the analysis of water content.

[0061] Furthermore, the wavelength interval of the imaging hyperspectral camera in the visible-near infrared wavelength range can be, but is not limited to, 2 nanometers, 2.5 nanometers, or 5 nanometers, and the wavelength interval of the imaging hyperspectral camera in the short-wave infrared wavelength range can be, but is not limited to, 5 nanometers or 10 nanometers.

[0062] Optionally, the computing platform 101 can perform image registration on the acquired visual inspection image of fresh tea leaves and the hyperspectral image of fresh tea leaves.

[0063] Step S202: Input the visual detection image of fresh tea leaves into the independent bud and leaf image segmentation model to generate independent bud and leaf image segmentation result information.

[0064] Specifically, the computing platform 101 can input the visual inspection image of fresh tea leaves acquired by the industrial camera into the independent bud and leaf image segmentation model mounted on the computing platform 101 to generate independent bud and leaf image segmentation result information.

[0065] Optionally, the independent bud and leaf image segmentation model can classify each pixel of the input visual detection image of fresh tea leaves. The classification result may include, but is not limited to, buds, leaves, and background. The independent bud and leaf image segmentation model can output a binary mask matrix of the same size as the input visual detection image of fresh tea leaves.

[0066] For example, in a binary mask matrix, 0 can be used to represent the background and 1 can be used to represent the bud.

[0067] Optionally, the independent bud and leaf image segmentation model can be constructed based on, but is not limited to, a U-shaped neural network model, a DeepLab model, a SLIC superpixel segmentation algorithm, a YOLO model, a deep experimental model (DeepLab), or a region convolutional neural network model (R-CNN).

[0068] Step S203: Based on the segmentation results of the independent bud and leaf images, extract the visual detection image set of independent buds and leaves and the hyperspectral image set of independent buds and leaves from the visual detection image of fresh tea leaves and the hyperspectral image of fresh tea leaves, respectively.

[0069] Specifically, the computing platform 101 can segment and extract the region of each individual bud and leaf from the visual inspection image of fresh tea leaves based on the segmentation results generated by the individual bud and leaf image segmentation model mounted on the computing platform 101, obtaining the corresponding visual inspection image of each individual bud and leaf, and constructing an individual bud and leaf visual inspection image set based on the corresponding visual inspection image of each individual bud and leaf. The computing platform 101 can also segment and extract the region of each individual bud and leaf from the hyperspectral image of fresh tea leaves based on the segmentation results, obtaining the corresponding hyperspectral image of each individual bud and leaf, and constructing an individual bud and leaf hyperspectral image set based on the corresponding hyperspectral image of each individual bud and leaf.

[0070] For example, when the binary mask matrix in the segmentation result information of the independent bud and leaf image uses 0 to represent the background and 1 to represent the bud and leaf, the computing platform 101 can multiply the segmentation mask pixel-by-pixel with the visual detection image of fresh tea leaves and the hyperspectral image of fresh tea leaves to preserve the bud and leaf region.

[0071] Indicatively, when the computing platform 101 multiplies the segmentation mask with a value of 0 with the pixel of the hyperspectral image of the fresh leaf, it can assign the spectral value of the pixel of the hyperspectral image of the fresh leaf to 0, which means that the pixel of the hyperspectral image of the fresh leaf is not selected as an independent bud or leaf region.

[0072] Optionally, the computing platform 101 can use Gaussian filtering or median filtering to optimize the bud-leaf boundary.

[0073] Step S204: Perform spectral analysis on each independent bud and leaf hyperspectral image in the independent bud and leaf hyperspectral image set to obtain the independent bud and leaf hyperspectral analysis result image.

[0074] Specifically, the computing platform 101 can perform spectral analysis on each independent bud hyperspectral image in the independent bud hyperspectral image set, calculate the absorption rate of each independent bud hyperspectral image in a specific spectrum, and reduce the dimensionality of the pixel data of each independent bud hyperspectral image in the independent bud hyperspectral image set to obtain the image of the independent bud hyperspectral analysis result.

[0075] Optionally, the dimension of the pixel data of each independent bud hyperspectral image in the independent bud hyperspectral image set can be reduced to two dimensions, including the feature values ​​of the independent bud hyperspectral water content analysis results and the feature values ​​of the independent bud hyperspectral chlorophyll content analysis results.

[0076] Step S205: Input the hyperspectral analysis result image of the independent bud and leaf corresponding to the same independent bud and leaf and the visual detection image of the independent bud and leaf in the independent bud and leaf visual detection image set into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0077] Specifically, the computing platform 101 can input the hyperspectral analysis result image of the independent bud and leaf in the set of registered independent bud and leaf hyperspectral analysis result images and the visual detection image of the independent bud and leaf in the set of independent bud and leaf visual detection images into the tea fresh leaf independent bud and leaf grading image analysis model mounted on the computing platform 101 to obtain the tea fresh leaf independent bud and leaf grading result information.

[0078] Optionally, the number of input channels of the tea leaf independent bud and leaf grading image analysis model mounted on the computing platform 101 can be five, corresponding to the blue channel, red channel and green channel of the visual detection image of independent buds and leaves, as well as the independent buds and leaves hyperspectral water content analysis result channel and independent buds and leaves hyperspectral chlorophyll content analysis result channel of the hyperspectral analysis result image of independent buds and leaves.

[0079] Optionally, the image analysis model for the independent bud and leaf grading of fresh tea leaves mounted on the computing platform 101 can be constructed, but is not limited to, based on a deep convolutional neural network, a region-based convolutional neural network (R-CNN), a residual network (ResNet), a generative adversarial network (GAN), or a densely connected convolutional network (DenseNet).

[0080] The aforementioned cross-modal fusion method for grading fresh tea leaves integrates two major data modalities: visual inspection images and hyperspectral images. This enables comprehensive capture of multi-dimensional features of fresh tea leaves. Visual inspection images accurately depict the appearance of fresh tea leaves, providing intuitive external features for grading. Hyperspectral images, on the other hand, reveal the differences in the internal chemical composition of fresh tea leaves, thereby improving grading accuracy and reliability. By introducing an independent bud and leaf image segmentation model to process the visual inspection images of fresh tea leaves, individual buds and leaves in the entire image can be accurately separated, generating independent bud and leaf image segmentation results. This improves the accuracy of bud and leaf feature extraction and reduces feature confusion caused by background. This approach aims to improve the overall performance of fresh tea leaf grading. By performing spectral analysis on individual bud and leaf hyperspectral images, the internal quality information of fresh tea leaves contained in the hyperspectral images can be fully extracted, resulting in hyperspectral analysis images of individual buds and leaves. This compensates for the quality assessment bias of appearance grading, thus providing a more comprehensive, in-depth, and accurate basis for fresh tea leaf grading, making the grading results more accurately reflect the actual value of tea. Through the image analysis model of independent bud and leaf grading of fresh tea leaves, the model can automatically learn and master the complex nonlinear relationship between the appearance characteristics and internal quality characteristics of fresh tea leaves. This can provide standardized and automated grading schemes, promoting the tea processing industry towards intelligence and modernization, and improving the production efficiency and product quality stability of the entire tea industry.

[0081] In an optional embodiment of this application, please refer to Figure 2 and Figure 3 Step S204 involves performing spectral analysis on the hyperspectral images of each independent bud and leaf in the set of independent bud and leaf hyperspectral images to obtain the hyperspectral analysis result images of the independent bud and leaf, which may include:

[0082] Step S301: Obtain target frequency band setting information.

[0083] Optionally, the target frequency band setting information can be used to characterize the target analysis frequency band for spectral analysis.

[0084] Step S302: Based on the target frequency band setting information, perform spectral analysis on the frequency band information corresponding to the target frequency band setting information in the spectral data of each pixel in the hyperspectral image of independent buds and leaves, and obtain the spectral analysis result parameters of each pixel in the hyperspectral image of independent buds and leaves.

[0085] Based on the spectral analysis parameters of each pixel in the hyperspectral image of an independent bud and leaf, a hyperspectral analysis result image of an independent bud and leaf is constructed.

[0086] The aforementioned cross-modal fusion method for grading fresh tea leaves improves the accuracy of internal quality assessment by setting target frequency bands and performing refined spectral analysis on the hyperspectral images of individual buds and leaves. It eliminates a large amount of interference information from irrelevant frequency bands, simplifying the processing of massive spectral data. This allows the analysis process to focus on core, effective information, significantly improving computational efficiency and shortening the spectral analysis cycle for a single bud and leaf sample. This effectively solves the problems of lengthy hyperspectral data analysis processes and susceptibility to invalid data interference, providing a more accurate and efficient spectral analysis foundation for the quality assessment of fresh tea leaves. Furthermore, through in-depth mining of the spectral data of each pixel within the target frequency band, it can fully release… This method enhances the richness and discriminative power of tea quality characteristics hidden in hyperspectral images, providing solid data support for accurately distinguishing the intrinsic quality differences of fresh tea leaves of different grades. By constructing independent hyperspectral analysis images of buds and leaves based on the spectral analysis parameters of each pixel, it is possible to transform abstract data into intuitive visual information, optimizing the quality and usability of model input data. By using image reconstruction algorithms, it is possible to accurately map scattered pixel-level spectral analysis results to two-dimensional image space, thereby meeting the model's needs for complex data pattern recognition, effectively improving the model's efficiency in parsing input data, and enhancing model adaptability and training efficiency.

[0087] In an optional embodiment of this application, the target frequency band setting information may include water content target frequency band setting information and chlorophyll content target frequency band setting information; the spectral analysis result parameters may include water content spectral analysis result parameters and chlorophyll content spectral analysis result parameters; and the hyperspectral analysis result images of individual buds and leaves may include an individual bud and leaf hyperspectral water content analysis result channel and an individual bud and leaf hyperspectral chlorophyll content analysis result channel. Please refer to... Figure 3 Based on the spectral analysis parameters of each pixel in the hyperspectral image of an independent bud and leaf, a hyperspectral analysis result image of the independent bud and leaf can be constructed, which may include:

[0088] Step S303: Based on the water content spectral analysis result parameters of each pixel in the hyperspectral image of independent buds and leaves, construct the hyperspectral water content analysis result channel of independent buds and leaves in the hyperspectral analysis result image of independent buds and leaves.

[0089] Step S304: Based on the chlorophyll content spectral analysis results parameters of each pixel in the hyperspectral image of independent buds and leaves, construct the hyperspectral chlorophyll content analysis result channel of independent buds and leaves in the hyperspectral analysis result image of independent buds and leaves.

[0090] Step S305: Construct an image of the hyperspectral analysis results of independent buds and leaves by combining the results of the hyperspectral water content analysis of independent buds and leaves and the results of the hyperspectral chlorophyll content analysis of independent buds and leaves.

[0091] Among them, the parameters of the water content spectral analysis result are obtained based on the target frequency band setting information of water content, and the parameters of the chlorophyll content spectral analysis result are obtained based on the target frequency band setting information of chlorophyll content.

[0092] Optionally, the target frequency band setting information for water content may include, but is not limited to, 950 nm to 1000 nm in the visible-near infrared wavelength range (400 nm to 1000 nm) and 1350 nm to 1450 nm and 1900 nm to 2000 nm in the short-wave infrared wavelength range (1000 nm to 2000 nm).

[0093] Optionally, the chlorophyll content target frequency band setting information in the visible-near infrared wavelength range (400 nm to 1000 nm) may include 650 nm to 750 nm.

[0094] In the aforementioned cross-modal fusion method for grading fresh tea leaves, by setting target frequency bands for moisture content and chlorophyll content respectively, targeted spectral analysis is performed on the hyperspectral images of individual buds and leaves to obtain spectral analysis parameters for moisture content and chlorophyll content. This enables simultaneous evaluation of two key quality indicators of fresh tea leaves. Moisture content accurately reflects the freshness and processing suitability of tea leaves, while chlorophyll content effectively indicates the maturity and potential flavor of tea leaves. By extracting and analyzing the spectral data from the target frequency bands for moisture content and chlorophyll content respectively, a more comprehensive and in-depth quality assessment of fresh tea leaves can be achieved. By constructing hyperspectral analysis result channels for moisture content and chlorophyll content of individual buds and leaves, the expression of spectral features can be effectively optimized, achieving accurate quantification of moisture content and chlorophyll content in fresh tea leaves and improving the model adaptability of hyperspectral images.

[0095] In an optional embodiment of this application, please refer to Figure 3 Step S302: Based on the target frequency band setting information, perform spectral analysis on the frequency band information corresponding to the target frequency band setting information in the spectral data of each pixel in the hyperspectral image of the independent bud and leaf, and obtain the spectral analysis result parameters of each pixel in the hyperspectral image of the independent bud and leaf, which may include:

[0096] Based on the target frequency band setting information for water content, the spectral absorption intensity parameters of the frequency band information corresponding to the target frequency band setting information for water content in the spectral data of each pixel in the hyperspectral image of independent buds and leaves are calculated, and the spectral absorption intensity parameters corresponding to the target frequency band setting information for water content are set as the parameters of the water content spectral analysis results of each pixel in the hyperspectral image of independent buds and leaves.

[0097] Based on the target frequency band setting information of chlorophyll content, the spectral absorption intensity parameters of the frequency band information corresponding to the target frequency band setting information of chlorophyll content in the spectral data of each pixel in the hyperspectral image of independent buds and leaves are calculated, and the spectral absorption intensity parameters corresponding to the target frequency band setting information of chlorophyll content are set as the spectral analysis result parameters of chlorophyll content in each pixel in the hyperspectral image of independent buds and leaves.

[0098] The expression for the spectral absorption intensity parameter can be:

[0099] ;

[0100] In the formula, For the first The spectral absorption intensity parameters corresponding to the project's standard frequency band settings. For the first The total number of wavelengths in the frequency band information corresponding to the project's target frequency band setting information. For the first The reflectivity of the reference frequency band for the project's frequency band setting information. For the first The frequency band information corresponding to the project's target frequency band setting information is the first one. wavelength, For the first The frequency band information corresponding to the project's target frequency band setting information is the first one. Reflectivity of wavelength.

[0101] Indicatively, target frequency band setting information may include target frequency band setting information for water content and target frequency band setting information for chlorophyll content.

[0102] In an optional embodiment of this application, the visual detection image of fresh tea leaves may include a red channel, a blue channel, and a green channel. Correspondingly, the visual detection image of independent buds and leaves segmented from the visual detection image of fresh tea leaves based on the segmentation result information of independent buds and leaves may also include a red channel, a blue channel, and a green channel. The input channels of the independent bud and leaf grading image analysis model for fresh tea leaves may have at least five channels. Please refer to [reference needed]. Figure 2 Step S205 involves inputting the hyperspectral analysis result image of the independent bud and leaf corresponding to the same independent bud and leaf and the visual detection image of the independent bud and leaf in the independent bud and leaf visual detection image set into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information, which may include:

[0103] The hyperspectral analysis results of the independent buds and leaves corresponding to the same independent bud and leaf, including the hyperspectral water content analysis result channel, the hyperspectral analysis result of the independent bud and leaf chlorophyll content analysis result channel, the red channel, the blue channel, and the green channel of the visual inspection image of the independent bud and leaf, are input into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0104] Please refer to the following: Figure 4 The image analysis model for the independent bud and leaf grading of fresh tea leaves can be obtained by training an improved YOLO model. The improved YOLO model is obtained by replacing the convolutional module in the cross-stage partial feature fusion module (C2F module) of the original YOLOV8 model with a depthwise separable convolutional module, and adding a channel attention module between the neck network and the head network.

[0105] Optionally, depthwise separable convolution is not only the foundation for lightweight models, but also allows for the efficient extraction of features from each channel of the input to an image analysis model for grading individual buds and leaves of fresh tea leaves. Depthwise separable convolution can include depthwise convolution and pointwise convolution. Depthwise convolution can extract spatial features channel by channel, while pointwise convolution can achieve cross-channel feature interaction.

[0106] As an illustration, the Channel Attention Module in the tea leaf grading model can accurately extract key features and improve grading accuracy. The image analysis model for independent bud and leaf grading of tea leaves can assign high weights to key channel features through the Channel Attention Module, suppressing noise and redundant channel data, and allowing the model to focus on the core features that truly affect grading.

[0107] In the aforementioned cross-modal fusion method for grading fresh tea leaves, by inputting the red, blue, and green channels of the visual detection image of fresh tea leaves, along with the moisture content analysis channel and chlorophyll content analysis channel of the hyperspectral analysis image, into the independent bud and leaf grading image analysis model for fresh tea leaves, deep fusion of multimodal data can be achieved. This allows for comprehensive analysis of fresh tea leaf grading using external features such as color and texture, as well as internal quality indicators such as moisture content and chlorophyll content, thereby significantly improving the accuracy of fresh tea leaf grading and providing a more comprehensive and accurate basis for the quality assessment of fresh tea leaves. Furthermore, by replacing the convolutional module in the cross-stage partial feature fusion module of the original YOLOV8 model with a depthwise separable convolutional module and adding a channel attention module between the neck network and the head network, the detection performance of the model can be significantly improved.

[0108] Specifically, in the aforementioned cross-modal fusion method for grading fresh tea leaves, the introduction of a depthwise separable convolutional module reduces the computational cost and number of parameters, improving the model's operational efficiency. Simultaneously, the channel attention module enables the model to automatically learn and focus on key feature channels relevant to fresh tea leaf grading, enhancing its ability to capture important features. This allows the model to maintain grading accuracy while achieving higher detection speed and better real-time performance, meeting the real-time grading requirements of tea production lines.

[0109] In an optional embodiment of this application, please refer to Figure 5 The output of the image analysis model for grading individual buds and leaves of fresh tea leaves includes information on the leaf-bud ratio and maturity of individual buds and leaves. The grading results for individual buds and leaves of fresh tea leaves include information on the leaf-bud ratio, maturity, and grading score of individual buds and leaves. The cross-modal fusion method for grading fresh tea leaves also includes:

[0110] Step S506: Based on the leaf-bud ratio information and maturity information of independent buds, calculate the independent bud grading score information.

[0111] Step S507: Generate comprehensive grading results for the tea leaf batch to which the independent buds and leaves belong based on the independent bud and leaf grading score information.

[0112] The expressions for the independent bud and leaf grading score information and the comprehensive grading result information are as follows:

[0113] ;

[0114] In the formula, For the first The grading and scoring information of independent buds and leaves. For the first The number of leaves with independent buds and leaves, For the first The number of buds with independent leaf buds, For the first The score corresponding to the maturity information of individual buds and leaves. For the first Information on the comprehensive grading results of fresh tea leaves batches belonging to individual buds and leaves. For the first The average value of the grading score information of the independent buds and leaves in the batch of fresh tea leaves to which the independent buds and leaves belong.

[0115] Optionally, the maturity information of buds and leaves may include, but is not limited to, initial unfolding and full opening, with a score of 0.2 for initial unfolding and a score of 0.5 for full opening.

[0116] In an optional embodiment of this application, please refer to Figure 6 The independent bud and leaf image segmentation model can be obtained by training an improved U-shaped neural network model. The downsampling level of the improved U-shaped neural network model can be three levels. The convolution module of the improved U-shaped neural network model can be a residual convolution module. A spatial attention module can be added between the two residual convolution modules with the largest downsampling factor of the improved U-shaped neural network model.

[0117] In one exemplary embodiment of this application, such as Figure 5 As shown, a cross-modal fusion method for grading fresh tea leaves is provided, including:

[0118] Step S501: Obtain visual detection images and hyperspectral images of fresh tea leaves.

[0119] Step S502: Input the visual detection image of fresh tea leaves into the independent bud and leaf image segmentation model to generate independent bud and leaf image segmentation result information.

[0120] Step S503: Based on the segmentation results of independent bud and leaf images, extract the visual detection image set of independent bud and leaf and the hyperspectral image set of independent bud and leaf from the visual detection image of fresh tea leaves and the hyperspectral image of fresh tea leaves, respectively.

[0121] Step S504: Perform spectral analysis on each independent bud and leaf hyperspectral image in the independent bud and leaf hyperspectral image set to obtain the independent bud and leaf hyperspectral analysis result image.

[0122] Step S505: Input the hyperspectral analysis result image of the independent bud and leaf corresponding to the same independent bud and leaf and the visual detection image of the independent bud and leaf in the independent bud and leaf visual detection image set into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0123] Step S506: Based on the leaf-bud ratio information and maturity information of independent buds, calculate the independent bud grading score information.

[0124] Step S507: Generate comprehensive grading results for the tea leaf batch to which the independent buds and leaves belong based on the independent bud and leaf grading score information.

[0125] The aforementioned cross-modal fusion method for grading fresh tea leaves integrates visual inspection images and hyperspectral images of fresh tea leaves with their appearance and internal spectral characteristics. This allows for a comprehensive capture of the internal and external quality characteristics of fresh tea leaves, enabling precise grading and ensuring that each grade of tea has clear and consistent quality characteristics, thus meeting the market's demand for refined grading of high-quality tea. Furthermore, by performing spectral analysis on the hyperspectral images of individual buds and leaves in a dataset, the method obtains hyperspectral analysis results images of individual buds and leaves. This significantly enhances the ability to assess the internal quality of fresh tea leaves, enabling multi-dimensional quality analysis and a more comprehensive and in-depth quality assessment, ensuring that the grading results accurately reflect the actual quality of the tea.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0127] Based on the same inventive concept, this application also provides a cross-modal fusion tea leaf grading device for implementing the cross-modal fusion tea leaf grading method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more cross-modal fusion tea leaf grading device embodiments provided below can be found in the limitations of the cross-modal fusion tea leaf grading method described above, and will not be repeated here.

[0128] In one exemplary embodiment, such as Figure 7 As shown, a cross-modal fusion tea leaf grading device 700 is provided, comprising:

[0129] The cross-modal data acquisition module 701 can be used to acquire visual inspection images and hyperspectral images of fresh tea leaves.

[0130] The bud and leaf image segmentation module 702 can be used to input visual detection images of fresh tea leaves into the independent bud and leaf image segmentation model to generate independent bud and leaf image segmentation result information.

[0131] The independent bud and leaf image generation module 703 can be used to extract the independent bud and leaf visual detection image set and the independent bud and leaf hyperspectral image set from the visual detection image and the hyperspectral image of the fresh tea leaves, respectively, based on the independent bud and leaf image segmentation result information.

[0132] The hyperspectral analysis module 704 can be used to perform spectral analysis on each independent bud and leaf hyperspectral image in the independent bud and leaf hyperspectral image set to obtain the hyperspectral analysis result image of the independent bud and leaf.

[0133] The independent bud and leaf grading module 705 can be used to input the hyperspectral analysis result image of the independent bud and leaf corresponding to the same independent bud and leaf and the visual detection image of the independent bud and leaf in the independent bud and leaf visual detection image set into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0134] In an optional embodiment of this application, the hyperspectral analysis module 704 can also be used for:

[0135] Obtain target frequency band setting information, which is used to characterize the target analysis frequency band for spectral analysis.

[0136] Based on the target frequency band setting information, spectral analysis is performed on the frequency band information corresponding to the target frequency band setting information in the spectral data of each pixel in the hyperspectral image of independent buds and leaves to obtain the spectral analysis result parameters of each pixel in the hyperspectral image of independent buds and leaves.

[0137] Based on the spectral analysis parameters of each pixel in the hyperspectral image of an independent bud and leaf, a hyperspectral analysis result image of an independent bud and leaf is constructed.

[0138] In an optional embodiment of this application, the hyperspectral analysis module 704 can also be used for:

[0139] Based on the water content spectral analysis parameters of each pixel in the hyperspectral image of independent buds and leaves, a channel for the hyperspectral water content analysis of independent buds and leaves in the hyperspectral analysis result image of independent buds and leaves is constructed.

[0140] Based on the chlorophyll content spectral analysis parameters of each pixel in the hyperspectral image of independent buds and leaves, a channel for analyzing the chlorophyll content of independent buds and leaves in the hyperspectral analysis result image of independent buds and leaves is constructed.

[0141] An image of the hyperspectral analysis results of independent buds and leaves was constructed by combining the results of the hyperspectral water content analysis of independent buds and leaves and the results of the hyperspectral chlorophyll content analysis of independent buds and leaves.

[0142] In an optional embodiment of this application, the hyperspectral analysis module 704 can also be used for:

[0143] Based on the target frequency band setting information for water content, the spectral absorption intensity parameters of the frequency band information corresponding to the target frequency band setting information for water content in the spectral data of each pixel in the hyperspectral image of independent buds and leaves are calculated, and the spectral absorption intensity parameters corresponding to the target frequency band setting information for water content are set as the parameters of the water content spectral analysis results of each pixel in the hyperspectral image of independent buds and leaves.

[0144] Based on the target frequency band setting information of chlorophyll content, the spectral absorption intensity parameters of the frequency band information corresponding to the target frequency band setting information of chlorophyll content in the spectral data of each pixel in the hyperspectral image of independent buds and leaves are calculated, and the spectral absorption intensity parameters corresponding to the target frequency band setting information of chlorophyll content are set as the spectral analysis result parameters of chlorophyll content in each pixel in the hyperspectral image of independent buds and leaves.

[0145] In an optional embodiment of this application, the independent bud and leaf grading module 705 can also be used for:

[0146] The hyperspectral analysis results of the independent buds and leaves corresponding to the same independent bud and leaf, including the hyperspectral water content analysis result channel, the hyperspectral analysis result of the independent bud and leaf chlorophyll content analysis result channel, the red channel, the blue channel, and the green channel of the visual inspection image of the independent bud and leaf, are input into the tea fresh leaf independent bud and leaf grading image analysis model to obtain the tea fresh leaf independent bud and leaf grading result information.

[0147] In an optional embodiment of this application, the cross-modal fusion tea leaf grading device 700 can also be used for:

[0148] Based on the leaf-bud ratio and maturity information of independent buds, the grading score information of independent buds is calculated.

[0149] Based on the grading and scoring information of individual buds and leaves, a comprehensive grading result is generated for the batch of fresh tea leaves to which the individual buds and leaves belong.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the cross-modal fusion method for grading fresh tea leaves as described above.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0152] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0153] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A cross-modal fusion tea fresh leaf grading method, characterized in that, The method comprises: obtaining a tea fresh leaf visual detection image and a tea fresh leaf hyperspectral image; inputting the tea fresh leaf visual detection image into an independent bud leaf image segmentation model to generate independent bud leaf image segmentation result information; based on the independent bud leaf image segmentation result information, extracting an independent bud leaf visual detection image set and an independent bud leaf hyperspectral image set from the tea fresh leaf visual detection image and the tea fresh leaf hyperspectral image respectively; performing spectral analysis on each independent bud leaf hyperspectral image in the independent bud leaf hyperspectral image set to obtain an independent bud leaf hyperspectral analysis result image; inputting the independent bud leaf hyperspectral analysis result image and the independent bud leaf visual detection image corresponding to the same independent bud leaf into a tea fresh leaf independent bud leaf grading image analysis model to obtain tea fresh leaf independent bud leaf grading result information of the independent bud leaf; wherein the tea fresh leaf independent bud leaf grading image analysis model is obtained by training an improved YOLO model, and the improved YOLO model is obtained by replacing the convolution module in the cross-stage partial feature fusion module of the original YOLOV8 model with a depth separable convolution module and adding a channel attention module between the neck network and the head network.

2. The method of claim 1, wherein, The method comprises: obtaining target frequency band setting information, wherein the target frequency band setting information is used to represent a target analysis frequency band of the spectral analysis; based on the target frequency band setting information, performing spectral analysis on the frequency band information corresponding to the target frequency band setting information in the spectral data of each pixel point in the independent bud leaf hyperspectral image to obtain a spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image; based on the spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image, constructing the independent bud leaf hyperspectral analysis result image.

3. The method of claim 2, wherein, The target frequency band setting information comprises water content target frequency band setting information and chlorophyll content target frequency band setting information, the spectral analysis result parameter comprises water content spectral analysis result parameter and chlorophyll content spectral analysis result parameter, the independent bud leaf hyperspectral analysis result image comprises an independent bud leaf hyperspectral water content analysis result channel and an independent bud leaf hyperspectral chlorophyll content analysis result channel, and the construction of the independent bud leaf hyperspectral analysis result image based on the spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image comprises: based on the water content spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image, constructing the independent bud leaf hyperspectral water content analysis result channel in the independent bud leaf hyperspectral analysis result image; based on the chlorophyll content spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image, constructing the independent bud leaf hyperspectral chlorophyll content analysis result channel in the independent bud leaf hyperspectral analysis result image; Combine the independent bud leaf hyperspectral water content analysis result channel and the independent bud leaf hyperspectral chlorophyll content analysis result channel to construct the independent bud leaf hyperspectral analysis result image; The water content spectral analysis result parameter is obtained based on the water content target frequency band setting information, and the chlorophyll content spectral analysis result parameter is obtained based on the chlorophyll content target frequency band setting information.

4. The method of claim 3, wherein, The target frequency band setting information of the spectral data of each pixel point in the independent bud leaf hyperspectral image is set, and the spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image is obtained, including: Based on the water content target frequency band setting information, the spectral absorption intensity parameter of the frequency band information corresponding to the water content target frequency band setting information in the spectral data of each pixel point in the independent bud leaf hyperspectral image is calculated, and the spectral absorption intensity parameter corresponding to the water content target frequency band setting information is set as the water content spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image; Based on the chlorophyll content target frequency band setting information, the spectral absorption intensity parameter of the frequency band information corresponding to the chlorophyll content target frequency band setting information in the spectral data of each pixel point in the independent bud leaf hyperspectral image is calculated, and the spectral absorption intensity parameter corresponding to the chlorophyll content target frequency band setting information is set as the chlorophyll content spectral analysis result parameter of each pixel point in the independent bud leaf hyperspectral image; The expression of the spectral absorption intensity parameter is: ； In the formula, For the first The spectral absorption intensity parameter corresponding to the target frequency band setting information mentioned above. For the first The total number of wavelengths in the frequency band information corresponding to the target frequency band setting information mentioned above. For the first The reflectivity of the reference frequency band for the target frequency band setting information mentioned above. For the first The target frequency band setting information mentioned above corresponds to the frequency band information of the first item. wavelength, For the first The target frequency band setting information mentioned above corresponds to the frequency band information of the first item. Reflectivity of wavelength.

5. The method of claim 3, wherein, The independent bud leaf visual detection image includes a red channel, a blue channel and a green channel, the input channel of the tea leaf independent bud leaf grading image analysis model has at least five, the independent bud leaf hyperspectral analysis result image and the independent bud leaf visual detection image corresponding to the same independent bud leaf in the independent bud leaf visual detection image set are input into the tea leaf independent bud leaf grading image analysis model to obtain the tea leaf independent bud leaf grading result information of the independent bud leaf, including: The independent bud leaf hyperspectral water content analysis result channel of the independent bud leaf hyperspectral analysis result image corresponding to the same independent bud leaf, the independent bud leaf hyperspectral chlorophyll content analysis result channel of the independent bud leaf hyperspectral analysis result image, the red channel of the independent bud leaf visual detection image, the blue channel of the independent bud leaf visual detection image and the green channel of the independent bud leaf visual detection image are input into the tea leaf independent bud leaf grading image analysis model to obtain the tea leaf independent bud leaf grading result information of the independent bud leaf.

6. The method of claim 5, wherein, The output of the tea fresh leaf independent bud leaf grading image analysis model includes leaf bud ratio information and maturity information of the independent bud leaf, and the tea fresh leaf independent bud leaf grading result information includes the leaf bud ratio information, the maturity information, and independent bud leaf grading score information. The independent bud leaf grading score information of the independent bud leaf is calculated based on the leaf bud ratio information and the maturity information of the independent bud leaf. The comprehensive grading result information of the tea fresh leaf batch to which the independent bud leaf belongs is generated based on the independent bud leaf grading score information of the independent bud leaf. The expression of the independent bud leaf grading score information and the comprehensive grading result information is as follows: ； ； In the formula, is the first the independent leaf grade score information of the independent leaf, is the first the leaf number of the independent leaf, is the first the bud number of the independent leaf, is the first the score value corresponding to the maturity information of the independent leaf, is the first the comprehensive grading result information of the tea leaf batch to which the independent leaf belongs, is the first the average value of the independent leaf grade score information in the tea leaf batch to which the independent leaf belongs.

7. The method according to any one of claims 1 to 6, characterized in that, The independent bud leaf image segmentation model is trained based on an improved U-shaped neural network model, the improved U-shaped neural network model has three down-sampling levels, the convolution module of the improved U-shaped neural network model is a residual convolution module, and a spatial attention module is added between the two residual convolution modules with the largest down-sampling multiple of the improved U-shaped neural network model.

8. A cross-modal fusion tea leaf grading device, characterized in that, The device comprises: A cross-modal data acquisition module is configured to acquire a tea fresh leaf visual detection image and a tea fresh leaf hyperspectral image. A bud leaf image segmentation module is configured to input the tea fresh leaf visual detection image into an independent bud leaf image segmentation model to generate independent bud leaf image segmentation result information. An independent bud leaf image generation module is configured to extract an independent bud leaf visual detection image set and an independent bud leaf hyperspectral image set from the tea fresh leaf visual detection image and the tea fresh leaf hyperspectral image, respectively, based on the independent bud leaf image segmentation result information. A hyperspectral analysis module is configured to perform spectral analysis on each independent bud leaf hyperspectral image in the independent bud leaf hyperspectral image set to obtain an independent bud leaf hyperspectral analysis result image. An independent bud leaf grading module is configured to input the independent bud leaf hyperspectral analysis result image corresponding to the same independent bud leaf and the independent bud leaf visual detection image in the independent bud leaf visual detection image set into a tea fresh leaf independent bud leaf grading image analysis model to obtain tea fresh leaf independent bud leaf grading result information of the independent bud leaf. The tea fresh leaf independent bud leaf grading image analysis model is trained based on an improved YOLO model, the improved YOLO model is obtained by replacing the convolution module in the cross-stage partial feature fusion module of the original YOLOV8 model with a depth separable convolution module and adding a channel attention module between the neck network and the head network. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images