Citrus huanglongbing detection device and detection method based on visual image

Abstract

The invention relates to a Citrus Huanglongbing detection device and detection method based on a visual image. According to the method, a visual image and a biophysical image of a to-be-detected citrus leaf are obtained, targeted preprocessing and feature extraction are performed on the visual image and the biophysical image to form a comprehensive feature vector, then the comprehensive feature vector is input into a trained citrus huanglongbing detection model, and finally a detection result is obtained. According to the method, multi-source image information can be fully fused, the limitation that early lesion features are not obvious when the citrus leaf huanglongbing is detected only by depending on a visual image in the immature fruit period is overcome, and multi-dimensional features of leaf appearance, physiological metabolism and cell structure changes caused by the huanglongbing are accurately captured; therefore, the accuracy and the early diagnosis capability of citrus huanglongbing detection are remarkably improved, the citrus planting industry is effectively assisted in timely disease prevention and control, the economic loss is reduced, and the fruit quality and yield are guaranteed.

Description

Technical Field

[0001] The invention belongs to the field of image recognition, and in particular relates to a citrus Huanglongbing detection device and a detection method based on visual images. Background Art

[0002] Citrus Huanglongbing is a highly devastating citrus disease caused by Bacillus subtilis. It can lead to systemic decline of citrus trees and, in severe cases, cause complete crop failure in an orchard.

[0003] With the development of computer vision and sensor technology, citrus Huanglongbing detection methods based on visual images have been gradually applied to the agricultural field. This type of method collects image data of citrus trees, uses image processing and analysis technology to extract features and identify diseases, and has the advantages of non-contact, rapid, and automated. However, existing citrus Huanglongbing detection methods based on visual images mostly focus on the analysis of the overall appearance of citrus trees or the characteristics of mature fruits, and have limited ability to capture subtle pathological features of citrus leaves during the period when the fruit is immature.

[0004] In the early stages of citrus growth, the disease may have already caused physiological changes in the leaf tissue. However, because the leaves are not yet fully mature, their color, texture and other appearance characteristics have not changed significantly. Traditional visual image detection methods have difficulty detecting these early lesions, resulting in the inability to detect the disease in time and take effective prevention and control measures, thus missing the best prevention and control opportunity, causing huge economic losses to the citrus planting industry.

[0005] In addition, the early stage of citrus Huanglongbing infection may cause changes in leaf water transport and transpiration, leading to damage to leaf cell structure and photosynthetic function, thus causing changes in the biophysical structure of citrus leaves. However, the current detection method does not take these factors into account, so the existing citrus Huanglongbing detection method based on visual images has limitations and cannot be accurately and quickly detected.

[0006] In summary, developing a method that can effectively utilize biophysical characteristics and detect Huanglongbing on citrus leaves during the unripe period has become a technical problem that needs to be urgently solved in the field of citrus cultivation. Summary of the invention

[0007] Based on this, it is necessary to provide a citrus Huanglongbing detection device and detection method based on visual images that can identify Huanglongbing in the early stage of citrus growth in order to address the above technical problems.

[0008] In a first aspect, the present application provides a citrus Huanglongbing detection method based on visual images, comprising: Acquire a visual image of a citrus leaf to be detected, and perform a first preprocessing on the visual image of the citrus leaf to be detected to obtain a preprocessed visual image of the citrus leaf to be detected; wherein the first preprocessing includes cropping and scaling; Perform visual feature extraction on the preprocessed visual image of the citrus leaf to be detected to obtain a visual feature vector of the citrus leaf to be detected; Acquire a biophysical image of a citrus leaf to be detected, and perform a second preprocessing on the biophysical image of the citrus leaf to be detected to obtain a preprocessed biophysical image of the citrus leaf to be detected; wherein the second preprocessing includes image conversion processing and image enhancement processing; Extracting biophysical features of the preprocessed biophysical image of the citrus leaf to be detected, and obtaining a biophysical feature vector of the citrus leaf to be detected; Combining the visual feature vector and the biophysical feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector of the citrus leaf to be detected; The comprehensive feature vector of the citrus leaves to be tested is input into the trained citrus Huanglongbing detection model to obtain the test results.

[0009] In a second aspect, the present application also provides a citrus Huanglongbing detection device based on visual images, comprising: A visual image acquisition module is used to acquire a visual image of a citrus leaf to be detected, and to perform a first preprocessing on the visual image of the citrus leaf to be detected to obtain a preprocessed visual image of the citrus leaf to be detected; wherein the first preprocessing includes cropping and scaling; A visual feature extraction module is used to extract visual features from the pre-processed visual image of the citrus leaves to be detected, and obtain a visual feature vector of the citrus leaves to be detected; A biophysical image acquisition module is used to acquire a biophysical image of a citrus leaf to be detected, and to perform a second preprocessing on the biophysical image of the citrus leaf to be detected to obtain a preprocessed biophysical image of the citrus leaf to be detected; the second preprocessing includes image conversion processing and image enhancement processing; A biophysical feature extraction module is used to extract biophysical features from the preprocessed biophysical image of the citrus leaves to be detected, and obtain a biophysical feature vector of the citrus leaves to be detected; A feature combination module is used to combine the visual feature vector and the biophysical feature vector of the citrus leaves to be detected to obtain a comprehensive feature vector of the citrus leaves to be detected; The Huanglongbing detection module is used to input the comprehensive feature vector of the citrus leaves to be detected into the trained citrus Huanglongbing detection model to obtain the detection results.

[0010] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a visual image-based citrus Huanglongbing detection method as in the first aspect is implemented.

[0011] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements a citrus Huanglongbing detection method based on visual images as in the first aspect.

[0012] The above-mentioned citrus Huanglongbing detection method, device, computer equipment and storage medium based on visual images obtain the visual image and biophysical image of the citrus leaves to be detected, respectively, and combine them into a comprehensive feature vector after targeted preprocessing and feature extraction, and then input the trained citrus Huanglongbing detection model to finally obtain the detection result. This method can fully integrate multi-source image information, overcome the limitation of relying solely on visual images when detecting citrus leaves with Huanglongbing in the immature period of fruit, and the early pathological characteristics are not obvious. It accurately captures the multi-dimensional characteristics of leaves caused by Huanglongbing from appearance to physiological metabolism and cell structure changes, thereby significantly improving the accuracy and early diagnosis ability of citrus Huanglongbing detection, effectively helping the citrus planting industry to prevent and control diseases in a timely manner, reduce economic losses and ensure fruit quality and yield. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A schematic diagram of a flow chart of a citrus Huanglongbing detection method based on visual images provided by the present invention; Figure 2 A schematic structural diagram of a citrus Huanglongbing detection device based on visual images provided by the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0016] refer to Figure 1 , which shows a flow chart of a citrus Huanglongbing detection method based on visual images provided by the present application, the method comprising the following steps: S101, obtaining a visual image of a citrus leaf to be detected, and performing a first preprocessing on the visual image of the citrus leaf to be detected to obtain a preprocessed visual image of the citrus leaf to be detected; wherein the first preprocessing includes cropping and scaling.

[0017] Specifically, you can use appropriate image acquisition equipment (such as high-definition digital cameras, professional agricultural testing instruments with image acquisition functions, etc.) to take photos of representative leaves selected from citrus plants in citrus planting areas or specific locations where samples are collected. When taking photos, pay attention to factors such as angle and light, and try to ensure that the images of leaves obtained are clear and complete, and can accurately reflect the actual appearance of the leaves and possible signs of disease characteristics, such as the color of the leaves, whether they are mottled, yellowed, etc.

[0018] The original images collected often contain redundant background information besides the leaves, such as branches, surrounding soil or other parts of the plant, etc. Through cropping operations, using image editing software or specialized image processing algorithms, the area containing only citrus leaves can be accurately selected based on the position of the leaves in the image to remove unnecessary background interference.

[0019] Due to different acquisition equipment, shooting distance and other factors, the sizes of the acquired leaf images are different. In order to make the image meet the unified requirements of subsequent feature extraction and other processing steps, the cropped leaf image needs to be scaled.

[0020] S102, extracting visual features from the preprocessed visual image of the citrus leaf to be detected, to obtain a visual feature vector of the citrus leaf to be detected.

[0021] Specifically, a suitable computer vision algorithm can be used to extract the visual features of the image. For example, a color histogram-based method can be used to count the pixel distribution of different color channels (such as RGB channels) in the leaf image to form a description vector of the color features, because citrus Huanglongbing often causes abnormal changes in leaf color, such as yellow-green spots.

[0022] Texture feature extraction algorithms, such as the gray-level co-occurrence matrix (GLCM), can also be used to obtain texture features such as the roughness and directionality of the leaf surface texture by analyzing the spatial distribution relationship of pixel gray levels in the image. There may be differences in texture between normal leaves and leaves infected with Huanglongbing. For example, the texture of diseased leaves may become disordered and irregular.

[0023] The edge detection algorithm can also be used to first determine the outline of the leaves, and then extract shape features such as the aspect ratio and roundness of the leaves. Huanglongbing may cause the leaves to shrink, deform, etc., affecting their shape characteristics.

[0024] These visual features such as color, texture, shape, etc. are combined in a certain quantitative way to finally form a visual feature vector that can comprehensively describe the visual characteristics of the citrus leaves to be tested.

[0025] S103, acquiring the biophysical image of the citrus leaves to be detected, and performing a second preprocessing on the biophysical image of the citrus leaves to be detected to obtain a preprocessed biophysical image of the citrus leaves to be detected; wherein the second preprocessing includes image conversion processing and image enhancement processing.

[0026] Specifically, we can use professional biophysical imaging equipment, such as near-infrared imagers and hyperspectral imagers. These devices can be used to image citrus leaves and obtain images that can reflect the biophysical properties of the leaves. For example, near-infrared imaging can detect the reflection and absorption of near-infrared light by the internal tissues of the leaves, while hyperspectral imaging can obtain the reflectivity information of the leaves in multiple spectral bands. This information is closely related to the physiological structure, composition, and whether the leaves are infected with diseases.

[0027] Image conversion processing: Depending on the original format of the acquired biophysical images and the needs of subsequent processing, format conversion may be required, such as converting the specific format collected by the device (such as some device-specific raw data formats) into a common image format (such as JPEG, PNG, etc.) to facilitate subsequent operations using common image processing algorithms and software. At the same time, for images such as hyperspectral images that contain multiple band information, conversion operations such as band selection or band combination may also be required to extract images of key band combinations that best reflect the characteristics of citrus Huanglongbing, reduce data redundancy, and highlight effective information.

[0028] Image enhancement processing: Since biophysical images may be affected by factors such as uneven ambient light and equipment noise during the acquisition process, the image quality may not be ideal. Image enhancement algorithms are used to improve image quality. For example, the histogram equalization method is used to adjust the image contrast, making the differences between different tissues and structures inside the leaves more obvious, which is convenient for subsequent feature extraction. Filtering algorithms (such as median filtering, Gaussian filtering, etc.) can also be used to remove noise points in the image, so that the image can more clearly and accurately reflect the biophysical characteristics of the leaves, thereby ensuring the reliability of the biophysical features extracted later.

[0029] S104, extracting biophysical features of the preprocessed biophysical image of the citrus leaf to be detected, and obtaining a biophysical feature vector of the citrus leaf to be detected.

[0030] Specifically, for the preprocessed biophysical image, feature extraction is performed based on different imaging principles and corresponding effective information.

[0031] If it is a near-infrared image, the characteristic parameters of the leaves' absorption and reflection of near-infrared light in different areas can be extracted, such as the reflectance value at a specific wavelength, the difference in reflectance in different areas, etc., because the changes in the internal cell structure, water content and other biophysical properties of leaves infected with Huanglongbing will lead to changes in the near-infrared light response characteristics.

[0032] For hyperspectral images, the spectral characteristic curves of each band are analyzed to extract features such as the spectral absorption peak position, absorption valley depth, and spectral slope. These spectral features are closely related to the pigment content and chemical composition in the leaves. Huanglongbing will affect the biochemical components in the leaves, which will then manifest as differences in spectral characteristics.

[0033] The multi-dimensional biophysical features obtained from the biophysical images are integrated and quantified to form a biophysical feature vector that can characterize the biophysical properties of the leaf.

[0034] S105, combining the visual feature vector and the biophysical feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector of the citrus leaf to be detected.

[0035] Specifically, the visual feature vectors and biophysical feature vectors extracted separately are combined in a certain order or rule. For example, the elements in the visual feature vector can be arranged first, and then the elements in the biophysical feature vector can be arranged to form a new, higher-dimensional comprehensive feature vector. This comprehensive feature vector combines the key information extracted from the visual appearance of the leaves and the internal biophysical characteristics, and can more comprehensively and three-dimensionally describe the state of the citrus leaves to be tested.

[0036] S106, inputting the comprehensive feature vector of the citrus leaves to be detected into a trained citrus Huanglongbing detection model to obtain a detection result.

[0037] Specifically, a large number of citrus leaf samples that are known to be healthy and infected with Huanglongbing are collected in advance (these samples are subjected to the same feature extraction steps as mentioned above to obtain the corresponding comprehensive feature vectors, and are marked with the real labels of whether they are infected with Huanglongbing), and machine learning or deep learning algorithms (such as support vector machines, convolutional neural networks, etc.) are used to train the model, so that the model can learn the mapping relationship between the comprehensive feature vector and whether it is infected with Huanglongbing.

[0038] When the comprehensive feature vector of the citrus leaves to be tested is input into the trained citrus Huanglongbing detection model, the model will judge whether the leaves are infected with Huanglongbing based on its internal parameters and learned rules, and output the corresponding test results, such as the classification results of "infected with Huanglongbing" or "not infected with Huanglongbing", or a probability value indicating the possibility of infection (such as outputting an infection probability of 0.8, indicating a high possibility of infection with Huanglongbing), so that relevant personnel can take further response measures based on this result, such as isolating and treating plants infected with Huanglongbing.

[0039] The above-mentioned citrus Huanglongbing detection method based on visual images obtains the visual image and biophysical image of the citrus leaves to be detected, respectively, and combines them into a comprehensive feature vector after targeted preprocessing and feature extraction, and then inputs the trained citrus Huanglongbing detection model to finally obtain the detection result. This method can fully integrate multi-source image information, overcome the limitation of relying solely on visual images when detecting citrus leaves with Huanglongbing in the immature period of fruit, and the early pathological characteristics are not obvious. It accurately captures the multi-dimensional characteristics of leaves caused by Huanglongbing from appearance to physiological metabolism and cell structure changes, thereby significantly improving the accuracy and early diagnosis ability of citrus Huanglongbing detection, effectively helping the citrus planting industry to prevent and control diseases in a timely manner, reduce economic losses and ensure fruit quality and yield.

[0040] In an optional embodiment, obtaining a biophysical image of a citrus leaf to be detected, performing a second preprocessing on the biophysical image of the citrus leaf to be detected, and obtaining a preprocessed biophysical image of the citrus leaf to be detected includes the following steps: Capturing thermal imaging images of the citrus leaves to be inspected, and obtaining temperature distribution images of the citrus leaves to be inspected as original thermal imaging images of the citrus leaves to be inspected; Collecting fluorescence images of the citrus leaves to be detected, and obtaining fluorescence images of the citrus leaves to be detected under excitation of a specific wavelength as original fluorescence images of the citrus leaves to be detected; Performing a third preprocessing on the original thermal imaging image of the citrus leaves to be detected to obtain a preprocessed thermal imaging image of the citrus leaves to be detected; performing a fourth preprocessing on the original fluorescent image of the citrus leaves to be detected to obtain a preprocessed fluorescent image of the citrus leaves to be detected; The preprocessed thermal imaging image and the preprocessed fluorescence image of the citrus leaf to be detected are used as the preprocessed biophysical image of the citrus leaf to be detected; Wherein, the second preprocessing includes a third preprocessing and a fourth preprocessing; The third preprocessing is: converting the original thermal imaging image into an actual temperature value matrix, removing abnormal temperature point data caused by equipment or environmental factors, cropping and scaling; The fourth preprocessing is: performing background subtraction processing on the original fluorescence image, removing the background fluorescence signal caused by the excitation light scattering factor, enhancing the contrast of the leaf fluorescence signal, cropping and scaling.

[0041] Specifically, (1) for obtaining preprocessed thermal imaging images: Professional thermal imaging equipment (such as infrared thermal imagers, etc.) can be used to shoot the citrus leaves to be tested. Thermal imaging equipment can sense the infrared radiation emitted by the surface of the leaves and generate corresponding temperature distribution images based on the intensity difference. During the acquisition process, it is necessary to ensure that the equipment is kept at a suitable distance from the leaves and the shooting angle is suitable to obtain a complete, clear and original thermal imaging image that can accurately reflect the actual temperature distribution of the leaves. For example, if the distance is too close, the image may only show part of the leaf, while if the distance is too far, the image resolution may be too low, affecting the subsequent analysis of temperature details. At the same time, when shooting, try to avoid other surrounding heat sources (such as local high temperature areas caused by direct sunlight, nearby heating equipment, etc.) from interfering with the leaf temperature measurement, so as to ensure that the temperature distribution of the leaves in the original thermal imaging image obtained is a true reflection of their own physiological state, because citrus Huanglongbing may affect the metabolism of the leaves, thereby causing abnormal changes in the leaf temperature.

[0042] Convert the original thermal imaging image into an actual temperature value matrix: Thermal imaging images are initially presented in the form of pixel grayscale values, which correspond to actual temperature values. Through the calibration algorithm built into the device or the conversion function in the specialized image processing software, the grayscale information corresponding to each pixel in the image is accurately converted into actual temperature values ​​according to the temperature calibration parameters of the thermal imaging device, thereby constructing a temperature value matrix. In this way, subsequent analysis and processing can be directly based on these specific temperature values.

[0043] Removing abnormal temperature point data caused by equipment or environmental factors: During the thermal imaging image acquisition process, due to the accuracy limitations of the equipment itself, occasional failures (such as abnormal sensitivity or failure of a certain pixel of the detector), or environmental factors (such as a sudden gust of hot air passing over the blade causing local short-term temperature fluctuations, strong electromagnetic interference around during acquisition affecting the accuracy of temperature measurement, etc.), some abnormal temperature points that do not conform to the actual temperature distribution law of the blade may be generated in the image. By setting a reasonable temperature threshold range (a reasonable upper and lower limit range can be obtained based on statistical analysis of a large number of normal blade thermal imaging temperature data in the past), and using data filtering and outlier detection algorithms (such as the 3σ principle based on statistics, etc.), these abnormal temperature point data can be identified and removed to ensure that the blade temperature distribution reflected by the temperature value matrix is ​​relatively accurate and reliable.

[0044] Cropping and scaling: Similar to the cropping and scaling in the visual image preprocessing, the original thermal imaging image may contain redundant background areas other than the leaves. Through the cropping operation, only the area where the leaves are located is retained to reduce the amount of unnecessary background data and possible interference. At the same time, according to the unified requirements of subsequent processing, the cropped thermal imaging image is scaled according to the established size standard, such as uniformly scaling the image to a size with a fixed pixel value in length and width, so that the preprocessed thermal imaging image specifications of all leaves to be detected are consistent.

[0045] (2) Acquisition of preprocessed fluorescence images: Instruments with fluorescence imaging functions can be used to select appropriate excitation light wavelengths to illuminate citrus leaves, so that specific fluorescent substances in the leaves are excited to produce fluorescence. The instrument then captures these fluorescence signals and generates corresponding fluorescence images, which are the original fluorescence images. Different fluorescent substances will emit fluorescence with characteristic wavelengths when excited by specific wavelengths. When citrus leaves are infected with Huanglongbing, their internal chemical composition (such as certain pigments, proteins and other fluorescent substance-related components) may change, resulting in differences in the fluorescence characteristics emitted under the same excitation conditions. Therefore, based on the characteristics of citrus leaves and the fluorescence changes related to Huanglongbing found in previous studies, the excitation light wavelength should be accurately selected. For example, a wavelength that can make fluorescent substances such as chlorophyll and flavonoids that are closely related to the health of the leaves produce obvious fluorescence responses is selected to ensure that the collected original fluorescence images can effectively reflect whether the leaves are infected with Huanglongbing.

[0046] Background subtraction is performed on the original fluorescence image: When collecting fluorescence images, in addition to the fluorescence signal generated by the leaves themselves, the excitation light may be scattered in the surrounding environment (such as slides, air, etc.) during the propagation process, thereby forming a background fluorescence signal superimposed on the real fluorescence signal of the leaves. By collecting the fluorescence image of the blank control area without leaves (also under the same excitation light conditions), the intensity distribution of the background fluorescence is obtained, and then the corresponding background fluorescence intensity value is subtracted pixel by pixel from the original fluorescence image to achieve background subtraction, and an image that only reflects the fluorescence characteristics of the leaves themselves is obtained, which can more accurately highlight the changes in the fluorescence signal caused by changes in the physiological state of the leaves (such as infection with Huanglongbing).

[0047] Removal of background fluorescence signals caused by excitation light scattering: In addition to the above-mentioned background subtraction processing, it is also necessary to further remove the weak background fluorescence signals remaining due to the scattering of the excitation light inside the leaves and in the surrounding complex media. Some methods based on the combination of optical principles and image processing can be used, such as using hardware devices such as optical filters to minimize the amount of scattered light entering the imaging system during acquisition, and using digital filtering algorithms at the image level to further filter out the background fluorescence signal from the image based on its frequency, intensity and other characteristics, making the fluorescence image of the leaves purer.

[0048] Enhance the contrast of leaf fluorescence signals: Since the fluorescence signal intensity of the leaves themselves may be relatively weak, or the difference in fluorescence signals in different areas is not obvious enough, it is not conducive to intuitively discovering the changes in fluorescence characteristics caused by Huanglongbing. Through image enhancement algorithms, such as histogram stretching and contrast-limited adaptive histogram equalization, the pixel grayscale values ​​of the fluorescence image are adjusted to expand the contrast between areas with different fluorescence intensities, so that those areas on the leaves with changes in fluorescence intensity due to Huanglongbing infection can be presented more clearly.

[0049] Cropping and scaling: Similarly, the original fluorescence image may also contain redundant background parts. These irrelevant areas are removed by cropping, and only the fluorescence image of the main part of the leaf is retained. Then, the cropped image is scaled according to a unified standard to make its size meet the requirements of the subsequent processing flow.

[0050] In an optional embodiment, extracting biophysical features from the preprocessed biophysical image of the citrus leaf to be detected to obtain the biophysical feature vector of the citrus leaf to be detected includes the following steps: According to the pre-processed thermal imaging images of the citrus leaves to be tested, the average temperature T of the leaves of the citrus with Huanglongbing was calculated. avg , temperature variance T var and the temperature gradient T grad ; Combine the average temperature, temperature variance and temperature gradient into the thermal imaging feature vector H of the citrus leaves to be detected; where H=[T avg , T var , T grad ]; According to the pre-processed fluorescence image of the citrus leaves to be tested, the average fluorescence intensity F of the leaves of the citrus with Huanglongbing was calculated. avg , peak fluorescence intensity F peak and the skewness F of the fluorescence intensity distribution skew , the average fluorescence intensity, the peak fluorescence intensity and the skewness of the fluorescence intensity distribution are combined into the fluorescence feature vector I of the citrus leaves to be detected; where I=[F avg , F peak ,Fskew ]; The thermal imaging feature vector H and the fluorescence feature vector I of the citrus leaves to be detected are used as the biophysical feature vectors of the citrus leaves to be detected.

[0051] Specifically, calculate the average temperature T avg : Traverse the temperature values ​​corresponding to all pixels in the preprocessed thermal imaging image (these values ​​are obtained after converting the original thermal imaging image into a temperature value matrix), add up all the temperature values, and then divide them by the total number of pixels to obtain the average temperature T of the entire blade avg The average temperature is an important indicator of the overall thermal state of the leaves. There may be differences in the average temperature between healthy leaves and leaves infected with Huanglongbing, because Huanglongbing may affect the physiological activities of the leaves, thereby changing the balance of heat generation and dissipation, causing the average temperature to increase or decrease. Therefore, T avg It is a basic characteristic for determining citrus Huanglongbing disease.

[0052] Calculate the temperature variance T var : For each pixel temperature value T in the image i , calculate its difference with the average temperature T avg The difference (T i -T avg ), then add the squares of these differences and divide by the total number of pixels to get the temperature variance T var The temperature variance can measure the degree of discreteness of the leaf temperature distribution, that is, the fluctuation of the temperature in different areas of the leaf. The temperature distribution of healthy leaves is relatively uniform, and the temperature variance is small; while leaves infected with Huanglongbing may have abnormal cell metabolism due to local lesions, resulting in large local temperature changes, thereby increasing the temperature variance. Therefore, the temperature variance is an important feature that helps to distinguish healthy from diseased leaves.

[0053] Calculate the temperature gradient T grad :The temperature gradient reflects the rate of change of leaf temperature in space. By calculating the temperature difference between adjacent pixels and combining the distance relationship between pixels (for example, the distance between adjacent pixels in the horizontal and vertical directions is usually 1 unit), the rate of change of temperature in different directions can be determined, and then the temperature gradient T can be obtained. grad In leaves infected with Huanglongbing, there may be a large temperature gradient between the local temperature abnormality area and the normal area. Because changes in physiological activities in the diseased area will affect heat transfer, the temperature gradient is also a key judgment feature.

[0054] The calculated average temperature T avg , temperature variance T var and temperature gradient T grad Combine them in order to form the thermal imaging feature vector H = [Tavg , T var , T grad ], this vector fully describes the temperature characteristics of the blade from a thermal point of view.

[0055] Calculate the mean fluorescence intensity F avg : Count the fluorescence intensity values ​​of all pixels in the preprocessed fluorescence image, add these values ​​and divide them by the total number of pixels to get the average fluorescence intensity F avg The mean fluorescence intensity reflects the overall fluorescence emission level of the leaves. Citrus Huanglongbing may affect the content or activity of fluorescent substances in the leaves, thereby changing the mean fluorescence intensity and making it different from that of healthy leaves. Therefore, it is a basic fluorescence feature for judging the health of leaves.

[0056] Calculate the peak fluorescence intensity F peak : In the distribution data of fluorescence intensity, find the fluorescence intensity value with the highest frequency, which is the peak value of fluorescence intensity F peak The peak fluorescence intensity can reflect the fluorescence emission characteristics of the main fluorescent substances in the leaves. Infection with Huanglongbing may cause changes in the types or concentrations of certain fluorescent substances in the leaves, thereby changing the peak fluorescence intensity. Therefore, it is one of the important characteristics to distinguish healthy from diseased leaves.

[0057] Calculate the skewness F of the fluorescence intensity distribution skew : The skewness of the fluorescence intensity distribution is used to measure the asymmetry of the fluorescence intensity data distribution. The skewness F is obtained by calculating the ratio of the third-order central moment of the fluorescence intensity data to the cube of the standard deviation. skew The fluorescence intensity distribution of healthy leaves and leaves infected with Huanglongbing may have different skewness characteristics. For example, the fluorescent substance in the diseased leaves may be unevenly distributed due to local lesions, making the fluorescence intensity distribution show different asymmetry from that of healthy leaves. Therefore, the skewness of the fluorescence intensity distribution is also an effective judgment feature.

[0058] The mean fluorescence intensity F avg , peak fluorescence intensity F peak and the skewness F of the fluorescence intensity distribution skew Combined into fluorescence feature vector I = [F avg , F peak , F skew ], this vector describes the state of the leaf in terms of its fluorescence properties.

[0059] The thermal imaging feature vector and the fluorescence feature vector describe the characteristics of the citrus leaves to be tested from two different but interrelated biophysical levels: thermal and fluorescence characteristics. Combining them together forms a more comprehensive and integrated biophysical feature vector, which can more fully reflect the physiological state of the leaves and the characteristic information related to citrus Huanglongbing. In this way, in the subsequent citrus Huanglongbing detection process, this comprehensive biophysical feature vector can be used in combination with the trained detection model to more accurately determine whether the leaves are infected with Huanglongbing, thereby improving the accuracy and reliability of the detection.

[0060] In an optional embodiment, performing visual feature extraction on the preprocessed visual image of the citrus leaf to be detected to obtain the visual feature vector of the citrus leaf to be detected includes the following steps: Performing color feature extraction on the preprocessed visual image of the citrus leaves to be detected to obtain a color feature vector of the citrus leaves to be detected; Extracting texture features of the preprocessed visual image of the citrus leaf to be detected, and obtaining a texture feature vector of the citrus leaf to be detected; Perform shape feature extraction on the preprocessed visual image of the citrus leaf to be detected to obtain a shape feature vector of the citrus leaf to be detected; The color feature vector, texture feature vector and shape feature vector of the detected citrus are used as the visual feature vector of the citrus leaves to be detected.

[0061] Specifically, (1) color feature extraction: For each image in the preprocessed visual image dataset, extract the color features of the image. Convert the image from RGB color space to HSV color space. Calculate the color histogram of the three channels H (hue), S (saturation), and V (brightness) in the HSV color space. The color histogram can be calculated by counting the frequency of different color values ​​in each color channel. Assuming that the color value range of a color channel in the image is [0, L], the calculation formula for the color histogram of the channel is: X(k) = n k / N , k=1 , 2 , … , L; Among them, X(k) represents the frequency of pixels with color value in the image, n k represents the number of pixels with color value k, and N represents the total number of pixels in the image. The color distribution characteristics of the image can be obtained through the color histogram, which is recorded as the color feature vector C, C=[X H , X S , X V ], where X H , X S , X VThey are the color histograms of the H, S, and V channels respectively.

[0062] (2) Texture feature extraction: The gray level co-occurrence matrix (GLCM) method can be used to extract the texture features of the preprocessed visual image. The image is converted to a grayscale image, and the conversion formula is: Gray(i, j)=0.299×R(i, j)+0.587×G(i, j)+0.114×B(i, j); Among them, Gray(i, j) represents the grayscale value of the pixel at the coordinate (i, j) in the grayscale image, R(i, j), G(i, j), and B(i, j) represent the red, green, and blue channel pixel values ​​at the coordinate (i, j) in the RGB color space of the original image, respectively.

[0063] Calculate the gray-level co-occurrence matrix, set different directions θ (such as 0°, 45°, 90°, 135°) and displacement distances d (such as d=1, 2, 3), and calculate the gray-level co-occurrence matrix element P(i, j, d, θ) under each direction and displacement distance, which represents the frequency of occurrence of pixel pairs with gray-level values ​​​​i and j under direction θ and displacement distance d.

[0064] Based on the gray level co-occurrence matrix, texture features such as contrast, energy, and entropy are extracted. For example, the calculation formula for contrast is: ;

[0065] The energy calculation formula is: ;

[0066] The calculation formula for entropy is: ;

[0067] The extracted texture features are combined into a texture feature vector B, B=[B1, B2, …, B m ], where B k Represents different texture feature values, k=1, 2, …, m.

[0068] (3) Shape feature extraction: For the citrus leaves and fruit parts in the preprocessed visual image, shape features are extracted. Citrus leaves and fruits can be separated from the background by image segmentation algorithms (such as threshold segmentation, edge detection-based segmentation algorithms, etc.), and the threshold used by the segmentation algorithm can be determined by experiments or calculated using an adaptive threshold method.

[0069] Extract shape features, such as the circumference of a leaf ,area , circularity , the length of the long axis of the fruit is L f , short axis length S f , aspect ratio R f = L f / S f Etc. These shape features are combined into a shape feature vector D, D = [D1, D2, …, D n ], where D t represents different shape eigenvalues, t=1, 2, …, n.

[0070] (4) Obtain visual feature vector: The color feature vector C, texture feature vector B, and shape feature vector D describe the characteristics of the citrus leaves to be detected from different visual perspectives. The color feature reflects the color information of the leaves, the texture feature reflects the microstructure of the leaf surface, and the shape feature describes the macroscopic outline of the leaves. Combining these three vectors together forms a comprehensive visual feature vector that can comprehensively reflect the visual characteristics of the leaf appearance, providing rich and multi-dimensional visual information for subsequent accurate judgment using the trained citrus Huanglongbing detection model.

[0071] In an optional embodiment, the visual feature vector and the biophysical feature vector of the citrus leaf to be detected are combined to obtain a comprehensive feature vector of the citrus leaf to be detected, comprising the following steps: The thermal imaging feature vector, fluorescence feature vector, color feature vector, texture feature vector and shape feature vector of the citrus leaf to be detected are combined to obtain a comprehensive feature vector M=[C, B, D, H, I] of the citrus leaf to be detected; Among them, C represents the color feature vector of the citrus leaf to be detected, B represents the texture feature vector of the citrus leaf to be detected, and D represents the shape feature vector of the citrus leaf to be detected.

[0072] Specifically, these five feature vectors are combined in order into a comprehensive feature vector M, which realizes the fusion of multi-dimensional information from visual appearance (color, texture, shape) to biophysical properties (thermal, fluorescence). This comprehensive feature vector can more comprehensively and deeply characterize the state of the citrus leaves to be tested. Compared with a single feature vector, it can provide richer information and reduce the risk of misjudgment caused by inaccurate judgment of a single feature. When the comprehensive feature vector is subsequently input into the trained citrus Huanglongbing detection model, the model can use these various information for comprehensive analysis, so as to more accurately determine whether the leaves are infected with Huanglongbing and improve the accuracy and reliability of detection.

[0073] In an optional embodiment, before the comprehensive feature vector of the citrus leaves to be tested is input into the trained citrus Huanglongbing detection model to obtain the test result, the following steps are also included: Constructing a citrus Huanglongbing detection model using convolutional neural networks; The citrus Huanglongbing detection model is trained to obtain a trained citrus Huanglongbing detection model; Among them, the structure of the convolutional neural network includes: An input layer, whose dimension matches the dimension of the comprehensive feature vector, is used to receive the color feature vector, texture feature vector, shape feature vector, thermal imaging feature vector and fluorescence feature vector of the citrus leaves to be detected; Convolutional layer module, including: In the first convolution layer, 32 convolution kernels are set, each of which has a size of 3×3, a step size of 1, and a ReLU activation function; The first batch of normalization layers is connected after the first convolutional layer and performs batch normalization on the data output by the first convolutional layer; In the second convolution layer, 64 convolution kernels are set, the kernel size is 3×3, the step size is 1, and the ReLU activation function is used; The second batch normalization layer is connected after the second convolutional layer and performs batch normalization on the data output by the second convolutional layer; In the third convolution layer, 128 convolution kernels are set, the kernel size is 3×3, the step size is 1, and the ReLU activation function is used; The third batch normalization layer is connected after the third convolutional layer and performs batch normalization on the data output by the third convolutional layer; Pooling layer module, including: The first pooling layer uses the maximum pooling method, with a pooling kernel size of 2×2 and a stride of 2, and is connected after the first batch of normalization layers; The second pooling layer uses the maximum pooling method, with a pooling kernel size of 2×2 and a step size of 2, and is connected after the second batch of normalization layers; The third pooling layer uses the maximum pooling method, with a pooling kernel size of 2×2 and a step size of 2, and is connected after the third batch of normalization layers; Fully connected layer module, including: The first fully connected layer flattens the data output by the third pooling layer and then connects them. It sets 256 neurons and uses the ReLU activation function. Dropout layer, connected after the first fully connected layer, with a dropout rate of 0.5; In the second fully connected layer, two neurons are set, corresponding to the healthy and diseased categories of citrus fruits. The Softmax activation function is used to output the probability that the sample belongs to the healthy and diseased categories.

[0074] Specifically, the Convolutional Neural Network (CNN) is used to build a citrus Huanglongbing detection model due to its outstanding performance in image recognition and other fields. Its structural design can effectively automatically learn the characteristic patterns related to citrus Huanglongbing from the input comprehensive feature vector, thereby achieving accurate classification judgment. The following is an explanation of the structure of the convolutional neural network: (1) Input layer: The dimension of the input layer matches the dimension of the comprehensive feature vector M=[C, B, D, H, I]. The input layer can receive all the information contained in the color feature vector C, texture feature vector B, shape feature vector D, thermal imaging feature vector H, and fluorescence feature vector I of the citrus leaves to be tested. For example, if the color feature vector C has n1 elements, the texture feature vector B has n2 elements, the shape feature vector D has n3 elements, the thermal imaging feature vector H has n4 elements (average temperature, temperature variance, and temperature gradient), and the fluorescence feature vector I has n5 elements (average fluorescence intensity, peak fluorescence intensity, and skewness of fluorescence intensity distribution), then the dimension of the input layer is n1+n2+n3+n4+n5. This ensures that the model can obtain complete information extracted from multiple aspects of the visual and biophysical characteristics of the leaves.

[0075] (2) Convolutional layer module: (21) The first convolutional layer: Set 32 ​​convolution kernels, each with a size of 3×3 and a step size of 1. The convolution kernel can be regarded as a filter that extracts different features by sliding on the input data (i.e., the image-form data after the comprehensive feature vector conversion, assuming that it has been properly reshaped) and performing convolution operations on the local area. The 3×3 convolution kernel size can capture the relationship between adjacent features. A step size of 1 means that the convolution kernel moves one pixel position each time, which ensures a comprehensive scan of the input data. The ReLU (Rectified Linear Unit) activation function is used, which is defined as f(x) = max(0, x). It can introduce nonlinear factors, increase the expressive power of the model, and enable the model to learn more complex feature relationships. For example, for input feature data such as color and texture, the first convolution layer can preliminarily extract some basic local feature patterns, such as local color change trends, simple structural features of textures, etc.

[0076] (22) The first batch of standardized layers: After connecting to the first convolutional layer, the data output by the first convolutional layer is batch normalized. The purpose of batch normalization is to accelerate the training process of the model and improve the stability and convergence speed of the model. It makes the distribution of data more stable by normalizing each feature dimension of each batch of data, reduces the problem of gradient disappearance or explosion caused by changes in data distribution, and helps subsequent convolutional layers to better learn features. For example, for the feature map output by the first convolutional layer, the batch normalization layer will normalize the values ​​of each channel (corresponding to the output of each convolution kernel) so that the mean value of the data in each channel is close to 0 and the variance is close to 1.

[0077] (23) Second convolutional layer: Set 64 convolution kernels with a kernel size of 3×3, a step size of 1, and a ReLU activation function. Based on the feature data processed by the first batch of normalization layers, the second convolution layer further extracts more advanced and complex features. 64 convolution kernels can learn more types of feature patterns, the 3×3 convolution kernel continues to focus on local feature relationships, and the ReLU activation function continues to introduce nonlinearity. For example, in terms of color features, it may be possible to learn the combination pattern of colors in a larger area; in terms of texture features, it can capture more complex texture structure changes, etc.

[0078] (24) The second batch of standardization layers: Similar to the first batch of normalization layers, batch normalization is performed on the output data of the second convolutional layer to stabilize the data distribution, speed up model training, and provide better input data for subsequent convolutional layers.

[0079] (25) The third convolutional layer: 128 convolution kernels are set with a kernel size of 3×3, a stride of 1, and a ReLU activation function. The third convolution layer continues to dig deeper into features based on the previous two convolutional and normalized layers. 128 convolution kernels can learn richer and more abstract feature representations, which may be critical for accurately distinguishing healthy and diseased citrus leaves. For example, complex association patterns between color, texture, shape, thermal, and fluorescence features related to citrus Huanglongbing may be learned.

[0080] (26) The third batch of standardization layers: The data output by the third convolutional layer is batch normalized to ensure that the data has good distribution characteristics before entering the pooling layer.

[0081] (3) Pooling layer module: (31) First pooling layer: The maximum pooling method is adopted, with a pooling kernel size of 2×2 and a stride of 2, which is connected after the first batch of normalization layers. The maximum pooling operation selects the maximum value in a 2×2 area as the output, which can reduce the dimension of the data, reduce the amount of calculation, and retain the most significant feature information. For example, for the feature map processed by the first convolutional layer and the first batch of normalization layers, the first pooling layer will reduce its size by half (because the pooling kernel size is 2×2 and the stride is 2), and highlight the local maximum values ​​in the feature map. These maximum values ​​may correspond to significant change areas in features such as color and texture, which helps the model focus on the most important feature information, while reducing the amount of data and improving computational efficiency.

[0082] (32) Second pooling layer: The same maximum pooling method is used, with a pooling kernel size of 2×2 and a step size of 2, connected after the second batch of normalization layers. Its function is similar to the first pooling layer. While further reducing the data dimension, it retains the important feature information extracted by the second convolutional layer. As the network layer deepens, higher-level feature representations are gradually abstracted through pooling operations.

[0083] (33) The third pooling layer: The maximum pooling method is adopted, with a pooling kernel size of 2×2 and a step size of 2, connected after the third batch of normalization layers. The maximum pooling operation is performed on the feature data processed by the third convolutional layer and the third batch of normalization layers to further compress the data scale, highlight the most important features, and provide more compact and representative feature inputs for the subsequent fully connected layers.

[0084] (4) Fully connected layer module: (41) First fully connected layer: The data output by the third pooling layer is flattened and connected, 256 neurons are set, and the ReLU activation function is used. The flattening operation converts the multi-dimensional output data of the pooling layer into a one-dimensional vector so that it can be connected to the neurons of the fully connected layer. The 256 neurons can perform more comprehensive integration and nonlinear transformation of the flattened feature vectors, and further learn the complex relationship between features. For example, this fully connected layer can comprehensively analyze the color, texture, shape, thermal and fluorescence features extracted by the previous convolutional layer and pooling layer, learn the synergistic mode between different features, and thus provide a more advanced feature representation for the final classification judgment.

[0085] (42) Dropout layer: After the first fully connected layer, the dropout rate is 0.5. The Dropout layer is a regularization technique that randomly drops half (with a dropout rate of 0.5) of the neuron connections during training to prevent the model from overfitting. Overfitting refers to a situation where the model performs well on the training data but performs poorly on new test data. With the Dropout layer, the model uses a different subset of neurons at each training iteration, increasing the model's generalization ability and allowing the model to better adapt to new data.

[0086] (43) Second fully connected layer: Two neurons are set to correspond to the healthy and diseased categories of citrus, and the Softmax activation function is used to output the probability that the sample belongs to the healthy and diseased categories. The Softmax activation function can convert the numerical value output by the first fully connected layer into a probability distribution, so that the outputs of the two neurons represent the probabilities that the sample belongs to the healthy category and the diseased category, respectively. For example, if the output is [0.8, 0.2], it means that the model predicts that the sample has an 80% probability of being healthy and a 20% probability of being diseased. This design enables the model to directly give the classification results, which is convenient for the subsequent detection and judgment of citrus Huanglongbing.

[0087] In an optional embodiment, training a citrus Huanglongbing detection model includes the following steps: S1: Obtain visual images and biophysical images of healthy citrus leaves, and obtain visual images and biophysical images of citrus leaves with Huanglongbing; S2: Based on the visual images and biophysical images of healthy citrus leaves, the comprehensive feature vector set K of healthy samples is obtained h =[M h1 , M h2 , … , M hn ]; Based on the visual images and biophysical images of citrus leaves infected with Huanglongbing, the comprehensive feature vector set K of diseased samples was obtained. d =[M d1 , M d2 , … , M dm ]; where M h1 To M hn Represents the comprehensive feature vector set K of healthy samples h The comprehensive feature vector in M d1 To M dm The comprehensive feature vector set K representing the diseased samples d The comprehensive feature vector in n represents the comprehensive feature vector set K of healthy samples. h The number of samples in, m represents the comprehensive feature vector set K of diseased samples d The number of samples in ; S3: Comprehensive feature vector set K based on healthy samples h and the comprehensive feature vector set K of diseased samples d , get the training set and the validation set; S4: The citrus Huanglongbing detection model is trained cyclically using the training set and the validation set, wherein each round of training includes: S41: extract a batch of sample data and corresponding true category labels from the training set according to a preset batch size; Among them, the sample data is the comprehensive feature vector set K of healthy samples h The comprehensive feature vector in and the comprehensive feature vector set K of diseased samples d The comprehensive feature vector in ; S42: Input the sample data into the convolutional neural network, perform forward propagation calculation, and output the probability that each sample in the sample data belongs to the healthy category and the diseased category; S43: Calculate the loss value using the cross entropy loss function based on the output probability of the healthy category, the probability of the diseased category, and the true category label; S44: Based on the loss value, back propagation is performed from the output layer to calculate the gradient of the network parameters; wherein the network parameters include weight parameters and bias parameters; S45: Update the network parameters using the stochastic gradient descent optimization algorithm combined with the momentum term according to the gradient of the network parameters; S5: After each round of training, the convolutional neural network is applied to the validation set to calculate the accuracy, recall and F1 value on the validation set; the accuracy, recall and F1 value on the validation set are used as the model performance evaluation indicators; Among them, the accuracy rate is the ratio of correctly classified samples to the total number of samples in the validation set, the recall rate is the ratio of diseased samples that are correctly classified as diseased, and the F1 value is an indicator of the comprehensive accuracy and recall rate; S6: Evaluate the performance of the citrus Huanglongbing detection model according to the model performance evaluation indicators. If the performance of the citrus Huanglongbing detection model no longer improves, stop training in advance, save the model parameters with the best performance during the training process, and obtain a trained citrus Huanglongbing detection model.

[0088] Specifically, (1) for step S1: For the image collection of healthy citrus leaves, it can be carried out in citrus orchards or experimental planting areas where it is known that there is no citrus Huanglongbing. Use image acquisition equipment such as high-definition digital cameras to photograph healthy citrus leaves at different growth stages and positions under uniform lighting conditions (such as a specific time period under natural light, avoiding direct strong light or heavy shadows) and shooting angles (such as perpendicular to the leaf plane) to obtain visual images. At the same time, use professional biophysical imaging equipment, such as near-infrared imagers and fluorescence imagers, to image the same leaves in accordance with the equipment operating specifications to obtain biophysical images.

[0089] Image acquisition of citrus leaves with Huanglongbing can be carried out in orchards where citrus Huanglongbing has been confirmed to have occurred. The same image acquisition equipment is used to photograph leaves showing typical symptoms of Huanglongbing (such as yellowing, mottled leaves, swollen veins, etc.) to ensure that the acquired images can accurately reflect the characteristics of Huanglongbing leaves. The equipment and methods for acquiring biophysical images are the same as those for healthy leaves to obtain comparable biophysical information.

[0090] In order to ensure the accuracy and generalization ability of the training model, a sufficient number of samples need to be collected. Generally, the number of healthy and diseased samples should reach a certain scale and try to cover citrus leaves of different varieties and different growth environments to reduce the risk of overfitting the model to specific samples. For example, the number of samples in each category (healthy and diseased) can range from hundreds to thousands.

[0091] (2) For step S2: For each healthy and diseased citrus leaf sample, the visual image was first preprocessed (cropping, scaling, color space conversion, etc.) according to the method described above, and then the color feature vector, texture feature vector, and shape feature vector were extracted respectively. For biophysical images (thermal imaging images and fluorescence images), after corresponding preprocessing (temperature value conversion, background subtraction, contrast enhancement, etc.), the thermal imaging feature vector and fluorescence feature vector were extracted. Finally, these feature vectors were combined into a comprehensive feature vector.

[0092] For healthy samples, the comprehensive feature vectors extracted from multiple healthy citrus leaf samples are combined into a comprehensive feature vector set K of healthy samples. h =[M h1 , M h2 , … , M hn ], where M h1 To M hn Represents the comprehensive feature vector set K of healthy samples h The comprehensive feature vector in n represents the comprehensive feature vector set K of healthy samples. h The number of samples in .

[0093] Similarly, for diseased samples, we construct a comprehensive feature vector set K of diseased samples. d =[M d1 , M d2 , … ,M dm ], where M d1 To M dm The comprehensive feature vector set K representing the diseased samples d The comprehensive feature vector in, m represents the comprehensive feature vector set K of the diseased sample d The number of samples in .

[0094] (3) For step S3: The comprehensive feature vector set of healthy samples and diseased samples can be divided into training set and validation set by random partitioning. The common partitioning ratio is 70%-80% of the data is used for training, and 20%-30% of the data is used for validation. For example, 70% of the comprehensive feature vector set of healthy samples (assuming 0.7n samples) and 70% of the comprehensive feature vector set of diseased samples (assuming 0.7m samples) are combined as the training set, and the remaining 30% (0.3n healthy samples and 0.3m diseased samples) are used as the validation set. Such a partitioning can ensure that the training set has enough data for the model to learn feature relationships, and the validation set can evaluate the model performance during the model training process to prevent overfitting.

[0095] Before or after dividing the data set, the data can be further preprocessed, such as normalization. For the numerical features in the comprehensive feature vector (such as temperature, fluorescence intensity, etc.), they can be normalized to a specific range (such as [0, 1] or [-1, 1]) to make different features comparable, which is helpful for model training. At the same time, ensure that the data format of the training set and the validation set is consistent with the input requirements of the convolutional neural network, for example, convert the comprehensive feature vector into a tensor format suitable for the input layer to receive.

[0096] (4) For step S4: S41: The choice of batch size affects the efficiency and stability of model training. Smaller batch sizes (such as 32 or 64) may result in more frequent parameter updates during model training, but may converge more slowly; larger batch sizes (such as 256 or 512) can speed up training, but may require more memory resources and, in some cases, may cause the model to converge to a local minimum. The appropriate batch size can be selected based on factors such as dataset size and hardware resources (such as GPU memory). For example, if hardware resources permit, a larger batch size such as 128 or 256 can be selected for larger datasets (such as thousands of samples); a smaller batch size such as 32 or 64 can be selected for smaller datasets.

[0097] Randomly extract sample data and the corresponding true category labels from the training set according to the set batch size. For example, if the batch size is 64, then 64 comprehensive feature vectors (including comprehensive feature vectors of healthy samples and diseased samples) and their corresponding true category labels (0 for healthy and 1 for diseased) are randomly selected from the training set. This random extraction method ensures that each sample has a chance to be selected during the training process, avoiding the impact of data order on model training, and enabling the model to learn the diversity of data.

[0098] S42: Sample data is input into the input layer of the convolutional neural network in the form of tensors. In the network, the data is processed by the convolutional layer module, the pooling layer module, and the fully connected layer module in turn. In the convolutional layer, the convolution kernel extracts features through convolution operations, introduces nonlinearity through the activation function, and then is normalized through the batch normalization layer; the pooling layer reduces the data dimension through the maximum pooling operation and retains important features; the fully connected layer integrates the previously extracted features and further nonlinearly transforms them. Finally, in the output layer (the second fully connected layer), the output of the model is converted into the probability distribution of the sample belonging to the healthy category and the diseased category through the Softmax activation function. For example, a tensor with a shape of (64,2) is output (assuming the batch size is 64), where each row represents the probability of a sample belonging to the healthy and diseased categories.

[0099] S43: For each sample, the predicted probability of the healthy category is p i0 , the probability of the disease category is p i1 , the true category label is y i (0 or 1). According to the calculation formula of the cross entropy loss function L i =-[y i log(p i1 )+(1-y i )log(p i0)], calculate the loss value of each sample, and then average the loss values ​​of this batch of samples (assuming the batch size is K) to get the average loss value L of this batch of samples. The calculation formula is: ;

[0100] This loss value reflects the degree of difference between the model prediction result and the true label. The smaller the loss value, the more accurate the model prediction.

[0101] S44: The backpropagation algorithm starts from the output layer and calculates the gradient of the loss function with respect to the parameters of the previous layer (fully connected layer, pooling layer, convolutional layer, etc.) based on the partial derivatives of the loss function with respect to the parameters of the output layer (weights and biases). The gradient is passed back to the previous part of the network layer by layer through the chain rule. For example, for the weight parameter W of the output layer out , whose gradient It can be calculated based on the derivative of the loss function L and the Softmax activation function and the output of the previous layer; then, based on this gradient and the output of the previous layer (the first fully connected layer), the weight parameter W of the first fully connected layer is calculated. fc1 Gradient , and so on, until the gradients of the weight parameters and bias parameters of the convolutional layer are calculated. These gradient values ​​​​indicate the degree of influence of each parameter on the loss value.

[0102] S45: The update formula of the stochastic gradient descent (SGD) optimization algorithm combined with the momentum term is: ;

[0103] ;

[0104] where θ t is the network parameter (weight or bias) at the current time t, is the gradient of the loss function with respect to this parameter, η is the learning rate (the step size of the control parameter update), μ is the momentum coefficient (usually between 0 and 1, such as 0.9), and υ t is the momentum term at time t. The role of the momentum term is to accelerate the convergence of the model, make the parameter update direction more stable, and avoid drastic fluctuations in the gradient direction. By calculating the gradient of each parameter and combining the momentum term and the learning rate, the network parameters are updated, so that the model can adjust the parameters in the direction of reducing the loss function in subsequent training, and gradually learn more accurate feature representations and classification boundaries.

[0105] (5) For step S5: Accuracy refers to the ratio of correctly classified samples to the total number of samples in the validation set. Input the samples in the validation set into the trained convolutional neural network one by one to obtain the model's predicted category (healthy or sick) for each sample. Then compare the predicted category with the true category of the sample and count the number of correctly classified samples.

[0106] Recall refers to the proportion of diseased samples that are correctly classified as diseased. First, count the number of real diseased samples in the validation set N d , and then count the number of samples N that are correctly classified as diseased by the model cd , then the recall rate Recall=N d / N cd For example, if there are 40 real diseased samples in the validation set and 35 of them are correctly classified as diseased, the recall rate is 0.875 (or 87.5%). The recall rate reflects the model's ability to identify diseased samples. A high recall rate means that the model can detect as many diseased samples as possible and reduce missed diagnoses.

[0107] The F1 value is an indicator of comprehensive accuracy and recall, and its calculation formula is: ;

[0108] Precision is the accuracy rate (accuracy = number of samples correctly classified as diseased / number of samples predicted as diseased by the model). The F1 value can more comprehensively evaluate the performance of the model and strike a balance between accuracy and recall.

[0109] For example, if the accuracy is 0.85, the recall is 0.875, and the precision is 0.9 (assuming that 90% of the samples predicted by the model as diseased are correctly classified), then: F1=(2×0.85×0.875) / (0.85+0.875)≈0.862.

[0110] (6) For step S6: After each round of training, the calculated validation set accuracy reflects the model's ability to correctly classify data that was not involved in the training. If the accuracy continues to rise, it means that the model is constantly learning effective features and can more accurately determine whether citrus leaves are infected with Huanglongbing. For example, the initial accuracy may be low, and as the number of training rounds increases, the model gradually learns to distinguish the characteristic patterns of healthy and diseased leaves, and the accuracy gradually increases.

[0111] During the training process, the changes of these indicators are continuously monitored. If these indicators do not improve significantly after multiple rounds of training (such as 10 rounds) (such as the improvement in accuracy is less than 0.005, the improvement in recall is less than 0.005, and the improvement in F1 value is less than 0.005), the model performance is considered to be no longer improved. At this point, continuing training may lead to overfitting of the model, that is, the model is too adapted to the training data, and the generalization ability to new data is reduced.

[0112] When it is determined that the model performance will no longer improve, stop training in advance to avoid unnecessary waste of computing resources and the risk of overfitting. Then, from the model parameters recorded during the training process, select the model parameters with the best performance (such as the highest accuracy, the highest F1 value, or the best case after considering the accuracy and recall rate) and save them. These saved model parameters define the final trained citrus Huanglongbing detection model, which can perform efficient and accurate Huanglongbing detection on newly collected citrus leaves to be tested in actual citrus planting scenarios.

[0113] The present application provides a method for detecting citrus Huanglongbing based on visual images. The method collects visual images and biophysical images of citrus leaves by multiple means, uses multiple preprocessing methods to improve image quality and highlight feature information, extracts feature vectors in multiple dimensions including color, texture, shape, thermal imaging, fluorescence, etc., and then uses convolutional neural networks to build and train a detection model. The model is able to fully learn the characteristic differences between healthy and diseased citrus leaves, effectively integrates information from multiple aspects, and thus achieves the technical effect of accurately detecting citrus Huanglongbing, improves the accuracy, reliability and generalization ability of detection, provides strong technical support for the early diagnosis and prevention of citrus Huanglongbing, and helps to ensure the healthy development of the citrus industry.

[0114] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0115] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the above-mentioned method for detecting citrus Huanglongbing based on visual images. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more citrus Huanglongbing detection devices based on visual images provided below can be referred to the limitations of the above-mentioned method for detecting citrus Huanglongbing based on visual images, and will not be repeated here.

[0116] In an exemplary embodiment, Figure 2 As shown, a citrus Huanglongbing detection device 200 based on visual images is provided, comprising: The visual image acquisition module 201 is used to acquire the visual image of the citrus leaf to be detected, and perform a first preprocessing on the visual image of the citrus leaf to be detected to obtain a preprocessed visual image of the citrus leaf to be detected; wherein the first preprocessing includes cropping and scaling.

[0117] The visual feature extraction module 202 is used to extract visual features from the pre-processed visual image of the citrus leaf to be detected, and obtain a visual feature vector of the citrus leaf to be detected.

[0118] The biophysical image acquisition module 203 is used to acquire the biophysical image of the citrus leaves to be detected, and perform a second preprocessing on the biophysical image of the citrus leaves to be detected to obtain the preprocessed biophysical image of the citrus leaves to be detected; wherein the second preprocessing includes image conversion processing and image enhancement processing.

[0119] The biophysical feature extraction module 204 is used to extract biophysical features from the pre-processed biophysical image of the citrus leaf to be detected, and obtain the biophysical feature vector of the citrus leaf to be detected.

[0120] The feature combination module 205 is used to combine the visual feature vector and the biophysical feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector of the citrus leaf to be detected.

[0121] The Huanglongbing detection module 206 is used to input the comprehensive feature vector of the citrus leaves to be detected into the trained citrus Huanglongbing detection model to obtain the detection result.

[0122] Optionally, the biophysical image acquisition module 203 includes: The thermal imaging image acquisition unit is used to collect thermal imaging images of the citrus leaves to be detected, and obtain the temperature distribution image of the citrus leaves to be detected as the original thermal imaging image of the citrus leaves to be detected.

[0123] The fluorescence image acquisition unit is used to collect fluorescence images of the citrus leaves to be detected, and obtain the fluorescence images of the citrus leaves to be detected under the excitation of a specific wavelength as the original fluorescence images of the citrus leaves to be detected.

[0124] The biophysical image preprocessing unit is used to perform a third preprocessing on the original thermal imaging image of the citrus leaves to be detected to obtain a preprocessed thermal imaging image of the citrus leaves to be detected; and perform a fourth preprocessing on the original fluorescent image of the citrus leaves to be detected to obtain a preprocessed fluorescent image of the citrus leaves to be detected.

[0125] The image integration unit is used to use the preprocessed thermal imaging image and the preprocessed fluorescence image of the citrus leaves to be detected as the preprocessed biophysical image of the citrus leaves to be detected.

[0126] Among them, the second preprocessing includes the third preprocessing and the fourth preprocessing; the third preprocessing is: converting the original thermal imaging image into an actual temperature numerical matrix, removing abnormal temperature point data caused by equipment or environmental factors, cropping and scaling; the fourth preprocessing is: performing background subtraction processing on the original fluorescence image, removing the background fluorescence signal caused by the excitation light scattering factor, enhancing the contrast of the leaf fluorescence signal, cropping and scaling.

[0127] Optionally, the biophysical feature extraction module 204 includes: The thermal imaging feature extraction unit is used to calculate the average temperature T of the leaves of citrus with Huanglongbing according to the pre-processed thermal imaging image of the citrus leaves to be detected. avg , temperature variance T var and the temperature gradient T grad ; Combine the average temperature, temperature variance and temperature gradient into the thermal imaging feature vector H of the citrus leaves to be detected; where H=[T avg , T var , T grad ].

[0128] The fluorescence feature extraction unit is used to calculate the average fluorescence intensity F of the leaves of citrus with Huanglongbing according to the pre-processed fluorescence image of the citrus leaves to be detected. avg , peak fluorescence intensity F peak and the skewness F of the fluorescence intensity distribution skew , the average fluorescence intensity, the peak fluorescence intensity and the skewness of the fluorescence intensity distribution are combined into the fluorescence feature vector I of the citrus leaves to be detected; where I=[F avg , F peak , F skew ].

[0129] The biophysical feature integration unit is used to use the thermal imaging feature vector H and the fluorescence feature vector I of the citrus leaves to be detected as the biophysical feature vectors of the citrus leaves to be detected.

[0130] Optionally, the visual feature extraction module 202 includes: The color feature extraction unit is used to extract color features from the preprocessed visual image of the citrus leaves to be detected, and obtain a color feature vector of the citrus leaves to be detected.

[0131] The texture feature extraction unit is used to extract texture features from the preprocessed visual image of the citrus leaf to be detected, so as to obtain a texture feature vector of the citrus leaf to be detected.

[0132] The shape feature extraction unit is used to extract shape features from the preprocessed visual image of the citrus leaf to be detected, and obtain a shape feature vector of the citrus leaf to be detected.

[0133] The visual feature integration unit is used to use the color feature vector, texture feature vector and shape feature vector of the detected citrus as the visual feature vector of the citrus leaf to be detected.

[0134] Optionally, the feature combination module 205 is specifically used to: combine the thermal imaging feature vector, the fluorescence feature vector, the color feature vector, the texture feature vector and the shape feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector M=[C, B, D, H, I] of the citrus leaf to be detected.

[0135] Among them, C represents the color feature vector of the citrus leaf to be detected, B represents the texture feature vector of the citrus leaf to be detected, and D represents the shape feature vector of the citrus leaf to be detected.

[0136] Optionally, the Huanglongbing detection module 206 further includes a detection model building unit 2061 and a detection model training unit 2062. Before the comprehensive feature vector of the citrus leaves to be detected is input into the trained citrus Huanglongbing detection model, the following operations are performed respectively: The detection model building unit 2061 is used to build a citrus Huanglongbing detection model using a convolutional neural network.

[0137] The detection model training unit 2062 is used to train the citrus Huanglongbing detection model to obtain a trained citrus Huanglongbing detection model.

[0138] Among them, the structure of the convolutional neural network includes: An input layer, whose dimension matches the dimension of the comprehensive feature vector, is used to receive the color feature vector, texture feature vector, shape feature vector, thermal imaging feature vector and fluorescence feature vector of the citrus leaves to be detected; Convolutional layer module, including: The first convolution layer sets 32 convolution kernels, each with a size of 3×3 and a step size of 1, and uses the ReLU activation function; the first batch standardization layer is connected after the first convolution layer, and performs batch standardization on the data output by the first convolution layer; the second convolution layer sets 64 convolution kernels with a size of 3×3 and a step size of 1, and uses the ReLU activation function; the second batch standardization layer is connected after the second convolution layer, and performs batch standardization on the data output by the second convolution layer; the third convolution layer sets 128 convolution kernels with a size of 3×3 and a step size of 1, and uses the ReLU activation function; the third batch standardization layer is connected after the third convolution layer, and performs batch standardization on the data output by the third convolution layer.

[0139] Pooling layer module, including: The first pooling layer adopts the maximum pooling method, the pooling kernel size is 2×2, the step size is 2, and it is connected after the first batch of standardization layers; the second pooling layer adopts the maximum pooling method, the pooling kernel size is 2×2, the step size is 2, and it is connected after the second batch of standardization layers; the third pooling layer adopts the maximum pooling method, the pooling kernel size is 2×2, the step size is 2, and it is connected after the third batch of standardization layers.

[0140] Fully connected layer module, including: In the first fully connected layer, the data output by the third pooling layer is flattened and connected, 256 neurons are set, and the ReLU activation function is used; the Dropout layer is connected to the first fully connected layer, and the dropout rate is 0.5; the second fully connected layer is set with 2 neurons, corresponding to the healthy and diseased categories of citrus, and the Softmax activation function is used to output the probability that the sample belongs to the healthy and diseased categories.

[0141] Optionally, the detection model training unit 2062 is specifically configured to perform the following steps: S1: Obtain visual images and biophysical images of healthy citrus leaves, and obtain visual images and biophysical images of citrus leaves with Huanglongbing.

[0142] S2: Based on the visual images and biophysical images of healthy citrus leaves, the comprehensive feature vector set K of healthy samples is obtained h =[M h1 , M h2 , … , M hn ]; Based on the visual images and biophysical images of citrus leaves infected with Huanglongbing, the comprehensive feature vector set K of diseased samples was obtained. d =[M d1 , M d2 , … , M dm ]; where Mh1 To M hn Represents the comprehensive feature vector set K of healthy samples h The comprehensive feature vector in M d1 To M dm The comprehensive feature vector set K representing the diseased samples d The comprehensive feature vector in n represents the comprehensive feature vector set K of healthy samples. h The number of samples in, m represents the comprehensive feature vector set K of diseased samples d The number of samples in .

[0143] S3: Comprehensive feature vector set K based on healthy samples h and the comprehensive feature vector set K of diseased samples d , and obtain the training set and the validation set.

[0144] S4: The citrus Huanglongbing detection model is trained cyclically using the training set and the validation set, wherein each round of training includes: S41: Extract a batch of sample data and the corresponding true category labels from the training set according to the preset batch size. The sample data is the comprehensive feature vector set K of the healthy sample. h The comprehensive feature vector in and the comprehensive feature vector set K of diseased samples d The comprehensive feature vector in .

[0145] S42: Input the sample data into the convolutional neural network, perform forward propagation calculation, and output the probability that each sample in the sample data belongs to the healthy category and the diseased category.

[0146] S43: According to the output probability of the healthy category, the probability of the diseased category and the true category label, the loss value is calculated using the cross entropy loss function.

[0147] S44: Based on the loss value, back propagation is performed from the output layer to calculate the gradient of the network parameters; wherein the network parameters include weight parameters and bias parameters.

[0148] S45: According to the gradient of the network parameters, the network parameters are updated using the stochastic gradient descent optimization algorithm combined with the momentum term.

[0149] S5: After each round of training, the convolutional neural network is applied to the validation set to calculate the accuracy, recall and F1 value on the validation set; the accuracy, recall and F1 value on the validation set are used as indicators for evaluating model performance. The accuracy is the ratio of correctly classified samples to the total number of validation set samples, the recall is the ratio of diseased samples that are correctly classified as diseased, and the F1 value is an indicator of the comprehensive accuracy and recall. S6: Evaluate the performance of the citrus Huanglongbing detection model according to the model performance evaluation indicators. If the performance of the citrus Huanglongbing detection model no longer improves, stop training in advance, save the model parameters with the best performance during the training process, and obtain a trained citrus Huanglongbing detection model.

[0150] An embodiment of the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0151] The embodiments of the present application further provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0152] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0153] The above-mentioned embodiments only express several implementation methods of the embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the embodiments of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the embodiments of the present application, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A citrus Huanglongbing detection method based on visual images, characterized in that: The method comprises: Acquire a visual image of a citrus leaf to be detected, and perform a first preprocessing on the visual image of the citrus leaf to be detected to obtain a preprocessed visual image of the citrus leaf to be detected; wherein the first preprocessing includes cropping and scaling; Extracting visual features from the preprocessed visual image of the citrus leaf to be detected to obtain a visual feature vector of the citrus leaf to be detected; Acquire the biophysical image of the citrus leaf to be detected, and perform a second preprocessing on the biophysical image of the citrus leaf to be detected to obtain a preprocessed biophysical image of the citrus leaf to be detected; wherein the second preprocessing includes image conversion processing and image enhancement processing; Extracting biophysical features of the preprocessed biophysical image of the citrus leaf to be detected to obtain a biophysical feature vector of the citrus leaf to be detected; Combining the visual feature vector and the biophysical feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector of the citrus leaf to be detected; The comprehensive feature vector of the citrus leaves to be detected is input into a trained citrus Huanglongbing detection model to obtain a detection result.

2. The method according to claim 1, characterized in that The step of acquiring the biophysical image of the citrus leaf to be detected and performing a second preprocessing on the biophysical image of the citrus leaf to be detected to obtain the preprocessed biophysical image of the citrus leaf to be detected includes: Performing thermal imaging image acquisition on the citrus leaves to be detected, obtaining a temperature distribution image of the citrus leaves to be detected as an original thermal imaging image of the citrus leaves to be detected; Collecting a fluorescence image of the citrus leaf to be detected, and obtaining a fluorescence image of the citrus leaf to be detected under excitation of a specific wavelength as an original fluorescence image of the citrus leaf to be detected; Performing a third preprocessing on the original thermal imaging image of the citrus leaves to be detected to obtain a preprocessed thermal imaging image of the citrus leaves to be detected; performing a fourth preprocessing on the original fluorescent image of the citrus leaves to be detected to obtain a preprocessed fluorescent image of the citrus leaves to be detected; Using the preprocessed thermal imaging image and the preprocessed fluorescence image of the citrus leaf to be detected as the preprocessed biophysical image of the citrus leaf to be detected; Wherein, the second preprocessing includes the third preprocessing and the fourth preprocessing; The third preprocessing includes: converting the original thermal imaging image into an actual temperature value matrix, removing abnormal temperature point data caused by equipment or environmental factors, and cropping and scaling; The fourth preprocessing includes: performing background subtraction processing on the original fluorescence image, removing background fluorescence signals generated by excitation light scattering factors, enhancing the contrast of leaf fluorescence signals, cropping and scaling.

3. The method according to claim 2, characterized in that The step of extracting biophysical features from the preprocessed biophysical image of the citrus leaf to be detected to obtain the biophysical feature vector of the citrus leaf to be detected includes: According to the pre-processed thermal imaging image of the citrus leaves to be detected, the average temperature T of the leaves of the citrus with Huanglongbing is calculated. avg , temperature variance T var and the temperature gradient T grad ; Combining the average temperature, the temperature variance and the temperature gradient into the thermal imaging feature vector H of the citrus leaves to be detected; wherein H=[T avg , T var , T grad ]; According to the pre-processed fluorescence image of the citrus leaves to be detected, the average fluorescence intensity F of the leaves of the citrus with Huanglongbing is calculated. avg , peak fluorescence intensity F peak and the skewness F of the fluorescence intensity distribution skew , the average fluorescence intensity, the peak fluorescence intensity and the skewness of the fluorescence intensity distribution are combined into the fluorescence feature vector I of the citrus leaves to be detected; wherein, I=[F avg , F peak , F skew ]; The thermal imaging feature vector H and the fluorescence feature vector I of the citrus leaf to be detected are used as the biophysical feature vectors of the citrus leaf to be detected.

4. The method according to claim 3, characterized in that The step of extracting visual features from the preprocessed visual image of the citrus leaf to be detected to obtain a visual feature vector of the citrus leaf to be detected includes: Extracting color features from the preprocessed visual image of the citrus leaves to be detected to obtain a color feature vector of the citrus leaves to be detected; Extracting texture features of the preprocessed visual image of the citrus leaf to be detected to obtain a texture feature vector of the citrus leaf to be detected; Performing shape feature extraction on the preprocessed visual image of the citrus leaf to be detected to obtain a shape feature vector of the citrus leaf to be detected; The color feature vector, texture feature vector and shape feature vector of the detected citrus are used as the visual feature vector of the citrus leaf to be detected.

5. The method according to claim 4, characterized in that The step of combining the visual feature vector and the biophysical feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector of the citrus leaf to be detected comprises: The thermal imaging feature vector, the fluorescence feature vector, the color feature vector, the texture feature vector and the shape feature vector of the citrus leaf to be detected are combined to obtain a comprehensive feature vector M=[C, B, D, H, I] of the citrus leaf to be detected; Among them, C represents the color feature vector of the citrus leaf to be detected, B represents the texture feature vector of the citrus leaf to be detected, and D represents the shape feature vector of the citrus leaf to be detected.

6. The method according to claim 5, characterized in that Before inputting the comprehensive feature vector of the citrus leaves to be detected into the trained citrus Huanglongbing detection model to obtain the detection result, the method further includes: Constructing the citrus Huanglongbing detection model using a convolutional neural network; Training the citrus Huanglongbing detection model to obtain the trained citrus Huanglongbing detection model; Wherein, the structure of the convolutional neural network includes: An input layer, whose dimension matches the dimension of the comprehensive feature vector, and is used to receive the color feature vector, the texture feature vector, the shape feature vector, the thermal imaging feature vector and the fluorescence feature vector of the citrus leaf to be detected; Convolutional layer module, including: In the first convolution layer, 32 convolution kernels are set, each of which has a size of 3×3, a step size of 1, and a ReLU activation function; A first batch normalization layer, connected after the first convolutional layer, performs batch normalization processing on the data output by the first convolutional layer; In the second convolution layer, 64 convolution kernels are set, the kernel size is 3×3, the step size is 1, and the ReLU activation function is used; A second batch normalization layer, connected to the second convolutional layer, performs batch normalization on the data output by the second convolutional layer; In the third convolution layer, 128 convolution kernels are set, the kernel size is 3×3, the step size is 1, and the ReLU activation function is used; A third batch normalization layer, connected to the third convolutional layer, performs batch normalization on the data output by the third convolutional layer; Pooling layer module, including: The first pooling layer uses the maximum pooling method, the pooling kernel size is 2×2, the step size is 2, and it is connected after the first batch of normalization layers; The second pooling layer uses the maximum pooling method, with a pooling kernel size of 2×2 and a step size of 2, and is connected after the second batch of normalization layers; The third pooling layer adopts the maximum pooling method, the pooling kernel size is 2×2, the step size is 2, and it is connected after the third batch normalization layer; Fully connected layer module, including: The first fully connected layer flattens the data output by the third pooling layer and then connects them, sets 256 neurons, and adopts the ReLU activation function; Dropout layer, connected after the first fully connected layer, with a dropout rate of 0.5; In the second fully connected layer, two neurons are set, corresponding to the healthy and diseased categories of citrus fruits. The Softmax activation function is used to output the probability that the sample belongs to the healthy and diseased categories.

7. The method according to claim 6, characterized in that The training of the citrus Huanglongbing detection model comprises: S1: Obtain visual images and biophysical images of healthy citrus leaves, and obtain visual images and biophysical images of citrus leaves with Huanglongbing; S2: Based on the visual image and biophysical image of the healthy citrus leaves, a comprehensive feature vector set K of healthy samples is obtained h =[M h1 , M h2 , … , M hn ]; Based on the visual image and biophysical image of the Huanglongbing citrus leaves, a comprehensive feature vector set K of the diseased samples is obtained d =[M d1 , M d2 , … , M dm ]; where M h1 To M hn Represents the comprehensive feature vector set K of healthy samples h The comprehensive feature vector in M d1 To M dm The comprehensive feature vector set K representing the diseased samples d The comprehensive feature vector in n represents the comprehensive feature vector set K of the healthy sample h The number of samples in, m represents the comprehensive feature vector set K of the diseased samples d The number of samples in ; S3: Comprehensive feature vector set K based on the healthy sample h and the comprehensive feature vector set K of the diseased sample d , get the training set and the validation set; S4: cyclically training the citrus Huanglongbing detection model using the training set and the validation set, wherein each round of training includes: S41: extracting a batch of sample data and corresponding true category labels from the training set according to a preset batch size; The sample data is the comprehensive feature vector set K of healthy samples. h The comprehensive feature vector in and the comprehensive feature vector set K of diseased samples d The comprehensive feature vector in ; S42: Input the sample data into the convolutional neural network, perform forward propagation calculation, and output the probability that each sample in the sample data belongs to the healthy category and the diseased category; S43: Calculate a loss value using a cross entropy loss function according to the output probability of the healthy category, the probability of the diseased category, and the true category label; S44: Based on the loss value, back propagation is performed from the output layer to calculate the gradient of the network parameters; wherein the network parameters include weight parameters and bias parameters; S45: updating the network parameters according to the gradient of the network parameters by using a stochastic gradient descent optimization algorithm combined with a momentum term; S5: After each round of training, the convolutional neural network is applied to the validation set, and the accuracy, recall and F1 value on the validation set are calculated; the accuracy, recall and F1 value on the validation set are used as model performance evaluation indicators; The accuracy rate is the ratio of the number of correctly classified samples to the total number of samples in the validation set, the recall rate is the ratio of the diseased samples that are correctly classified as diseased, and the F1 value is an indicator that combines the accuracy rate and the recall rate; S6: Evaluate the performance of the citrus Huanglongbing detection model according to the model performance evaluation index. If the performance of the citrus Huanglongbing detection model no longer improves, stop training in advance, save the model parameters with the best performance during the training process, and obtain the trained citrus Huanglongbing detection model.

8. A citrus Huanglongbing detection device based on visual images, characterized in that: The device comprises: A visual image acquisition module, used to acquire a visual image of a citrus leaf to be detected, and perform a first preprocessing on the visual image of the citrus leaf to be detected to obtain a preprocessed visual image of the citrus leaf to be detected; wherein the first preprocessing includes cropping and scaling; A visual feature extraction module, used for extracting visual features from the preprocessed visual image of the citrus leaf to be detected, to obtain a visual feature vector of the citrus leaf to be detected; A biophysical image acquisition module, used to acquire the biophysical image of the citrus leaf to be detected, and perform a second preprocessing on the biophysical image of the citrus leaf to be detected to obtain a preprocessed biophysical image of the citrus leaf to be detected; wherein the second preprocessing includes image conversion processing and image enhancement processing; A biophysical feature extraction module is used to extract biophysical features from the preprocessed biophysical image of the citrus leaf to be detected, so as to obtain a biophysical feature vector of the citrus leaf to be detected; A feature combination module, used for combining the visual feature vector and the biophysical feature vector of the citrus leaf to be detected to obtain a comprehensive feature vector of the citrus leaf to be detected; The Huanglongbing detection module is used to input the comprehensive feature vector of the citrus leaves to be detected into the trained citrus Huanglongbing detection model to obtain the detection result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images