Pest detection and management system based on machine vision and AR

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a pest detection and management system based on machine vision and AR. The system includes: an image acquisition part, which is used to acquire super-resolution images and spectral images of target crops; image preprocessing is performed on the super-resolution images and spectral images respectively; an image area extraction part, which is used to identify the leaf area and stem area of ​​the target crop from the preprocessed super-resolution image; a pest judgment part, which is used to perform spectral analysis on the leaf area and stem area in the spectral image; a pest analysis part, which is used to extract the leaf area from the preprocessed super-resolution image when the judgment result is true; an AR display part, which is used to display the pest area of ​​the target crop in the AR device. The present invention realizes automatic identification and real-time monitoring of crop pests and diseases. Combined with advanced image processing and machine learning algorithms, it can quickly and accurately extract features and analyze pest and disease conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an insect pest detection and management system based on machine vision and AR. Background Art

[0002] The prevention and control of crop diseases and pests has always been one of the important issues in agricultural production. The frequent occurrence of diseases and pests will not only lead to reduced crop yields and quality decline, but may also damage the ecological environment and pose a threat to farmers' economic benefits and livelihood security. Therefore, effective monitoring and management of crop diseases and pests has become one of the urgent problems to be solved in agricultural production. At present, traditional pest and disease monitoring and management methods mainly rely on manual inspections and sampling, which have problems such as low efficiency, high cost, and long time. Manual inspections require a lot of manpower and material resources, with a large workload and a long cycle. It is difficult to detect the occurrence and spread of pests and diseases in time, which can easily cause losses. In addition, due to the complexity of the crop growth environment and the diversity of pests and diseases, traditional manual inspection methods often find it difficult to accurately identify different types of pests and diseases, resulting in inaccurate formulation of prevention and control measures.

[0003] Traditional pest and disease detection methods mainly rely on manual observation and empirical judgment. For example, farmers or professionals observe the appearance of plant leaves and stems to determine whether they are affected by pests and diseases. The main problem with this method is that manual observation takes a lot of time and energy, and is affected by the subjective factors of the observer, resulting in inaccurate judgment and missed detection. In addition, manual observation can only detect surface damage caused by pests and diseases, and it is difficult to find early signs of pests and diseases, resulting in delayed governance. In recent years, with the development of computer vision, machine learning and augmented reality technology, some pest and disease detection technologies based on image processing and intelligent recognition have been gradually applied to agricultural production. These technologies use digital image processing technology to analyze and identify plant images, which can realize automatic detection and diagnosis of plant pests and diseases, greatly improving detection efficiency and accuracy. However, the existing pest and disease detection technologies based on image processing and intelligent recognition still have some problems and challenges. First, due to the complexity of the plant growth environment and the instability of lighting conditions, the image acquisition quality may be affected, resulting in unclear or blurred pest and disease features in the image. Second, traditional image processing algorithms may have limitations when processing complex plant images, and it is difficult to accurately extract the characteristic information of pests and diseases. In addition, existing pest and disease detection technologies can often only detect known types of pests and diseases, and cannot effectively identify new or unknown pests and diseases, and lack versatility and flexibility. Summary of the invention

[0004] The main purpose of this invention is to provide a pest detection and management system based on machine vision and AR, which realizes the automatic identification and real-time monitoring of crop pests and diseases. Combined with advanced image processing and machine learning algorithms, it can quickly and accurately extract features and analyze pest and disease conditions.

[0005] To solve the above technical problems, the present invention provides a pest detection and management system based on machine vision and AR, the system comprising: an image acquisition part, used to acquire a super-resolution image and a spectral image of a target crop, the super-resolution image and the spectral image are of equal size, and the relative positions of the crops in the super-resolution image and the spectral image are consistent; image preprocessing is performed on the super-resolution image and the spectral image respectively to obtain a preprocessed super-resolution image and a preprocessed spectral image; an image region extraction part, used to identify the leaf region and the stem region of the target crop from the preprocessed super-resolution image, and thereby extract the leaf region and the stem region of the target crop from the preprocessed spectral image; a pest judgment part, used to perform spectral analysis on the leaf region and the stem region in the spectral image, judge whether the target crop has pests and diseases, and obtain a judgment result; a pest analysis part, used to perform pest and disease identification in the leaf region and the stem region in the preprocessed super-resolution image when the judgment result is true, and obtain the pest and disease region in the leaf region and the stem region; an AR display part, used to display the pest and disease region of the target crop in an AR device.

[0006] Furthermore, the method of performing image preprocessing on the super-resolution image to obtain the preprocessed super-resolution image includes: performing bidirectional wavelet decomposition on the super-resolution image by using a pair of orthogonal wavelet filters to obtain low-frequency sub-bands in the horizontal and vertical directions, high-frequency sub-bands in the horizontal direction, high-frequency sub-bands in the vertical direction, and high-frequency sub-bands in the diagonal direction; using a soft threshold processing method based on wavelet coefficients to denoise the high-frequency sub-bands in the horizontal direction, the high-frequency sub-bands in the vertical direction, and the high-frequency sub-bands in the diagonal direction; and then reconstructing the image using the low-frequency sub-bands in the horizontal and vertical directions, the high-frequency sub-bands in the horizontal direction, the high-frequency sub-bands in the vertical direction, and the high-frequency sub-bands in the diagonal direction to obtain the preprocessed super-resolution image.

[0007] Furthermore, the spectral image is preprocessed to obtain a method for a preprocessed spectral image, including: normalizing the pixel values ​​of the spectral image to [-1,1]; converting the normalized spectral image into a data matrix, in which each column represents a pixel point and each row represents a band; performing low-rank matrix decomposition on the data matrix; setting the number of main singular values ​​to r, and setting the singular values ​​other than the main singular values ​​to zero to achieve low-rank approximation and complete truncated singular values; using the singular values ​​after truncated singular values ​​to reconstruct the data matrix, thereby obtaining the preprocessed spectral image.

[0008] Furthermore, in the image region extraction part, the method for identifying the leaf region and stem region of the target crop from the pre-processed super-resolution image includes: obtaining a labeled sample, extracting each labeled sample K i The labels form a label vector:

[0009] B={B1,B2,...,B N},

[0010] Among them B i is the label of the i-th labeled sample; N is the number of labeled samples; using the conditional Bayes method, a classifier is trained based on the labeled samples (K, B) and the sparse representation s of the labeled samples is transformed into i and the corresponding label y i Establish association; the labeled sample is a small image with leaf and stem labels; for each labeled sample K i , calculate its sparse representation s i ; Divide the preprocessed super-resolution image into small blocks of size n×n, expressed as:

[0011] X={x1,x2,...,x N},

[0012] where x i is the i-th small block; a dictionary matrix D is constructed using the sparse coding method, where each column of the dictionary represents a feature; the size of the dictionary is m×k, where m is the number of features in the dictionary and k is the dimension of the feature; for each small block x i Perform sparse coding and find its sparse representation vector a i , so that x i ≈Da i ; According to the trained conditional Bayes classifier, calculate each small block x i The conditional probability P(y) of belonging to leaves and stems i |a i ); According to Bayes’ theorem, calculate each small block x i The posterior probability P(y) of belonging to leaves and stems i |x i ); according to the posterior probability, each small block is thresholded to determine whether it belongs to the category of leaf or stem; adjacent small blocks with the same label are merged to form leaf area and stem area.

[0013] Furthermore, x is made to i ≈Da i :

[0014]

[0015] Among them, a iis a sparse representation vector, which represents the sparse representation of the i-th small block; λ is a sparsity parameter, which controls the strength of sparsity; a ij represents the sparse representation vector a i The j-th element of ;∈ is a smooth term.

[0016] Furthermore, in the pest judgment part, spectral analysis is performed on the leaf area and stem area in the spectral image to judge whether the target crops have pests and diseases. The method for obtaining the judgment result includes: performing spectral analysis on the leaf area and stem area in the spectral image to extract the chlorophyll absorption characteristic peak, and extracting the intensity value of the chlorophyll absorption characteristic peak; setting the leaf area characteristic peak threshold range and the stem area characteristic peak threshold range; respectively comparing the intensity value of the chlorophyll absorption characteristic peak of the leaf area and the intensity value of the chlorophyll absorption characteristic peak of the stem area with the leaf area characteristic peak threshold range and the stem area characteristic peak threshold range. If any one of the two is not within the corresponding set threshold range, the judgment result is true; otherwise, the judgment result is false.

[0017] Furthermore, the method of performing spectral analysis on the leaf area and stem area in the spectral image to extract the chlorophyll absorption characteristic peak includes: using a Gaussian smoothing algorithm to smooth the leaf area and stem area in the spectral image to make the characteristic peak clearer, and performing baseline correction to improve the signal-to-noise ratio; using a first-order derivative method to detect the chlorophyll absorption characteristic peak in the leaf area and stem area in the spectral image, that is, by taking the first-order derivative of the spectrum, finding a change rate exceeding a set change value as the chlorophyll absorption characteristic peak; using a local maximum point detection method to locate the position of the chlorophyll absorption characteristic peak, that is, finding a local maximum point on the first-order derivative curve and using it as the position of the chlorophyll absorption characteristic peak; according to the position of the chlorophyll absorption characteristic peak, extracting the intensity value of the chlorophyll absorption characteristic peak.

[0018] Furthermore, in the pest analysis part, when the judgment result is true, pests and diseases are identified from the leaf area and the stem area, and the method for obtaining the pest and disease areas in the leaf area and the stem area includes: using an image instance segmentation algorithm to segment the leaves and stems in the image, and generating a binary mask image, wherein the leaf area and the stem area are respectively marked with different pixel values; for the leaf area and the stem area, using a grayscale co-occurrence matrix to extract their respective feature vectors; using a machine learning or deep learning method to establish a classifier, and mapping the extracted feature vectors to pest and disease categories; for the segmented leaf area and stem area, using a trained classifier to identify pests and diseases for each pixel or area; according to the output of the classifier, marking the area identified as pests and diseases; using an image connectivity analysis method to remove small connected areas, merge the marked pest and disease areas together, and obtain the pest and disease areas in the leaf area and the stem area.

[0019] Furthermore, the AR device is a mobile phone, a tablet, a head-mounted display or a display.

[0020] The pest detection and management system based on machine vision and AR of the present invention has the following beneficial effects:

[0021] First, the present invention uses machine vision technology to realize automatic recognition and analysis of farmland images, which can quickly and accurately extract the characteristic information of crops, including leaf area and stem area, etc., laying the foundation for subsequent pest and disease analysis and management. Compared with traditional manual inspections, machine vision technology has the advantages of high efficiency, speed and precision, which can greatly improve the efficiency and accuracy of pest and disease monitoring.

[0022] Secondly, the present invention uses augmented reality technology to superimpose pest and disease information on the real farmland in a virtual form, providing farmers with an intuitive and vivid perception experience. Through AR devices such as mobile phones, tablets, head-mounted displays or monitors, farmers can intuitively observe the health status of crops and the distribution of pests and diseases in real scenes, helping them to take corresponding prevention and control measures in a timely manner.

[0023] In addition, the present invention uses advanced image processing and analysis technologies such as bidirectional wavelet decomposition and spectral analysis, which can perform efficient and accurate preprocessing and feature extraction on farmland images and spectral images, providing reliable data support for the identification and analysis of pests and diseases. By optimizing the image processing algorithm and spectral analysis method, the present invention can overcome the problems of uneven lighting conditions and background interference in traditional methods, and improve the identification accuracy and stability of pests and diseases.

[0024] In addition, the present invention also adopts advanced machine learning algorithms such as conditional Bayesian method and sparse coding technology, which can realize automatic identification and classification of pests and diseases, and provide farmers with convenient and efficient pest and disease management tools. By establishing a classifier and optimizing the recognition algorithm, the present invention can quickly and accurately mark the pest and disease areas, and provide farmers with timely and effective prevention and control suggestions. Finally, the present invention combines image analysis technologies such as image instance segmentation and grayscale co-occurrence matrix, which can carry out refined identification and segmentation of pest and disease areas, provide farmers with more accurate pest and disease distribution information, and help them formulate targeted prevention and control strategies. By analyzing the morphological characteristics and grayscale distribution of pest and disease areas, the present invention can help farmers better understand the severity and spread trend of pests and diseases, so as to take effective measures to control the occurrence of pests and diseases in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0026] Figure 1 A schematic diagram of the system structure of a pest detection and management system based on machine vision and AR provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The method of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

[0028] Example 1: Reference Figure 1, a pest detection and management system based on machine vision and AR, the system comprising: an image acquisition part, used to acquire a super-resolution image and a spectral image of a target crop, the super-resolution image and the spectral image are of equal size, and the relative position of the crop in the super-resolution image and the spectral image is consistent; image preprocessing is performed on the super-resolution image and the spectral image respectively to obtain a preprocessed super-resolution image and a preprocessed spectral image; an image region extraction part, used to identify the leaf region and the stem region of the target crop from the preprocessed super-resolution image, and thereby extract the leaf region and the stem region of the target crop from the preprocessed spectral image; a pest judgment part, used to perform spectral analysis on the leaf region and the stem region in the spectral image, judge whether the target crop has pests and diseases, and obtain a judgment result; a pest analysis part, used to perform pest and disease identification in the leaf region and the stem region in the preprocessed super-resolution image when the judgment result is true, and obtain the pest and disease region in the leaf region and the stem region; an AR display part, used to display the pest and disease region of the target crop in an AR device.

[0029] Specifically, super-resolution images refer to images with higher resolution than ordinary images. They can provide more detailed information and help to observe the growth status of crops more carefully. Spectral images are images obtained by recording spectral reflection or absorption information of different wavelengths. They can provide spectral characteristics of crops in different bands, which helps to analyze the physiological condition and health status of crops. In the image acquisition part, the target farmland needs to be photographed or scanned using the corresponding equipment or sensors to obtain images containing target crops. For super-resolution images, high-resolution cameras or professional imaging equipment may be used to capture images to ensure image quality and details. For spectral images, sensors or spectrometers with different wavelength ranges are needed to capture spectral information in different bands. Once these raw images are acquired, preprocessing may be required to improve image quality and accuracy. The preprocessing steps may include removing noise from the image, adjusting the brightness and contrast of the image, correcting the color deviation of the image, etc. This ensures that subsequent image processing and analysis can be performed based on high-quality input data. During the image acquisition process, it is also necessary to pay attention to maintaining the consistency of super-resolution images and spectral images, that is, their sizes are equal and the relative positions of crops in the two images are consistent. This is important because subsequent image processing and analysis require accurate positioning and alignment based on this consistency.

[0030] In the workflow of the image region extraction part, the first task is to analyze the super-resolution image. This usually involves image segmentation technology, the purpose of which is to segment the image into different regions in order to identify the leaves and stems of the target crop. Common image segmentation methods include threshold segmentation, edge detection, and region growing. Through these methods, different pixels in the image can be grouped to form regions with clear boundaries, so as to achieve the distinction between leaves and stems. Secondly, in the segmented regions, the leaves and stems of the target crop need to be further identified. This step usually uses morphological processing and feature extraction methods. Morphological processing can help remove noise and unnecessary details in the image, thereby extracting clearer leaf and stem contours. At the same time, by extracting the features of leaves and stems, such as shape, texture, and color, their recognition and distinction can be further enhanced. These features can be calculated and described by various algorithms and techniques, such as gray level co-occurrence matrix, local binary pattern, and color histogram. Next, the corresponding region extraction is performed on the pre-processed spectral image. Since the spectral image provides information about the spectral characteristics of crops, in this step, the spectral features can be used to further determine the location of leaves and stems. Spectral analysis can help identify areas with specific spectral features, thereby locating and extracting different parts of crops. Finally, by establishing a correspondence between the leaf and stem areas extracted from the super-resolution image and the spectral image, the consistency and accuracy between them can be ensured. In this way, it is possible to accurately extract the leaf and stem areas corresponding to the super-resolution image in the spectral image, providing a reliable data basis for subsequent pest judgment and analysis.

[0031] Spectral analysis is the basis of pest judgment. Spectral images provide reflective spectral information of crops at different wavelengths, which is of great significance for the detection of pests and diseases. Spectral analysis involves extracting and analyzing the spectral information of each pixel in the image to obtain the reflective spectral curves of crops at different wavelengths. These spectral curves can reflect the physiological state of crop tissues, such as chlorophyll content, water content, and leaf structure. By comparing the differences in spectral curves between infected areas and healthy areas, the characteristic spectral signals of infected areas can be found, thereby realizing the detection and judgment of pests and diseases. Secondly, based on the results of spectral analysis, techniques such as machine learning and pattern recognition can be used to judge pests and diseases. This step usually involves building a classification model, which can automatically learn the spectral features under different pest and disease states by training the spectral data of known samples. Once the model training is completed, it can be applied to new spectral data to judge pests and diseases based on its characteristics. Commonly used classification algorithms include support vector machines, random forests, and convolutional neural networks, which can effectively classify and identify complex spectral data, thereby realizing automatic judgment of pests and diseases. In the workflow of pest judgment, some issues need to be considered, such as the influence of lighting conditions, interference from environmental noise, and preprocessing of spectral data. To address these issues, spectral correction, noise removal, and data standardization techniques can be used to improve the accuracy and stability of judgment. In addition, image processing technology can be combined to match spectral analysis results with leaf and stem areas in super-resolution images to obtain more accurate pest judgment results.

[0032] The pest analysis part starts with preprocessing the leaf and stem regions in the super-resolution image. The extraction of these regions usually involves image segmentation and object detection techniques. Image segmentation aims to segment the image into different regions, while object detection is to detect the location and boundaries of specific targets (such as leaves and stems). Common image segmentation algorithms include threshold segmentation, edge detection and region growing, while object detection can be achieved through deep learning methods such as convolutional neural networks (CNN). Secondly, once the leaf and stem regions are extracted, the next step is to identify pests and diseases. This process usually includes two steps: feature extraction and classification recognition. Feature extraction aims to extract features that can characterize pests and diseases from the image, such as shape, color, texture, etc. These features can be extracted through various image processing techniques and feature descriptors, such as gray level co-occurrence matrix, local binary pattern and oriented gradient histogram. Once the features are extracted, they can be classified and identified using classifiers. Commonly used classifiers include support vector machines, random forests and deep learning networks, which can compare the extracted features with known pest and disease categories to achieve pest and disease recognition and classification. In the pest analysis workflow, some issues need to be considered, such as the impact of image quality, the diversity of pest types, and the accuracy of data annotation. To address these issues, image enhancement technology, multi-classifier fusion, and data enhancement can be used to improve the accuracy and stability of the analysis. In addition, expert knowledge and field observations can be combined to provide feedback and adjustments to the system to further improve its performance and reliability.

[0033] The workflow of the AR display part starts with the location of the pest area. In the previous pest analysis part, the pest areas in the leaves and stems of the crops have been determined. The next task is to convert the location information of these areas into a coordinate system that the AR device can understand. This process usually involves coordinate conversion and spatial positioning technology. By converting the pixel coordinates of the pest area in the image into the corresponding position in the virtual coordinate system of the AR device, the pest area can be accurately located in the AR device. Secondly, once the location information of the pest area is determined, the next step is to display these areas in real time in the AR device. This process usually involves computer graphics and augmented reality technology. By superimposing the location information of the pest area with the camera image of the AR device, the pest area can be dynamically displayed in the actual scene. In this way, users can directly observe the pest situation of crops through AR devices without relying on traditional images or text reports. In the workflow of the AR display part, some issues need to be considered, such as the compatibility and performance requirements of AR devices, the display effect and interaction method of the pest area, etc. To address these issues, the AR development platform and engine can be used to realize the dynamic display of the pest area, and the user interface design and interaction design can be combined to improve the user experience and operation convenience. In addition, virtual reality technology and augmented reality technology can be combined to achieve a richer and more immersive pest display effect, thereby further improving user participation and usage effects.

[0034] Embodiment 2: A method for performing image preprocessing on a super-resolution image to obtain a preprocessed super-resolution image includes: performing bidirectional wavelet decomposition on the super-resolution image by using a pair of orthogonal wavelet filters to obtain low-frequency sub-bands in the horizontal and vertical directions, high-frequency sub-bands in the horizontal direction, high-frequency sub-bands in the vertical direction, and high-frequency sub-bands in the diagonal direction; using a soft threshold processing method based on wavelet coefficients to denoise the high-frequency sub-bands in the horizontal direction, the high-frequency sub-bands in the vertical direction, and the high-frequency sub-bands in the diagonal direction; and then reconstructing the image using the low-frequency sub-bands in the horizontal and vertical directions, the high-frequency sub-bands in the horizontal direction, the high-frequency sub-bands in the vertical direction, and the high-frequency sub-bands in the diagonal direction to obtain the preprocessed super-resolution image.

[0035] Specifically, first, bidirectional wavelet decomposition is performed. Wavelet transform is a signal decomposition and analysis method that can decompose a signal into components of different frequencies and scales. In this example, a pair of orthogonal wavelet filters is used to perform bidirectional wavelet decomposition on the super-resolution image. This pair of filters includes a low-pass filter and a high-pass filter, which are used to extract low-frequency information and high-frequency information in the image, respectively. Through bidirectional wavelet decomposition, the image is decomposed into low-frequency subbands in the horizontal and vertical directions, high-frequency subbands in the horizontal direction, high-frequency subbands in the vertical direction, and high-frequency subbands in the diagonal direction. Next, soft threshold processing is performed. In this step, the high-frequency subbands in the horizontal direction, the high-frequency subbands in the vertical direction, and the high-frequency subbands in the diagonal direction are denoised. Here, a soft threshold processing method based on wavelet coefficients is used. Soft threshold processing is a commonly used image denoising method, and its principle is to suppress noise in the image by threshold processing the wavelet coefficients. Specifically, soft threshold processing is performed on each wavelet coefficient, and when the absolute value of the wavelet coefficient is less than the set threshold, the coefficient is set to zero; otherwise, it remains unchanged. This can effectively remove the noise in the image while retaining the main features and structure of the image. Finally, the image is reconstructed. In this step, the processed low-frequency sub-band and the denoised high-frequency sub-band are used to reconstruct the image. This step uses the inverse wavelet transform to synthesize the processed sub-bands into a new image, that is, the pre-processed super-resolution image. Specifically, the inverse wavelet transform is used to synthesize the processed sub-bands into a new image, in which the low-frequency information retains the main structure and features of the image, and the high-frequency information retains the detail information of the image after denoising. Through the image reconstruction process, the noise and unnecessary details in the image can be effectively reduced, thereby improving the quality and clarity of the image.

[0036] Bidirectional wavelet decomposition is a signal decomposition method based on wavelet transform. Wavelet transform is a mathematical tool that decomposes a signal into components of different frequencies and scales, similar to Fourier transform, but wavelet transform has better local properties. In bidirectional wavelet decomposition, a pair of orthogonal wavelet filters (usually high-pass and low-pass filters) are first selected for signal decomposition in the horizontal and vertical directions, respectively. Then, by performing convolution operations on the signal, the signal is decomposed into low-frequency and high-frequency sub-bands in the horizontal and vertical directions. Through bidirectional wavelet decomposition, the signal can be decomposed into components of different frequencies and scales, so as to better understand the characteristics and structure of the signal. High-frequency sub-bands usually contain details and noise information in the image. By filtering the high-frequency sub-bands, the noise in the image can be effectively removed, thereby improving the quality and clarity of the image.

[0037] The workflow of bidirectional wavelet decomposition is usually divided into the following steps: Select a pair of orthogonal wavelet filters, such as Haar wavelet or Daubechies wavelet. Convolve the image signal with the filters in the horizontal and vertical directions respectively to obtain low-frequency and high-frequency sub-bands in the horizontal and vertical directions. Downsample the low-frequency and high-frequency sub-bands in the horizontal and vertical directions to reduce the amount of data. Repeat the above steps until the desired decomposition level is reached.

[0038] Embodiment 3: Performing image preprocessing on a spectral image to obtain a preprocessed spectral image comprises: normalizing the pixel values ​​of the spectral image to [-1, 1]; converting the normalized spectral image into a data matrix, in which each column represents a pixel point and each row represents a band; performing low-rank matrix decomposition on the data matrix; setting the number of main singular values ​​to r, setting the singular values ​​other than the main singular values ​​to zero to achieve low-rank approximation and complete truncated singular values; using the singular values ​​after truncated singular values ​​to reconstruct the data matrix, thereby obtaining the preprocessed spectral image.

[0039] Specifically, first, the pixel values ​​of the spectral image are normalized. The purpose of pixel value normalization is to limit the pixel value range in the spectral image to [-1,1] so as to unify the pixel value range of different bands and different images. This can avoid the instability of image processing caused by the brightness difference between different bands and different images, and make the subsequent processing more unified and reliable. Next, the normalized spectral image is converted into a data matrix. The construction of this data matrix is ​​to facilitate the mathematical analysis and processing of the spectral image. In the data matrix, each column represents a pixel point and each row represents a band, so the entire data matrix represents the entire spectral image. By constructing the data matrix, the complex information of the spectral image can be converted into a mathematically processable form, which provides a basis for subsequent processing. Next, the data matrix is ​​subjected to low-rank matrix decomposition. Low-rank matrix decomposition is a dimensionality reduction method, the principle of which is to decompose the original data matrix into the product of two low-rank matrices. Through low-rank matrix decomposition, the dimension of the data can be reduced, the redundant information can be reduced, and the main features of the data can be extracted. The role of this step is to reduce the dimensionality of the spectral image in order to better understand and analyze the spectral data. After the low-rank matrix is ​​decomposed, the number of main singular values ​​is set to r, and the singular values ​​other than the main singular values ​​are set to zero to achieve low-rank approximation and complete the truncated singular values. The role of this step is to further reduce the dimension of the data and retain the main features of the data. By retaining the main singular values, the storage space and computational complexity of the data can be reduced while ensuring the data quality. Finally, the singular values ​​after the truncated singular values ​​are used to reconstruct the data matrix to obtain the preprocessed spectral image. By multiplying the truncated singular values ​​with the corresponding singular vectors, a data matrix similar to the original data can be reconstructed to obtain the preprocessed spectral image. The role of this step is to remap the reduced-dimensional data into the original space, so that the preprocessed spectral image has better quality and interpretability.

[0040] Embodiment 4: Image region extraction part, a method for identifying the leaf region and stem region of a target crop from a pre-processed super-resolution image comprises: obtaining a labeled sample, extracting each labeled sample K i The labels form a label vector:

[0041] B={B1,B2,...,B N},

[0042] Among them B i is the label of the i-th labeled sample; using the conditional Bayes method, a classifier is trained based on the labeled sample (K, B), and the sparse representation s of the labeled sample is i and the corresponding label y iEstablish association; N is the number of labeled samples; the labeled samples are small pieces of images with leaf and stem labels; for each labeled sample K i , calculate its sparse representation s i ; Divide the preprocessed super-resolution image into small blocks of size n×n, expressed as:

[0043] X={x1,x2,...,x N},

[0044] where x i is the i-th small block; a dictionary matrix D is constructed using the sparse coding method, where each column of the dictionary represents a feature; the size of the dictionary is m×k, where m is the number of features in the dictionary and k is the dimension of the feature; for each small block x i Perform sparse coding and find its sparse representation vector a i , so that x i ≈Da i ; According to the trained conditional Bayes classifier, calculate each small block x i The conditional probability P(y) of belonging to leaves and stems i |a i ); According to Bayes’ theorem, calculate each small block x i The posterior probability P(y) of belonging to leaves and stems i |x i ); according to the posterior probability, each small block is thresholded to determine whether it belongs to the category of leaf or stem; adjacent small blocks with the same label are merged to form leaf area and stem area.

[0045] Specifically, first, by obtaining labeled samples and training a classifier using the conditional Bayesian method, automatic recognition of leaves and stems in crop images is achieved. The recognition of leaves and stems in crop images is of great significance for the detection and management of crop diseases and insect pests. The traditional manual extraction of leaf and stem regions requires a lot of time and energy, and is easily affected by subjective factors, while the method in Example 4 can efficiently identify leaves and stems in an automated manner, greatly improving the efficiency of detection and management of crop diseases and insect pests. Secondly, by performing small block segmentation and sparse coding on the pre-processed super-resolution image, efficient processing and analysis of crop images are achieved. The pre-processed super-resolution image is obtained after a series of pre-processing steps of the original crop image, and has a higher resolution and clearer image quality. The method of small block segmentation and sparse coding in Example 4 can effectively perform feature extraction and data dimension reduction on the pre-processed super-resolution image, so as to better understand and analyze the information in the crop image. In addition, by calculating the posterior probability that each small block belongs to a leaf and a stem, and performing threshold processing and regional merging, the segmentation and classification of the crop image are achieved. In crop images, leaves and stems usually have different color, texture and shape characteristics. Through the method in Example 4, the image can be segmented into leaf areas and stem areas according to these characteristics, and these areas can be classified and marked. In this way, the leaves and stems in the crop image are clearly distinguished, which provides strong support for subsequent disease and insect pest detection and management. Finally, the method in Example 4 is universal and scalable. This method does not depend on specific crops or image types, but by utilizing conditional Bayesian classifiers and sparse representation technology, it can be applied to image processing and analysis tasks of various crops. In addition, this method can be combined with other image processing and machine learning technologies to further improve the accuracy and efficiency of crop image processing and provide more technical support and assistance for agricultural production.

[0046] Example 5: Make x by minimizing the following sparsity criterion i ≈Da i :

[0047]

[0048] Among them, a i is a sparse representation vector, which represents the sparse representation of the i-th small block; λ is a sparsity parameter, which controls the strength of sparsity; a ij represents the sparse representation vector a i The j-th element of ;∈ is a smooth term.

[0049] At the same time, if the number of features in the dictionary of the super-resolution image is less than n, x can be minimized by minimizing the following sparsity criterion: i ≈Dai , to improve accuracy:

[0050]

[0051] Specifically, the sparsity criterion in the first case aims to achieve a sparse representation of the image patch, that is, to minimize the loss function so that the image patch x i Instead of sparse representation Da i The error between is minimized and the sparsity is maintained as much as possible. Let's take a deeper look at the principle and role of this sparsity criterion. First, let's explain the various parts of the loss function. The loss function consists of two parts. The first part is the squared error term. Represents a small image patch x i Instead of sparse representation Da i The square of the Euclidean distance between is used to measure the size of the reconstruction error. The second part is the sparsity penalty term Where λ is the sparsity parameter, which controls the strength of sparsity, and a ij represents the sparse representation vector a i The jth element of ∈ is a smoothing term. The sparsity penalty term is used to represent the sparse representation vector a i Penalize each element in to make it as sparse as possible, that is, try to ij Set to zero to reduce the data dimension. Next, let’s explain the square root term in the sparsity penalty term. This term is to avoid numerical instability caused by excessive sparsity while penalizing sparsity. When the sparsity is higher, the sparse representation vector a i The elements in are close to zero, but directly setting them to zero may cause numerical instability, so the square root term is introduced to balance the relationship between sparsity and numerical stability. By penalizing the square root of the sparsity penalty term, the degree of sparsity can be effectively controlled while maintaining numerical stability. Finally, let us explain the role of the sparsity parameter λ. The sparsity parameter λ is used to adjust the degree of penalty of the sparsity penalty term, and its numerical value determines the degree of influence of sparsity on the loss function. When λ is large, the penalty of the sparsity penalty term is greater, which will emphasize sparsity more, thereby making the sparse representation vector a i More sparse; when λ is small, the penalty of the sparsity penalty term is smaller, and more emphasis will be placed on minimizing the reconstruction error, making the sparse representation vector a iCloser to the original data. By adjusting the sparsity parameter λ, the degree of sparse representation can be flexibly controlled to meet the needs of different application scenarios. The sparsity criterion in the first case achieves sparse representation of small blocks of the image by minimizing the loss function, including the squared error term and the sparsity penalty term. This criterion can reduce the dimension of sparse representation as much as possible while maintaining image quality, thereby improving the efficiency of image processing and analysis. By reasonably selecting the values ​​of the sparsity parameter λ and the smoothness term ∈, the relationship between sparsity and numerical stability can be effectively balanced, thereby obtaining a more ideal sparse representation result.

[0052] The sparsity criterion in the second case is designed to further improve the accuracy of sparse representation when the number of features in the dictionary is less than n. This sparsity criterion achieves sparse representation of small patches of the image while maintaining sparsity by minimizing the loss function. Let's take a closer look at the principle and role of this sparsity criterion. First, let's explain the two parts of the loss function. The loss function consists of two parts. The first part is the squared error term. It represents the square of the Euclidean distance between the image patch xi and its sparse representation Dai, which is used to measure the size of the reconstruction error. The second part is the sparsity penalty term Where λ is the sparsity parameter, which controls the strength of sparsity, and aij represents the sparse representation vector a i The jth element of ∈ is the smoothing term. The sparsity penalty term is used to penalize each element in the sparse representation vector ai to make it as sparse as possible, that is, to set aij to zero as much as possible to achieve the purpose of reducing the data dimension. Next, let's explain the three parts of the sparsity penalty term. arctan(a ij )and These three parts represent the effects of a ij The logarithm, inverse tangent and reciprocal transformations of . These transformations are introduced to better balance the relationship between sparsity and numerical stability, and to better punish excessive and low sparsity. and the inverse tangent transform arctan(a ij ) is used to punish a ij The absolute value of is small and the absolute value is large, while the reciprocal transformation Used to maintain the stability of the value and avoid ijThe numerical instability caused by the sparsity penalty term is close to zero. Through these transformations of the sparsity penalty term, the degree of sparse representation can be controlled more accurately, thereby obtaining a more accurate sparse representation result. Finally, let us explain the role of the sparsity parameter λ. The sparsity parameter λ is used to adjust the degree of penalty of the sparsity penalty term, and its numerical value determines the degree of influence of sparsity on the loss function. When λ is large, the penalty of the sparsity penalty term is larger, which will emphasize sparsity more, thereby making the sparse representation vector a i More sparse; when λ is small, the penalty of the sparsity penalty term is smaller, and more emphasis will be placed on minimizing the reconstruction error, making the sparse representation vector a i Closer to the original data. By adjusting the sparsity parameter λ, the degree of sparse representation can be flexibly controlled to meet the needs of different application scenarios. The sparsity criterion in the second case achieves sparse representation of small blocks of the image by minimizing the loss function, including the squared error term and the sparsity penalty term. This criterion can further improve the accuracy of sparse representation while maintaining image quality, and is suitable for situations where the number of features in the dictionary is small. By reasonably selecting the values ​​of the sparsity parameter λ and the smoothing term ∈, a more ideal sparse representation result can be obtained, thereby improving the efficiency and accuracy of image processing and analysis.

[0053] Embodiment 6: In the pest judgment part, spectral analysis is performed on the leaf area and the stem area in the spectral image to judge whether the target crops have pests and diseases. The method for obtaining the judgment result includes: performing spectral analysis on the leaf area and the stem area in the spectral image to extract the chlorophyll absorption characteristic peak, and extracting the intensity value of the chlorophyll absorption characteristic peak; setting the leaf area characteristic peak threshold range and the stem area characteristic peak threshold range; respectively comparing the intensity value of the chlorophyll absorption characteristic peak of the leaf area and the intensity value of the chlorophyll absorption characteristic peak of the stem area with the leaf area characteristic peak threshold range and the stem area characteristic peak threshold range. If any one of the two is not within the corresponding set threshold range, the judgment result is true; otherwise, the judgment result is false.

[0054] Specifically, first, for the leaf area and stem area in the spectral image, spectral analysis is performed to extract the chlorophyll absorption characteristic peaks therein. Chlorophyll is an important photosynthetic pigment in plants, which exhibits absorption peaks at specific wavelengths. Through spectral analysis, the chlorophyll absorption characteristic peaks in the spectral curve can be detected and their intensity values ​​can be obtained. The intensity values ​​of these characteristic peaks can reflect the content of chlorophyll in plant tissues, and then be used to judge the health status of the plant. Next, the threshold range of the characteristic peaks in the leaf area and the threshold range of the characteristic peaks in the stem area are set. These threshold ranges are determined based on existing knowledge or experimental data, and are used to judge whether the intensity of the chlorophyll absorption characteristic peaks in the leaf and stem areas is normal. Generally, healthy plants and plants affected by pests and diseases will differ in the intensity of the chlorophyll absorption characteristic peaks, so different threshold ranges need to be set according to actual conditions. Then, for each leaf area and stem area, the intensity value of the corresponding chlorophyll absorption characteristic peak is compared with the set threshold range. If the intensity value of the characteristic chlorophyll absorption peak in the leaf region or the intensity value of the characteristic chlorophyll absorption peak in the stem region is not within the corresponding set threshold range, the judgment result is true, that is, it is considered that the target crop has pests and diseases. Conversely, if the intensity values ​​of the characteristic chlorophyll absorption peaks in the leaf region and the stem region are both within the set threshold range, the judgment result is false, that is, it is considered that the target crop has no pests and diseases.

[0055] Embodiment 7: A method for performing spectral analysis on leaf areas and stem areas in a spectral image to extract characteristic peaks of chlorophyll absorption comprises: using a Gaussian smoothing algorithm to smooth the leaf areas and stem areas in the spectral image to make the characteristic peaks clearer, and performing baseline correction to improve the signal-to-noise ratio; using a first-order derivative method to detect the chlorophyll absorption characteristic peaks in the leaf areas and stem areas in the spectral image, that is, by taking the first-order derivative of the spectrum, finding a change rate exceeding a set change value as the chlorophyll absorption characteristic peak; using a local maximum point detection method to locate the position of the chlorophyll absorption characteristic peak, that is, finding a local maximum point on the first-order derivative curve, and using it as the position of the chlorophyll absorption characteristic peak; extracting the intensity value of the chlorophyll absorption characteristic peak according to the position of the chlorophyll absorption characteristic peak.

[0056] Specifically, first, the leaf area and stem area in the spectral image are smoothed using the Gaussian smoothing algorithm. The Gaussian smoothing algorithm is a commonly used signal processing method. By smoothing the spectral image, the influence of noise can be reduced, making the characteristic peak clearer. At the same time, baseline correction can further improve the signal-to-noise ratio and eliminate the influence of background signals, thereby more accurately detecting the characteristic peak. Next, the first-order derivative method is used to detect the chlorophyll absorption characteristic peak in the leaf area and stem area in the spectral image. The first-order derivative is a differential operation on the spectral curve, and the rate of change of the curve can be found. In the spectral image, the chlorophyll absorption characteristic peak is usually manifested as a local maximum point of the curve, that is, a position where the curve has a large rate of change. Therefore, by obtaining the first-order derivative of the spectrum, the point where the rate of change exceeds the set change value can be found as the position of the chlorophyll absorption characteristic peak. Then, the local maximum point detection method is used to locate the position of the chlorophyll absorption characteristic peak. Find the local maximum point on the first-order derivative curve and use it as the position of the chlorophyll absorption characteristic peak. The local maximum point refers to a point with a maximum value within a certain range, which usually represents a characteristic peak in the spectral curve. By detecting the local maximum point, the position of the chlorophyll absorption characteristic peak can be accurately determined. Finally, according to the position of the chlorophyll absorption characteristic peak, the intensity value of the chlorophyll absorption characteristic peak is extracted. After determining the position of the characteristic peak, the intensity value of the characteristic peak can be obtained by the signal intensity at the corresponding position on the spectrum curve. These intensity values ​​can be used for subsequent analysis and judgment, such as judging whether the plant is affected by pests and diseases.

[0057] Embodiment 8: In the pest analysis part, when the judgment result is true, pests and diseases are identified from the leaf area and the stem area, and the method for obtaining the pest and disease areas in the leaf area and the stem area includes: using an image instance segmentation algorithm to segment the leaves and stems in the image, and generating a binary mask image, wherein the leaf area and the stem area are respectively marked with different pixel values; for the leaf area and the stem area, using a grayscale co-occurrence matrix to extract their respective feature vectors; using a machine learning or deep learning method to establish a classifier, and mapping the extracted feature vectors to pest and disease categories; for the segmented leaf area and stem area, using a trained classifier to identify pests and diseases for each pixel or area; according to the output of the classifier, marking the area identified as pests and diseases; using an image connectivity analysis method to remove small connected areas, merge the marked pest and disease areas together, and obtain the pest and disease areas in the leaf area and the stem area.

[0058] Specifically, first, the image instance segmentation algorithm is used to segment the leaves and stems in the image to generate a binary mask image. This process marks the leaf area and stem area in the image with different pixel values, which is convenient for subsequent processing. Then, for the segmented leaf area and stem area, the gray level co-occurrence matrix (GLCM) is used to extract their respective feature vectors. The gray level co-occurrence matrix is ​​a statistical method used to describe the texture characteristics of an image. By analyzing the grayscale relationship between pixels, the texture information of the image can be extracted, which helps to distinguish different tissue structures and pests and diseases. Then, a classifier is established using machine learning or deep learning methods to map the extracted feature vectors to pest and disease categories. This classifier can be based on traditional machine learning algorithms such as support vector machines (SVM), random forests, etc., or it can be based on deep learning algorithms such as convolutional neural networks (CNN). The establishment of the classifier requires the use of labeled pest and disease samples for training. Next, for the segmented leaf area and stem area, pest and disease identification is performed on each pixel or area through the trained classifier. According to the output of the classifier, the area identified as pest and disease is marked. Finally, the image connectivity analysis method is used to remove small connected areas and merge the marked pest and disease areas together to form the pest and disease areas in the leaf area and stem area. Image connectivity analysis is a method used to identify and merge connected areas in an image. By analyzing the connectivity relationship between pixels, adjacent pest and disease areas can be merged into a larger area, thereby obtaining a more accurate pest and disease area.

[0059] Embodiment 9: The AR device is a mobile phone, a tablet, a head-mounted display or a display.

[0060] Specifically, AR (augmented reality) technology is a technology that superimposes virtual information on the real world, superimposes virtual objects or information on the real world through specific devices, and enables users to see virtual objects or information in real scenes. Mobile phones, tablets, head-mounted displays or monitors are all common AR devices. They have different characteristics and uses, but the principles are basically the same. These AR devices can capture the real environment around the user in real time through built-in cameras or sensors. Then, through AR applications or software, virtual objects or information are superimposed and displayed with the real environment captured in real time. This superimposed display method can be achieved through a transparent display of a screen or head-mounted display, so that users can see the real world and virtual objects or information at the same time. When displaying the pest and disease area of ​​the target animal or plant in the AR device, the pest and disease information that has been analyzed and processed must first be presented in a virtual form. This information can be the boundary, outline or marking point of the pest and disease area. Then, using AR technology, this virtual information is superimposed on the display screen or head-mounted display of the AR device, and superimposed and displayed with the real scene of the target animal or plant captured in real time. By observing the screen of the AR device or the head-mounted display, the user can intuitively see the location and shape of the pest and disease areas of the target animals and plants in the real scene. For portable AR devices such as mobile phones and tablets, users can directly observe the real scene through the camera and see the superimposed pest and disease information on the screen. For head-mounted AR devices such as head-mounted displays or monitors, users wear the device to project virtual information onto a transparent display in front of them, allowing users to see the real scene and the superimposed pest and disease information while wearing the device.

[0061] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these specific embodiments are only illustrative, and those skilled in the art may omit, replace, and change the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, merging the above method steps so as to perform substantially the same functions in substantially the same manner to achieve substantially the same results is within the scope of the present invention. Therefore, the scope of the present invention is limited only by the appended claims.

Claims

1. Pest detection and management system based on machine vision and AR, characterized by: The system comprises: an image acquisition part, which is used to acquire a super-resolution image and a spectral image of a target crop, wherein the super-resolution image and the spectral image are of equal size, and the relative positions of the crops in the super-resolution image and the spectral image are consistent; image preprocessing is performed on the super-resolution image and the spectral image respectively to obtain a preprocessed super-resolution image and a preprocessed spectral image; an image region extraction part, which is used to identify the leaf region and the stem region of the target crop from the preprocessed super-resolution image, and thereby extract the leaf region and the stem region of the target crop from the preprocessed spectral image; a pest judgment part, which is used to perform spectral analysis on the leaf region and the stem region in the preprocessed spectral image to judge whether pests and diseases occur in the target crop and obtain a judgment result; a pest analysis part, which is used to perform pest and disease identification in the leaf region and the stem region in the preprocessed super-resolution image when the judgment result is true, and obtain the pest and disease region in the leaf region and the stem region; an AR display part, which is used to display the pest and disease region of the target crop in an AR device; The method of performing image preprocessing on a super-resolution image to obtain a preprocessed super-resolution image includes: performing bidirectional wavelet decomposition on the super-resolution image by using a pair of orthogonal wavelet filters to obtain low-frequency subbands in horizontal and vertical directions, high-frequency subbands in horizontal direction, high-frequency subbands in vertical direction, and high-frequency subbands in diagonal direction; performing denoising operation on the high-frequency subbands in horizontal direction, high-frequency subbands in vertical direction, and high-frequency subbands in diagonal direction by using a soft threshold processing method based on wavelet coefficients; and reconstructing the image by using the low-frequency subbands in horizontal and vertical directions, high-frequency subbands in horizontal direction, high-frequency subbands in vertical direction, and high-frequency subbands in diagonal direction to obtain a preprocessed super-resolution image; The method for preprocessing the spectral image to obtain the preprocessed spectral image includes: normalizing the pixel values ​​of the spectral image to [-1, 1]; converting the normalized spectral image into a data matrix, in which each column represents a pixel point and each row represents a band; performing low-rank matrix decomposition on the data matrix; setting the number of main singular values ​​to , setting the singular values ​​other than the main singular values ​​to zero to achieve low-rank approximation and complete the truncated singular values; reconstructing the data matrix using the singular values ​​after the truncated singular values, and then obtaining the preprocessed spectral image; In the image region extraction part, the method for identifying the leaf region and stem region of the target crop from the preprocessed super-resolution image includes: obtaining a labeled sample, extracting each labeled sample The labels form a label vector: ， in It is The labels of the labeled samples; using the conditional Bayes method, according to the labeled samples Train a classifier to represent the sparse representation of labeled samples and the corresponding labels Establish association; the labeled sample is a small image with leaf and stem labels; for each labeled sample , calculate its sparse representation ; Divide the preprocessed super-resolution image into A small block is represented as: ， in It is a small piece; is the number of labeled samples; a dictionary matrix is ​​constructed using sparse coding method , each column of the dictionary represents a feature; the size of the dictionary is ,in is the number of features in the dictionary, is the dimension of the feature; for each small block Perform sparse coding and find its sparse representation vector , so that ; According to the trained conditional Bayes classifier, calculate each small block Conditional probability of belonging to leaves and stems ; According to Bayes' theorem, calculate each small block The posterior probability of belonging to leaves and stems ; According to the posterior probability, threshold processing is performed on each small block to determine whether it belongs to the category of leaf or stem; adjacent small blocks with the same label are merged to form leaf areas and stem areas; By minimizing the following sparsity criterion : ； in, is a sparse representation vector, representing the Sparse representation of small blocks; is the sparsity parameter, which controls the strength of sparsity; Represents a sparse representation vector No. elements; is a smoothing term; In the pest judgment part, spectral analysis is performed on the leaf area and the stem area in the pre-processed spectral image to judge whether the target crop has pests and diseases. The method for obtaining the judgment result includes: performing spectral analysis on the leaf area and the stem area in the pre-processed spectral image to extract the chlorophyll absorption characteristic peak, and extracting the intensity value of the chlorophyll absorption characteristic peak; setting the threshold range of the characteristic peak of the leaf area and the threshold range of the characteristic peak of the stem area; respectively comparing the intensity value of the chlorophyll absorption characteristic peak of the leaf area and the intensity value of the chlorophyll absorption characteristic peak of the stem area with the threshold range of the characteristic peak of the leaf area and the threshold range of the characteristic peak of the stem area, if any one of the two is not within the corresponding set threshold range, the judgment result is true; otherwise, the judgment result is false; The method for performing spectral analysis on the leaf area and the stem area in the preprocessed spectral image to extract the chlorophyll absorption characteristic peak includes: using a Gaussian smoothing algorithm to smooth the leaf area and the stem area in the preprocessed spectral image to make the characteristic peak clearer, and performing baseline correction to improve the signal-to-noise ratio; using a first-order derivative method to detect the chlorophyll absorption characteristic peak in the leaf area and the stem area in the preprocessed spectral image, that is, by obtaining the first-order derivative of the spectrum, finding a change rate exceeding a set change value as the chlorophyll absorption characteristic peak; using a local maximum point detection method to locate the position of the chlorophyll absorption characteristic peak, that is, finding a local maximum point on the first-order derivative curve and using it as the position of the chlorophyll absorption characteristic peak; according to the position of the chlorophyll absorption characteristic peak, extracting the intensity value of the chlorophyll absorption characteristic peak; In the pest analysis part, when the judgment result is true, pests and diseases are identified from the leaf area and the stem area, and the method for obtaining the pest and disease areas in the leaf area and the stem area includes: using an image instance segmentation algorithm to segment the leaves and stems in the image, and generating a binary mask image, wherein the leaf area and the stem area are respectively marked with different pixel values; for the leaf area and the stem area, using a grayscale co-occurrence matrix to extract their respective feature vectors; using a machine learning or deep learning method to establish a classifier, and mapping the extracted feature vectors to the pest and disease category; for the segmented leaf area and stem area, using a trained classifier to identify pests and diseases for each pixel or area; according to the output of the classifier, marking the area identified as pests and diseases; using an image connectivity analysis method to remove small connected areas, merge the marked pest and disease areas together, and obtain the pest and disease areas in the leaf area and the stem area.

2. The pest detection and management system based on machine vision and AR as claimed in claim 1, characterized in that: The AR device is a mobile phone, a tablet or a display.

Images