A method and system for separating and identifying crops and weeds based on image processing and machine vision

Abstract

The present invention belongs to the technical field of separation and identification of crops and weeds, and specifically is a method and system for separation and identification of crops and weeds based on image processing and machine vision. The present invention makes full use of advanced image processing technology, deep learning, machine vision algorithms and intelligent post-processing mechanisms, and describes in detail the complete process from image acquisition, preprocessing, segmentation, feature extraction and identification to intelligent post-processing and comprehensive management decision support. This method has the advantages of high-precision identification technology, real-time processing capability, adaptability to complex environments, self-learning and adaptive mechanisms, and integrated management system integration, thereby achieving efficient and accurate identification of crops and weeds, and providing strong technical support for precision agriculture.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop and weed separation and identification, and specifically relates to a crop and weed separation and identification method and system based on image processing and machine vision. Background Art

[0002] With the acceleration of globalization and the rapid development of science and technology, smart agricultural technology has gradually become a key force in improving agricultural production efficiency and promoting sustainable agricultural development. In particular, efficient image processing and machine vision technology are particularly important in accurately distinguishing crops from weeds and achieving precise pesticide application and irrigation. However, in the face of complex agricultural environments and diverse crop and weed species, existing technologies still face many challenges in terms of accuracy, real-time and adaptability.

[0003] Existing crop and weed recognition technologies mainly rely on simple image processing techniques and limited machine learning methods. These technologies often have difficulty achieving high accuracy and high reliability in complex practical application scenarios, such as variable lighting conditions and high visual similarity between crops and weeds. In addition, when processing large amounts of farmland image data, traditional methods often fail to meet the requirements of real-time or near-real-time processing due to low algorithm efficiency, thus limiting their application scope and effects in modern smart agriculture. Specifically, existing technologies have significant deficiencies in the following aspects:

[0004] 1. Lack of high-precision recognition technology: Existing methods have difficulty in handling scenes with large lighting changes and high visual similarity between crops and weeds, resulting in low recognition accuracy;

[0005] 2. Insufficient real-time processing capabilities: Due to low algorithm processing efficiency, existing technologies are difficult to achieve real-time or near real-time image processing capabilities, which affects the timeliness and effectiveness of agricultural production;

[0006] 3. Lack of adaptability and robustness: Existing technologies lack sufficient adaptability and robustness to different types of crops, crops and weeds at different growth stages, and changing weather and light conditions;

[0007] 4. Lack of effective self-learning and adaptive mechanisms: Most existing technologies are unable to self-learn and optimize from new data, making it difficult to cope with new weed species and new crop varieties;

[0008] 5. Insufficient integration of integrated management systems: Existing solutions often only focus on the identification technology itself, while ignoring the integration with agricultural production management and decision support systems, which limits the application effect and scope of the technology;

[0009] Therefore, how to provide a method and system for separating and identifying crops and weeds based on image processing and machine vision is a problem that those skilled in the art need to solve urgently. Summary of the invention

[0010] One object of the present invention is to provide a method for separating and identifying crops and weeds based on image processing and machine vision. One object of the present invention is to provide a system for separating and identifying crops and weeds based on image processing and machine vision.

[0011] In order to solve the technical problems raised in the background technology, the present invention provides the following technical solutions:

[0012] A method for separating and identifying crops and weeds based on image processing and machine vision comprises the following steps:

[0013] S1. Collect high-quality image data of crops and weeds in farmland;

[0014] S3, using the image segmentation network, combined with the growth characteristics of crops and weeds and environmental background knowledge, segmenting the candidate areas of crops and weeds from the preprocessed image;

[0015] S4. Apply hybrid feature extraction technology, combining machine vision and deep learning algorithms, to comprehensively extract multi-dimensional features of crops and weeds from candidate areas;

[0016] S5. Use an adaptive deep learning model to dynamically adjust the network structure according to the characteristics of crops and weeds to achieve high-precision classification and identification of crops and weeds;

[0017] The S3 specifically includes:

[0018] S31. First, use the image segmentation algorithm based on convolutional neural network to perform preliminary segmentation of crops, weeds and background:

[0019] C i = argmax c P(c|X i ;θ);

[0020] Among them, C i represents the classification label of pixel i, P(c|X i ; θ) is the given pixel feature X i The probability that a pixel belongs to category c under the model parameters θ;

[0021] S32, introduce multi-scale feature fusion strategy, by combining different levels of feature information F local,k With the global feature F global To enhance the model's ability to identify crops and weeds in different sizes and shapes:

[0022]

[0023] Among them, σ represents the activation function, which is used to enhance the nonlinear expression ability, λ k is the importance weight of the k-th layer feature, Represents the feature fusion operation, combining global and local features to optimize the final feature representation F final ;

[0024] S33, use conditional random fields as a post-processing step to refine the segmentation boundaries and enhance the spatial coherence of the segmentation results:

[0025] E(L)=∑ i ψ u (L i ; X,θ u )+∑ i<j ψ p (L i ,L j ; X,θ p );

[0026] Among them, E(L) is the energy of the CRF model, L i and L j are the labels of pixels i and j, respectively, ψ u and ψ p denote the unary and pairwise potential functions, θ u and θ p is the parameter of the potential function, which is used to adjust the consistency and edge accuracy of pixel labels;

[0027] S34, completing the segmentation of crops and weeds in the enhanced image, and generating accurate candidate regions;

[0028] The S4 specifically includes:

[0029] S41. From the segmented image, for each candidate region of crops and weeds, use feature extraction technology to comprehensively extract multi-dimensional features:

[0030]

[0031] Among them, F total represents the comprehensive feature vector, w n is the nth feature F n The weight of N is the total number of features, which aims to obtain a comprehensive feature description and provide necessary feature data for subsequent analysis;

[0032] S42. Calculate the shape feature F based on the features extracted in S41 shape Geometric properties, such as circularity:

[0033]

[0034] Among them, A represents the area of ​​the object, P represents the perimeter of the object, and Circularity represents the roundness, which is used to describe the smoothness of the object shape;

[0035] S43, color feature F color Consider the color histogram uniformity U and color entropy Entropy:

[0036]

[0037] Where M is the number of entries in the color histogram, and p i is the normalized frequency of the i-th color value. The color entropy represents the uncertainty of the color distribution, which helps to identify the uncertainty of the color distribution;

[0038] S44, texture feature F texture It is calculated by the contrast Contrast and homogeneity Homogeneity in the gray level co-occurrence matrix:

[0039] Contrast=∑ i,j (ij) 2 P(i,j);

[0040]

[0041] Among them, P(i,j) is the probability that pixel values ​​i and j appear at the same time at a specific direction and distance, which helps to identify subtle differences in the plant surface;

[0042] S45, the comprehensive feature vector F after refinement total The data is input into a pre-trained deep learning model for classification, and the output of the model is used to determine whether the candidate area is a crop or a weed, aiming to improve the accuracy and efficiency of recognition.

[0043] The S5 specifically includes:

[0044] S51, using the obtained comprehensive feature F total The input is sent to the adaptive deep learning model, which dynamically adjusts its network structure and parameters according to the input features to achieve high-precision classification and identification of crops and weeds. The model outputs the category label L for each candidate area. category :

[0045] L category =DeepLearningModel(F total ; Θ);

[0046] Among them, Θ represents the model parameters;

[0047] S52. The adaptive deep learning model improves the generalization ability and adaptability of the model on new data sets:

[0048]

[0049] Among them, Θ old and θ new are the parameters before and after the model is updated, η is the learning rate, is the gradient of the loss function L with respect to the model parameters Θ, Y true is the true label;

[0050] S53, use the back propagation algorithm to optimize the model parameters Θ, minimize the classification error, and measure the difference between the model prediction value and the true value:

[0051]

[0052] Where C is the number of categories, y c is 1 if the category is c, otherwise 0, p c is the probability that the model predicts that the candidate region belongs to category c;

[0053] S54. Assign a category label to each candidate region, which indicates whether the region is a crop, a weed, or a background.

[0054] Preferably, in S1, adaptive illumination adjustment technology is used to automatically adjust digital camera parameters under various illumination conditions;

[0055] Specifically include:

[0056] S11, determine the light level L, first define the minimum value of light intensity I min and the maximum value I max , and then use the light sensor to measure the light intensity I of the current environment env , based on the measurement results, the current light level L is calculated, which aims to quantify the ambient lighting conditions so as to make appropriate camera parameter adjustments in subsequent steps:

[0057]

[0058] in, Indicates the division interval of light intensity levels, n is the total number of light levels;

[0059] S12, based on the measured light level L, set preliminary exposure time parameters for the current light conditions, according to the current light intensity I env and aperture value f number , calculate the ideal exposure time t exp:

[0060]

[0061] Among them, K is a constant for achieving ideal exposure at a specific ISO sensitivity;

[0062] S13. Use the image quality assessment algorithm to evaluate the image acquisition effect after adjusting the parameters and calculate the image quality score Q:

[0063] Q = w1·C + w2·S + w3·D;

[0064] Among them, C represents the contrast score of the image, S represents the clarity score, D represents the saturation score, w1, w2, and w3 are weight coefficients. Through the quality evaluation of the actual image, it is confirmed whether the preliminary parameter settings meet the expected image quality standards;

[0065] S14. Compare the image quality score Q with the preset quality threshold Q min For comparison, when Q min Adjust exposure time:

[0066] t exp,new =t exp ·(1+α·(Q min -Q));

[0067] Among them, α is the adjustment coefficient, which is used to control the adjustment range of the exposure time. According to the feedback of the image quality score, the camera exposure setting is further optimized to ensure that the required image quality standard is achieved;

[0068] S15. After the image quality score Q reaches a predetermined standard, continuous image acquisition of the farmland scene is performed using the finally determined camera parameter settings. During the acquisition process, the camera parameter settings, the illumination level L at that time, and the image quality score Q are recorded for each image data.

[0069] Preferably, S2 is provided between S1 and S3, and an image preprocessing algorithm based on deep learning is used to perform denoising, enhancement and intelligent color space conversion on the collected image;

[0070] Specifically include:

[0071] S21, perform preliminary denoising on the collected image data, use the median filter algorithm to reduce the random noise in the image, and calculate the new value of each pixel:

[0072] P new =median{P neighbor};

[0073] Among them, P new Represents the pixel value after filtering, P​neighbor Represents the set of pixel values ​​in the neighborhood of the pixel point;

[0074] S22, perform image enhancement processing to improve the contrast and details of the image, use histogram equalization technology to adjust the brightness distribution of the image to make the image clearer, and calculate the new value of the brightness of each pixel:

[0075]

[0076] Among them, L new Indicates the enhanced brightness value, L max and L min are the maximum and minimum brightness values ​​in the histogram, respectively. N is the total number of brightness levels in the image. C(L) is the cumulative histogram value of the current brightness L. min is the minimum cumulative value in the histogram;

[0077] S23. Apply color space conversion to convert the image from the RGB color space to a color space more suitable for plant identification:

[0078]

[0079] Among them, H represents hue, S represents saturation, I represents brightness, and R, G, and B represent the pixel values ​​of the red, green, and blue channels respectively;

[0080] S24, further enhance the image according to the hue H and saturation S in the HSI color space to enhance the color saturation and contrast of the green plant part:

[0081] S new =S·(1+β·(S target -S));

[0082] I new =I·(1+γ·(I target -I));

[0083] Among them, S new and I new Represent the adjusted saturation and brightness values, S target and I target are the target saturation and brightness values ​​respectively, and β and γ are adjustment coefficients used to control the adjustment intensity of saturation and brightness.

[0084] Preferably, S6 is provided after S5, and an intelligent post-processing mechanism is adopted to automatically optimize the recognition result and correct errors based on the comparative analysis between the recognition result and the actual situation;

[0085] Specifically include:

[0086] S61. Adopt intelligent post-processing mechanism, automatically optimize the recognition results and correct errors based on comparative analysis between the recognition results and the actual situation, and evaluate the confidence score C for each candidate area identified as crops or weeds. score :

[0087]

[0088] Where N is the number of pixels in the candidate area, p i,c is the probability that pixel i belongs to category c;

[0089] S62: For the confidence level lower than the predetermined threshold T confidence The recognition result starts the error correction process and uses the spatial context information to correct the category:

[0090] L corrected =Mode{L neighbor};

[0091] Among them, L corrected is the corrected category label, L neightbor It is a set of category labels of pixels around the candidate area, and Mode calculates the most frequently occurring category;

[0092] S63, by combining the recognition results of adjacent areas and the characteristic distribution of the original recognition area, adjusting the misclassified area to increase the recognition accuracy:

[0093] L final =f(L corrected ,F neighbor );

[0094] Among them, L final represents the final category label, and f is based on the neighborhood feature F neighbor and the modified category L corrected Comprehensive judgment function of

[0095] S64. Update the classification information of the candidate area and generate an optimized recognition result, which will directly affect subsequent decision-making, enhance the adaptability and robustness of the system to complex environments, and ensure that reliable recognition results can be obtained under different conditions.

[0096] Preferably, S7 is provided after S6 to output the optimized recognition result to the integrated management system in real time through the data interface, so as to provide instant and accurate decision support for weed control and crop management in agricultural production.

[0097] Specifically include:

[0098] S71. Output the optimized recognition results to the integrated management system in real time through the data interface. The system is responsible for integrating the data from the recognition module to provide instant and accurate decision support for weed control and crop management in agricultural production:

[0099] D output ={(L final ,C score ,Position)};

[0100] Among them, D output represents the output data set, L final represents the final category label of each candidate region, C score is the confidence score, and Position represents the location information of the candidate region in the image;

[0101] S72. The integrated management system uses intelligent algorithms to automatically plan and adjust agricultural operation strategies based on the received recognition results:

[0102] Strategy = f(S crop ,S growth ,E cond );

[0103] Among them, f is the strategy optimization function, S crop Indicates the crop type, S growth Represents the growth stage of the crop, E cond Indicates the current environment conditions, and Strategy is the subsequent operation based on these contents;

[0104] S73, the integrated management system is also responsible for generating detailed operation reports and future operation suggestions to support farm managers in making more informed decisions:

[0105] Report={Effectiveness,Recommendations};

[0106] Among them, Effectiveness evaluates the execution effect of the current job, Recommendations provides future improvement suggestions based on the current job results, and Relport records the entire process of subsequent operations.

[0107] Preferably, in S45, the pre-trained deep learning model is any one of a convolutional neural network, a support vector machine or a random forest.

[0108] A crop and weed separation and identification system based on image processing and machine vision, comprising:

[0109] The image acquisition unit is responsible for collecting image data of crops and weeds in the farmland under various lighting conditions;

[0110] An image preprocessing unit, which preprocesses the collected image;

[0111] An image segmentation unit, which uses deep learning-based image segmentation technology to separate candidate regions of crops and weeds from preprocessed images;

[0112] A feature extraction and recognition unit extracts multi-dimensional features from the candidate regions obtained by the image segmentation unit;

[0113] Intelligent post-processing unit, which automatically optimizes recognition results and corrects errors based on comparative analysis between recognition results and actual conditions;

[0114] The integrated management and decision support unit outputs the optimized recognition results to the integrated management system in real time through the data interface, which integrates the data from the recognition module;

[0115] The user interface unit provides a user-friendly interface that allows the user to monitor image acquisition status, view recognition results and reports, and adjust system settings and parameters.

[0116] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0117] (1) The present invention achieves high-precision recognition of crops and weeds by combining advanced image processing technology with deep learning algorithms. In particular, the use of adaptive light adjustment technology and deep learning-based image preprocessing algorithms significantly improves the image quality under variable light conditions. Furthermore, through hybrid feature extraction and adaptive deep learning models, the visual similarity problem between crops and weeds is effectively addressed, greatly improving the recognition accuracy.

[0118] (2) The real-time or near real-time recognition system designed by the present invention ensures a rapid response to weeds in agricultural mechanized operations. By introducing efficient image processing algorithms and optimized deep learning model structures, the system can quickly process large amounts of image data and achieve instant recognition of crops and weeds, thereby significantly improving the efficiency of agricultural production.

[0119] (3) The present invention takes into account the complexity of the farmland environment and effectively copes with different types of crops, crops and weeds at different growth stages, and changing weather and light conditions by introducing a recognition system with strong environmental adaptability and robustness. The flexibility and adaptability of this system ensures that it can work stably and efficiently under various agricultural production conditions.

[0120] (4) The present invention also proposes an integrated management system that combines high-precision identification technology with agricultural production management and decision support systems. This integrated approach not only achieves accurate identification of weeds and crops, but also provides technical support for precision agricultural management, achieves more efficient and sustainable crop management, and significantly improves the overall level of intelligent agricultural production.

[0121] The present invention makes full use of advanced image processing technology, deep learning, machine vision algorithms and intelligent post-processing mechanisms, and describes in detail the complete process from image acquisition, preprocessing, segmentation, feature extraction and recognition to intelligent post-processing and comprehensive management decision support. This method has the advantages of high-precision recognition technology, real-time processing capability, adaptability to complex environments, self-learning and adaptive mechanisms, and integrated management system integration, thereby achieving efficient and accurate identification of crops and weeds, and providing strong technical support for precision agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0122] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0123] Figure 1 This is an overall flow chart of the crop and weed separation and identification method based on image processing and machine vision proposed in the present invention. DETAILED DESCRIPTION

[0124] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0125] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0126] S1, using adaptive light adjustment technology to automatically adjust digital camera parameters under various lighting conditions to collect high-quality image data of crops and weeds in the field;

[0127] S2, using deep learning-based image preprocessing algorithms to denoise, enhance, and intelligently convert color spaces of collected images to improve the image quality of subsequent processing;

[0128] S3, using the image segmentation network, combined with the growth characteristics of crops and weeds and environmental background knowledge, segmenting the candidate areas of crops and weeds from the preprocessed image;

[0129] S4. Apply hybrid feature extraction technology, combining machine vision and deep learning algorithms, to comprehensively extract multi-dimensional features of crops and weeds from candidate areas;

[0130] S5. Use an adaptive deep learning model to dynamically adjust the network structure according to the characteristics of crops and weeds to achieve high-precision classification and identification of crops and weeds;

[0131] S6. Adopt intelligent post-processing mechanism, automatically optimize the recognition results and correct errors based on comparative analysis between the recognition results and the actual situation, and improve the recognition accuracy of the system;

[0132] S7. The optimized recognition results are output to the integrated management system in real time through the data interface, providing instant and accurate decision support for weed control and crop management in agricultural production.

[0133] refer to Figure 1 A method for separating and identifying crops and weeds based on image processing and machine vision, characterized in that it comprises the following steps:

[0134] S1, using adaptive light adjustment technology to automatically adjust digital camera parameters under various lighting conditions to collect high-quality image data of crops and weeds in the field;

[0135] In this implementation, S1 specifically includes:

[0136] S11, determine the light level L, first define the minimum value of light intensity I min and the maximum value I max , and then use the light sensor to measure the light intensity I of the current environment env , based on the measurement results, the current light level L is calculated, which aims to quantify the ambient lighting conditions so as to make appropriate camera parameter adjustments in subsequent steps:

[0137]

[0138] in, Indicates the division interval of light intensity levels, n is the total number of light levels;

[0139] S12, based on the measured light level L, set preliminary exposure time parameters for the current light conditions, according to the current light intensity I env and aperture value f number , calculate the ideal exposure time T exp :

[0140]

[0141] Among them, K is a constant for achieving ideal exposure at a specific ISO sensitivity;

[0142] S13. Use the image quality assessment algorithm to evaluate the image acquisition effect after adjusting the parameters and calculate the image quality score Q:

[0143] Q = w1·C + w2·S + w3·D;

[0144] Among them, C represents the contrast score of the image, S represents the clarity score, D represents the saturation score, w1, w2, and w3 are weight coefficients. Through the quality evaluation of the actual image, it is confirmed whether the preliminary parameter settings meet the expected image quality standards;

[0145] S14. Compare the image quality score Q with the preset quality threshold Q min For comparison, when Q min Adjust exposure time:

[0146] t exp,new =t exp ·(1+α·(Q min -Q));

[0147] Among them, α is the adjustment coefficient, which is used to control the adjustment range of the exposure time. According to the feedback of the image quality score, the camera exposure setting is further optimized to ensure that the required image quality standard is achieved;

[0148] S15. After the image quality score Q reaches the predetermined standard, the finalized camera parameter settings are used to continuously collect images of the farmland scene. During the collection process, the camera parameter settings of each image data are recorded in detail, including key information such as exposure time, aperture value, focal length, etc. At the same time, the light level L at that time is recorded. By evaluating the image scores Q under different camera parameter settings and lighting conditions, a set of optimization strategies are explored and developed to improve the overall performance of image acquisition and ensure that high-quality images can be obtained under various environmental conditions.

[0149] S2, using deep learning-based image preprocessing algorithms to denoise, enhance, and intelligently convert color spaces of collected images to improve the image quality of subsequent processing;

[0150] In this implementation, S2 specifically includes:

[0151] S21, perform preliminary denoising on the collected image data, use the median filter algorithm to reduce the random noise in the image, and calculate the new value of each pixel:

[0152] P new ​=median{P neighbor};

[0153] Among them, P new Represents the pixel value after filtering, P neighbor Represents the set of pixel values ​​in the neighborhood of the pixel point;

[0154] S22, perform image enhancement processing to improve the contrast and details of the image, use histogram equalization technology to adjust the brightness distribution of the image to make the image clearer, and calculate the new value of the brightness of each pixel:

[0155]

[0156] Among them, L new Indicates the enhanced brightness value, L max and L min are the maximum and minimum brightness values ​​in the histogram, respectively. N is the total number of brightness levels in the image. C(L) is the cumulative histogram value of the current brightness L. min is the minimum cumulative value in the histogram;

[0157] S23. Apply color space conversion to convert the image from the RGB color space to a color space more suitable for plant identification:

[0158]

[0159] Among them, H represents hue, S represents saturation, I represents brightness, and R, G, and B represent the pixel values ​​of the red, green, and blue channels respectively;

[0160] S24, further enhance the image according to the hue H and saturation S in the HSI color space to enhance the color saturation and contrast of the green plant part:

[0161] S new =S·(1+β·(S target -S));

[0162] I new =I·(1+γ·(L target -I));

[0163] Among them, S new and I new Represent the adjusted saturation and brightness values, S target and I target are the target saturation and brightness values ​​respectively, and β and γ are adjustment coefficients used to control the adjustment intensity of saturation and brightness.

[0164] S3, using the image segmentation network, combined with the growth characteristics of crops and weeds and environmental background knowledge, segmenting the candidate areas of crops and weeds from the preprocessed image;

[0165] The growth characteristics and environmental background knowledge of crops and weeds, for example, the soil in East China is yellow, while the soil in Northeast China is black. In terms of weed growth characteristics, weeds are divided into grasses and broadleaf weeds. The seedling period of broadleaf weeds is 30-40 days, while that of grasses is 20-30 days.

[0166] In this implementation, S3 specifically includes:

[0167] S31. First, use the image segmentation algorithm based on convolutional neural network to perform preliminary segmentation of crops, weeds and background:

[0168] C i = argmax c P(c|X i ;θ);

[0169] Among them, C i represents the classification label of pixel i, P(c|X i ; θ) is the given pixel feature X i The probability that a pixel belongs to category c under the model parameters θ;

[0170] S32, introduce multi-scale feature fusion strategy, by combining different levels of feature information F local,k With the global feature F global To enhance the model's ability to identify crops and weeds in different sizes and shapes:

[0171]

[0172] Among them, σ represents the activation function, which is used to enhance the nonlinear expression ability, λ k is the importance weight of the k-th layer feature, Represents the feature fusion operation, combining global and local features to optimize the final feature representation F final ;

[0173] S33, use conditional random fields as a post-processing step to refine the segmentation boundaries and enhance the spatial coherence of the segmentation results:

[0174] E(L)=∑ i ψ u (L i ; X,θ u )+∑ i<j ψ p (L i ,L j ; X,θ p );

[0175] Among them, E(L) is the energy of the CRF model, L i and L j are the labels of pixels i and j, respectively, ψ u and ψ p denote the unary and pairwise potential functions, θ u and θ p is the parameter of the potential function, which is used to adjust the consistency and edge accuracy of pixel labels;

[0176] S34, completing the segmentation of crops and weeds in the enhanced image, and generating accurate candidate regions.

[0177] S4. Apply hybrid feature extraction technology, combining machine vision and deep learning algorithms, to comprehensively extract multi-dimensional features of crops and weeds from candidate areas;

[0178] In this implementation, S4 specifically includes:

[0179] S41. From the segmented image, for each candidate region of crops and weeds, use feature extraction technology to comprehensively extract multi-dimensional features:

[0180]

[0181] Among them, F total represents the comprehensive feature vector, w n is the nth feature F n The weight of N is the total number of features, which aims to obtain a comprehensive feature description and provide necessary feature data for subsequent analysis;

[0182] S42. Calculate the shape feature F based on the features extracted in S41 shape Geometric properties, such as circularity:

[0183]

[0184] Among them, A represents the area of ​​the object, P represents the perimeter of the object, and Circularity represents the roundness, which is used to describe the smoothness of the object shape;

[0185] S43, color feature F color Consider the color histogram uniformity U and color entropy Entropy:

[0186]

[0187] Where M is the number of entries in the color histogram, and p iis the normalized frequency of the i-th color value. The color entropy represents the uncertainty of the color distribution, which helps to identify the uncertainty of the color distribution;

[0188] S44, texture feature F texture It is calculated by the contrast Contrast and homogeneity Homogeneity in the gray level co-occurrence matrix:

[0189] Contrast=∑ i,j (ij) 2 P(i,j);

[0190]

[0191] Among them, P(i,j) is the probability that pixel values ​​i and j appear at the same time at a specific direction and distance, which helps to identify subtle differences in the plant surface;

[0192] S45 and S41 obtain preliminary multi-dimensional feature vectors F through feature extraction technology total S42 to S44 further extract specific shape, color and texture features respectively, and finally the comprehensive feature vector F after refinement is total The input is sent to a pre-trained deep learning model for classification, and the output of the model is used to determine whether the candidate area is a crop or a weed. These models may include convolutional neural networks (CNN), support vector machines (SVM), or random forests, etc. This aims to improve the accuracy and efficiency of recognition.

[0193] S5. Use an adaptive deep learning model to dynamically adjust the network structure according to the characteristics of crops and weeds to achieve high-precision classification and identification of crops and weeds;

[0194] In this implementation, S5 specifically includes:

[0195] S51, using the obtained comprehensive feature F total The input is sent to the adaptive deep learning model, which dynamically adjusts its network structure and parameters according to the input features to achieve high-precision classification and identification of crops and weeds. The model outputs the category label L for each candidate area. category :

[0196] L category =DeepLearningModel(F total ; Θ);

[0197] Among them, Θ represents the model parameters;

[0198] S52. The adaptive deep learning model improves the generalization ability and adaptability of the model on new data sets:

[0199]

[0200] Among them, Θ old and θ new are the parameters before and after the model is updated, η is the learning rate, is the gradient of the loss function L with respect to the model parameters Θ, Y true is the true label;

[0201] S53, use the back propagation algorithm to optimize the model parameters Θ, minimize the classification error, and measure the difference between the model prediction value and the true value:

[0202]

[0203] Where C is the number of categories, y c is 1 if the category is c, otherwise 0, p c is the probability that the model predicts that the candidate region belongs to category c;

[0204] S54. Assign a category label to each candidate region, which indicates whether the region is a crop, a weed, or a background.

[0205] S6. Adopt intelligent post-processing mechanism, automatically optimize the recognition results and correct errors based on comparative analysis between the recognition results and the actual situation, and improve the recognition accuracy of the system;

[0206] In this implementation, S6 specifically includes:

[0207] S61. Adopt intelligent post-processing mechanism, automatically optimize the recognition results and correct errors based on comparative analysis between the recognition results and the actual situation, and evaluate the confidence score C for each candidate area identified as crops or weeds. score :

[0208]

[0209] Where N is the number of pixels in the candidate area, p i,c is the probability that pixel i belongs to category c;

[0210] S62: For the confidence level lower than the predetermined threshold T confidence The recognition result starts the error correction process and uses the spatial context information to correct the category:

[0211] L corrected =Mode{L neighbor};

[0212] Among them, L corrected is the corrected category label, Lneightbor It is a set of category labels of pixels around the candidate area, and Mode calculates the most frequently occurring category;

[0213] S63, by combining the recognition results of adjacent areas and the characteristic distribution of the original recognition area, adjusting the misclassified area to increase the recognition accuracy:

[0214] L final =f(L corrected ,F neighbor );

[0215] Among them, L final represents the final category label, and f is based on the neighborhood feature F neighbor and the modified category L corrected Comprehensive judgment function of

[0216] S64. Update the classification information of the candidate area and generate an optimized recognition result, which will directly affect subsequent decision-making, enhance the adaptability and robustness of the system to complex environments, and ensure that reliable recognition results can be obtained under different conditions.

[0217] S7. The optimized recognition results are output to the integrated management system in real time through the data interface, providing instant and accurate decision support for weed control and crop management in agricultural production.

[0218] In this implementation, S7 specifically includes:

[0219] S71. Output the optimized recognition results to the integrated management system in real time through the data interface. The system is responsible for integrating the data from the recognition module to provide instant and accurate decision support for weed control and crop management in agricultural production:

[0220] D output ={(L final ,C score ,Position)};

[0221] Among them, D output represents the output data set, L final represents the final category label of each candidate region, C score is the confidence score, and Position represents the location information of the candidate region in the image;

[0222] S72. The integrated management system uses intelligent algorithms to automatically plan and adjust agricultural operation strategies based on the received recognition results:

[0223] Strategy = f(S crop ,S growth ,E cond );

[0224] Among them, f is the strategy optimization function, S crop Indicates the crop type, S growth Represents the growth stage of the crop, E cond Indicates the current environmental conditions. Strategy is the subsequent operation based on these contents. For example, robot weeding is to conduct tracking and attack.

[0225] S73, the integrated management system is also responsible for generating detailed operation reports and future operation suggestions to support farm managers in making more informed decisions:

[0226] Report={Effectiveness,Recommendations};

[0227] Among them, Effectiveness evaluates the execution effect of the current job, Recommendations provides future improvement suggestions based on the current job results, and Report records the entire process of subsequent operations.

[0228] A crop and weed separation and identification system based on image processing and machine vision, characterized by comprising:

[0229] The image acquisition unit is responsible for collecting image data of crops and weeds in the farmland under various lighting conditions;

[0230] An image preprocessing unit, which preprocesses the collected image;

[0231] An image segmentation unit, which uses deep learning-based image segmentation technology to separate candidate regions of crops and weeds from preprocessed images;

[0232] A feature extraction and recognition unit extracts multi-dimensional features from the candidate regions obtained by the image segmentation unit;

[0233] Intelligent post-processing unit, which automatically optimizes recognition results and corrects errors based on comparative analysis between recognition results and actual conditions;

[0234] The integrated management and decision support unit outputs the optimized recognition results to the integrated management system in real time through the data interface, which integrates the data from the recognition module;

[0235] The user interface unit provides a user-friendly interface that allows the user to monitor image acquisition status, view recognition results and reports, and adjust system settings and parameters.

[0236] Embodiment 1:

[0237] In order to verify the effectiveness of the present invention, this embodiment selects a farm in a province in China with advanced agricultural development as an application scenario. The farm mainly grows wheat and corn, and often faces the problem of weed infestation. Especially during the spring growth period, the rapid growth of weeds poses a threat to the healthy growth of crops. In addition, the lighting conditions in the area where the farm is located are changeable, which increases the difficulty of separating and identifying crops and weeds. In this context, the crop and weed separation and identification method based on image processing and machine vision proposed in the present invention is adopted.

[0238] The process of applying the present invention in this scenario includes using a high-resolution digital camera to capture images of farmland at different time periods, and then preprocessing these images through adaptive light adjustment technology to adapt to different lighting conditions. After that, a deep learning model is used to segment the images and accurately identify crops and weeds from the images. Then, the system extracts multidimensional features such as shape, color and texture, and uses these features for accurate identification. Finally, the recognition results are optimized through an intelligent post-processing mechanism, and the results are fed back to the farm management system in real time to support decision-making. In order to demonstrate the beneficial effects of the present invention, a three-month field application test was carried out and relevant data was recorded. The specific data is as follows:

[0239] Table 1 Report on improvement in separation and identification of crops and weeds on farms

[0240]

[0241]

[0242] Before deploying this system, the process used in the experiment includes the following steps:

[0243] Image acquisition, fix camera parameters, and obtain farmland images;

[0244] Image preprocessing, using median filtering algorithm to remove image noise and improve image quality;

[0245] Feature extraction, extracting features of crops and weeds from images;

[0246] Classification and recognition, using deep learning models to distinguish between crops and weeds, such as ResNet, YOLO, Mask R-CNN, etc.;

[0247] The result is output and the recognition result is provided to the management system.

[0248] As can be seen from the above table, after deploying the system of the present invention, the number of times crops and weeds are separated and identified each month has increased significantly, the real-time processing capability has been greatly improved, and more images can be processed within 1 hour. At the same time, the accuracy of the deep learning model has increased from 80% to 95%, and the misrecognition rate has also decreased from 15% to 3%, showing a significant improvement in recognition accuracy and efficiency of the present invention. In addition, the adaptive lighting adjustment technology enables the system to maintain high quality of image acquisition under different lighting conditions, further improving the accuracy and reliability of recognition. Through accurate identification and timely weed management, the healthy growth of crops has been significantly improved, and the use of pesticides has become more accurate and efficient, reducing the use of pesticides by 30%, which is conducive to environmental protection and sustainable development.

[0249] It can be clearly seen from these data that the present invention not only improves the accuracy and efficiency of separation and identification of crops and weeds, but also optimizes agricultural production management, provides strong technical support for precision agriculture, and has significant economic and social benefits.

[0250] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0251] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for separating and identifying crops and weeds based on image processing and machine vision, characterized in that: The following steps are involved: S1. Collect high-quality image data of crops and weeds in farmland; S3, using the image segmentation network, combined with the growth characteristics of crops and weeds and environmental background knowledge, segmenting the candidate areas of crops and weeds from the preprocessed image; S41. From the segmented image, for each candidate region of crops and weeds, use feature extraction technology to comprehensively extract multi-dimensional features: Among them, F total represents the comprehensive feature vector, w n is the nth feature F n The weight of N is the total number of features, which aims to obtain a comprehensive feature description and provide necessary feature data for subsequent analysis; S42. Calculate the shape feature F based on the features extracted in S41 shape Circularity: Among them, A represents the area of ​​the object, P represents the perimeter of the object, and Circularity represents the roundness, which is used to describe the smoothness of the object shape; S43, color feature F color Consider the color histogram uniformity U and color entropy Entropy: Where M is the number of entries in the color histogram, and p i is the normalized frequency of the i-th color value. The color entropy represents the uncertainty of the color distribution, which helps to identify the uncertainty of the color distribution; S44, texture feature F texture It is calculated by the contrast Contrast and homogeneity Homogeneity in the gray level co-occurrence matrix: Contrast=∑ i，j (i-j) 2 P(i,j); Among them, P(i, j) is the probability that pixel values ​​i and j appear at the same time at a specific direction and distance, which helps to identify subtle differences in plant surfaces; S5. Use an adaptive deep learning model to dynamically adjust the network structure according to the characteristics of crops and weeds to achieve high-precision classification and identification of crops and weeds; The S3 specifically includes: S31. First, use the image segmentation algorithm based on convolutional neural network to perform preliminary segmentation of crops, weeds and background: C i 1argmax c P(c|X i (θ)) Among them, C i represents the classification label of pixel i, P(c|X i ; θ) is the given pixel feature X i The probability that a pixel belongs to category c under the model parameters θ; S32, introduce multi-scale feature fusion strategy, by combining different levels of feature information F local，k With the global feature F global To enhance the model's ability to identify crops and weeds in different sizes and shapes: Among them, σ represents the activation function, which is used to enhance the nonlinear expression ability, λ k is the importance weight of the k-th layer feature, Represents the feature fusion operation, combining global and local features to optimize the final feature representation F final ; S33, use conditional random fields as a post-processing step to refine the segmentation boundaries and enhance the spatial coherence of the segmentation results: E(L)=∑ i ψ u (L i ;X,θ u )+∑ i＜j ψ p (L i ,L j ;X,θ p ); Among them, E(L) is the energy of the CRF model, L i and L j are the labels of pixels i and j, respectively, ψ u and ψ p denote the unary and pairwise potential functions, θ u and θ p is the parameter of the potential function, which is used to adjust the consistency and edge accuracy of pixel labels; S34, completing the segmentation of crops and weeds in the enhanced image, and generating accurate candidate regions; The S5 specifically includes: S51, using the obtained comprehensive feature F total The input is sent to the adaptive deep learning model, which dynamically adjusts its network structure and parameters according to the input features to achieve high-precision classification and identification of crops and weeds. The model outputs the category label L for each candidate area. category : L category =DeepLearningModel(F total ;Θ); Among them, Θ represents the model parameters; S52, the adaptive deep learning model improves the generalization ability and adaptability of the model on new data sets: Among them, Θ old and θ new are the parameters before and after the model is updated, η is the learning rate, is the gradient of the loss function L with respect to the model parameters Θ, Y true is the true label; S53, use the back propagation algorithm to optimize the model parameters Θ, minimize the classification error, and measure the difference between the model prediction value and the true value: Where C is the number of categories, y c is 1 if the category is c, otherwise 0, p c is the probability that the model predicts that the candidate region belongs to category c; S54, assigning a category label to each candidate region, the label indicating whether the region is a crop, a weed or a background; S55, the comprehensive feature vector F after refinement total The input is fed into a pre-trained deep learning model for classification, and the output of the model is used to determine whether the candidate area is a crop or a weed, aiming to improve the accuracy and efficiency of recognition.

2. The method for separating and identifying crops and weeds based on image processing and machine vision according to claim 1, characterized in that: In the S1, adaptive illumination adjustment technology is used to automatically adjust digital camera parameters under various illumination conditions; Specifically include: S11, determine the light level L, first define the minimum value of light intensity I min and the maximum value I max , and then use the light sensor to measure the light intensity I of the current environment env , based on the measurement results, the current light level L is calculated, which aims to quantify the ambient lighting conditions so as to make appropriate camera parameter adjustments in subsequent steps: in, Indicates the division interval of light intensity levels, n is the total number of light levels; S12, based on the measured light level L, set preliminary exposure time parameters for the current light conditions, according to the current light intensity I env and aperture value f number , calculate the ideal exposure time t exp : Among them, K is a constant for achieving ideal exposure at a specific ISO sensitivity; S13. Use the image quality assessment algorithm to evaluate the image acquisition effect after adjusting the parameters and calculate the image quality score Q: Q = w1·C + w2·S + w3·D; Among them, C represents the contrast score of the image, S represents the clarity score, D represents the saturation score, w1, w2, and w3 are weight coefficients. The quality evaluation of the actual image is used to confirm whether the preliminary parameter settings meet the expected image quality standards; S14. Compare the image quality score Q with the preset quality threshold Q min For comparison, when Q<Q min Adjust exposure time: t exp,new =t exp ·(1+α·(Q min -Q)); Among them, α is the adjustment coefficient, which is used to control the adjustment range of the exposure time. According to the feedback of the image quality score, the camera exposure setting is further optimized to ensure that the required image quality standard is achieved; S15. After the image quality score Q reaches a predetermined standard, continuous image acquisition of the farmland scene is performed using the finally determined camera parameter settings. During the acquisition process, the camera parameter settings, the illumination level L at that time, and the image quality score Q are recorded for each image data.

3. The method for separating and identifying crops and weeds based on image processing and machine vision according to claim 1, characterized in that: S2 is arranged between S1 and S3, and uses an image preprocessing algorithm based on deep learning to perform denoising, enhancement and intelligent color space conversion on the collected image; Specifically include: S21, perform preliminary denoising on the collected image data, use the median filter algorithm to reduce the random noise in the image, and calculate the new value of each pixel: P new =median{P neighbor }; Among them, P new Represents the pixel value after filtering, P neighbor Represents the set of pixel values ​​in the neighborhood of the pixel point; S22, perform image enhancement processing to improve the contrast and details of the image, use histogram equalization technology to adjust the brightness distribution of the image to make the image clearer, and calculate the new value of the brightness of each pixel: Among them, L new Indicates the enhanced brightness value, L max and L min are the maximum and minimum brightness values ​​in the histogram, respectively. N is the total number of brightness levels in the image. C(L) is the cumulative histogram value of the current brightness L. min is the minimum cumulative value in the histogram; S23. Apply color space conversion to convert the image from the RGB color space to a color space more suitable for plant identification: Among them, H represents hue, S represents saturation, I represents brightness, and R, G, and B represent the pixel values ​​of the red, green, and blue channels respectively; S24, further enhance the image according to the hue H and saturation S in the HSI color space to enhance the color saturation and contrast of the green plant part: S new =S·(1+β·(s target -S)); I new =I·(1+γ·(I target -I)); Among them, S new and I new Represent the adjusted saturation and brightness values, S target and I target are the target saturation and brightness values ​​respectively, and β and γ are adjustment coefficients used to control the adjustment intensity of saturation and brightness.

4. The method for separating and identifying crops and weeds based on image processing and machine vision according to claim 1, characterized in that: S6 is arranged after S5, and adopts an intelligent post-processing mechanism to automatically optimize the recognition result and correct errors based on the comparative analysis between the recognition result and the actual situation; Specifically include: S61. Adopt intelligent post-processing mechanism, automatically optimize the recognition results and correct errors based on comparative analysis between the recognition results and the actual situation, and evaluate the confidence score C for each candidate area identified as crops or weeds. score : Where N is the number of pixels in the candidate area, p i，c is the probability that pixel i belongs to category c; S62: For the confidence level lower than the predetermined threshold T confidence The recognition result starts the error correction process and uses the spatial context information to correct the category: L corrected =Mode{L neighbor }; Among them, L corrected is the corrected category label, L neighbor It is a set of category labels of pixels around the candidate area, and Mode calculates the most frequently occurring category; S63, by combining the recognition results of adjacent areas and the characteristic distribution of the original recognition area, adjusting the misclassified area to increase the recognition accuracy: L final =f(L corrected ,F neighbor ); Among them, L final represents the final category label, and f is based on the neighborhood feature F neighbor and the modified category L corrected Comprehensive judgment function of S64. Update the classification information of the candidate area and generate an optimized recognition result, which will directly affect subsequent decision-making, enhance the adaptability and robustness of the system to complex environments, and ensure that reliable recognition results can be obtained under different conditions.

5. The method for separating and identifying crops and weeds based on image processing and machine vision according to claim 4, characterized in that: S7 is arranged after S6, and the optimized recognition result is output to the integrated management system in real time through the data interface, so as to provide instant and accurate decision support for weed control and crop management in agricultural production. Specifically include: S71. Output the optimized recognition results to the integrated management system in real time through the data interface. The system is responsible for integrating the data from the recognition module to provide instant and accurate decision support for weed control and crop management in agricultural production: D output ={(L final ,C score ,Position)}; Among them, D output represents the output data set, L final represents the final category label of each candidate region, C score is the confidence score, and Position represents the location information of the candidate region in the image; S72. The integrated management system uses intelligent algorithms to automatically plan and adjust agricultural operation strategies based on the received recognition results: Strategy=f(S crop ,S growth ,E cond ); Among them, f is the strategy optimization function, S crop Indicates the crop type, S growth Represents the growth stage of the crop, E cond Indicates the current environment conditions, and Strategy is the subsequent operation based on these contents; S73, the integrated management system is also responsible for generating detailed operation reports and future operation suggestions to support farm managers in making more informed decisions: Report={Effectiveness, Recommendations}; Among them, Effectiveness evaluates the execution effect of the current job, Recommendations provides future improvement suggestions based on the current job results, and Report records the entire process of subsequent operations.

6. The method for separating and identifying crops and weeds based on image processing and machine vision according to claim 1, characterized in that: In S45, the pre-trained deep learning model is any one of a convolutional neural network, a support vector machine or a random forest.

7. A crop and weed separation and identification system based on image processing and machine vision, characterized in that: The method for separating and identifying crops and weeds based on image processing and machine vision as claimed in any one of claims 1 to 6 comprises: The image acquisition unit is responsible for collecting image data of crops and weeds in the farmland under various lighting conditions; An image preprocessing unit, which preprocesses the collected image; An image segmentation unit, which uses deep learning-based image segmentation technology to separate candidate regions of crops and weeds from preprocessed images; A feature extraction and recognition unit extracts multi-dimensional features from the candidate regions obtained by the image segmentation unit; Intelligent post-processing unit, which automatically optimizes recognition results and corrects errors based on comparative analysis between recognition results and actual conditions; The integrated management and decision support unit outputs the optimized recognition results to the integrated management system in real time through the data interface, which integrates the data from the recognition module; The user interface unit provides a user-friendly interface that allows the user to monitor image acquisition status, view recognition results and reports, and adjust system settings and parameters.

Images