Crop determination method and system based on combination of scene recognition model and target detection

Abstract

The invention discloses a crop determination method and system based on combination of a scene recognition model and target detection, and belongs to the technical field of agricultural information, and the method specifically comprises the steps: collecting crop image data, recording time-space information, and carrying out the preprocessing of a crop image; the method comprises the following steps: extracting crop image features, constructing a scene classification model and a target crop detection model, taking a to-be-detected image as input, firstly determining a scene category through the scene classification model, and then loading corresponding target crop detection model optimization parameters for detection; and carrying out weighted fusion on confidence scores of target detection and scene classification, matching with a crop growth cycle model database in combination with spatio-temporal information, and finally determining a crop variety and outputting a result. And the accuracy and efficiency of crop identification are improved.

Description

Technical Field

[0001] The present invention belongs to the field of agricultural information technology, and in particular to a crop determination method and system based on the combination of scene recognition model and target detection. Background Art

[0002] In agricultural production, accurately identifying crop types facilitates rational field management, resource allocation, and yield estimation. Traditional crop identification methods rely primarily on manual field observation and empirical judgment, which is inefficient. With the advancement of computer vision technology, crop identification using image or video data has become possible. However, existing single-technique approaches often have limitations. For example, relying solely on object detection can result in high misjudgment rates in complex scenarios, while simple scene recognition struggles to accurately identify specific individual crop information. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention proposes a crop identification method and system based on the combination of scene recognition model and target detection, which collects crop image data, records spatiotemporal information, and preprocesses the crop images; extracts crop image features, constructs a scene classification model and a target crop detection model, takes the image to be detected as input, first determines the scene category through the scene classification model, and then loads the corresponding target crop detection model to optimize parameters for detection; the confidence scores of target crop detection and scene classification are weighted and fused, and the spatiotemporal information is combined with the crop growth cycle model database for matching, finally determining the crop type and outputting the result, thereby improving the accuracy and efficiency of crop identification.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] The crop identification method based on the combination of scene recognition model and target detection includes:

[0006] Step S1: Collect crop image data and perform preprocessing;

[0007] Step S2: Based on the preprocessed crop image data, a neural network-based scene classification model and a target crop detection model are constructed respectively;

[0008] Step S3: Inputting the image to be detected into a pre-trained neural network-based scene classification model to obtain the scene category and corresponding confidence score to which the image to be detected belongs. Simultaneously, based on the scene classification result, the optimized parameters of the target crop detection model for the corresponding scene are loaded from a pre-established target crop detection model parameter library. The adjusted target crop detection model is used to perform target detection on the image to be detected, and a detection result for the image to be detected is output; the detection result includes the category and location of the target crop and the corresponding confidence score.

[0009] Step S4: performing weighted fusion on the confidence score of the target crop and the confidence score of the corresponding scene output by the scene classification model to obtain a fused confidence score;

[0010] Step S5: According to the geographical location and time information recorded when the image is collected, the detection result is matched with the pre-established crop growth cycle model database to determine the crop type in the image to be detected, and the final determined crop type and detection information are output as the result.

[0011] Specifically, the specific steps of step S2 include:

[0012] S2.1: Load the pre-trained convolutional neural network model and use the convolutional neural network's hierarchical feature extraction mechanism to input the pre-processed crop image data one by one into the pre-trained convolutional neural network model. Through the nonlinear transformation of multiple convolutional layers and pooling layers, feature vectors containing spatial semantic information are extracted.

[0013] S2.2: Load the pre-built multi-layer perceptron model architecture and train it using the feature vectors obtained in S2.1 to generate a neural network-based scene classification model.

[0014] S2.3: At the same time, use the LabelImg annotation tool to draw anchor boxes on the preprocessed crop image data, circle the target crop area, and add the corresponding crop type label to each anchor box;

[0015] S2.4: Load the pre-built Faster R-CNN model and input the labeled crop image data into the pre-built Faster R-CNN model. Use the labeled crop image data to train the region proposal network and classification regression layer of the pre-built Faster R-CNN model to generate a target crop detection model.

[0016] Specifically, the specific steps of step S3 include:

[0017] S3.1: Obtain the preprocessed image to be detected and input it into the scene classification model to obtain the probability distribution of each scene category through the neural network forward reasoning Based on the probability distribution of each scene category, the category with the largest probability value is selected as the scene category to which the image to be detected belongs, as predicted by the scene classification model. , the probability distribution of the scene category is the corresponding confidence score, where, represents the probability distribution of the i-th scene category, Represents the output vector of the last fully connected layer of the scene classification model The i-th element in n represents the total number of scene categories, represents the predicted scene category, Represents the maximum independent variable point set function, which means when Pick When The maximum value of the value range;

[0018] S3.2: Based on the scene classification result obtained by the scene classification model, locate the pre-established target crop detection model parameter library for different scenes, and search the target crop detection model parameter library for the location of the matching target crop detection model optimization parameters;

[0019] S3.3: Use the parameter loading function to load the found target crop detection model optimization parameters into memory. By traversing each key-value pair in the loaded optimization parameter dictionary, the original parameters with corresponding names in the target crop detection model are replaced with the loaded optimization parameters to complete the adjustment of the target crop detection model.

[0020] Specifically, the specific steps of step S3 also include:

[0021] S3.4: Input the preprocessed image to be detected into the adjusted target crop detection model and output the category of the target crop detected in the image , the corresponding confidence score and the final position of the target crop .

[0022] Specifically, the specific steps of step S4 include:

[0023] S4.1: Extract the confidence score of each target crop detection result from the output of the target crop detection model ,At the same time, the confidence score of the corresponding scene is obtained from the output of the ,scene classification model;

[0024] S4.2: Setting the weighting coefficient for the target crop detection confidence score and the weighted coefficient of the scene classification confidence score , and satisfies ;

[0025] S4.3: For each target crop q, the fusion confidence score is obtained by calculating the weighted cumulative sum of the confidence scores of each target crop and the corresponding scene .

[0026] Specifically, the specific steps of step S5 include:

[0027] S5.1: Extracting geographic location and time information from the image acquisition record, and using the extracted geographic location and time information as an index to query a pre-established crop growth cycle model database;

[0028] S5.2: Obtain the target crop scene information and the calculated fusion confidence score from the target crop detection results , for each crop obtained from the crop growth cycle model database ,Each crop obtained from the query is evaluated based on the scene information and fusion confidence score;

[0029] If the crop obtained by querying the crop growth cycle model database The query result of is the same as the scene in the target crop detection result, and ,but Belong to actual crops, among which, Indicates the preset confidence threshold;

[0030] S5.3: For each crop that actually exists , matching its growth cycle model with the spatiotemporal information and crop appearance features in the target crop detection results to obtain a matching result;

[0031] S5.4: Based on the matching results, the crop type with the highest matching degree is selected as the final crop type, and the final crop type and detection information are output as the result, where the detection information includes the crop position, confidence level, and scene information.

[0032] The crop identification system based on the combination of scene recognition model and target detection includes: data processing module, model building module, crop detection and classification module, confidence fusion module, and crop type determination module;

[0033] The data processing module is used to collect crop image data and perform preprocessing;

[0034] The model building module is used to build a scene classification model and a target crop detection model based on the preprocessed crop image data;

[0035] The crop detection and classification module is used to input the image to be detected into the pre-trained scene classification model and target crop detection model, perform scene classification and target crop detection respectively, and output the detection results;

[0036] The confidence fusion module is used to perform weighted fusion of the confidence score output by the target crop detection model and the confidence score output by the scene classification model;

[0037] The crop type determination module is used to combine the spatiotemporal information during image acquisition, query the crop growth cycle model database, determine the crop type in the image, and output the final result.

[0038] Specifically, the crop detection and classification module includes: a scene classification unit, a target crop detection unit, and a parameter loading unit;

[0039] The scene classification unit is used to input the image to be detected into a pre-trained scene classification model, and the scene classification model determines the scene category to which the image to be detected belongs and outputs a corresponding confidence score;

[0040] The target crop detection unit is configured to load the corresponding target crop detection model optimization parameters according to the scene classification result, perform target crop detection on the image to be detected, and output the category, position, and corresponding confidence score of the target crop detected in the image to be detected;

[0041] The parameter loading unit is used to search and load the target crop detection model optimization parameters of the corresponding scene from a pre-established target crop detection model parameter library according to the scene classification result output by the scene classification unit.

[0042] Specifically, the confidence fusion module includes: a confidence calculation unit and a weighted fusion unit;

[0043] The confidence calculation unit is used to calculate the confidence score output by the target crop detection model and the confidence score output by the scene classification model;

[0044] The weighted fusion unit is used to perform weighted fusion on the confidence score output by the target crop detection model and the confidence score output by the scene classification model according to preset weights to obtain a fused confidence score.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The present invention proposes a crop identification system based on the combination of scene recognition model and target detection, and optimizes and improves the architecture, operation steps and processes. The system has the advantages of simple process, low investment and operation costs, and low production work costs.

[0047] 2. The present invention proposes a crop identification method based on the combination of scene recognition model and target detection. Through the collaborative operation of multiple steps and multiple models, it achieves relatively accurate and comprehensive acquisition of crop information. First, the crop image data is collected and preprocessed, and features are extracted from it to construct a scene classification model and a target crop detection model. Then, during actual detection, the scene classification model is used to first determine the scene category and confidence score of the image to be detected, and then the parameters of the target crop detection model are optimized based on this before detection is performed. It can output the detailed category, location and corresponding confidence score of the target crop, effectively improving the accuracy and reliability of crop identification, so that it can adapt to the crop detection needs in different scenarios.

[0048] 3. The present invention proposes a crop identification method based on the combination of scene recognition models and target detection. It further performs a weighted fusion of the confidence scores of target crop detection and scene classification, combines the spatiotemporal information of image acquisition, and matches it with a pre-established crop growth cycle model database to ultimately determine the crop type in the image and output the integrated detection information. This comprehensive processing method fully considers the various factors that affect crop judgment. It can not only accurately determine the crop type, but also combine information from different dimensions to more scientifically grasp the crop status, thereby improving the practicality of the entire crop identification process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of a crop identification method based on a combination of scene recognition model and target detection according to the present invention;

[0050] Figure 2 This is a flow chart showing the principle of the crop identification method based on the combination of scene recognition model and target detection in the present invention;

[0051] Figure 3 This is a diagram of the crop determination system architecture based on the combination of scene recognition model and target detection in the present invention. DETAILED DESCRIPTION

[0052] Example 1:

[0053] See also Figure 1 and Figure 2 The present invention provides an embodiment of a crop identification method based on a combination of a scene recognition model and target detection, comprising the following steps:

[0054] Step S1: Collect crop image data and perform preprocessing;

[0055] Furthermore, the specific steps of step S1 include:

[0056] (1) Start image data collection, start acquiring crop images according to the determined data source and collection plan, and store the acquired images in the designated location;

[0057] (2) For each stored image, extract and record its corresponding geographic location and time information through the positioning and time recording functions of the acquisition device, and store it in association with the image;

[0058] (3) The images with spatiotemporal information are sequentially fed into the image preprocessing process:

[0059] 1) First, perform image denoising. Select a filtering denoising method based on the image noise level and configure the parameters to obtain a denoised image. The filtering denoising method is a prior art in this field and does not constitute the inventive solution of this application, so it will not be described in detail here.

[0060] 2) Next, determine whether grayscale processing is required. If so, convert the color image into a grayscale image using a weighted averaging method. The weighted averaging method is a prior art in this field and does not constitute an inventive solution of the present application, so it will not be described in detail here.

[0061] 3) Finally, the image after the previous steps is normalized, and the image pixel values ​​are adjusted using a linear normalization method to complete the preprocessing of the crop image data. The preprocessed image can be used for subsequent feature extraction, model building, and other operations. Among them, the linear normalization method is the existing technology content in this field and is not an inventive solution of this application, and will not be described in detail here.

[0062] Step S2: Based on the preprocessed crop image data, a neural network-based scene classification model and a target crop detection model are constructed respectively;

[0063] Step S3: Inputting the image to be detected into a pre-trained neural network-based scene classification model to obtain the scene category and corresponding confidence score to which the image to be detected belongs. Simultaneously, based on the scene classification result, the optimized parameters of the target crop detection model for the corresponding scene are loaded from a pre-established target crop detection model parameter library. The adjusted target crop detection model is used to perform target detection on the image to be detected, and a detection result for the image to be detected is output; the detection result includes the category and location of the target crop and the corresponding confidence score.

[0064] Step S4: performing weighted fusion on the confidence score of the target crop and the confidence score of the corresponding scene output by the scene classification model to obtain a fused confidence score;

[0065] Step S5: According to the geographical location and time information recorded when the image is collected, the detection result is matched with the pre-established crop growth cycle model database to determine the crop type in the image to be detected, and the final determined crop type and detection information are output as the result.

[0066] For example, in different agricultural production areas across the country, including the Northeast Plain, the North China Plain, the middle and lower reaches of the Yangtze River Plain, and the South China region, drones equipped with high-definition cameras were used to capture images of farmland, orchards, vegetable fields, and other scenes in different seasons and weather conditions, such as sunny, cloudy, overcast, and light rain, as well as different time periods, such as morning, noon, afternoon, and evening. A total of 100,000 image data were collected, and the latitude and longitude, date and time, light intensity, and other information of each image were recorded. At the same time, the crop types in the images were accurately labeled, covering more than 50 common crops such as wheat, rice, corn, apples, citrus, cabbage, and tomatoes. The collected images are preprocessed by using image normalization to adjust the pixel value range to [0, 1]. Unrelated background areas at the edges of the images are removed by cropping to make the images clearer. ResNet-50 is used as the basic network architecture to extract features from the preprocessed images. An attention mechanism module is added to this architecture to focus on key scene features in the image, such as ridges and irrigation facilities in farmland and the arrangement of fruit trees in orchards. The output layer of the model uses a softmax function to output five common agricultural scene categories, such as farmland, orchard, vegetable field, greenhouse, and other, as well as their corresponding confidence scores.

[0067] The scene classification model was then trained using 80,000 images labeled with scene categories. The dataset was split into training and validation sets in an 8:2 ratio, and a stochastic gradient descent algorithm was used as the optimizer. The initial learning rate was set to 0.001, the momentum parameter was set to 0.9, and the batch size was set to 32. During training, the learning rate was decayed by 0.5 times after every 10 epochs. When the accuracy on the validation set did not improve for five consecutive epochs, training was stopped and the model parameters were saved. After training, the scene classification model achieved an accuracy of over 95% on the validation set.

[0068] Furthermore, based on the preprocessed image data, the LabelImg tool was used to annotate the target crops in the images. This annotation information included the crop category, bounding box coordinates, and growth stage, including seedling, growing, and mature stages. A target detection dataset containing 60,000 images was constructed and divided into training and test sets in a 7:3 ratio.

[0069] Secondly, when an image to be detected is received, it is input into the trained scene classification model. Assume that the image is classified as a farmland scene with a confidence score of 0.98. According to the farmland scene category, the target crop detection model parameters optimized for the farmland scene are loaded from the pre-established target crop detection model parameter library. For example, the anchor box size is adjusted to [32, 64, 128, 256, 512], which is more suitable for the size of crops in the farmland. The number of channels and convolution kernel parameters of the feature extraction layer are adjusted accordingly to improve the detection effect of the target crop detection model in the farmland scene. At the same time, the target crop detection model with dynamically adjusted parameters is used to perform target detection on the image. Assuming that a target crop is detected, the target crop detection model outputs its category as wheat with a confidence score of 0.85.

[0070] An adaptive weight distribution strategy is used to calculate the fusion confidence score based on the confidence score of the scene classification model and the confidence score of the target crop detection model. In the farmland scenario, the scene confidence weight is set to 0.4 and the target crop detection confidence weight is set to 0.6. The fusion confidence score is 0.4×0.98 + 0.6×0.85 = The fusion confidence score is 0.902, and this fusion confidence score is used as one of the important bases for the final judgment of crop type. Based on the geographic location and time information recorded during image acquisition, a pre-established crop growth cycle model database is queried. For example, wheat is in the seedling stage in mid-March, and its image characteristics are slender leaves, light green color, and low plant height. The target crop detected is matched and compared with the wheat growth cycle model. By constructing a similarity measurement function based on spatiotemporal characteristics, the similarity score between the target crop and the wheat growth cycle model is calculated to be 0.92, which is higher than the similarity scores of other crops. At the same time, considering that the local soil type is loam and the climate conditions are relatively humid, which are suitable for wheat cultivation, the possibility of the crop type being wheat is further verified.

[0071] Finally, the final determined crop type is wheat and the related detection information, including the target location, the fusion confidence score of 0.902, and the scene category of farmland, is organized into JSON format data and output to the agricultural production management system through the network interface; the actual planting data and crop growth information of the local agricultural departments are regularly collected and compared and analyzed with the output results of the method of the present invention.

[0072] The specific steps of step S2 include:

[0073] S2.1: Load a pre-trained convolutional neural network model and, utilizing the convolutional neural network's hierarchical feature extraction mechanism, input the pre-processed crop image data one by one into the pre-trained convolutional neural network model. Through nonlinear transformations using multiple convolutional and pooling layers, feature vectors containing spatial semantic information are extracted. The convolutional neural network model is prior art in this field and does not constitute an inventive solution of this application, and is not described in detail here.

[0074] S2.2: Load a pre-built multi-layer perceptron model architecture and train the multi-layer perceptron model architecture using the feature vectors obtained in S2.1 to generate a neural network-based scene classification model. The multi-layer perceptron model is prior art in this field and does not constitute an inventive solution of this application, so its detailed description is omitted here.

[0075] S2.3: At the same time, use the LabelImg annotation tool to draw anchor boxes on the preprocessed crop image data, circle the target crop area, and add the corresponding crop type label to each anchor box;

[0076] Furthermore, the specific steps of S2.3 include:

[0077] (1) First, download the LabelImg tool from the official channel and complete the installation. The installation process may vary depending on the operating system. Generally, after the installation is complete, execute the labelimg command in the terminal or command prompt to open the LabelImg software;

[0078] (2) In the LabelImg tool interface, click the Open Directory button to select the folder where the pre-processed crop image data is stored. The tool will load all the images in the folder and display them one by one in the interface in the order of the file names.

[0079] (3) Set the basic annotation parameters in the menu bar or toolbar, for example, determine the shape of the annotation box and the format of the annotation. In this invention, the anchor box and PASCAL VOC annotation format are selected;

[0080] (4) For each image, press the left button of the mouse at the upper left corner of the target crop area, then drag the mouse to the lower right corner of the target crop area. Release the mouse to draw an anchor box to enclose the target crop area.

[0081] (5) After the anchor frame is drawn, enter the corresponding crop type label in the pop-up label input box, such as rice, wheat, and corn;

[0082] (6) Repeat the above steps to mark all target crop areas in the image;

[0083] (7) After completing the annotation of an image, click the Save button to save the annotation information to an XML file with the same name as the image. The XML file will record the image file name, size, anchor box coordinates of each target crop area, crop type label and other information;

[0084] (8) In the same way, all crop image data in the folder are labeled and saved.

[0085] S2.4: Load a pre-built Faster R-CNN model and input the labeled crop image data into the pre-built Faster R-CNN model. Use the labeled crop image data to train the region proposal network and classification regression layer of the pre-built Faster R-CNN model to generate a target crop detection model. The Faster R-CNN model is prior art in this field and is not an inventive solution of this application. It is not described in detail here.

[0086] The specific steps of step S3 include:

[0087] S3.1: Obtain the preprocessed image to be detected and input it into the scene classification model to obtain the probability distribution of each scene category through the neural network forward reasoning Based on the probability distribution of each scene category, the category with the largest probability value is selected as the scene category to which the image to be detected belongs, as predicted by the scene classification model. , the probability distribution of the scene category is the corresponding confidence score, where, represents the probability distribution of the i-th scene category, Represents the output vector of the last fully connected layer of the scene classification model The i-th element in n represents the total number of scene categories, represents the predicted scene category, Represents the maximum independent variable point set function, which means when Pick When The maximum value of the value range;

[0088] S3.2: Based on the scene classification result obtained by the scene classification model, locate the pre-established target crop detection model parameter library for different scenes, and search the target crop detection model parameter library for the location of the matching target crop detection model optimization parameters;

[0089] It should be understood that, assuming that the scene classification model outputs three scene categories, namely the farmland scene, orchard scene, and vegetable field scene, the corresponding probabilities are 0.6, 0.3, and 0.1 respectively, then the farmland scene is determined to be the classification result this time; and the pre-established target crop detection model parameter library for different scenes is divided into different folders according to the scene name, and each folder stores the model parameter file of the corresponding scene, such as the PyTorch model parameter file saved in .pth format. According to the determined scene category, the location of the target crop detection model optimization parameters that match it is found in the parameter library. For example, after determining that it is a farmland scene, the parameter file path specially saved for farmland scene optimization is found in the parameter library.

[0090] S3.3: Use the parameter loading function to load the found target crop detection model optimization parameters into memory. By traversing each key-value pair in the loaded optimization parameter dictionary, the original parameters with corresponding names in the target crop detection model are replaced with the loaded optimization parameters, thus completing the adjustment of the target crop detection model.

[0091] S3.4: Input the preprocessed image to be detected into the adjusted target crop detection model and output the category of the target crop detected in the image , the corresponding confidence score and the final position coordinates of the target crop , where the final position coordinates satisfy: , Indicates the category of target crop c, represents the original output of the classification layer of the target crop detection model for category c, c represents the category of the target crop, m represents the total number of categories, Indicates the final position coordinates of the target crop, Represents the coordinate and size information of the anchor box, 、 Represent the horizontal and vertical coordinate positions of the anchor box respectively, 、 Represent the length and width of the anchor box respectively, Represents the predicted position offset, and Indicates the offset of the target crop center coordinate relative to the candidate region center coordinate. and Indicates the scaling ratio of the width and height of the target crop relative to the candidate region, is an exponential function, Indicates environmental factors, represents the light intensity factor, represents the wind speed factor, represents the soil moisture factor, 、 、 Represents the corresponding weight parameter.

[0092] Furthermore, the specific steps of S3.4 include:

[0093] (1) First, the pre-processed image to be detected is resized to meet the input size requirements of the target crop detection model. Then, the pixel values ​​are normalized to ensure that the data is in an appropriate numerical range for model calculation. Finally, the image data format is adjusted to a tensor format that can be accepted by the deep learning framework where the model is located. This completes the image preprocessing preparation so that it can be smoothly input into the target crop detection model.

[0094] (2) The pre-processed image to be detected is input into the adjusted target crop detection model. The model performs forward propagation calculations based on its internal convolutional layer, pooling layer, fully connected layer and corresponding algorithm mechanism, and outputs result data containing relevant information such as target crop category and location. However, this data is usually encoded and organized in a specific way by the model, and further analysis is required to obtain intuitive and usable information.

[0095] (3) Based on the output of the model, the category, location and confidence score of the target crop are analyzed respectively. The crop category is determined by finding the index with the highest category probability. The output position parameters are converted into actual image coordinates according to the coordinate transformation method defined by the model, and the corresponding confidence score data is extracted from the output. Finally, the category, location and corresponding confidence score of the target crop detected in the image are obtained as the final output result for subsequent analysis, application, etc.

[0096] The specific steps of step S4 include:

[0097] S4.1: Extract the confidence score of each target crop detection result from the output of the target crop detection model At the same time, the confidence score of the corresponding scene is obtained from the output of the scene classification model ,in, represents the scene category predicted by the scene classification model;

[0098] S4.2: Setting the weighting coefficient for the target crop detection confidence score and the weighted coefficient of the scene classification confidence score , and satisfies ;

[0099] S4.3: For each target crop q, calculate the fusion confidence score .

[0100] The specific steps of step S5 include:

[0101] S5.1: Extracting geographic location and time information from the image acquisition record, and using the extracted geographic location and time information as an index to query a pre-established crop growth cycle model database;

[0102] For example, assuming that the database stores information in a key-value data structure, relevant data can be obtained by constructing a query statement. For example, the query obtains the set of crop types that can be planted at the location (lon, lat) and time t and their growth cycle related parameters.

[0103] S5.2: Obtain the target crop scene information and the calculated fusion confidence score from the target crop detection results , for each crop obtained from the crop growth cycle model database ,Each crop obtained from the query is evaluated based on the scene information and fusion confidence score;

[0104] If the crop obtained by querying the crop growth cycle model database The query result of is the same as the scene in the target crop detection result, and ,but For crops that actually exist, Indicates the preset confidence threshold;

[0105] For example, if a certain crop A is usually grown in an orchard scene, and the scene in the target crop detection result is also an orchard, and the fusion confidence score The higher the value, the greater the possibility that A is the actual crop type.

[0106] S5.3: For each crop that actually exists , matching its growth cycle model with the spatiotemporal information and crop appearance features in the target crop detection results to obtain a matching result;

[0107] Among them, the growth cycle model includes parameters such as sowing time, growth stage, and maturity time.

[0108] For example, if the growth cycle model of crop A shows that it should be in the middle growth stage at time t, and the crop appearance characteristics in the target crop detection results, such as plant height, leaf morphology, etc., are consistent with the middle growth stage characteristics, then A has a high matching degree.

[0109] S5.4: Based on the matching results, the crop type with the highest matching degree is selected as the final crop type, and the final crop type and detection information are output as the result, where the detection information includes the crop position, confidence level, and scene information.

[0110] Example 2

[0111] See also Figure 3 Another embodiment of the present invention provides a crop identification system based on a combination of a scene recognition model and target detection, comprising:

[0112] Data processing module, model building module, crop detection and classification module, confidence fusion module, crop type determination module;

[0113] Data processing module, used to collect crop image data and perform preprocessing;

[0114] A model building module is used to build a scene classification model and a target crop detection model based on the preprocessed crop image data;

[0115] The crop detection and classification module is used to input the image to be detected into the pre-trained scene classification model and target crop detection model, perform scene classification and target crop detection respectively, and output the detection results;

[0116] The confidence fusion module is used to perform weighted fusion of the confidence scores output by the target crop detection model and the confidence scores output by the scene classification model to improve the accuracy of crop recognition;

[0117] The crop type determination module is used to combine the spatiotemporal information during image acquisition, query the crop growth cycle model database, determine the crop type in the image, and output the final result.

[0118] The data processing module includes: an image data collection unit, a spatiotemporal information recording unit, and an image preprocessing unit;

[0119] Image data collection unit, used to collect crop image data from farmland monitoring cameras and agricultural drones in different regions and time periods;

[0120] A spatiotemporal information recording unit, used to record the geographical location and time information of each image, such as the geographical location and time of image capture;

[0121] The image preprocessing unit is used to perform preprocessing operations on the collected crop image data, such as removing noise interference in the image to make the image clearer; performing grayscale processing and normalization processing to unify image features such as pixel values ​​into a specific range, so that the data format meets the requirements of subsequent model training and processing, and improve the overall stability and accuracy of the system.

[0122] The model building module includes: feature extraction unit, target crop labeling unit, and model building unit;

[0123] A feature extraction unit is used to extract features from the preprocessed crop image data using a feature extraction algorithm. These features will be used to train the model;

[0124] The target crop labeling unit is used to manually label the pre-processed crop image data using professional image labeling tools, select the target crop area in the image, and label the corresponding crop type label;

[0125] The model building unit is used to build a scene classification model using the extracted features, and at the same time, build a target crop detection model based on the labeled crop image data.

[0126] The crop detection and classification module includes: scene classification unit, target crop detection unit, and parameter loading unit;

[0127] The scene classification unit is used to input the image to be detected into a pre-trained scene classification model. The scene classification model determines the scene category to which the image belongs and outputs the corresponding confidence score. This determines the type of crop growth scene the image is likely to belong to, such as farmland, orchard, or greenhouse, and provides a scenario basis for subsequent targeted adjustments to the target crop detection model.

[0128] The target crop detection unit is used to load the corresponding target crop detection model optimization parameters based on the scene classification results, perform target crop detection on the image to be detected, and ultimately output the category, location, and corresponding confidence score of the target crop detected in the image to be detected, completing the preliminary detection and positioning of the target crop in the image;

[0129] The parameter loading unit is used to search and load the target crop detection model optimization parameters of the corresponding scene from the pre-established target crop detection model parameter library according to the scene category results output by the scene classification unit, so as to realize adaptive parameter adjustment of the target crop detection model according to different scenes, so that it can better adapt to the scene characteristics of the current image to be detected and improve the accuracy of detection.

[0130] The confidence fusion module includes: a confidence calculation unit and a weighted fusion unit;

[0131] A confidence calculation unit, used to calculate the confidence score output by the target crop detection model and the confidence score output by the scene classification model;

[0132] The weighted fusion unit is used to perform weighted fusion on the confidence scores output by the target crop detection model and the confidence scores output by the scene classification model according to preset weights to obtain a fused confidence score.

[0133] The crop type determination module includes: a spatiotemporal information query unit, a crop type matching unit, and a result output unit;

[0134] The spatiotemporal information query unit is used to query a pre-established crop growth cycle model database based on the geographical location and time information recorded during image acquisition. The database stores the growth cycle characteristics of various crops in different regions and at different times.

[0135] The crop species matching unit is used to combine the detected target crops, fusion confidence scores and corresponding spatiotemporal information with the information in the crop growth cycle model database to determine the crop species and realize the core function of the entire system;

[0136] The result output unit is used to output the final determined crop type and detection information as a result for user viewing or further analysis, wherein the detection information includes the location of the target crop, the original confidence score, and the fused confidence score.

[0137] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiments are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also change, modify, replace and modify the above-mentioned embodiments without departing from the purpose and scope of protection of the present invention. These are all protected by the present invention.

Claims

1. A crop identification method based on a combination of scene recognition model and target detection, characterized in that: include: Step S1: Collect crop image data and perform preprocessing; Step S2: Based on the preprocessed crop image data, a neural network-based scene classification model and a target crop detection model are constructed respectively; Step S3: Inputting the image to be detected into a pre-trained neural network-based scene classification model to obtain the scene category and corresponding confidence score to which the image to be detected belongs. Simultaneously, based on the scene classification result, the optimized parameters of the target crop detection model for the corresponding scene are loaded from a pre-established target crop detection model parameter library. The adjusted target crop detection model is used to perform target detection on the image to be detected, and a detection result for the image to be detected is output; the detection result includes the category and location of the target crop and the corresponding confidence score. Step S4: performing weighted fusion on the confidence score of the target crop and the confidence score of the corresponding scene output by the scene classification model to obtain a fused confidence score; Step S5: According to the geographical location and time information recorded when the image is collected, the detection result is matched with the pre-established crop growth cycle model database to determine the crop type in the image to be detected, and the final determined crop type and detection information are output as the result.

2. The crop identification method based on the combination of scene recognition model and target detection according to claim 1, characterized in that: The specific steps of step S2 include: S2.1: Load the pre-trained convolutional neural network model and use the convolutional neural network's hierarchical feature extraction mechanism to input the pre-processed crop image data one by one into the pre-trained convolutional neural network model. Through the nonlinear transformation of multiple convolutional layers and pooling layers, feature vectors containing spatial semantic information are extracted. S2.2: Load the pre-built multi-layer perceptron model architecture and train it using the feature vectors obtained in S2.1 to generate a neural network-based scene classification model. S2.3: At the same time, use the LabelImg annotation tool to draw anchor boxes on the preprocessed crop image data, circle the target crop area, and add the corresponding crop type label to each anchor box; S2.4: Load the pre-built Faster R-CNN model and input the labeled crop image data into the pre-built Faster R-CNN model. Use the labeled crop image data to train the region proposal network and classification regression layer of the pre-built Faster R-CNN model to generate a target crop detection model.

3. The crop identification method based on the combination of scene recognition model and target detection according to claim 2, characterized in that: The specific steps of step S3 include: S3.1: Obtain the preprocessed image to be detected and input it into the scene classification model to obtain the probability distribution of each scene category through the neural network forward reasoning Based on the probability distribution of each scene category, the category with the largest probability value is selected as the scene category to which the image to be detected belongs, as predicted by the scene classification model. , the probability distribution of the scene category is the corresponding confidence score, where, represents the probability distribution of the i-th scene category, Represents the output vector of the last fully connected layer of the scene classification model The i-th element in n represents the total number of scene categories, represents the predicted scene category, Represents the maximum independent variable point set function, which means when Pick When The maximum value of the value range; S3.2: Based on the scene classification result obtained by the scene classification model, locate the pre-established target crop detection model parameter library for different scenes, and search the target crop detection model parameter library for the location of the matching target crop detection model optimization parameters; S3.3: Use the parameter loading function to load the found target crop detection model optimization parameters into memory. By traversing each key-value pair in the loaded optimization parameter dictionary, the original parameters with corresponding names in the target crop detection model are replaced with the loaded optimization parameters to complete the adjustment of the target crop detection model.

4. The crop identification method based on the combination of scene recognition model and target detection according to claim 3, characterized in that: The specific steps of step S3 also include: S3.4: Input the preprocessed image to be detected into the adjusted target crop detection model and output the category of the target crop detected in the image , the corresponding confidence score and the final position coordinates of the target crop .

5. The crop identification method based on the combination of scene recognition model and target detection according to claim 4, characterized in that: The specific steps of step S4 include: S4.1: Extract the confidence score of each target crop detection result from the output of the target crop detection model ,At the same time, the confidence score of the corresponding scene is obtained from the output of the ,scene classification model; S4.2: Setting the weighting coefficient for the target crop detection confidence score and the weighted coefficient of the scene classification confidence score , and satisfies ; S4.3: For each target crop q, the fusion confidence score is obtained by calculating the weighted cumulative sum of the confidence scores of each target crop and the corresponding scene .

6. The crop identification method based on the combination of scene recognition model and target detection according to claim 5, characterized in that: The specific steps of step S5 include: S5.1: Extracting geographic location and time information from the image acquisition record, and using the extracted geographic location and time information as an index to query a pre-established crop growth cycle model database; S5.2: Obtain the target crop scene information and the calculated fusion confidence score from the target crop detection results , for each crop obtained from the crop growth cycle model database ,Each crop obtained from the query is evaluated based on the scene information and fusion confidence score; If the crop obtained by querying the crop growth cycle model database The query result of is the same as the scene in the target crop detection result, and ,but Belong to actual crops, among which, Indicates the preset confidence threshold; S5.3: For each crop that actually exists , matching its growth cycle model with the spatiotemporal information and crop appearance features in the target crop detection results to obtain a matching result; S5.4: Based on the matching results, the crop type with the highest matching degree is selected as the final crop type, and the final crop type and detection information are output as the result, where the detection information includes the crop position, confidence level, and scene information.

7. A crop determination system based on a combination of scene recognition model and target detection, which is used to implement the crop determination method based on a combination of scene recognition model and target detection according to any one of claims 1 to 6, characterized in that: include: Data processing module, model building module, crop detection and classification module, confidence fusion module, crop type determination module; The data processing module is used to collect crop image data and perform preprocessing; The model building module is used to build a scene classification model and a target crop detection model based on the preprocessed crop image data; The crop detection and classification module is used to input the image to be detected into the pre-trained scene classification model and target crop detection model, perform scene classification and target crop detection respectively, and output the detection results; The confidence fusion module is used to perform weighted fusion of the confidence score output by the target crop detection model and the confidence score output by the scene classification model; The crop type determination module is used to combine the spatiotemporal information during image acquisition, query the crop growth cycle model database, determine the crop type in the image, and output the final result.

8. The crop identification system based on the combination of scene recognition model and target detection according to claim 7, characterized in that: The crop detection and classification module includes: a scene classification unit, a target crop detection unit, and a parameter loading unit; The scene classification unit is used to input the image to be detected into a pre-trained scene classification model, and the scene classification model determines the scene category to which the image to be detected belongs and outputs a corresponding confidence score; The target crop detection unit is configured to load the corresponding target crop detection model optimization parameters according to the scene classification result, perform target crop detection on the image to be detected, and output the category, position, and corresponding confidence score of the target crop detected in the image to be detected; The parameter loading unit is used to search and load the target crop detection model optimization parameters of the corresponding scene from a pre-established target crop detection model parameter library according to the scene classification result output by the scene classification unit.

9. The crop identification system based on the combination of scene recognition model and target detection according to claim 8, characterized in that: The confidence fusion module includes: a confidence calculation unit and a weighted fusion unit; The confidence calculation unit is used to calculate the confidence score output by the target crop detection model and the confidence score output by the scene classification model; The weighted fusion unit is used to perform weighted fusion on the confidence score output by the target crop detection model and the confidence score output by the scene classification model according to preset weights to obtain a fused confidence score.

Images