Tea leaf yield estimation method, device, equipment and medium
Through deep learning tea green yield estimation method, multi-scale feature extraction and density map generation are used to solve the accuracy and dense occlusion problems of tea green yield estimation in tea gardens, and the accurate and efficient estimation of tea leaf yield is achieved, providing more accurate information and rapid collection scheme.
Patent Information
- Application Number
- CN202411127924.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-19
AI Technical Summary
The existing tea green yield estimation method has limited fitting ability, and the estimation accuracy is difficult to guarantee. Especially in unstructured tea gardens, it is difficult to solve the problem of dense crops or occlusion, resulting in missed inspections and missed inspections, which are difficult to meet the production estimation requirements.
Using the deep learning-based tea green yield estimation method, a tea green yield estimation model is constructed through multi-scale feature extraction, ROI Align pooling layer and density prediction module, and a density map is generated to determine the tea green yield in the target area of the tea garden. Use the drone platform to quickly and efficiently collect data.
It realizes the estimation of tea green yield in complex environments, provides more accurate and rich yield information, reduces the labeling workload, is suitable for tea planting and research, and provides new estimation ideas.
Smart Images

Figure CN118982121A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of agricultural production, and in particular to a method for estimating green tea yield, a corresponding device, an electronic device and a computer-readable storage medium. Background Art
[0002] As an important cash crop, tea is widely planted around the world. As one of the origins of tea, China has a long history of tea drinking and tea planting culture, and its tea planting area, output and sales volume are all world-leading. At present, affected by uncertain factors such as natural disasters, short-term shocks and market supply, in order to ensure the safety of the tea industry and respond to changes in market demand for tea consumption, it is necessary to estimate tea production in advance so as to formulate relevant measures and regulatory policies to ensure market stability and people's normal life needs. As a crucial link in the tea production process, timely and accurate estimation of tea production and accurate acquisition of tea garden tea production information can not only provide decision-making support for field management such as light, water and fertilizer during tea growth, but also these information are important data for realizing modern precision agriculture.
[0003] At present, the main method of measuring tea yield is to estimate the yield by manual sampling, picking and weighing. This method is not only labor-intensive and subjective, but also tea leaves are high-density, highly overlapping, of the same color, and of different sizes and shapes. In addition, the tea garden background in the natural environment is complex, the weather is changeable, and there are many interference factors, which makes the confidence of manual field yield estimation data low and difficult to provide a scientific reference. The existing statistical yield estimation method is to describe the relationship between crop yield and various meteorological factors by combining meteorological data with crop yields of previous years to form a crop statistical regression model. Although this method can easily and intuitively estimate crop yields, the method has limited fitting ability. In addition to being affected by the accuracy and amount of meteorological data, it is also affected by various environmental factors such as soil, pests and diseases, and the accuracy of the estimation is difficult to guarantee.
[0004] The recognition accuracy of the tea yield estimation method based on machine learning of shallow features is easily affected by factors such as field environment and natural light, making it difficult to provide technical support for tea yield estimation. Compared with the above methods, the tea yield estimation method based on deep learning of target detection has achieved certain results, but in unstructured tea gardens, the model is difficult to solve the problem of dense crops or occlusion, resulting in false detection and missed detection, resulting in poor detection results and difficulty in meeting the estimation requirements. In addition, in standard machine learning tasks, a large amount of labeled data is usually relied on to train the model. However, this is not practical in many actual scenarios and consumes a lot of energy and time.
[0005] To sum up, the tea yield estimation method used in the prior art has limited fitting ability, and its estimation accuracy is difficult to guarantee. In addition, in unstructured tea gardens, the model is difficult to solve the problem of dense crops or occlusion, resulting in false detection and missed detection, resulting in poor detection results and difficulty in meeting the estimation requirements. The applicant has made corresponding explorations to solve this problem. Summary of the invention
[0006] The purpose of the present application is to solve the above-mentioned problems and to provide a method for estimating green tea yield, a corresponding device, an electronic device and a computer-readable storage medium.
[0007] In order to meet the various objectives of this application, this application adopts the following technical solutions:
[0008] A method for estimating green tea yield proposed to meet one of the purposes of this application comprises:
[0009] In response to the tea leaves yield estimation instruction, an image of a target area of a tea garden containing tea leaves is acquired;
[0010] The multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergent state performs feature extraction on the target area image of the tea garden to extract a multi-scale feature map corresponding to the target area image of the tea garden;
[0011] Extracting a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map;
[0012] Generate a feature correlation graph based on the multi-scale feature graph and the sample feature graph based on the feature matching layer in the green tea yield estimation model;
[0013] The feature correlation map is input into the density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and the green tea yield in the target area of the tea garden is determined based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
[0014] Optionally, the step of extracting features of the target area image of the tea garden by using a multi-scale feature extraction module of a backbone network of a feature extraction module in a green tea yield estimation model that has been trained to a convergent state to extract a multi-scale feature map corresponding to the target area image of the tea garden includes:
[0015] Inputting the tea garden target area image into the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergence state;
[0016] The multi-scale feature extraction module in the backbone network generates feature maps of different resolutions through different convolutional layers and pooling layers to capture features of different scales and levels in the image;
[0017] Output a set of multi-scale feature maps, wherein the multi-scale feature maps contain detailed information of the tea garden image at different scales.
[0018] Optionally, the step of extracting a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map includes:
[0019] Inputting the multi-scale feature map into the ROI Align pooling layer in the green tea yield estimation model to determine the region of interest corresponding to the green tea, wherein the region of interest represents the position of the green tea in the target area image of the tea garden;
[0020] Fine-grained spatial alignment is performed on each region of interest, a green tea feature map corresponding to the green tea area in the multi-scale feature map is extracted, and the green tea feature map is used as a sample feature map.
[0021] Optionally, the step of generating a feature correlation map based on the multi-scale feature map and the sample feature map based on the feature matching layer in the green tea yield estimation model includes:
[0022] Determine the multi-scale feature map output by the multi-scale feature extraction module and the sample feature map output by the ROI Align pooling layer;
[0023] Inputting the multi-scale feature map and the sample feature map into the feature matching layer in the green tea yield estimation model, and calculating and determining the similarity value between the multi-scale feature map and the sample feature map;
[0024] The similarity values between the multi-scale feature map and the sample feature map are arranged in a matrix form to form a feature correlation map, wherein each element of the feature correlation map represents the matching degree of a certain position of the sample feature map in the multi-scale feature map.
[0025] Optionally, the step of inputting the feature correlation map into a density prediction module in the green tea yield estimation model to determine a density map corresponding to the green tea image includes:
[0026] In the density prediction module of the green tea yield estimation model, the feature correlation graph is subjected to convolution operation, nonlinear activation function, and pooling layer processing to extract higher-level features;
[0027] The processed feature correlation map is converted into a density map corresponding to the green tea image, wherein the density map shows the green tea density information at each position in the target area image of the tea garden.
[0028] Optionally, the step of determining the yield of green tea in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image comprises:
[0029] Extracting a pixel density value of each pixel in the density map corresponding to the green tea image;
[0030] Adding the pixel density values of all pixels in the density map corresponding to the green tea image to obtain the total pixel density value of the density map corresponding to the green tea image;
[0031] Determine the actual weight corresponding to each unit density value, and based on the actual weight corresponding to each unit density value and the total pixel density value, convert the total pixel density value of the density map corresponding to the green tea image into the actual weight of green tea to complete the estimation of green tea yield.
[0032] Optionally, the basic network architecture of the green tea yield estimation model is a few-sample density map estimation neural network.
[0033] A tea leaf yield estimation device provided for another purpose of the present application comprises:
[0034] The target area image acquisition module is configured to respond to the tea leaf yield estimation instruction and acquire the target area image of the tea garden containing the tea leaves;
[0035] A multi-scale feature map determination module is configured to extract features of the target area image of the tea garden based on a multi-scale feature extraction module of a backbone network of a feature extraction module in a green tea yield estimation model that has been trained to a convergence state, so as to extract a multi-scale feature map corresponding to the target area image of the tea garden;
[0036] A sample feature map determination module is configured to extract a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and use the green tea feature map as a sample feature map;
[0037] A feature correlation graph determination module is configured to generate a feature correlation graph based on the multi-scale feature graph and the sample feature graph based on the feature matching layer in the green tea yield estimation model;
[0038] The green tea yield estimation module is configured to input the feature correlation map into the density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and determine the green tea yield in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
[0039] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the tea leaf yield estimation method described in the present application.
[0040] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the tea leaf yield estimation method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.
[0041] Compared with the prior art, the present application aims at the problems that the tea yield estimation method in the prior art has limited fitting ability, its estimation accuracy is difficult to guarantee, and in the unstructured tea garden, the model is difficult to solve the problem of dense crops or occlusion, resulting in false detection and missed detection, thus resulting in poor detection effect and difficulty in meeting the estimation requirements. The present application includes but is not limited to the following beneficial effects:
[0042] The green tea yield estimation method of the present application collects data from multiple dimensions in the tea garden and constructs a green tea yield estimation model based on density map estimation. The green tea yield estimation model can accurately and efficiently estimate the yield in the target area of the orchard. Compared with the target detection estimation method based on deep learning, it can realize the green tea yield estimation in complex environments such as tea leaves being illuminated, blocked, and dense, and can provide more accurate and rich yield information. In addition, compared with the tedious labeling work based on the target detection counting model, this application does not need to label all samples, but only provides a solution for labeling a small number of samples;
[0043] Furthermore, this method realizes fast and efficient data collection based on the UAV platform, and realizes tea yield estimation through the constructed green tea yield estimation model, providing some guiding opinions and references for tea growers and researchers, and providing a new idea and method for accurate tea yield estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0045] Figure 1 This is a schematic diagram of the process of estimating the yield of green tea leaves in the embodiment of the present application;
[0046] Figure 2 This is an exemplary network architecture of the green tea yield estimation model in the embodiment of the present application;
[0047] Figure 3 This is a diagram showing the effect of the density map corresponding to the green tea image in the embodiment of the present application;
[0048] Figure 4 This is a principle block diagram of the tea leaf yield estimation device in the embodiment of the present application;
[0049] Figure 5 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION
[0050] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0051] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0052] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.
[0053] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.
[0054] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.
[0055] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.
[0056] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.
[0057] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.
[0058] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.
[0059] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.
[0060] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.
[0061] See also Figure 1 , Figure 2 as well as Figure 3 In one embodiment, the method for estimating green tea yield of the present application comprises:
[0062] Step S10, responding to the green tea yield estimation instruction, acquiring an image of a target area of a tea garden containing green tea;
[0063] The green tea yield estimation system in the computer terminal device can respond to the green tea yield estimation instruction and obtain the target area image of the tea garden containing green tea;
[0064] In some embodiments, the tea garden target area image can be acquired by a binocular camera or other camera equipment in a drone. The tea garden target area image can be an image of the tea garden target area under weak light or strong light, an image of the tea garden target area on a cloudy day or a sunny day, an image of the tea garden target area at 2m, 3m, or 4m, an image of the tea garden target area in summer, autumn, or winter, etc. The above image data can be used as a training set for a green tea yield estimation model.
[0065] In some embodiments, the present application adopts a density map estimation counting technology based on deep learning to realize the counting and yield estimation of green tea leaves in a natural environment, and combines multiple dimensions such as light, weather, season, and altitude for data collection, so that the yield assessment of green tea leaves is based on the actual growth environment of green tea leaves, effectively improving the generalization and accuracy of green tea yield assessment.
[0066] The steps of constructing the tea yield estimation model include:
[0067] Step S101, collecting tea leaf data set;
[0068] Step S102: data preprocessing;
[0069] Step S103, dividing the data set;
[0070] Step S104, generating a true value density map;
[0071] Step S105: establishing a green tea yield estimation model to evaluate green tea yield.
[0072] The step S101 specifically includes the following steps:
[0073] The tea garden is divided into image collection areas; data is collected based on the real growth environment of green tea leaves in different dimensions such as light, weather, altitude and season based on drones; and images of the target area of the tea garden containing green tea leaves are acquired by using automatic cruising and vertical shooting of drones.
[0074] The step S102 specifically includes the following steps:
[0075] Eliminate invalid data such as over-blurred, severely distorted, and repeated shots; crop the high-resolution tea images collected by the drone through sliding windows to generate multiple sub-images; annotate the tea leaves that meet the picking standards in the cropped images based on the Computer Vision Annotation Tool annotation platform; construct a tea leaf dataset by combining rectangular frame annotation and point annotation during the annotation process; rectangular frame annotation extracts relevant features of the tea leaves, and generates sample frame features through pooling, usually three to five random annotations; point annotation annotates the center points of all the tea leaves in the image for density map generation.
[0076] The step S103 specifically includes the following steps:
[0077] The labeled data set is divided into a training set, a test set, and a validation set in a ratio of 6:2:2; there is no duplicate data in the training set, the test set, and the validation set, and the combined number set of the training set, the test set, and the validation set is less than or equal to the data set.
[0078] The step S104 specifically includes the following steps:
[0079] The positions of the tea leaves marked manually have certain deviations from the actual tea leaves, which affects the overall performance of the trained model. Compared with the total number of tea leaves, the density map can more intuitively reflect the spatial distribution of tea leaves. The sample box marked in each picture is used as the Gaussian window size for generating the true value density map, and the point labels are processed to generate the density map.
[0080] The step S105 specifically includes the following steps:
[0081] Establish a green tea yield estimation model to predict green tea yield, use the training data set to train the green tea yield estimation model, input the training set into the model to generate the green tea yield estimation model; input the validation set data features into the model for prediction to verify the model accuracy and adjust the network structure, and optimize the green tea yield prediction model;
[0082] The mean absolute error (MAE) and root mean square error (RMSE) are used as evaluation indicators of the tea yield estimation model, and the yield estimation model with the smallest MAE and RMSE values is selected as the optimal yield estimation model. The evaluation indicator formula is as follows:
[0083]
[0084] In the formula represents the actual number of green tea leaves in the i-th image, Indicates the number of predicted green tea leaves in the i-th image. Input some pictures and a small amount of sample labeled test set data into the optimal yield estimation model. Visually display the difference ratio between the green tea yield value output by the green tea yield estimation model and the actual yield value and compare it with the preset difference ratio to intuitively reflect the accuracy of the green tea yield estimation model.
[0085] Based on the above steps, the green tea yield estimation model is trained. When the green tea yield estimation model is trained to a convergence state, it can be put into actual production use to estimate the density map corresponding to the green tea image in the target area image of the tea garden.
[0086] Step S20, performing feature extraction on the target area image of the tea garden based on the multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergent state, so as to extract a multi-scale feature map corresponding to the target area image of the tea garden;
[0087] After obtaining the target area image of the tea garden containing green tea leaves, the multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergence state performs feature extraction on the target area image of the tea garden to extract the multi-scale feature map corresponding to the target area image of the tea garden; the basic network architecture of the green tea yield estimation model is a few-sample density map estimation neural network.
[0088] In some embodiments, the few-sample density map estimation neural network includes a feature extraction module, a ROI Align pooling module, a feature matching layer, a density prediction module, etc.
[0089] Feature extraction module: In deep convolutional neural networks, the receptive field of shallow networks is small, and the geometric detail information is strong, but the semantic information representation ability is weak, while the receptive field of deep networks is relatively large, the semantic information representation ability is strong, but the spatial geometric feature details are lacking. Therefore, the multi-scale feature extraction module in the feature extraction module extracts better features in the image by fusing features of different pyramid scales.
[0090] ROI Align pooling layer: used to extract sample features. It only focuses on the features of the area of interest, finds the feature map of the area of interest, that is, the features of the rectangular box in the figure, upsamples or downsamples the sample feature map, and then splices it.
[0091] Feature matching layer: Using the correlation between the sample feature map and the entire image feature, the sample is used as a convolution kernel to match the original image and convolve to obtain a feature correlation map, which is used as the input of the density prediction module for prediction.
[0092] Density prediction module: This module realizes the prediction of tea leaf density map.
[0093] Further, the multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergent state performs feature extraction on the target area image of the tea garden to extract a multi-scale feature map corresponding to the target area image of the tea garden, comprising:
[0094] Step S201, inputting the tea garden target area image into the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergence state;
[0095] Step S202: The multi-scale feature extraction module in the backbone network generates feature maps of different resolutions through different convolutional layers and pooling layers to capture features of different scales and levels in the image;
[0096] Step S203: output a set of multi-scale feature maps, wherein the multi-scale feature maps contain detailed information of the tea garden image at different scales.
[0097] Step S30, extracting a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map;
[0098] After obtaining the multi-scale feature map corresponding to the target area image of the tea garden, extracting the green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map;
[0099] Further, the step of extracting the green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map includes:
[0100] Step S301, inputting the multi-scale feature map into the ROI Align pooling layer in the green tea yield estimation model to determine the region of interest corresponding to the green tea, wherein the region of interest represents the position of the green tea in the target area image of the tea garden;
[0101] Step S302: Perform fine-grained spatial alignment on each region of interest, extract the green tea feature map corresponding to the green tea area in the multi-scale feature map, and use the green tea feature map as a sample feature map.
[0102] Step S40, generating a feature correlation graph based on the multi-scale feature graph and the sample feature graph based on the feature matching layer in the green tea yield estimation model;
[0103] Extracting a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map, generating a feature correlation map based on the multi-scale feature map and the sample feature map based on the feature matching layer in the green tea yield estimation model;
[0104] Furthermore, the step of generating a feature correlation map based on the multi-scale feature map and the sample feature map based on the feature matching layer in the green tea yield estimation model includes:
[0105] Step S401, determining the multi-scale feature map output by the multi-scale feature extraction module and the sample feature map output by the ROIAlign pooling layer;
[0106] Step S402, inputting the multi-scale feature map and the sample feature map into the feature matching layer in the green tea yield estimation model, and calculating and determining the similarity value between the multi-scale feature map and the sample feature map;
[0107] Step S403: Arrange the similarity values between the multi-scale feature map and the sample feature map in a matrix form to form a feature correlation map, wherein each element of the feature correlation map represents the matching degree of a certain position of the sample feature map in the multi-scale feature map.
[0108] Step S50, input the feature correlation map into the density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and determine the green tea yield in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
[0109] Based on the feature matching layer in the green tea yield estimation model, after generating a feature correlation map according to the multi-scale feature map and the sample feature map, the feature correlation map is input into the density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and the green tea yield in the target area of the tea garden is determined based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
[0110] Furthermore, the step of inputting the feature correlation graph into the density prediction module in the green tea yield estimation model to determine the density graph corresponding to the green tea image includes:
[0111] Step S501: In the density prediction module of the green tea yield estimation model, the feature correlation graph is subjected to convolution operation, nonlinear activation function, and pooling layer processing to extract higher-level features;
[0112] Step S502: converting the processed feature correlation map into a density map corresponding to the green tea image, wherein the density map displays the green tea density information at each position in the target area image of the tea garden.
[0113] Furthermore, the step of determining the yield of green tea in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image includes:
[0114] Step S5001, extracting the pixel density value of each pixel in the density map corresponding to the green tea image;
[0115] Step S5002, adding the pixel density values of all pixels in the density map corresponding to the green tea image to obtain the total pixel density value of the density map corresponding to the green tea image;
[0116] Step S5003, determine the actual weight corresponding to each unit density value, and based on the actual weight corresponding to each unit density value and the total pixel density value, convert the total pixel density value of the density map corresponding to the green tea image into the actual weight of green tea to complete the estimation of green tea yield.
[0117] Specifically, if the density value unit is the amount of green tea per square unit (such as grams), the density values of all pixels in the area can be added to obtain the total density value of the target area, and the calculated total density value can be compared with the actual sampling data to convert the density value into the actual weight or quantity of green tea.
[0118] It can be seen from the above embodiments that, compared with the prior art, the present application aims at the problems that the tea yield estimation method in the prior art has limited fitting ability, its estimation accuracy is difficult to guarantee, and in the unstructured tea garden, the model is difficult to solve the problem of dense crops or occlusion, resulting in false detection and missed detection, thus resulting in poor detection effect and difficulty in meeting the estimation requirements. The present application includes but is not limited to the following beneficial effects:
[0119] The green tea yield estimation method of the present application collects data from multiple dimensions in the tea garden and constructs a green tea yield estimation model based on density map estimation. The green tea yield estimation model can accurately and efficiently estimate the yield in the target area of the orchard. Compared with the target detection estimation method based on deep learning, it can realize the green tea yield estimation in complex environments such as tea leaves being illuminated, blocked, and dense, and can provide more accurate and rich yield information. In addition, compared with the tedious labeling work based on the target detection counting model, this application does not need to label all samples, but only provides a solution for labeling a small number of samples;
[0120] Furthermore, this method realizes fast and efficient data collection based on the UAV platform, and realizes tea yield estimation through the constructed green tea yield estimation model, providing some guiding opinions and references for tea growers and researchers, and providing a new idea and method for accurate tea yield estimation.
[0121] See also Figure 4 , a green tea yield estimation device provided to meet one of the purposes of the present application, includes a target area image acquisition module 1100, a multi-scale feature map determination module 1200, a sample feature map determination module 1300, a feature correlation map determination module 1400 and a green tea yield estimation module 1500. Among them, the target area image acquisition module 1100 is configured to respond to the green tea yield estimation instruction and acquire a target area image of a tea garden containing green tea; the multi-scale feature map determination module 1200 is configured to extract features from the target area image of the tea garden based on the multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergence state, so as to extract the multi-scale feature map corresponding to the target area image of the tea garden; the sample feature map determination module 1300 is configured to extract the multi-scale feature map based on the ROIAlign pooling layer in the green tea yield estimation model. The green tea feature map in the green tea image is used as a sample feature map; a feature correlation map determining module 1400 is configured to generate a feature correlation map based on the feature matching layer in the green tea yield estimation model according to the multi-scale feature map and the sample feature map; a green tea yield estimation module 1500 is configured to input the feature correlation map into a density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and determine the green tea yield in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
[0122] Based on any embodiment of this application, please refer to Figure 5 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 5As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a method for estimating the yield of green tea. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the method for estimating the yield of green tea of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0123] In this embodiment, the processor is used to execute Figure 4 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the tea yield estimation device of the present application, and the server can call the program code and data of the server to execute the functions of all submodules.
[0124] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the tea leaf yield estimation method described in any embodiment of the present application.
[0125] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the method for estimating green tea yield described in any embodiment of the present application.
[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0127] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
[0128] In summary, this method achieves fast and efficient data collection based on the UAV platform, and realizes tea yield estimation through the constructed green tea yield estimation model, providing some guiding opinions and references for tea growers and researchers, and providing a new idea and method for accurate tea yield estimation.
Claims
1. A method for estimating green tea yield, characterized in that: include: In response to the tea leaf yield estimation instruction, an image of a target area of a tea garden containing tea leaves is obtained; The multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergent state performs feature extraction on the target area image of the tea garden to extract a multi-scale feature map corresponding to the target area image of the tea garden; Extracting a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and using the green tea feature map as a sample feature map; Generate a feature correlation graph based on the multi-scale feature graph and the sample feature graph based on the feature matching layer in the green tea yield estimation model; The feature correlation map is input into the density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and the green tea yield in the target area of the tea garden is determined based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
2. The method for estimating green tea yield according to claim 1, characterized in that: The step of extracting features of the target area image of the tea garden by a multi-scale feature extraction module of the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergent state to extract a multi-scale feature map corresponding to the target area image of the tea garden includes: Inputting the target area image of the tea garden into the backbone network of the feature extraction module in the green tea yield estimation model that has been trained to a convergent state; The multi-scale feature extraction module in the backbone network generates feature maps of different resolutions through different convolutional layers and pooling layers to capture features of different scales and levels in the image; Output a set of multi-scale feature maps, wherein the multi-scale feature maps contain detailed information of the tea garden image at different scales.
3. The method for estimating green tea yield according to claim 1, characterized in that: The step of extracting a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model and using the green tea feature map as a sample feature map includes: Inputting the multi-scale feature map into the ROI Align pooling layer in the green tea yield estimation model to determine the region of interest corresponding to the green tea, wherein the region of interest represents the position of the green tea in the target area image of the tea garden; Fine-grained spatial alignment is performed on each region of interest, a green tea feature map corresponding to the green tea area in the multi-scale feature map is extracted, and the green tea feature map is used as a sample feature map.
4. The method for estimating green tea yield according to claim 1, characterized in that: The step of generating a feature correlation map based on the multi-scale feature map and the sample feature map based on the feature matching layer in the green tea yield estimation model comprises: Determine the multi-scale feature map output by the multi-scale feature extraction module and the sample feature map output by the ROIAlign pooling layer; Inputting the multi-scale feature map and the sample feature map into the feature matching layer in the green tea yield estimation model, and calculating and determining the similarity value between the multi-scale feature map and the sample feature map; The similarity values between the multi-scale feature map and the sample feature map are arranged in a matrix form to form a feature correlation map, wherein each element of the feature correlation map represents the matching degree of a certain position of the sample feature map in the multi-scale feature map.
5. The method for estimating green tea yield according to claim 1, characterized in that: The step of inputting the feature correlation graph into the density prediction module in the green tea yield estimation model to determine the density graph corresponding to the green tea image comprises: In the density prediction module of the green tea yield estimation model, the feature correlation graph is subjected to convolution operation, nonlinear activation function, and pooling layer processing to extract higher-level features; The processed feature correlation map is converted into a density map corresponding to the green tea image, wherein the density map shows the green tea density information at each position in the target area image of the tea garden.
6. The method for estimating green tea yield according to claim 1, characterized in that: The step of determining the yield of green tea in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image comprises: Extracting a pixel density value of each pixel in the density map corresponding to the green tea image; Adding the pixel density values of all pixels in the density map corresponding to the green tea image to obtain the total pixel density value of the density map corresponding to the green tea image; Determine the actual weight corresponding to each unit density value, and based on the actual weight corresponding to each unit density value and the total pixel density value, convert the total pixel density value of the density map corresponding to the green tea image into the actual weight of green tea to complete the estimation of green tea yield.
7. The method for estimating green tea yield according to any one of claims 1 to 6, characterized in that: The basic network architecture of the green tea yield estimation model is a few-sample density map estimation neural network.
8. A device for estimating green tea yield, characterized in that: include: The target area image acquisition module is configured to respond to the tea leaf yield estimation instruction and acquire the target area image of the tea garden containing the tea leaves; A multi-scale feature map determination module is configured to extract features of the target area image of the tea garden based on a multi-scale feature extraction module of a backbone network of a feature extraction module in a green tea yield estimation model that has been trained to a convergence state, so as to extract a multi-scale feature map corresponding to the target area image of the tea garden; A sample feature map determination module is configured to extract a green tea feature map in the multi-scale feature map based on the ROI Align pooling layer in the green tea yield estimation model, and use the green tea feature map as a sample feature map; A feature correlation graph determination module is configured to generate a feature correlation graph based on the multi-scale feature graph and the sample feature graph based on the feature matching layer in the green tea yield estimation model; The green tea yield estimation module is configured to input the feature correlation map into the density prediction module in the green tea yield estimation model to determine the density map corresponding to the green tea image, and determine the green tea yield in the target area of the tea garden based on the total pixel density value of the density map corresponding to the green tea image to complete the estimation of the green tea yield.
9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.
