A flower and fruit thinning method and device based on image recognition

Abstract

The present application provides a method and device for thinning flowers and fruits based on image recognition. The method obtains a first image of a fruit tree to be processed captured by an image acquisition device. Based on the first image, the fruit tree type of the fruit tree to be processed is determined. According to the fruit tree type, a flower and fruit thinning model that matches the fruit tree type is determined. Based on the flower and fruit thinning model, a plurality of target information to be processed corresponding to the fruit tree to be processed and the attribute value of each target to be processed are determined. Among them, the target to be processed includes at least a flower bud image and a young fruit image. The attribute value represents the fruit setting rate of the target to be processed after processing. The first image, the second image of each target to be processed and the attribute value of each target to be processed are sent to a preset expert planting model to determine the processing risk value of the target to be processed. Based on the attribute value of each target to be processed and the corresponding processing risk value, the processing target is determined so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit tree to be processed according to the processing target.

Description

Technical Field

[0001] The present application relates to the technical field of flower and fruit thinning, and in particular to a flower and fruit thinning method and equipment based on image recognition. Background Art

[0002] In fruit tree cultivation, the quality of the fruit depends primarily on the tree's nutritional status and orchard management. Flower and fruit thinning plays a crucial role in ensuring high-quality fruit and a high fruit set rate. Flower and fruit thinning is a key measure for regulating the tree's nutrient balance. Proper flower and fruit thinning helps improve fruit yield and quality. Flower and fruit thinning can prevent over 50% nutrient waste. The nutrients saved by proper flower and fruit thinning are then concentrated on the remaining flowers and young fruits, ensuring adequate nutrition for the flowers and fruits and achieving higher economic benefits.

[0003] At present, flower and fruit thinning is mainly done manually. However, due to the strong seasonality and high labor intensity of flower and fruit thinning, manual thinning is inefficient and labor shortages are prone to occur during peak employment periods, causing large-scale fruit tree cultivation bases to often miss the best time for flower and fruit thinning.

[0004] In addition, with the development of the economy and society, a large number of inexperienced people have begun to use idle land to plant fruit trees. In the absence of planting experience, these people are more likely to miss or ignore thinning flowers and fruits.

[0005] Therefore, manually managing fruit trees and thinning fruits has low efficiency and long operation time. It is also labor-intensive and wastes human resources. Manual flower and fruit thinning may also affect the growers' planting income. Summary of the Invention

[0006] Based on this, the embodiments of the present application provide a method and equipment for thinning flowers and fruits based on image recognition, which is used to improve the efficiency of flower and fruit thinning, reduce operation time, reduce labor costs, and increase growers' income.

[0007] On the one hand, an embodiment of the present application provides a method for thinning flowers and fruits based on image recognition, the method comprising:

[0008] Obtain a first image of a fruit tree to be processed captured by an image acquisition device. Based on the first image, determine the type of the fruit tree to be processed. According to the type of the fruit tree, determine a flower and fruit thinning model that matches the fruit tree type. Based on the flower and fruit thinning model, determine information on a number of targets to be processed corresponding to the fruit tree to be processed and attribute values ​​of each target to be processed. The targets to be processed include at least flower bud images and young fruit images. The attribute values ​​represent the fruit setting rate of the targets to be processed after processing. Send the first image, the second image of each target to be processed, and the attribute values ​​of each target to be processed to a preset expert planting model to determine the processing risk value of the target to be processed. The second image is an image of the target to be processed captured by the image acquisition device. The processing risk value represents the degree of influence of processing the target to be processed on the fruit setting rate. Based on the attribute values ​​of each target to be processed and the corresponding processing risk values, determine the processing target so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit tree to be processed according to the processing target.

[0009] In an embodiment of the present application, a first image is input into a flower and fruit thinning model to determine information of several targets to be processed corresponding to the fruit trees to be processed. A second image of each target to be processed is obtained respectively. The second image includes at least two sub-images of the target to be processed at different shooting angles. According to each second image and the first image, the first weight of each target to be processed is determined respectively through the flower and fruit thinning model. The first weight is the predicted proportion of the nutrient content of the branch where the target to be processed is located. And according to each second image, the first image and the preset position comparison table, the second weight of each target to be processed is determined respectively, wherein the second weight is the proportion of the growth position of the target to be processed on the branch where it is located. According to the first weight and the second weight, the third weight of the target to be processed is determined. The third weight is used to characterize the processing weight of the target to be processed. Based on the third weight and several historical fruit setting rates of the fruit trees to be processed, the attribute value of the target to be processed is determined.

[0010] In an embodiment of the present application, a determination is made based on the first image as to whether a historical fruit set record exists for the fruit tree to be processed. If a historical fruit set record exists for the fruit tree to be processed, a number of historical fruit set rates for the fruit tree to be processed is determined based on the historical fruit set records. Otherwise, the first image is compared with a preset fruit tree sample library, and based on the comparison results, the number of historical fruit set rates for the corresponding fruit tree sample is determined as the number of historical fruit set rates for the fruit tree to be processed.

[0011] In an embodiment of the present application, several fruit setting targets corresponding to each historical fruit setting rate are determined. Among them, the fruit setting targets are at least flower buds and young fruits. Each target to be processed is matched with each fruit setting target. The matching includes at least: matching the predicted proportion of the nutrient content of the branch where it is located, and matching the proportion of the growth position on the branch where it is located. When the degree of matching between the second image of the target to be processed and the image of the fruit setting target is greater than the first preset threshold, the third weight is used as the fruit setting weight of the fruit setting target. According to each fruit setting weight and the corresponding historical fruit setting rate, the historical fruit setting rate is updated. Determine whether the updated historical fruit setting rate is in the preset fruit setting rate range. When it is determined that the updated historical fruit setting rate is in the preset fruit setting rate range, the updated historical fruit setting rate is used as the attribute value of the target to be processed. Otherwise, the attribute value of the target to be processed is determined to be a preset value. The preset value is zero.

[0012] In an embodiment of the present application, the Euclidean distance between the current position and the fruit tree to be processed is determined using images captured in real time by an image acquisition device and the current position coordinates of the image acquisition device. If the Euclidean distance is within a preset distance range, an image acquisition instruction is generated and sent to the image acquisition device, causing the image acquisition device to capture a pending first image. A sliding window algorithm is used to determine the outline image of the fruit tree to be processed in the pending first image. A scanning curve is set within the outline image and moved in a preset direction and step size until, at the edge of the outline image, the pixel value on the scanning curve exceeds a second preset threshold and the difference between the pixel value and the pixel value at the previous position of the scanning curve is less than a third preset threshold, thereby determining that the pending first image is the outline of a fruit tree that does not include the fruit tree to be processed. If it is determined that the pending first image is the outline of a fruit tree that does not include the fruit tree to be processed, a movement instruction is generated and sent to a flower and fruit thinning management terminal to increase the Euclidean distance between the current position and the fruit tree to be processed by a preset distance value, causing the image acquisition device to capture the pending first image until the outline image corresponding to the pending first image includes the outline of the fruit tree to be processed. A to-be-determined first image including the outline of the fruit tree to be processed is used as the first image.

[0013] In an embodiment of the present application, the first image and each second image are sent to an expert implant model to obtain a comparison attribute value for each target to be processed. A determination is made as to whether the attribute value and the comparison attribute value meet a preset condition. The preset condition is that the attribute value is within a preset neighborhood of the comparison attribute value. If the attribute value and the comparison attribute value do not meet the preset condition, the reciprocal of the difference between the attribute value and the comparison attribute value is determined as the treatment risk value.

[0014] In the embodiment of the present application, if the attribute value and the comparison attribute value meet the preset conditions, the processing risk value is determined to be 1. If the attribute value and the comparison attribute value do not meet the preset conditions, the comparison attribute value is used as a model training label to update the flower and fruit thinning model.

[0015] In this embodiment of the present application, the first image is binarized, and inflorescence features and young fruit features are extracted from the first image. These inflorescence features and young fruit features are then compared with various types of comparison images in a preset image library. The type of fruit tree in the type comparison image with the highest cosine similarity in the feature comparison is determined as the fruit tree type of the fruit tree to be processed.

[0016] In the embodiment of the present application, the absolute value of the product of the attribute value and the processing risk value is calculated, and the target to be processed corresponding to the absolute value of the product smaller than the fourth preset threshold is determined as the processing target.

[0017] On the other hand, an embodiment of the present application further provides a flower and fruit thinning device based on image recognition, the device comprising:

[0018] At least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0019] Obtain a first image of a fruit tree to be processed captured by an image acquisition device. Based on the first image, determine the type of the fruit tree to be processed. According to the type of the fruit tree, determine a flower and fruit thinning model that matches the fruit tree type. Based on the flower and fruit thinning model, determine information on a number of targets to be processed corresponding to the fruit tree to be processed and attribute values ​​of each target to be processed. The targets to be processed include at least flower bud images and young fruit images. The attribute values ​​represent the fruit setting rate of the targets to be processed after processing. Send the first image, the second image of each target to be processed, and the attribute values ​​of each target to be processed to a preset expert planting model to determine the processing risk value of the target to be processed. The second image is an image of the target to be processed captured by the image acquisition device. The processing risk value represents the degree of influence of processing the target to be processed on the fruit setting rate. Based on the attribute values ​​of each target to be processed and the corresponding processing risk values, determine the processing target so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit tree to be processed according to the processing target.

[0020] By such scheme, the application can utilize the first image of the fruit tree collected, judge the fruit tree type, and can determine the pending target (pending flower bud and / or young fruit) on this fruit tree by the flower and fruit thinning model corresponding to this fruit tree type.And obtain the attribute value of this flower bud or young fruit, this attribute value is after processing this target, the predicted fruit setting rate of fruit tree.Then by expert planting model, obtain being used to characterize the pending target of fruit tree after being processed, to the processing risk value of fruit setting rate influence degree, according to this processing risk value and attribute value, can determine whether each pending target can be processing target, and then by flower and fruit thinning management terminal, process processing target, carry out fruit tree flower and fruit thinning.Such scheme can carry out flower and fruit thinning by management terminal, liberates manpower, reduces labor cost, speeds up the operating time of flower and fruit thinning, thereby improves grower's income. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A flow chart of a method for thinning flowers and fruits based on image recognition in an embodiment of the present application;

[0023] Figure 2 A flow chart of a method for thinning flowers and fruits based on image recognition in an embodiment of the present application;

[0024] Figure 3 This is a structural schematic diagram of a flower and fruit thinning device based on image recognition in an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] For fruit tree growers, thinning flowers and fruits of fruit trees is a job that consumes both physical and financial resources. If the thinning is done manually, the labor cost will be high during this period of labor shortage and the farming season will be easily delayed.

[0027] Based on this, the embodiment of the present application provides a method and equipment for thinning flowers and fruits based on image recognition, which is used to improve the efficiency of flower and fruit thinning, reduce operation time, reduce labor costs, and increase growers' income.

[0028] The system corresponding to the flower and fruit thinning method based on image recognition provided in the embodiment of the present application includes: a central controller, an image acquisition device, and a flower and fruit thinning device (a flower and fruit thinning management terminal can be a pruning robot for flower and fruit thinning with a lighting system). The image acquisition device can collect information such as the position and relative size of fruit tree branches, flowers and fruits within a reasonable range up and down, left and right, and transmit the information to the central controller. The central controller can generate a strategy for thinning flowers and fruits based on the information collected by the image acquisition device and in combination with the flower and fruit thinning experience strategy, and then perform the thinning operation through the fruit and fruit thinning device. In addition, the central controller can record the abnormality when it finds abnormal inflorescences or abnormal young fruits such as pests and diseases in the image information, and send the abnormal data to the terminal device of the park administrator. The above-mentioned reasonable ranges can be set according to the administrator. For example, if the planting spacing is 1 meter and the plant height is 3 meters, the range of 50cm-300cm up and down and 50cm left and right are set.

[0029] Flower thinning strategy: Flower thinning should be started at the inflorescence separation stage and completed in one go; remove the side flowers of each flower cluster and retain the central flower; Fruit thinning strategy: Determine the number of fruits to be left per mu based on the variety, fruit setting, tree vigor, site conditions, and fertilizer and water conditions. For large-fruit varieties (such as Fuji), leave one fruit every 20-25 cm; for medium and small-fruit varieties (such as Gala), leave one fruit every 15-20 cm. Leave more fruits if the tree is strong, and fewer fruits if the tree is weak. (1) Leave one fruit every 20-25 cm and remove all excess fruits; (2) Prioritize removing axillary flowers and top flowers, leaving more short-branch flowers; (3) Prioritize removing deformed fruits, diseased and insect-infested fruits, and damaged fruits.

[0030] The abnormal data refers to images collected by the image collector. When the central processor identifies an image as a diseased insect, an incomplete inflorescence, or a young fruit, pruning is not performed and an abnormality is recorded. The park manager can refer to the distribution and quantity of abnormal data to deal with plant protection, abnormal weather, etc. in the park as appropriate. The fruit trees in this application can be pear trees, peach trees, or apple trees.

[0031] The following describes in detail various embodiments of the present application with reference to the accompanying drawings.

[0032] The embodiment of the present application provides a method for thinning flowers and fruits based on image recognition, and the execution subject of the method is a central controller, such as Figure 1 As shown, the method may include steps S101-S106:

[0033] S101: The central controller obtains a first image of a fruit tree to be processed captured by an image capture device.

[0034] In the embodiment of the present application, the fruit tree to be processed can be any fruit tree in the plantation or a fruit tree specified by the user, and the present application does not make any specific limitation on this. The central controller obtains the first image of the fruit tree to be processed captured by the image acquisition device, such as Figure 2 As shown, the specific steps include:

[0035] S201, the central controller determines the Euclidean distance between the current position and the fruit tree to be processed through the image captured in real time by the image acquisition device and the current position coordinates of the image acquisition device.

[0036] The image acquisition device can send the real-time captured image and the current position coordinates of the image acquisition device to the central controller, and the image acquisition device can calculate the distance between the image acquisition device and the branches of the fruit tree to be processed. The central controller can also calculate the Euclidean distance between the current position coordinates and the fruit tree to be processed based on the current position coordinates. The Euclidean distance calculation formula is as follows:

[0037]

[0038] Among them, d is the Euclidean distance between the current position coordinate and the fruit tree to be processed, (x1, y1) is the position coordinate of the fruit tree to be processed, and (x2, y2) is the current position coordinate.

[0039] S202: When the Euclidean distance is within a preset distance interval, the central controller generates an image acquisition instruction and sends it to the image acquisition device, so that the image acquisition device acquires a pending first image.

[0040] The central controller can perform distance judgment to determine whether the Euclidean distance is within a preset distance range, for example, the preset distance range is (20cm, 30cm) (cm is centimeter); if it is within the preset distance range, a shooting instruction can be sent to the image acquisition device to capture the first image to be determined.

[0041] In some embodiments of the present application, the image acquisition device can determine the distance between the image acquisition device and the branch. When the image acquisition device determines that the distance between the image acquisition device and the branch is within a preset distance interval, the image acquisition device can also capture the pending first image.

[0042] S203: The central controller determines the contour image of the fruit tree to be processed in the pending first image by using a sliding window algorithm.

[0043] The central controller can determine the outline image of the fruit tree to be processed in the pending first image by a sliding window algorithm. The outline image can be a binary image, where the outline of the fruit tree to be processed is white and the background is black.

[0044] S204, the central controller sets a scanning curve in the contour image, and moves the scanning curve in a preset direction and a preset step size until, at the edge of the contour image, the pixel value on the scanning curve is greater than the second preset threshold and the difference between the pixel value and the pixel value of the scanning curve at the previous position is less than the third preset threshold, and determines that the first image to be determined is a fruit tree contour that does not include the fruit tree to be processed.

[0045] The scanning curve in the contour image can be scanned sequentially along the growth direction of the fruit tree to be processed starting from the bottom of the contour image, and in the process of scanning upwards, the length of the scanning curve will be shortened in proportion to the thickness of the fruit tree to be processed. In the process of scanning the fruit tree to be processed, if there are multiple branches on the same horizontal line, the scanning curve will be divided into scanning curves identical to the branches of the tree, and scanned along the growth direction of the branches. If the end of a branch ends before reaching the edge of the contour image (the pixel value is 0), then the branch is a complete branch in the contour image; if at the edge of the contour image, the pixel value on the scanning curve corresponding to the branch is still greater than the second preset threshold, and the difference between the pixel value and the pixel value of the scanning curve at the previous position is less than the third preset threshold, then it is determined that the pending first image does not include the complete fruit tree contour of the fruit tree to be processed.

[0046] In actual use, the specific values ​​of the second and third preset thresholds can be set according to actual use. The second preset threshold can be set to ensure that branches capable of bearing fruit are included in the first image to be determined. For example, branches with pixel values ​​less than the second preset threshold cannot bear flowers or fruits. The third preset threshold can be set to ensure that branches are branches of the same fruit tree to be processed or are not affected by background noise.

[0047] S205. When the central controller determines that the pending first image does not include the fruit tree outline of the fruit tree to be processed, it generates a movement instruction and sends it to the flower and fruit thinning management terminal to increase the Euclidean distance between the current position and the fruit tree to be processed by a preset distance value, and enables the image acquisition device to acquire the pending first image until the outline image corresponding to the pending first image includes the fruit tree outline of the fruit tree to be processed.

[0048] Based on the above determination, if the central controller determines that the pending first image does not include all the fruit tree outlines, it can generate a movement control instruction, causing the flower and fruit thinning management terminal to move the tree, increasing the Euclidean distance. The preset distance value for each increase can be a fixed value or a manually set distance value, automatically adjusting to different distance values ​​as conditions are met. After the distance value is increased, the pending first image is recaptured to determine whether it contains all the outlines.

[0049] S206 , the central controller uses the to-be-determined first image including the outline of the fruit tree to be processed as the first image.

[0050] Through the above scheme, the present application can ensure that a suitable first image is obtained, and ensure that the outlines of the fruit trees are all in the first image, thereby avoiding misjudgment during the flower and fruit thinning operation, or affecting the normal implementation of the flower and fruit thinning strategy.

[0051] S102: The central controller determines the type of the fruit tree to be processed based on the first image.

[0052] In the embodiment of the present application, the central controller determines the type of fruit tree to be processed, specifically including:

[0053] First, the central controller performs binarization processing on the first image and extracts inflorescence features and young fruit features in the first image.

[0054] The central controller can perform binarization processing on the first image. After the binarization processing, the image of the fruit tree is white and the background is black. The central controller obtains inflorescence features and young fruit features from the first image after the binarization processing. The feature extraction can be implemented by a neural network model.

[0055] Next, the central controller compares the inflorescence characteristics and young fruit characteristics with various types of comparison images in the preset image library.

[0056] After obtaining the inflorescence or young fruit characteristics, the central controller compares them with various types of comparison images in a preset image library. The preset image library stores images of inflorescence characteristics and young fruit characteristics of various fruit tree types. By comparing the inflorescence and young fruit characteristics, the fruit tree type corresponding to the first image can be determined.

[0057] Subsequently, the central controller determines the fruit tree type of the type comparison image with the highest cosine similarity of the feature comparison as the fruit tree type of the fruit tree to be processed.

[0058] During the feature comparison process, the central controller can calculate the type comparison image with the highest cosine similarity between the inflorescence features and the young fruit features, and determine the type of the fruit tree. In the case where the fruit tree to be processed has an inflorescence but no young fruit, the central controller calculates the cosine similarity between the inflorescence features and the various type comparison images in the preset image library. The type comparison image with the highest cosine similarity is used as the fruit tree type of the fruit tree to be processed.

[0059] If the fruit tree to be processed has inflorescences and young fruits, the central controller compares the inflorescence and young fruit features with the comparison images of each type, calculates cosine similarity, and determines whether the type comparison image with the highest cosine similarity is the same fruit tree type. If so, the fruit tree type is determined as the fruit tree type of the fruit tree to be processed. If not, the first image is sent to a user terminal (such as a mobile phone or computer) to obtain the fruit tree type selected by the user terminal.

[0060] S103: The central controller determines a flower and fruit thinning model that matches the fruit tree type according to the fruit tree type.

[0061] In an embodiment of the present application, after obtaining the fruit tree type, the central controller can determine a flower and fruit thinning model based on the fruit tree type. The flower and fruit thinning model can be preset in the central controller, or the central controller can send the fruit tree type to a server, and the server can send the flower and fruit thinning model for the fruit tree type to the central controller. This application does not specifically limit the specific method for obtaining the flower and fruit thinning model.

[0062] S104: The central controller determines information of several to-be-processed targets corresponding to the to-be-processed fruit trees and attribute values ​​of each to-be-processed target based on the flower and fruit thinning model.

[0063] The target information to be processed includes at least a flower bud image and a young fruit image. The attribute value represents the fruit setting rate of the target to be processed after processing.

[0064] In the embodiment of the present application, the central controller determines a number of to-be-processed targets corresponding to the to-be-processed fruit trees and the attribute values ​​of each to-be-processed target based on the flower and fruit thinning model, specifically including:

[0065] First, the central controller inputs the first image into the flower and fruit thinning model to determine a number of target information to be processed corresponding to the fruit trees to be processed.

[0066] In other words, the flower and fruit thinning model can identify the target to be processed in the first image, which can be a flower bud or young fruit. That is, the flower and fruit thinning model can locate the flower bud or young fruit in the first image within the complete image of the fruit tree to be processed and identify them. The output of the target to be processed is the output of the flower and fruit thinning model without the second image input.

[0067] Next, the central controller obtains the second image of each target to be processed.

[0068] The second image at least includes two sub-images of the target to be processed at different shooting angles.

[0069] The second image can be determined before or after the target is processed. In other words, before the target is determined, the image acquisition device can capture multiple images of the fruit trees to be processed in real time. The central controller can select two images of the same target from different shooting angles from these images as the second image. After the target is determined, the central controller can control the image acquisition device to capture two images of the target from different shooting angles and use these images as the second image.

[0070] Subsequently, the central controller determines the first weight of each target to be processed based on each second image and the first image using the flower and fruit thinning model. The central controller also determines the second weight of each target to be processed based on each second image, the first image, and the preset position comparison table.

[0071] The first weight is the predicted proportion of the nutrient content of the branch where the target to be processed is located, and the second weight is the proportion of the growth position of the target to be processed on the branch where it is located.

[0072] The central controller can input each second image into a flower and fruit thinning model. The flower and fruit thinning model uses the first image and each second image to determine a first weight for each target to be processed. The flower and fruit thinning model is a pre-trained image recognition neural network model, trained on images of fruit trees and corresponding nutrient content labels for their branches.

[0073] The central controller can also determine the growth position proportion of the target to be processed corresponding to the position of the target to be processed based on the target to be processed in each second image, the position of the fruit tree to be processed in the first image and the preset position comparison table, and use the growth position proportion as the second weight.

[0074] Subsequently, the central controller determines a third weight of the target to be processed based on the first weight and the second weight. The third weight is used to represent a processing weight for processing the target to be processed.

[0075] After the central controller obtains the first weight and the second weight, it can multiply the first weight and the second weight. For example, the first weight is a, the second weight is b, and the third weight is a*b. Both the first weight and the second weight are not greater than 1.

[0076] Then, the central controller determines the attribute value of the target to be processed based on the third weight and several historical fruit setting rates of the fruit trees to be processed.

[0077] In the embodiment of the present application, before the central controller determines the attribute value of the target to be processed based on the third weight and the historical fruit setting rate of the fruit tree to be processed, the following steps are further included:

[0078] The central controller determines whether there is a historical fruit setting record for the fruit tree to be processed based on the first image.

[0079] After the central controller identifies the first image, it can determine in the preset database whether there is a historical fruit setting record for the first image, that is, a record of past fruiting. For example, if fruit tree A had a fruit setting record last year, through the first image of fruit tree A, the central controller can query the fruit setting record of fruit tree A last year in the preset database.

[0080] When it is determined that the fruit trees to be processed have historical fruit setting records, the central controller determines several historical fruit setting rates of the fruit trees to be processed based on the historical fruit setting records.

[0081] If the central controller can retrieve historical fruit set records, it can determine the historical fruit set rate of the fruit tree being processed. For example, the fruit set rates for the past three years could be: t1, t2, and t3. In actual fruit tree cultivation, the historical fruit set rate for individual trees may not be recorded, and the fruit set data from the previous year is not very useful. Therefore, the historical fruit set rate can be calculated by taking the average fruit set rate per tree based on the average yield per mu and the number of trees in the orchard.

[0082] Otherwise, the central controller compares the first image with a preset fruit tree sample library to determine, based on the comparison result, several historical fruit setting rates of the corresponding fruit tree samples as several historical fruit setting rates of the fruit trees to be processed.

[0083] That is to say, if the central controller fails to find the historical fruit setting record corresponding to the first image, the central controller will compare the first image with the preset fruit tree sample library. The preset fruit tree sample library contains images of several fruit trees and the historical fruit setting rates corresponding to the fruit trees. Based on the comparison results, the historical fruit setting rate of the matching fruit tree image will be used as the historical fruit setting rate of the fruit tree to be processed.

[0084] Comparison involves comparing the appearance, branch thickness, branch number, growth height, trunk thickness, and target location and number of trees to be treated with a pre-set sample library. This comparison helps better predict the historical fruit set rate of the trees to be treated.

[0085] In the embodiment of the present application, the central controller determines the attribute value of the target to be processed based on the third weight and several historical fruit setting rates of the fruit trees to be processed, specifically including:

[0086] First, the central controller determines several fruit setting targets corresponding to various historical fruit setting rates.

[0087] Among them, the fruit setting targets are at least flower buds and young fruits.

[0088] After obtaining the historical fruit setting rates, the central controller can determine the images of the fruit trees with each historical fruit setting rate, as well as the flower buds and young fruits on the fruit tree images.

[0089] Secondly, the central controller matches each target to be processed with each fruit setting target.

[0090] The matching includes at least: matching of the predicted proportion of nutrient content of the branch and matching of the proportion of growth position on the branch.

[0091] The central processing unit can match the target to be processed with the flower buds and young fruits, and the matching is to perform image feature comparison and matching.

[0092] Again, when the degree of matching between the second image of the target to be processed and the image of the fruit-setting target is greater than the first preset threshold, the central controller uses the third weight as the fruit-setting weight of the fruit-setting target.

[0093] The second image is matched with the fruit-setting target image. The matching can be performed by calculating the pixel cosine similarity of the two images. When the cosine similarity is greater than a first preset threshold, the third weight of the target to be processed is assigned to the fruit-setting target matched with the target to be processed as its fruit-setting weight.

[0094] Then, the central controller updates the historical fruit setting rate according to each fruit setting weight and the corresponding historical fruit setting rate.

[0095] The central controller can multiply the fruit setting weight of each fruit setting target with the historical fruit setting rate of the fruit tree corresponding to each fruit setting target. For example, if there are 3 fruit setting targets on a tree, the historical fruit setting rate is m, and the fruit setting weights corresponding to each fruit setting target are x1, x2, and x3, then the updated historical fruit setting rate is

[0096] x1*m+x2*m+x3*m.

[0097] Next, the central controller determines whether the updated historical fruit setting rate is within a preset fruit setting rate range.

[0098] The preset fruit setting rate range can be set by the user himself, or the fruit setting rate range can be obtained from a breeding expert through the Internet. The fruit setting rate range includes the optimal fruit setting rate of the fruit tree.

[0099] When the central controller determines that the updated historical fruit setting rate is within the preset fruit setting rate range, the updated historical fruit setting rate is used as the attribute value of the target to be processed.

[0100] Otherwise, the attribute value of the target to be processed is determined to be a preset value, which is zero.

[0101] When the central controller determines that the updated historical fruit setting rate is not within the preset fruit setting rate range, the attribute value of the target to be processed is set to 0.

[0102] S105 , the central controller sends the first image, the second image of each target to be processed, and the attribute value of each target to be processed to a preset expert implant model to determine a treatment risk value of the target to be processed.

[0103] The second image is an image of the target to be processed captured by the image acquisition device. The processing risk value represents the degree of impact of processing the target to be processed on the fruit setting rate.

[0104] In the embodiment of the present application, the central controller sends the first image, the second image of each target to be processed, and the attribute value of each target to be processed to a preset expert implant model to determine the treatment risk value of the target to be processed, specifically including:

[0105] First, the central controller sends the first image and each second image to the expert implant model to obtain a comparison attribute value of each target to be processed.

[0106] The expert planting model is an image recognition model trained using expert planting data and images of cultivated fruit trees. It can output a comparison attribute value corresponding to the image. This comparison attribute value corresponds to the fruit set rate of the fruit tree.

[0107] Then, the central controller determines whether the attribute value and the comparison attribute value meet a preset condition, wherein the preset condition is that the attribute value is within a preset neighborhood of the comparison attribute value.

[0108] The preset neighborhood can be set by the user and is not specifically limited here.

[0109] When the attribute value and the comparison attribute value do not meet the preset conditions, the central controller determines the inverse of the difference between the attribute value and the comparison attribute value as the processing risk value.

[0110] For example, if the attribute value is q and the comparison attribute value is p, the treatment risk value is 1 / (qp). The comparison attribute value is based on expert cultivation data, and the attribute value is based on historical fruit set targets. This comparison can determine the accuracy of the attribute value, allowing subsequent flower and fruit thinning operations to be performed based on an accurate flower and fruit thinning model, without relying on data from expert cultivation models.

[0111] In some embodiments of the present application, further comprising:

[0112] When the attribute value and the comparison attribute value meet the preset conditions, the central controller determines that the processing risk value is 1.

[0113] When the attribute value and the comparison attribute value do not meet the preset conditions, the central controller uses the comparison attribute value as the model training label to update the flower and fruit thinning model.

[0114] Update the flower and fruit thinning model to be more accurate when determining attribute values ​​for subsequent targets to be processed.

[0115] S106, the central controller determines the processing target based on the attribute value of each target to be processed and the corresponding processing risk value, so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit trees to be processed according to the processing target.

[0116] In the embodiment of the present application, the central controller determines the processing target based on the attribute value of each target to be processed and the corresponding processing risk value, specifically including:

[0117] The central controller can calculate the absolute value of the product of the attribute value and the processing risk value. For example, if the attribute value is q and the processing risk value is 1 / (qp), then the absolute value of the product is |q / (qp)|.

[0118] The central controller determines the target to be processed corresponding to the absolute value of the product less than a fourth preset threshold as the processing target. The fourth preset threshold can be set according to actual use.

[0119] The flower and fruit thinning management terminal can be a robot or robotic arm that performs automatic flower and fruit thinning, and performs pruning processing targets according to the instructions of the central controller.

[0120] Through the above scheme, the application can utilize image acquisition equipment to collect fruit tree images, and by identifying the fruit tree images and processing the flower and fruit thinning model, obtain the predicted fruit setting rate of each pending target (flower bud, young fruit) in the image. Then obtain the processing risk value of the pending target corresponding to the predicted fruit setting rate through the expert planting model, and then determine the processing target in each pending target. The application can automatically perform the judgment of the processing target, and can perform flower and fruit thinning by an automatically operated machine without manpower, thereby reducing the operation time, and can alternately operate day and night, thereby improving the income of the grower.

[0121] Figure 3 A structural diagram of a flower and fruit thinning device based on image recognition provided in an embodiment of the present application, the device comprising:

[0122] At least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0123] Obtain a first image of a fruit tree to be processed captured by an image acquisition device. Based on the first image, determine the type of the fruit tree to be processed. According to the type of the fruit tree, determine a flower and fruit thinning model that matches the fruit tree type. Based on the flower and fruit thinning model, determine information on a number of targets to be processed corresponding to the fruit tree to be processed and attribute values ​​of each target to be processed. The targets to be processed include at least flower bud images and young fruit images. The attribute values ​​represent the fruit setting rate of the targets to be processed after processing. Send the first image, the second image of each target to be processed, and the attribute values ​​of each target to be processed to a preset expert planting model to determine the processing risk value of the target to be processed. The second image is an image of the target to be processed captured by the image acquisition device. The processing risk value represents the degree of influence of processing the target to be processed on the fruit setting rate. Based on the attribute values ​​of each target to be processed and the corresponding processing risk values, determine the processing target so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit tree to be processed according to the processing target.

[0124] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0125] The device and method provided in the embodiments of the present application correspond one to one, and therefore, the device also has similar beneficial technical effects as its corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device will not be repeated here.

[0126] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0127] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A flower and fruit thinning method based on image recognition, characterized in that: The method comprises: Acquire a first image of the fruit tree to be processed captured by an image capture device; determining the fruit tree type of the fruit tree to be processed based on the first image; Determining a flower and fruit thinning model that matches the fruit tree type according to the fruit tree type; Based on the flower and fruit thinning model, determining a plurality of target information to be processed corresponding to the fruit tree to be processed and an attribute value of each target to be processed; wherein the target information to be processed includes at least a flower bud image and a young fruit image; and the attribute value represents the fruit setting rate of the target to be processed after processing; Sending the first image, the second image of each of the objects to be processed, and the attribute value of each of the objects to be processed to a preset expert planting model to determine a treatment risk value of the object to be processed; wherein the second image is an image of the object to be processed captured by the image acquisition device; and the treatment risk value represents the degree of impact of treating the object to be processed on the fruit setting rate; Based on the attribute value of each target to be processed and the corresponding processing risk value, a processing target is determined so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit trees to be processed according to the processing target.

2. The method according to claim 1, characterized in that Based on the flower and fruit thinning model, information of several to-be-processed targets corresponding to the to-be-processed fruit trees and attribute values ​​of each to-be-processed target are determined, specifically including: Inputting the first image into the flower and fruit thinning model to determine a plurality of target information to be processed corresponding to the fruit tree to be processed; Acquire a second image of each of the objects to be processed respectively; wherein the second image includes at least two sub-images of the object to be processed taken at different angles; Determining a first weight of each of the targets to be processed based on the second image and the first image using the flower and fruit thinning model; wherein the first weight is a predicted proportion of the nutrient content of the branch where the target to be processed is located; and Determining a second weight of each of the objects to be processed based on each of the second images, the first image, and a preset position comparison table, wherein the second weight is a proportion of the growth position of the object to be processed on the branch where it is located; Determining a third weight of the target to be processed based on the first weight and the second weight; the third weight is used to represent a processing weight for processing the target to be processed; Based on the third weight and several historical fruit setting rates of the fruit trees to be processed, the attribute value of the target to be processed is determined.

3. The method according to claim 2, characterized in that Before determining the attribute value of the target to be processed based on the third weight and the historical fruit setting rate of the fruit tree to be processed, the method further includes: Determining, based on the first image, whether there is a historical fruit setting record for the fruit tree to be processed; If so, determining several historical fruit setting rates of the fruit trees to be processed based on the historical fruit setting records; Otherwise, the first image is compared with a preset fruit tree sample library to determine, based on the comparison results, several historical fruit setting rates of the corresponding fruit tree samples as several historical fruit setting rates of the fruit tree to be processed.

4. The method according to claim 3, characterized in that Determining the attribute value of the target to be processed based on the third weight and several historical fruit setting rates of the fruit tree to be processed, specifically including: Determining a plurality of fruit setting targets corresponding to each of the historical fruit setting rates; wherein the fruit setting targets include at least flower buds and young fruits; Matching each of the to-be-processed targets with each of the fruit-setting targets; the matching at least includes: matching the predicted nutrient content proportion of the branches, and matching the growth position proportion on the branches; When the degree of matching between the second image of the target to be processed and the image of the fruit setting target is greater than a first preset threshold, using the third weight as the fruit setting weight of the fruit setting target; updating the historical fruit setting rate according to each of the fruit setting weights and the corresponding historical fruit setting rate; determining whether the updated historical fruit setting rate is within a preset fruit setting rate range; If yes, use the updated historical fruit setting rate as the attribute value of the target to be processed; Otherwise, it is determined that the attribute value of the target to be processed is a preset value; the preset value is zero.

5. The method according to claim 1, characterized in that: Acquiring a first image of the fruit tree to be processed captured by an image capture device specifically includes: Determine the Euclidean distance between the current position and the fruit tree to be processed by using the image captured in real time by the image capture device and the current position coordinates of the image capture device; When the Euclidean distance is within a preset distance interval, generating an image acquisition instruction and sending the instruction to the image acquisition device so that the image acquisition device acquires a pending first image; Determine, by a sliding window algorithm, a contour image of the fruit tree to be processed in the pending first image; In the contour image, a scanning curve is set, and the scanning curve is moved in a preset direction and a preset step size until it reaches the edge of the contour image, a pixel value on the scanning curve is greater than a second preset threshold value and a difference between the pixel value and a pixel value of the scanning curve at a previous position is less than a third preset threshold value, and the to-be-determined first image is determined to be a fruit tree contour that does not include the fruit tree to be processed; If it is determined that the pending first image does not include the fruit tree outline of the fruit tree to be processed, a movement instruction is generated and sent to the flower and fruit thinning management terminal to increase the Euclidean distance between the current position and the fruit tree to be processed by a preset distance value, and the image acquisition device is caused to acquire the pending first image until the outline image corresponding to the pending first image includes the fruit tree outline of the fruit tree to be processed; The undetermined first image including the fruit tree outline of the fruit tree to be processed is used as the first image.

6. The method according to claim 1, characterized in that Sending the first image, the second image of each of the objects to be processed, and the attribute value of each of the objects to be processed to a preset expert implant model to determine a treatment risk value of the object to be processed, specifically including: Sending the first image and each of the second images to the expert implant model to obtain a comparison attribute value of each of the objects to be processed; Determining whether the attribute value and the comparison attribute value meet a preset condition; wherein the preset condition is that the attribute value is within a preset neighborhood of the comparison attribute value; In the case that the attribute value and the comparison attribute value do not satisfy a preset condition, the inverse of the difference between the attribute value and the comparison attribute value is determined as the processing risk value.

7. The method according to claim 6, characterized in that The method further comprises: If the attribute value and the comparison attribute value meet a preset condition, determining the processing risk value to be 1; In the case that the attribute value and the comparison attribute value do not satisfy a preset condition, the comparison attribute value is used as a model training label to update the flower and fruit thinning model.

8. The method according to claim 1, characterized in that: Determining the fruit tree type of the fruit tree to be processed based on the first image specifically includes: Binarizing the first image and extracting inflorescence features and young fruit features from the first image; Perform feature comparison between the inflorescence feature, the young fruit feature and various types of comparison images in a preset image library; Determine the type comparison image with the highest cosine similarity in feature comparison as the fruit tree type.

9. The method according to claim 1, characterized in that: Determining a processing target based on the attribute value of each target to be processed and the corresponding processing risk value specifically includes: Calculating a product value of the attribute value and the processing risk value; The target to be processed corresponding to the product value smaller than a fourth preset threshold is determined as the processing target.

10. A flower and fruit thinning device based on image recognition, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Acquire a first image of the fruit tree to be processed captured by an image capture device; determining the fruit tree type of the fruit tree to be processed based on the first image; Determining a flower and fruit thinning model that matches the fruit tree type according to the fruit tree type; Based on the flower and fruit thinning model, determining a plurality of target information to be processed corresponding to the fruit tree to be processed and an attribute value of each target to be processed; wherein the target information to be processed includes at least a flower bud image and a young fruit image; and the attribute value represents the fruit setting rate of the target to be processed after processing; Sending the first image, the second image of each of the objects to be processed, and the attribute value of each of the objects to be processed to a preset expert planting model to determine a treatment risk value of the object to be processed; wherein the second image is an image of the object to be processed captured by the image acquisition device; and the treatment risk value represents the degree of impact of treating the object to be processed on the fruit setting rate; Based on the attribute value of each target to be processed and the corresponding processing risk value, a processing target is determined so that the flower and fruit thinning management terminal corresponding to the image acquisition device thins the flowers and fruits of the fruit trees to be processed according to the processing target.

Images