Flower pollination method and device, electronic equipment and storage medium

Abstract

The invention provides a flower pollination method and device, electronic equipment and a storage medium, relates to the technical field of facility agriculture, and realizes automatic identification of a growth stage of a flower through an identification model. By setting the recognition model, the recognition efficiency of the recognition model is improved. Through the initial position and the 3D point cloud data, the three-dimensional coordinate of the flower is automatically obtained, and the accuracy of flower pollination is improved. By determining the sequence of the pollination nozzles, the flowers are subjected to airflow vibration, powder spraying and hormone spraying pollination according to the three-dimensional coordinates corresponding to the pollination nozzles, so that uninterrupted precise pollination is realized, and the pollination efficiency of the flowers is improved. According to the invention, airflow vibration, powder spraying and hormone spraying pollination are carried out on flowers, so that the success rate of flower pollination is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of facility agriculture, and in particular to a flower pollination method, device, electronic equipment and storage medium. Background Art

[0002] At present, the demand for manual labor in greenhouse facilities in agriculture still accounts for a high proportion. With the aging of my country's population and the shortage of social labor, it is inevitable that intelligent operation robots in facility agriculture will replace manual labor. This will not only greatly reduce labor output, but will also be more intelligent and precise than manual operation, effectively improving agricultural production efficiency and product quality.

[0003] Tomatoes are hermaphroditic self-pollinating plants. In natural cultivation environments, pollination can be completed through natural wind, insects and other media. However, in the process of facility cultivation, due to the airtightness of the greenhouse, the lack of natural wind and insect activity, tomatoes are often poorly pollinated, resulting in widespread phenomena such as flower and fruit drop and deformed fruit. At present, the commonly used pollination methods for facility tomato cultivation include bumblebee pollination, spot hormone pollination, and vibrator pollination. Bumblebee pollination relies on bumblebees to randomly pollinate flowers. Bumblebees have a short lifespan, and pollination is accidental and uncertain, which leads to the problem of delayed maturity of tomato fruits. Spot hormone pollination and vibrator pollination mainly rely on manual labor, which has problems such as strong subjectivity, low work efficiency, and high labor intensity. In addition, spot hormone pollination and vibrator pollination have high requirements for flower identification and positioning, and are less suitable for installation on automatic machinery or robots.

[0004] In summary, it can be seen that the pollination efficiency of tomatoes in existing greenhouses is low. Summary of the invention

[0005] The present invention provides a flower pollination method, device, electronic device and storage medium, which are used to solve the defect of low pollination efficiency of tomato flowers in a greenhouse in the prior art and improve the pollination efficiency of tomato flowers in the greenhouse.

[0006] The present invention provides a flower pollination method, comprising: acquiring image data of at least one flower based on a binocular infrared depth camera; inputting the image data into a recognition model, acquiring a recognition result of at least one flower output by the recognition model, the recognition result including a growth stage and an initial position; wherein the recognition model is obtained based on a preset attention neural network and trained based on sample image data and labels of sample growth stages; when the growth stage includes a growth stage requiring pollination, matching the initial position with 3D point cloud data of the flower to obtain the three-dimensional coordinates of at least one flower; sorting the pollination nozzles of each three-dimensional coordinate, selecting the pollination nozzles in turn according to the sorting results, and performing airflow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and precise pollination.

[0007] According to the flower pollination method provided by the present invention, air flow vibration, powder spraying and hormone spraying are performed on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzle, including: according to the three-dimensional coordinates corresponding to the pollination nozzle, the pollination nozzle is moved up and down, swayed and telescopically until the pollination nozzle moves to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle; based on the pollination nozzle with a fixed time period and set duration, intermittent airflow, powder and hormones are sprayed to the flowers at the three-dimensional coordinates corresponding to the pollination nozzle to complete air flow vibration, powder spraying and hormone spraying.

[0008] According to the flower pollination method provided by the present invention, image data of at least one flower is obtained based on a binocular infrared depth camera, including: obtaining RGB image information of at least one flower based on the binocular infrared depth camera; grayscale processing is performed on the RGB image information according to a weighted average algorithm to obtain an initial grayscale image; Gaussian filtering is performed on the initial grayscale image to obtain a denoised image; histogram equalization processing and pixel grayscale mapping change processing are performed on the denoised image to obtain image data, and the image data is a grayscale image with uniform probability distribution.

[0009] According to the flower pollination method provided by the present invention, the preset attention neural network includes a Yolov8 network, and the recognition model is constructed based on the following steps: flipping, rotating, scaling or randomly cropping the initial sample image data to obtain diversified sample image data; adding a label of the sample growth stage to each sample image data to obtain a training sample carrying the label; training the Yolov8 network based on the training sample, adjusting the weights of the initial flower features in the Yolov8 network until the error of the predicted growth stage output by the Yolov8 network is less than the set error, thereby obtaining a recognition model.

[0010] According to the flower pollination method provided by the present invention, the recognition model is used to determine the growth stage of at least one flower: feature extraction is performed on at least one flower in the image data to obtain flower features and weights of the flower features, the flower features including morphology, color and texture; weighted fusion is performed on the flower features and the weights of the flower features to obtain the growth stage of at least one flower, the growth stages including bud stage, full bloom stage, flowering stage and initial fruit stage.

[0011] According to the flower pollination method provided by the present invention, the pollination nozzles of each three-dimensional coordinate are sorted to obtain the sorting result, including: arranging the three-dimensional coordinates in order from near to far according to the K nearest neighbor algorithm to obtain the sorted three-dimensional coordinates; obtaining the pollination nozzle corresponding to each three-dimensional coordinate, and sorting the pollination nozzles based on the sorted three-dimensional coordinates.

[0012] The present invention also provides a flower pollination device, comprising: an image acquisition module, used to acquire image data of at least one flower based on a binocular infrared depth camera; a control module, used to input the image data into a recognition model, and acquire a recognition result of at least one flower output by the recognition model, the recognition result including a growth stage and an initial position; wherein the recognition model is obtained based on a preset attention neural network and trained based on sample image data and labels of sample growth stages; the control module is also used to match the initial position with the 3D point cloud data of the flower when the growth stage includes a growth stage requiring pollination, so as to obtain the three-dimensional coordinates of at least one flower; a pollination module, used to sort the pollination nozzles of each three-dimensional coordinate, select the pollination nozzles in turn according to the sorting results, and perform airflow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and precise pollination.

[0013] The flower pollination device provided by the present invention also includes a lifting slide, a medicine barrel and an air pump. The pollination module includes multiple pollination nozzles, and each pollination nozzle includes a pollination telescopic rod, a pollination servo and a pollination box; the lifting slide is used to move the pollination module up and down to control the height of the pollination nozzle to move to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle, the pollination telescopic rod and the pollination servo are used to swing and telescopically move the pollination nozzle according to the three-dimensional coordinates corresponding to the pollination nozzle until the pollination nozzle moves to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle; the air pump is used to convey airflow to the pollination nozzle, the pollination box is used to convey powder to the pollination nozzle, and the medicine barrel is used to convey hormones to the pollination nozzle, so that the pollination nozzle can spray intermittent airflow, powder and hormones to the flower at the three-dimensional coordinates corresponding to the pollination nozzle at a fixed time period and set duration, so as to complete airflow vibration, powder spraying and hormone spraying pollination.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned flower pollination methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the pollination method of any of the flowers described above is implemented.

[0016] The flower pollination method, device, electronic device and storage medium provided by the present invention realize automatic recognition of the growth stage of the flower through the recognition model. By setting the recognition model, the recognition efficiency of the recognition model is improved. Through the initial position and 3D point cloud data, the three-dimensional coordinates of the flower are automatically acquired, which is conducive to improving the accuracy of flower pollination. By determining the order of the pollination nozzles, the flowers are subjected to air flow vibration, powder spraying and hormone spraying pollination according to the three-dimensional coordinates corresponding to the pollination nozzles, uninterrupted and accurate pollination is achieved, and the efficiency of flower pollination is improved. The present invention greatly improves the success rate of flower pollination by performing air flow vibration, powder spraying and hormone spraying on the flowers for pollination. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 The present invention is a schematic diagram of the process of the flower pollination method provided by the present invention.

[0019] Figure 2 This is one of the structural schematic diagrams of the flower pollination device provided by the present invention.

[0020] Figure 3 This is the second structural schematic diagram of the flower pollination device provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the pollination module provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention.

[0023] Reference numerals: 1001: pollination nozzle; 1002: lifting slide; 1003: air pump; 1004: pollination box; 1005: nozzle box; 1006: universal ball; 1007: pollination servo; 1008: pollination telescopic rod; 1009: binocular infrared depth camera. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Combine the following Figure 1-Figure 5 The present invention describes a method, a device and an electronic device for pollinating flowers.

[0026] Figure 1 It is one of the flow diagrams of the flower pollination method provided by the present invention, such as Figure 1 As shown, the flower pollination method includes S100 to S400, and each step is specifically as follows.

[0027] S100: Acquire image data of at least one flower based on a binocular infrared depth camera.

[0028] The execution subject of the embodiment of the present invention is a pollination device for flowers. First, a pollination operation area is selected and the pollination operation is started. The power chassis of the flower pollination device runs along the laid inter-row track, and the road conditions are detected in real time by a laser sensor during the movement. If the laser sensor detects an obstacle in the direction of the road, the information is fed back to the control module of the flower pollination device, and the control module finds a movement path from the current position to avoid the obstacle; if the road is impassable, the power chassis stops moving and triggers an alarm.

[0029] When the power chassis enters the inter-row track, the binocular infrared depth camera of the flower pollination device detects the tomato flowers in real time, and the dual cameras capture images to improve the efficiency of flower detection. When the tomato flowers are detected, the power chassis stops moving forward.

[0030] S200: Inputting image data into a recognition model, obtaining a recognition result of at least one flower output by the recognition model, wherein the recognition model is obtained by training based on a preset attention neural network, based on sample image data and labels of sample growth stages.

[0031] The flowers of the present invention include tomato flowers, such as tomato flowers in a greenhouse. The control module determines whether there are flowers in the bouquet that are in full bloom (a growth stage that requires pollination) based on red, green, and blue (RGB) image information of the tomato flowers collected by the binocular infrared depth camera. The preset attention neural network includes a Yolov8 network with an added attention mechanism.

[0032] Since the growth environment of tomato flowers is an unstructured environment, the flowers present small and multi-target distribution, one plant often contains multiple bouquets, and the same bouquet contains flowers of different flowering periods; traditional convolutional neural networks usually use a single network structure for feature extraction, and cannot extract bouquet area and flowering period information at the same time, resulting in insufficient extracted flower feature information and low network recognition accuracy. In response to these problems, the present invention proposes a Yolov8 method with an added attention mechanism to achieve the extraction of tomato bouquet areas and accurate identification of flowering periods in bouquets, and calculate the proportion of fully open flowers in the bouquet.

[0033] Furthermore, the recognition model can also obtain the initial position of the flower.

[0034] S300: When the growth stage includes a growth stage requiring pollination, matching the initial position with the 3D point cloud data of the flower to obtain the three-dimensional coordinates of at least one flower.

[0035] If there are flowers in the growth stage that need pollination (full bloom), the control module of the flower pollination device will fuse the RGB image information with the corresponding xyz coordinates in the flower point cloud image (3D point cloud data) to obtain the posture and three-dimensional coordinates of the target tomato bouquet. If there are no flowers in full bloom, the power chassis will continue to move forward.

[0036] The 3D point cloud data of the flower is collected by a binocular infrared depth camera.

[0037] S400: sorting the pollination nozzles of each three-dimensional coordinate, selecting the pollination nozzles in turn according to the sorting results, and performing air flow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and precise pollination.

[0038] The pollination nozzles of each three-dimensional coordinate are sorted to obtain the sorting result. Specifically, according to the K nearest neighbor algorithm, the three-dimensional coordinates are arranged in order from near to far to obtain the sorted three-dimensional coordinates; the pollination nozzles corresponding to each three-dimensional coordinate are obtained, and the pollination nozzles are sorted based on the sorted three-dimensional coordinates.

[0039] One three-dimensional coordinate corresponds to one flower that needs to be pollinated. The pollination device of the flower includes multiple pollination nozzles, and one pollination nozzle is assigned to each three-dimensional coordinate to obtain the pollination nozzle of the three-dimensional coordinate. Optionally, the same pollination nozzle can be assigned to two or more three-dimensional coordinates with similar positions.

[0040] The current position of the pollination device of the flower is obtained, and all three-dimensional coordinates are sorted from near to far according to the distance from the current position according to the K nearest neighbor algorithm to obtain the sorted three-dimensional coordinates.

[0041] The pollination nozzles are sorted according to the sorted three-dimensional coordinates to obtain a sorting result of the pollination nozzles.

[0042] The present invention realizes the continuous pollination of multiple flowers at the same time by setting the sequencing results of the pollination nozzles.

[0043] According to the sorting results, the pollination nozzles are selected in sequence to continuously perform air flow vibration, powder spraying and hormone spraying on at least one flower for pollination.

[0044] The flower pollination method provided in the embodiment of the present invention realizes automatic recognition of the growth stage of the flower through the recognition model. By setting the recognition model, the recognition efficiency of the recognition model is improved. Through the initial position and 3D point cloud data, the three-dimensional coordinates of the flower are automatically acquired, which is conducive to improving the accuracy of flower pollination. By determining the order of the pollination nozzles, the flowers are pollinated by air flow vibration, powder spraying and hormone spraying according to the three-dimensional coordinates corresponding to the pollination nozzles, thereby achieving uninterrupted and accurate pollination and improving the efficiency of flower pollination. The present invention greatly improves the success rate of flower pollination by performing air flow vibration, powder spraying and hormone spraying on the flowers for pollination.

[0045] Based on the three-dimensional coordinates corresponding to the pollination nozzle, the flowers are subjected to airflow vibration, powder spraying and hormone spraying for pollination, including S410 to S420, and each step is specifically as follows.

[0046] S410: According to the three-dimensional coordinates corresponding to the pollination nozzle, the pollination nozzle is moved up and down, swung and telescopically moved until the pollination nozzle moves to the optimal pollination position corresponding to the three-dimensional coordinates of the pollination nozzle.

[0047] S420: Based on the pollination nozzle, intermittent airflow, powder and hormones are sprayed to the flowers at the three-dimensional coordinates corresponding to the pollination nozzle at a fixed time cycle and set duration to complete airflow vibration, powder spraying and hormone spraying pollination.

[0048] The pollination nozzle currently ready for pollination and the corresponding current three-dimensional coordinates are selected according to the sorting results. The lifting slide of the pollination device of the flower moves the pollination module up and down to control the pollination nozzle in the pollination module to move to the height of the best pollination position corresponding to the current three-dimensional coordinates of the pollination nozzle. Each pollination nozzle includes a pollination telescopic rod and a pollination servo. The pollination telescopic rod and the pollination servo swing and telescope the pollination nozzle according to the current three-dimensional coordinates until the pollination nozzle moves to the best pollination position of the current three-dimensional coordinates.

[0049] Each pollination nozzle also includes a pollination box. The air pump of the flower's pollination device delivers airflow to the pollination nozzle, the pollination box delivers powder to the pollination nozzle, and the medicine barrel of the flower's pollination device delivers hormones to the pollination nozzle. The pollination nozzle sprays intermittent airflow, powder and hormones to the flower corresponding to the three-dimensional coordinates at a fixed time cycle and set duration to complete the current airflow vibration, powder spraying and hormone spraying pollination.

[0050] For example, the fixed time period is 0.5s, and the set duration is 20-30s. The pollination nozzle opens and closes at intervals of 0.5s, and the pollination nozzle sprays intermittent airflow, powder, and hormones with a period of 0.5s to the bouquet. The airflow speed of the intermittent airflow, powder, and hormones when reaching the flowers is 1.5-3.5 (meters per second) m / s, and the duration of the intermittent airflow, powder, and hormones is 20-30s.

[0051] The flowers are vibrated by intermittent airflow, powder and hormones. Driven by wind and vibration, pollen will fall from the stamens to the pistils. The airflow vibration simulates natural wind to promote pollen scattering. Powder spraying can evenly spread the pollen, and spraying hormones can be replenished on demand. During the application process, the pollination time, pollination method, airflow intensity and airflow oscillation interval can be adjusted according to different types of tomato flowers.

[0052] The next pollination nozzle to be prepared for pollination and the corresponding next three-dimensional coordinate are selected according to the sorting results. The lifting slide moves the pollination module up and down to control the height of the pollination nozzle in the pollination module to move to the optimal pollination position of the next three-dimensional coordinate. The pollination telescopic rod and the pollination servo swing and telescope the pollination nozzle according to the current three-dimensional coordinate until the pollination nozzle moves to the optimal pollination position of the next three-dimensional coordinate. The flowers corresponding to the next three-dimensional coordinate are vibrated by air flow, sprayed with pollen and sprayed with hormones to achieve continuous pollination.

[0053] During the pollination process, the binocular infrared depth camera continuously obtains RGB image information of flowers in the unpollinated area. The control module further identifies the growth stage of the flowers. The control module obtains the three-dimensional coordinates of the flowers that need to be pollinated and sends the three-dimensional coordinates to the pollination module, thereby realizing the uninterrupted operation of the flower pollination device.

[0054] Optionally, multiple pollination nozzles can be selected simultaneously according to the sorting results, and multiple flowers can be pollinated by air flow vibration, powder spraying and hormone spraying at the same time.

[0055] The present invention improves the accuracy of pollination by moving the pollination nozzle up and down, swinging and telescopically. The pollination nozzles are selected in sequence according to the sorting results for pollination, thereby improving the efficiency of continuous pollination of multiple flowers. Through a fixed time period and a set duration, interstitial airflow, powder and hormones are sprayed to the flowers corresponding to the three-dimensional coordinates, thereby improving the efficiency of flower pollination.

[0056] Based on the above embodiment, the image data of at least one flower is acquired based on a binocular infrared depth camera, including S110 to S140, and each step is specifically as follows.

[0057] S110: Acquire RGB image information of at least one flower based on a binocular infrared depth camera.

[0058] S120: grayscale processing is performed on the RGB image information according to a weighted average algorithm to obtain an initial grayscale image.

[0059] S130: Perform Gaussian filtering on the initial grayscale image to obtain a denoised image.

[0060] S140: performing histogram equalization processing and pixel grayscale mapping change processing on the denoised image to obtain image data, where the image data is a grayscale image with uniform probability distribution.

[0061] The RGB image information of at least one flower is obtained according to the binocular infrared depth camera. The RGB image information is gray-scaled according to the weighted average algorithm to obtain an initial gray-scale image. The initial gray-scale image is Gaussian-filtered to obtain a denoised image. The Gaussian-filtered process is used to reduce the blur of the initial gray-scale image and denoise the initial gray-scale image. The denoised image is subjected to histogram equalization and pixel gray-scale mapping to obtain image data, which is a gray-scale image with uniform probability distribution. The gray-scale image with uniform probability distribution can improve the contrast of the image data.

[0062] The present invention improves the quality and purity of image data by performing grayscale processing, Gaussian filtering processing, histogram equalization processing and pixel grayscale mapping change processing on RGB image information, eliminates background interference, makes the flower target more prominent, and is beneficial to improving the accuracy of the flower growth stage output by the recognition model.

[0063] Based on the above embodiment, the preset attention neural network includes the Yolov8 network, and the recognition model is constructed based on steps S210 to S230, and the details of each step are as follows.

[0064] S210: Flip, rotate, scale, or randomly crop the initial sample image data to obtain diversified sample image data.

[0065] S220: adding a label of the sample growth stage to each sample image data to obtain a training sample carrying the label.

[0066] S230: Training the Yolov8 network based on the training samples, adjusting the weights of the initial flower features in the Yolov8 network until the error of the predicted growth stage output by the Yolov8 network is less than the set error, and obtaining a recognition model.

[0067] The sample RGB image information of the sample flower is obtained. The sample RGB image is subjected to grayscale processing, Gaussian filtering processing, histogram equalization processing and pixel grayscale mapping change processing to obtain initial sample image data.

[0068] In the actual pollination process, due to various factors, the image data obtained may have many unknown changes. The initial sample image data is flipped, rotated, scaled or randomly cropped to obtain diversified sample image data. In this way, the number of sample image data can be increased, the diversity of sample image data can be increased, the robustness of the recognition model can be improved, and overfitting can be avoided.

[0069] A label of the sample growth stage is added to each sample image data to obtain a training sample with a label. The Yolov8 network is trained based on the training sample until the error of the predicted growth stage of the Yolov8 network is less than the set error, and a recognition model is obtained. The Yolov8 network is a Yolov8 network for fine detection of small targets with an added attention mechanism. During the training process, the weights of the features of various initial flowers in the Yolov8 network are adjusted to improve the accuracy of the growth stage of the flowers output by the Yolov8 network.

[0070] When the error of the predicted growth stage of the flower output by the Yolov8 network is less than the set error, the training is terminated, the weights of the features of multiple flowers are obtained, and the recognition model is obtained.

[0071] The present invention obtains diversified sample image data by flipping, rotating, scaling or randomly cropping the initial sample image data, thereby increasing the diversity of the sample image data, improving the robustness of the recognition model and avoiding overfitting. During the training process, by adjusting the weights of the features of multiple initial flowers, it is beneficial to improve the accuracy of the growth stage of the flower output by the recognition model.

[0072] Based on the above embodiment, the recognition model is used to determine the growth stage of at least one flower: feature extraction is performed on at least one flower in the image data to obtain flower features and weights of the flower features, the flower features including shape, color and texture; weighted fusion is performed on the flower features and the weights of the flower features to obtain the growth stage of at least one flower, the growth stages including bud stage, fully open stage, flowering stage and initial fruit stage.

[0073] The image data obtained after preprocessing is input into the recognition model (Yolov8 network with added attention mechanism). Based on the traditional Yolov8, the recognition model incorporates a carefully designed attention mechanism to enable it to focus more on the key feature areas of the flower. The attention mechanism automatically strengthens the representation of key features such as the flower's morphology, color, and texture by learning the importance weights of different areas in the image, and weakens the influence of irrelevant information. During the forward propagation of the network, the feature extraction layer first extracts multi-scale features from the image to generate rich feature maps (including multiple flower features). The attention module calculates the attention weight (the weight of the flower feature) of each position based on the information of these feature maps, and applies it to the feature map to enhance the expression of important features.

[0074] The recognition model is based on the efficient detection architecture of the Yolov8 network, which can quickly locate and classify tomato flowers in image data. By predicting the position and category probability of the bounding box, it can accurately identify the position of the flower and its growth stage, including the bud stage, full bloom stage, flowering stage and initial fruit stage.

[0075] The present invention is based on the unique network structure and algorithm design of the Yolov8 network with an added attention mechanism, so that high-precision detection can be achieved in complex environments even if the flowers are partially blocked or have morphological changes.

[0076] The flower pollination device provided by the present invention is described below. The flower pollination device described below and the flower pollination method described above can be referred to each other.

[0077] like Figure 2 As shown, a flower pollination device includes: an image acquisition module 201, which is used to acquire image data of at least one flower based on a binocular infrared depth camera 1009.

[0078] The control module 202 is used to input the image data into the recognition model and obtain the recognition result of at least one flower output by the recognition model, wherein the recognition model is obtained by training based on the sample image data and the labels of the sample growth stage on the basis of a preset attention neural network.

[0079] The control module 202 is further configured to match the initial position with the 3D point cloud data of the flower to obtain the three-dimensional coordinates of at least one flower when the growth stage includes a growth stage requiring pollination.

[0080] The pollination module 203 is used to sort the pollination nozzles at each three-dimensional coordinate, select the pollination nozzles in turn according to the sorting results, and perform air flow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and accurate pollination.

[0081] The flower pollination device provided in the embodiment of the present invention realizes automatic recognition of the growth stage of the flower through the recognition model. By setting the recognition model, the recognition efficiency of the recognition model is improved. Through the initial position and 3D point cloud data, the three-dimensional coordinates of the flower are automatically acquired, which is conducive to improving the accuracy of flower pollination. By determining the order of the pollination nozzles, the flowers are pollinated by air flow vibration, powder spraying and hormone spraying according to the three-dimensional coordinates corresponding to the pollination nozzles, thereby achieving uninterrupted and accurate pollination and improving the efficiency of flower pollination. The present invention greatly improves the success rate of flower pollination by pollinating the flowers by air flow vibration, powder spraying and hormone spraying.

[0082] In one embodiment, the pollination device of the flower also includes a lifting slide 1002, a medicine barrel and an air pump 1003, the pollination module includes a plurality of pollination nozzles 1001, each pollination nozzle 1001 includes a pollination telescopic rod 1008, a pollination servo 1007 and a pollination box 1004; the lifting slide is used to move the pollination module up and down to control the height of the pollination nozzle to move to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle, the pollination telescopic rod and the pollination servo are used to swing and telescopically move the pollination nozzle according to the three-dimensional coordinates corresponding to the pollination nozzle until the pollination nozzle moves to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle; the air pump is used to convey airflow to the pollination nozzle, the pollination box is used to convey powder to the pollination nozzle, and the medicine barrel is used to convey hormones to the pollination nozzle, so that the pollination nozzle can spray intermittent airflow, powder and hormones to the flower at the three-dimensional coordinates corresponding to the pollination nozzle at a fixed time period and set duration to complete airflow vibration, powder spraying and hormone spraying pollination.

[0083] like Figure 3 As shown, further, each pollination nozzle 1001 also includes a universal ball 1006, a driving motor and a nozzle box 1005. The pollination nozzle 1001 is connected to the pollination telescopic rod 1008 through the nozzle box 1005, and the air flow is controlled by adjusting the opening and closing of the air valve of the pollination nozzle; the air volume and air pressure of the pollination nozzle are changed by adjusting the built-in air valve of the nozzle box, thereby realizing the pressure and air flow control of the air outlet.

[0084] like Figure 2As shown, the pollination nozzle is an air spray nozzle, which is installed on a nozzle box. The nozzle box has a built-in pollination air pipe and an air valve connected to the air pump in the chassis of the flower pollination device. When pollination begins, the pollination servo 1007 smoothly adjusts the appropriate angle through the universal ball 1006 under the guidance of the control module. The pollination telescopic rod moves the nozzle box and the pollination nozzle 1001 to the front of the flower (the best pollination position of the current three-dimensional coordinates) under the guidance of the control module. The air valve is opened and closed according to a fixed cycle under the control of the control module. The high-pressure air generated by the air pump (such as a multi-functional air pump) can drive the gas, powder and hormones to be intermittently sprayed through the pollination air pipe and the pollination nozzle, and the flower is vibrated through intermittent airflow vibration. Driven by wind and vibration force, the pollen will fall from the stamen to the pistil. The airflow vibration simulates natural wind to promote pollen scattering. The spray powder can be evenly spread, and the spray hormone can be replenished on demand. During the application process, the pollination time, pollination method, airflow intensity and airflow oscillation interval can be adjusted according to different tomato flower types. The distance between the pollination nozzle and the flower during the pollination process can be set to improve the controllability, applicability and diversity of the pollination operation.

[0085] like Figure 3 As shown, the pollination nozzle and the nozzle box are installed on the pollination telescopic rod, and the pollination telescopic rod is installed on the chassis inside the pollination box. The pollination box is placed horizontally in the pollination box through a universal ball, and the pollination box is 0.5m above the ground. The lifting slide is two sections of slides connected to each other. The pollination box is placed at an initial height of 0.5m, and the telescopic height can be adjusted according to the system settings, such as Figure 3 As shown, the drive motor can drive the lifting slide to a maximum height of 2.4m. Figure 4 As shown, the pollination nozzle, pollination servo, and pollination telescopic rod can adjust the spraying angle and distance. The multifunctional pollination module that combines flower identification and positioning with airflow vibration, powder spraying, and hormone spraying is installed on the pollination box. The height of the pollination module is adjusted by driving the combined slide table driven by the drive motor to adapt to the pollination needs of different growth stages. The maximum lifting height is 2.9m, and the pollination height range is 0.5m-2.9m.

[0086] The pollination module of the present invention is a multifunctional tomato pollination module that combines airflow vibration, powder spraying, and hormone spraying. As a brand-new pollination method for eggplant flowers, the pollination module of the present invention improves the applicability and controllability of pollination and the efficiency of pollination operation compared with bumblebee pollination, spot-application hormone pollination, and vibrator pollination. It effectively avoids the contingency and uncertainty of male pollination, and at the same time avoids the problems of strong subjectivity, low work efficiency, high labor intensity, and poor applicability of spot-application hormone pollination and vibrator pollination.

[0087] In one embodiment, the image acquisition module 201 is used to: acquire RGB image information of at least one flower based on a binocular infrared depth camera; perform grayscale processing on the RGB image information according to a weighted average algorithm to obtain an initial grayscale image; perform Gaussian filtering on the initial grayscale image to obtain a denoised image; perform histogram equalization processing and pixel grayscale mapping change processing on the denoised image to obtain image data, and the image data is a grayscale image with uniform probability distribution.

[0088] In one embodiment, the preset attention neural network includes a Yolov8 network, and the control module 202 is further used to: flip, rotate, scale or randomly crop the initial sample image data to obtain diversified sample image data; add a label of the sample growth stage to each sample image data to obtain a training sample carrying the label; train the Yolov8 network based on the training sample, adjust the weights of the initial flower features in the Yolov8 network, until the error of the predicted growth stage output by the Yolov8 network is less than the set error, and obtain a recognition model.

[0089] In one embodiment, the recognition model is used to determine the growth stage of at least one flower: feature extraction is performed on at least one flower in the image data to obtain flower features and weights of the flower features, the flower features include morphology, color and texture; weighted fusion is performed on the flower features and the weights of the flower features to obtain the growth stage of at least one flower, the growth stages include bud stage, full bloom stage, flowering stage and initial fruit stage.

[0090] In one embodiment, the pollination module 203 is used to: arrange the three-dimensional coordinates in order from near to far according to the K nearest neighbor algorithm to obtain sorted three-dimensional coordinates; obtain the pollination nozzle corresponding to each three-dimensional coordinate, and sort the pollination nozzles based on the sorted three-dimensional coordinates.

[0091] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute a pollination method for flowers, which method includes: acquiring image data of at least one flower based on a binocular infrared depth camera; inputting the image data into a recognition model to obtain a recognition result of at least one flower output by the recognition model, and the recognition result includes a growth stage and an initial position; wherein the recognition model is obtained based on a preset attention neural network and is trained based on sample image data and labels of sample growth stages; when the growth stage includes a growth stage that requires pollination, the initial position is matched with the 3D point cloud data of the flower to obtain the three-dimensional coordinates of at least one flower; the pollination nozzles of each three-dimensional coordinate are sorted, and the pollination nozzles are selected in turn according to the sorting results, and the flowers are pollinated by air flow vibration, powder spraying and hormone spraying based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and precise pollination.

[0092] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the pollination method of flowers provided by the above-mentioned methods, the method comprising: acquiring image data of at least one flower based on a binocular infrared depth camera; inputting the image data into a recognition model, and acquiring a recognition result of at least one flower output by the recognition model, the recognition result including a growth stage and an initial position; wherein the recognition model is obtained based on a preset attention neural network and trained based on sample image data and labels of sample growth stages; when the growth stage includes a growth stage requiring pollination, matching the initial position with the 3D point cloud data of the flower to obtain the three-dimensional coordinates of at least one flower; sorting the pollination nozzles of each three-dimensional coordinate, selecting the pollination nozzles in turn according to the sorting results, and performing air flow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and precise pollination.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for pollinating a flower, characterized in that: include: Acquire image data of at least one flower based on a binocular infrared depth camera; Input the image data into a recognition model, and obtain a recognition result of at least one of the flowers output by the recognition model, wherein the recognition result includes a growth stage and an initial position; wherein the recognition model is obtained by training based on a preset attention neural network and labels of sample image data and sample growth stages; When the growth stage includes a growth stage requiring pollination, matching the initial position with 3D point cloud data of the flower to obtain at least one three-dimensional coordinate of the flower; The pollination nozzles at each of the three-dimensional coordinates are sorted, and the pollination nozzles are selected in turn according to the sorting results. The flowers are pollinated by air flow vibration, powder spraying and hormone spraying based on the three-dimensional coordinates corresponding to the pollination nozzles, so as to achieve uninterrupted and precise pollination.

2. The flower pollination method according to claim 1, characterized in that: The method of performing airflow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles includes: According to the three-dimensional coordinates corresponding to the pollination nozzle, the pollination nozzle is moved up and down, swayed and telescopically moved until the pollination nozzle moves to the optimal pollination position corresponding to the three-dimensional coordinates of the pollination nozzle; Based on the pollination nozzle, intermittent airflow, powder and hormones are sprayed to the flowers at the three-dimensional coordinates corresponding to the pollination nozzle at a fixed time cycle and set duration to complete airflow vibration, powder spraying and hormone spraying pollination.

3. The flower pollination method according to claim 1, characterized in that: The method of acquiring image data of at least one flower based on a binocular infrared depth camera includes: Acquire RGB image information of at least one of the flowers based on the binocular infrared depth camera; Performing grayscale processing on the RGB image information according to a weighted average algorithm to obtain an initial grayscale image; Performing Gaussian filtering on the initial grayscale image to obtain a denoised image; The denoised image is subjected to histogram equalization processing and pixel grayscale mapping change processing to obtain the image data, which is a grayscale image with uniform probability distribution.

4. The flower pollination method according to claim 1, characterized in that: The preset attention neural network includes a Yolov8 network, and the recognition model is constructed based on the following steps: Flip, rotate, scale or randomly crop the initial sample image data to obtain diversified sample image data; Adding a label of the sample growth stage to each of the sample image data to obtain a training sample carrying the label; The Yolov8 network is trained based on the training sample, and the weights of the initial flower features in the Yolov8 network are adjusted until the error of the predicted growth stage output by the Yolov8 network is less than the set error, thereby obtaining the recognition model.

5. The flower pollination method according to claim 1, characterized in that: The recognition model is used to determine the growth stage of at least one of the flowers: Extracting features of at least one of the flowers in the image data to obtain flower features and weights of the flower features, wherein the flower features include shape, color, and texture; The flower features and the weights of the flower features are weightedly fused to obtain at least one growth stage of the flower, wherein the growth stage includes a bud stage, a full-bloom stage, a flowering stage, and an initial fruit stage.

6. The flower pollination method according to claim 1, characterized in that: The step of sorting the pollination nozzles at each of the three-dimensional coordinates comprises: According to the K nearest neighbor algorithm, the three-dimensional coordinates are arranged in order from near to far to obtain sorted three-dimensional coordinates; The pollination nozzle corresponding to each of the three-dimensional coordinates is obtained, and the pollination nozzles are sorted based on the sorted three-dimensional coordinates.

7. A flower pollination device, characterized in that: include: An image acquisition module, used to acquire image data of at least one flower based on a binocular infrared depth camera; A control module, used for inputting the image data into a recognition model, and obtaining a recognition result of at least one of the flowers output by the recognition model, wherein the recognition result includes a growth stage and an initial position; wherein the recognition model is obtained by training based on a preset attention neural network and labels of sample image data and sample growth stages; The control module is further configured to match the initial position with the 3D point cloud data of the flower to obtain at least one three-dimensional coordinate of the flower when the growth stage includes a growth stage requiring pollination; The pollination module is used to sort the pollination nozzles at each of the three-dimensional coordinates, select the pollination nozzles in turn according to the sorting results, and perform air flow vibration, powder spraying and hormone spraying on the flowers based on the three-dimensional coordinates corresponding to the pollination nozzles to achieve uninterrupted and precise pollination.

8. The flower pollination device according to claim 7, characterized in that: It also includes a lifting slide, a medicine barrel and an air pump. The pollination module includes a plurality of pollination nozzles, each of which includes a pollination telescopic rod, a pollination steering gear and a pollination box; The lifting slide is used to move the pollination module up and down to control the height of the pollination nozzle to move to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle. The pollination telescopic rod and the pollination servo are used to swing and telescopically move the pollination nozzle according to the three-dimensional coordinates corresponding to the pollination nozzle until the pollination nozzle moves to the optimal pollination position of the three-dimensional coordinates corresponding to the pollination nozzle; The air pump is used to convey airflow to the pollination nozzle, the pollination box is used to convey powder to the pollination nozzle, and the medicine barrel is used to convey hormones to the pollination nozzle, so that the pollination nozzle can spray intermittent airflow, powder and hormones to the flowers at the three-dimensional coordinates corresponding to the pollination nozzle at a fixed time period and set duration, so as to complete airflow vibration, powder spraying and hormone spraying pollination.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the flower pollination method according to any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the flower pollination method according to any one of claims 1 to 6 is implemented.

Images