Combined control system of unmanned aerial vehicle and ground agricultural machinery

Abstract

The invention provides a combined control system for an unmanned aerial vehicle and a ground agricultural machine, and relates to the technical field of intelligent agriculture, and the system comprises an obtaining unit which is used for obtaining a crop image of a farm harvesting area by the unmanned aerial vehicle; the detection unit is used for performing area detection on the crop image and dividing the crop image into a mature area and an immature area; the analysis unit is used for analyzing the mature areas and the immature areas, generating a harvesting operation path and sending the harvesting operation path to the ground agricultural machine so as to control the ground agricultural machine to perform harvesting operation according to the harvesting operation path; the optimization unit is used for performing harvesting optimization according to the crop data corresponding to the harvesting operation path in the harvesting operation process of the ground agricultural machine to obtain harvesting control parameters of the ground agricultural machine; and the regulation and control unit is used for regulating and controlling the ground agricultural machine to continuously complete the harvesting operation of the harvesting operation path based on the harvesting control parameters. The control system provided by the invention can ensure the harvesting yield of crops while ensuring the harvesting efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent agricultural technology, and in particular to a combined control system for unmanned aerial vehicles (UAVs) and ground agricultural machinery. Background Technology

[0002] To improve convenience, before harvesting, agricultural machinery now uses drones to take simple pictures of the crop growing area for manual reference. Workers then use these images to determine if harvesting is possible, and only proceed with the ground-based harvesting operations if it is deemed feasible.

[0003] However, the existing manual harvesting confirmation method is inefficient in viewing the collected images and is not convenient for timely and accurate analysis of the optimal harvesting time for mature crops in the corresponding crop area. This not only affects the harvesting efficiency, but also leads to a significant reduction in harvesting yield due to uncontrollable factors such as unharvested mature crops and the high speed of agricultural machinery harvesting. Summary of the Invention

[0004] The joint control system for drones and ground agricultural machinery provided by this invention solves the technical problem that the existing manual harvesting confirmation method has low efficiency in viewing acquired images and is not convenient for timely and accurate analysis of the optimal harvesting time for mature crops in the corresponding crop area, which not only affects harvesting efficiency but also leads to a significant reduction in harvesting yield.

[0005] To achieve the above and other related objectives, the present invention provides a joint control system for a drone and ground agricultural machinery, comprising: an acquisition unit for acquiring images of crops in a farm harvesting area from the drone; a detection unit for performing region detection on the crop images, dividing the crop images into mature and immature areas; an analysis unit for analyzing the mature and immature areas, generating a harvesting operation path, and sending it to the ground agricultural machinery to control the ground agricultural machinery to perform harvesting operations according to the harvesting operation path; an optimization unit for optimizing the harvesting operation based on the crop data corresponding to the harvesting operation path during the harvesting operation of the ground agricultural machinery, obtaining harvesting control parameters for the ground agricultural machinery; and a control unit for controlling the ground agricultural machinery to continue completing the harvesting operation along the harvesting operation path based on the harvesting control parameters.

[0006] In one embodiment of the present invention, the detection unit includes: a conversion subunit, used to convert the crop image to point coordinates corresponding to ground coordinates to obtain the actual coordinates of each point in the crop image; a cutting subunit, used to cut the crop image into segments to obtain multiple crop segments; a first judgment subunit, used to judge whether the similarity between the crop segment and the crop maturity reference image reaches a preset similarity; if yes, the crop segment is regarded as a mature segment; if no, the crop segment is regarded as an immature segment, the immature segment including immature crops and disaster crops; and a segmentation subunit, used to splice the mature segment and the immature segment according to their corresponding actual coordinates to divide the crop image into mature areas and immature areas, wherein the mature area is obtained by splicing mature segments and the immature area is obtained by splicing immature segments.

[0007] In one embodiment of the present invention, the analysis unit includes: a second judgment subunit, used to judge whether mature and immature areas in the monitoring area meet the harvesting requirements and obtain a judgment result, wherein the monitoring area is an area selected from the farm's harvesting area; a first operation control subunit, used to generate a harvesting operation path based on the mature and immature areas and send it to the ground agricultural machinery when the judgment result indicates that the mature and immature areas in the monitoring area meet the harvesting requirements, so as to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path; and a second operation control subunit, used to continue to control the UAV to periodically collect crop images of the farm's harvesting area when the judgment result indicates that the mature and immature areas in the monitoring area do not meet the harvesting requirements.

[0008] In one embodiment of the present invention, the second judgment subunit includes: a ratio calculation module, used to calculate the ratio of mature areas according to the monitoring area to obtain a maturity ratio; a threshold judgment module, used to determine whether the maturity ratio exceeds a ratio threshold; a first output module, used to predict the maturity time of immature crops in immature areas of the monitoring area when the maturity ratio exceeds the ratio threshold, to obtain a predicted maturity time, and outputting that the mature and immature areas of the monitoring area meet the harvesting requirements as a judgment result when the predicted maturity time exceeds a set time requirement; and a second output module, used to output that the mature and immature areas of the monitoring area do not meet the harvesting requirements as a judgment result when the maturity ratio does not exceed the ratio threshold.

[0009] In one embodiment of the present invention, the proportion calculation module includes: a generation submodule, used to generate an additional harvesting area based on the disaster area of ​​the disaster-stricken crops in the immature area, wherein the additional harvesting area consists of other mature areas outside the monitoring area and adjacent to the monitoring area; an adding submodule, used to add the additional harvesting area to the monitoring area to obtain an updated area; and a calculation submodule, used to perform a proportion calculation on the mature area based on the updated area to obtain a maturity proportion, wherein the formula for calculating the maturity proportion is: , This indicates the total area corresponding to the monitored area. This represents the first area corresponding to the mature zone. This represents the second area corresponding to the immature zone. This indicates the third area corresponding to the increased harvesting area. This indicates the total updated area of ​​the updated region.

[0010] In one embodiment of the present invention, the first output module, in the process of predicting the maturity time of immature crops in immature areas of a monitoring area, includes: a region identification submodule, used to identify immature crops in immature areas to obtain immature areas; a comparison submodule, used to compare the similarity between immature areas and growth images in a crop growth image library to obtain the growth image in the crop growth image library that is closest to the immature area as the target growth image; and a calculation submodule, used to calculate the maximum superposition time among all immature areas as the predicted maturity time based on the image similarity between the immature area and the target growth image, the time conversion factor, and the growth time required for the target growth image to grow into a mature crop. The formula for calculating the predicted maturity time is as follows: , Indicates image similarity. Indicates the time conversion factor. This indicates the growth time required for a target growth image to grow into a mature crop.

[0011] In one embodiment of the present invention, the first operation control subunit generates a harvesting operation path based on mature and immature areas and sends it to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path. The subunit includes: a path generation module, which is used to take the mature areas in the monitoring area that meet the harvesting requirements as the harvesting operation areas, and generate an optimal harvesting path that avoids immature areas as the harvesting operation path according to the harvesting operation areas; and a sending module, which is used to send the harvesting operation path to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path.

[0012] In one embodiment of the present invention, the crop data includes crop sparsity data per unit length of the harvesting path, crop lodging data, and actual crop harvesting rate data. The optimization unit includes: a first generation subunit for generating a first harvesting control parameter based on the crop sparsity data; a second generation subunit for generating a second harvesting control parameter based on the crop lodging data; a third generation subunit for generating a third harvesting control parameter based on the actual crop harvesting rate data; and a comprehensive calculation subunit for performing harvesting optimization based on the first, second, and third harvesting control parameters to obtain the harvesting control parameters for the ground agricultural machinery. The calculation formula for the harvesting control parameters is as follows: ,in, Represents crop sparsity data. This represents data on crop dumping rate. This represents the actual harvest rate of crops. This represents the first transformation factor for crop sparsity data to harvest control parameters. This represents the second conversion factor for crop lodging data to harvest control parameters. This represents the third conversion factor for the actual crop harvest rate data to the harvest control parameters.

[0013] In one embodiment of the present invention, the crop data includes crop sparsity data, crop lodging data, and actual crop harvesting rate data per unit length of the harvesting path; the optimization unit further includes: a sparsity calculation subunit, used to identify missing crop points in mature areas per unit length of the harvesting path, obtain the area of ​​the missing points, and obtain crop sparsity data based on the harvested area and the area of ​​the missing points per unit length of the harvesting path. The calculation formula for crop sparsity data is as follows: , This represents the area of ​​each missing point. This represents the harvested area per unit length of the harvesting path; the lodging calculation subunit is used to identify the lodging points of crops in the mature area per unit length of the harvesting path, obtain the area of ​​each lodging point, identify the degree of lodging at each lodging point, obtain lodging degree data, and obtain crop lodging degree data based on the harvested area, lodging point area, and lodging degree data per unit length of the harvesting path. The formula for calculating crop lodging degree data is as follows: , This represents the area of ​​each dumping point. This represents the harvested area per unit length of the harvesting path. This represents the dumping degree data corresponding to each dumping point. This represents the first weight value corresponding to the area of ​​the dumping point. This represents the second weight value corresponding to the crop lodging data; and a harvest rate calculation subunit, used to compare crop images in mature areas per unit length of the harvesting path with historical harvest images. When a crop image reaches a set similarity, the actual crop harvest rate data is obtained based on the historical crop harvest rate corresponding to the historical harvest image. The formula for calculating the actual crop harvest rate data is as follows: , This represents the historical harvest rate of crops corresponding to historical harvest images. This represents the similarity value between historical harvest images and corresponding crop images in mature areas. This represents the similarity compensation coefficient.

[0014] In one embodiment of the present invention, it further includes: a harvesting statistics unit, used to take the crop images after harvesting in mature areas within a unit length of the harvesting operation path as historical harvesting images, collect corresponding weight data, and calculate the historical harvesting rate of crops in the historical harvesting images based on the baseline weight and weight data. The formula for calculating the historical harvesting rate of crops is as follows: , Indicates weight data, Indicates the reference weight.

[0015] The beneficial effects of this invention are as follows: This invention proposes a joint control system for unmanned aerial vehicles (UAVs) and ground agricultural machinery. By controlling the UAV to periodically collect crop images of the harvesting area, and based on the maturity of the crops in the images, the system can accurately divide the crop images into mature and immature areas. Further analysis determines whether the mature and immature areas are suitable for harvesting. When harvesting is suitable, a harvesting path that avoids immature areas is generated and sent to the ground agricultural machinery. This enables joint control of the ground agricultural machinery to perform harvesting operations according to the harvesting path. Through this joint control of harvesting operations by UAVs and ground agricultural machinery, the simultaneous harvesting of immature or diseased crops can be avoided, ensuring the uniformity of crop maturity and preventing a significant reduction in yield due to untimely harvesting. Furthermore, during the harvesting process, in order to ensure harvesting efficiency and results, when ground agricultural machinery is harvesting according to the harvesting operation path, it can optimize the harvesting by utilizing data on the crops to be harvested along the harvesting operation path. This generates harvesting control parameters for controlling the harvesting of ground agricultural machinery, and by continuously sending the harvesting control parameters to the ground agricultural machinery, the machinery can harvest along the harvesting operation path while also harvesting according to the harvesting control parameters, ensuring both harvesting efficiency and crop yield. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram: Figure 1 This is a structural block diagram of the joint control system for unmanned aerial vehicles and ground agricultural machinery provided in an embodiment of the present invention; Figure 2 The diagram shown illustrates the monitoring area update process according to an embodiment of the present invention.

[0018] Figure 3 The diagram shown illustrates a harvesting optimization process provided in an embodiment of the present invention.

[0019] The attached figures are labeled as follows: Acquisition unit 111; detection unit 112; analysis unit 113; optimization unit 114; control unit 115. Detailed Implementation

[0020] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0021] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0022] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0023] Please see Figure 1The present invention provides a joint control system for a drone and ground agricultural machinery, comprising: an acquisition unit 111 for acquiring crop images of a farm harvesting area from the drone; a detection unit 112 for performing region detection on the crop images, dividing the crop images into mature and immature areas; an analysis unit 113 for analyzing the mature and immature areas, generating a harvesting operation path and sending it to the ground agricultural machinery to control the ground agricultural machinery to perform harvesting operations according to the harvesting operation path; an optimization unit 114 for optimizing the harvesting operation based on the crop data corresponding to the harvesting operation path during the harvesting operation of the ground agricultural machinery, obtaining harvesting control parameters for the ground agricultural machinery; and a control unit 115 for controlling the ground agricultural machinery to continue completing the harvesting operation along the harvesting operation path based on the harvesting control parameters.

[0024] As can be seen from the above, in the process of joint control of drones and ground agricultural machinery to complete the harvesting operation, the drone is first controlled to periodically collect crop images of the harvesting area of ​​the farm, and the collected crop images are sent to the acquisition unit 111. Of course, the acquisition unit 111 can also actively send an image acquisition signal to the drone system to retrieve the corresponding crop images. Then, the detection unit 112 divides the crop images into mature areas and immature areas according to the maturity of the crops. The immature areas include not only immature crop segments, but also damaged crops, and other segments that are difficult to harvest. After obtaining the mature and immature areas, the analysis unit 113 determines whether the current mature and immature areas are suitable for harvesting. When it is suitable for harvesting, a harvesting operation path that avoids the immature areas is generated and sent to the ground agricultural machinery to jointly control the ground agricultural machinery to carry out the harvesting operation according to the harvesting operation path. By combining drones and ground agricultural machinery in the aforementioned harvesting operation, it is possible to avoid harvesting immature or diseased crops, which are unsuitable for harvesting, simultaneously with mature crops, while also ensuring the uniformity of maturity of the harvested crops. Furthermore, to ensure harvesting efficiency and effectiveness, the ground agricultural machinery, while following the harvesting path, can be optimized by the optimization unit 114 based on the data of the crops to be harvested along the path. This optimization generates harvesting control parameters for controlling the ground agricultural machinery's harvesting and continuously sends these parameters to the ground agricultural machinery through the control unit 115. This allows the ground agricultural machinery to harvest along the path while simultaneously adhering to the harvesting control parameters, such as the harvesting speed, and possibly the harvesting height.

[0025] In the joint control system of the UAV and ground agricultural machinery of the present invention, the detection unit 112 includes: a conversion subunit, used to convert the crop image into points corresponding to ground coordinates to obtain the actual coordinates of each point in the crop image; a cutting subunit, used to cut the crop image into segments to obtain multiple crop segments; a first judgment subunit, used to judge whether the similarity between the crop segment and the crop maturity reference image reaches a preset similarity; if yes, the crop segment is regarded as a mature segment; if no, the crop segment is regarded as an immature segment, the immature segment including immature crops and disaster crops; and a division subunit, used to splice the mature segment and the immature segment according to the corresponding actual coordinates to divide the crop image into mature areas and immature areas, wherein the mature area is obtained by splicing mature segments and the immature area is obtained by splicing immature segments.

[0026] During the process of dividing the crop image into mature and immature areas by the detection unit 112, the actual ground coordinates of the crop image can be assigned by the conversion subunit, that is, the actual coordinates of each point in the crop image are marked. Specifically, the actual coordinates of each point in the crop image can be further determined by obtaining the shooting altitude coordinates when the drone took the crop image and based on the shooting wide-angle situation. Then, the crop image is first segmented into multiple crop segments by the cutting subunit. Then, the first judgment subunit compares the similarity of each crop segment with the crop maturity benchmark image pre-stored by the system. The similarity between the obtained crop segment and the crop maturity benchmark image is then compared with a preset similarity. This allows the crop segment to be identified as a mature segment when the similarity is less than the preset similarity, and as an immature segment when the similarity is greater than the preset similarity. The immature segment can include immature crops and disaster crops, and of course, it can also include other situations that are not suitable for harvesting. After dividing the crop into mature and immature segments, the mature segments are then pieced together sequentially to form mature regions, and the immature segments are pieced together to form immature regions. This method of region division effectively ensures the accuracy of the division between mature and immature regions, guaranteeing the stability and consistency of the maturity level of harvested crops.

[0027] In the joint control system of the UAV and ground agricultural machinery of the present invention, the analysis unit 113 includes: a second judgment subunit, used to judge whether the mature and immature areas in the monitoring area meet the harvesting requirements and obtain a judgment result, wherein the monitoring area is an area selected from the farm's harvesting area; a first operation control subunit, used to generate a harvesting operation path based on the mature and immature areas and send it to the ground agricultural machinery when the judgment result is that the mature and immature areas in the monitoring area meet the harvesting requirements, so as to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path; and a second operation control subunit, used to continue to control the UAV to periodically collect crop images of the farm's harvesting area when the judgment result is that the mature and immature areas in the monitoring area do not meet the harvesting requirements.

[0028] During the process of planning the harvesting operation path through the analysis unit 113, the second judgment subunit can select a monitoring area in the farm's harvesting area. Then, it can determine whether the distribution of mature and immature areas in the monitoring area is suitable for harvesting. If harvesting is possible, the first operation control subunit can then use the mature and immature areas to generate a harvesting operation path and send it to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path. Of course, if harvesting is not possible, but as the crops grow, the number of mature areas in the monitoring area increases, and the mature and immature areas in the monitoring area gradually meet the harvesting requirements, then the UAV can continue to periodically collect crop images of the farm's harvesting area until the mature and immature areas in the monitoring area meet the harvesting requirements. It is worth noting that the monitoring area can be selected sequentially along the harvesting direction of the farm's harvesting area. For example, after a harvested area or an area that has not met the harvesting requirements, a monitoring area can be selected again along the harvesting direction of the farm's harvesting area. In addition, a monitoring area can be used as a harvesting area.

[0029] The second judgment subunit may further include: a ratio calculation module, used to calculate the ratio of mature areas based on the monitoring area to obtain the maturity ratio; a threshold judgment module, used to determine whether the maturity ratio exceeds the ratio threshold; a first output module, used to predict the maturity time of immature crops in immature areas of the monitoring area when the maturity ratio exceeds the ratio threshold, obtain the predicted maturity time, and output that the mature and immature areas in the monitoring area meet the harvesting requirements as a judgment result when the predicted maturity time exceeds the set time requirement; and a second output module, used to output that the mature and immature areas in the monitoring area do not meet the harvesting requirements as a judgment result when the maturity ratio does not exceed the ratio threshold.

[0030] In the process of using the second judgment subunit to determine whether the mature and immature areas in the monitoring area meet the harvesting requirements, the proportion of mature areas in the monitoring area can be calculated first through the proportion calculation module to obtain the maturity ratio. Then, the threshold judgment module determines whether the maturity ratio of the mature areas in the monitoring area reaches the proportion threshold. If it exceeds the proportion threshold, the first output module can further predict the time taken for immature crops in the immature areas to grow from the current state to the mature state, thereby obtaining the predicted maturity time. Then, if the predicted maturity time is greater than the set time requirement, it means that the crops in the mature area cannot wait until the immature areas are fully mature before harvesting. This may cause the crops in the mature area to suffer from problems such as aging and grain loss (for example, if wheat is not harvested in time after maturity, a large number of wheat grains will fall off the ears), insect infestation, and rot after maturity. Therefore, it can be determined that in this case, the already mature areas should be harvested directly. If the predicted maturity time is less than the set time requirement, the harvesting can be carried out according to the predicted maturity time until the immature areas mature before the set time, and then the ground agricultural machinery can be controlled to carry out the harvesting operation.

[0031] Furthermore, the proportion calculation module includes: a generation submodule, used to generate additional harvesting areas based on the disaster area of ​​disaster-stricken crops in immature areas, wherein the additional harvesting areas consist of other mature areas outside the monitoring areas but adjacent to the monitoring areas; an addition submodule, used to add the additional harvesting areas to the monitoring areas to obtain updated areas; and a calculation submodule, used to perform proportion calculation on the mature areas based on the updated areas to obtain the maturity proportion, wherein the formula for calculating the maturity proportion is: , This indicates the total area corresponding to the monitored area. This represents the first area corresponding to the mature zone. This represents the second area corresponding to the immature zone. This indicates the third area corresponding to the increased harvesting area. This indicates the total updated area of ​​the updated region.

[0032] In calculating the maturity ratio of mature areas within a monitoring area using the proportion calculation module, since there may be disaster-affected areas due to crop damage in immature areas, to further ensure the harvesting task of a single monitoring area, additional harvesting areas can be generated based on the disaster-affected areas where crops cannot be harvested. These additional harvesting areas are then used to expand the monitoring area; that is, the additional harvesting areas can be added to the monitoring area through an adding submodule, regenerating an updated area. Finally, the calculation submodule calculates the maturity ratio of the mature areas based on the updated area, obtaining the maturity ratio. The formula can be expressed as: By using the above methods, the required harvesting area can be guaranteed, and the harvestable amount of ground agricultural machinery in a single harvest can be guaranteed.

[0033] Please see Figure 2 , Figure 2 In one embodiment, the left side of the farm harvesting area is the harvested area. Of course, there can also be areas to the left of the harvested area that do not meet harvesting requirements. The right side of the harvested area can be designated as a selected monitoring area to achieve continuous harvesting. When dividing the monitoring area into mature and immature areas, the presence of damaged crops in the immature areas can lead to a reduction in the harvest volume during harvesting of the monitoring area. Therefore, an additional harvesting area can be added after the monitoring area in the harvesting direction of the farm harvesting area (this additional harvesting area is a mature area, or it can be a combination of mature and immature areas. However, if immature areas exist, the maturity ratio and the predicted maturity time of the immature areas need to be further calculated before adding the additional area). This adjusts for insufficient harvest volume in the monitoring area. After the additional harvesting area is added to the monitoring area, an updated area is formed to generate the harvesting operation path. Of course, if the predicted maturity time of immature areas exceeds the set time, the harvesting area can be further set to be added after the monitored area based on the immature area to ensure the single harvesting volume of ground agricultural machinery.

[0034] Next, the first output module, in the process of predicting the maturity time of immature crops in immature areas of the monitoring area, includes: a region identification submodule, used to identify immature crops in immature areas to obtain immature areas; a comparison submodule, used to compare the similarity between immature areas and growth images in the crop growth image library to obtain the growth image in the crop growth image library that is closest to the immature area as the target growth image; and a calculation submodule, used to calculate the maximum superposition time among all immature areas as the predicted maturity time based on the image similarity between the immature area and the target growth image, the time conversion factor, and the growth time required for the target growth image to grow into mature crops. The formula for calculating the predicted maturity time is: , Indicates image similarity. Indicates the time conversion factor. This indicates the growth time required for a target growth image to grow into a mature crop.

[0035] When predicting the maturity time of immature crops through the first output module, the region identification submodule first identifies the immature crops in the immature areas to obtain the immature areas. Then, the comparison subunit compares the similarity of the immature areas with growth images in the crop growth image database to find the crop growth image most similar to the immature area, which is then used as the target growth image for calculating the predicted maturity time. The calculation submodule then uses the image similarity between the immature areas and the target growth image, the time conversion factor for converting image similarity to predicted maturity time, and the growth time required for the target growth image to grow into a mature crop to calculate the superposition time of all immature areas, expressed by the formula: Then, among the superimposed times corresponding to all immature areas, the maximum superimposed time is found as the predicted maturity time, expressed by the formula: By using the above method, the predicted maturity time can be calculated by superimposing the image similarity between the immature area and the target growth image and the growth time required for the target growth image to grow into a mature crop. This can effectively ensure the accuracy of the predicted maturity time.

[0036] Among these methods, various crop growth images and the time required for them to grow to maturity can be pre-calibrated using manual calibration and stored in the joint control system of drones and ground agricultural machinery, so as to compare and judge them with the corresponding images of immature areas.

[0037] Furthermore, the first operation control subunit, in the process of generating a harvesting operation path based on mature and immature areas and sending it to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path, includes: a path generation module, used to take the mature areas in the monitored area that meet the harvesting requirements as the harvesting operation area, and generate the optimal harvesting path to avoid immature areas as the harvesting operation path according to the harvesting operation area; and a sending module, used to send the harvesting operation path to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path.

[0038] During the generation of the harvesting path through the first operation control subunit, the path generation module utilizes the regional coordinates of mature and immature areas within the monitoring zone. The mature areas are used as the harvesting area to generate the optimal harvesting path that avoids the immature areas. This can be achieved by manually setting an initial path without any avoidance areas. Then, whenever an immature area appears on the initial path, the path is modified based on the size of the immature area and a pre-set avoidance path for that area, ultimately forming the harvesting path. Alternatively, geographic information, remote sensing data, or other data can be directly incorporated into the intelligent algorithm to generate the harvesting path. Finally, the generated harvesting path is sent to the ground-based agricultural machinery via the sending module, controlling the machinery to perform harvesting operations according to the path. This achieves joint control of the ground-based machinery to complete the harvesting task.

[0039] Preferably, the crop data includes crop density data per unit length of the harvesting path, crop lodging data, and actual crop harvesting rate data. Of course, it may also include data on other factors affecting harvesting.

[0040] Please see Figure 3 , Figure 3 In one embodiment, the ground-based agricultural machinery performs harvesting operations along the harvesting direction. When the ground-based agricultural machinery harvests mature areas along the harvesting direction, it optimizes the harvesting process based on crop sparseness data, crop lodging data, and actual crop harvesting rate data for that area. It then calculates harvesting control parameters for regulating the ground-based agricultural machinery's harvesting operations and continues to control the machinery to harvest along the harvesting direction.

[0041] In the joint control system of the UAV and ground agricultural machinery of the present invention, the optimization unit 114 includes: a first generation subunit, used to generate a first harvesting control parameter based on crop sparseness data; a second generation subunit, used to generate a second harvesting control parameter based on crop lodging data; a third generation subunit, used to generate a third harvesting control parameter based on actual crop harvesting rate data; and a comprehensive calculation subunit, used to perform harvesting optimization based on the first, second, and third harvesting control parameters to obtain the harvesting control parameters of the ground agricultural machinery, wherein the calculation formula for the harvesting control parameters is as follows: ,in, Represents crop sparsity data. This represents data on crop dumping rate. This represents the actual harvest rate of crops. This represents the first transformation factor for crop sparsity data to harvest control parameters. This represents the second conversion factor for crop lodging data to harvest control parameters. This represents the third conversion factor for the actual crop harvest rate data to the harvest control parameters.

[0042] When optimizing harvesting using optimization unit 114, a first harvesting control parameter can be generated based on crop sparseness data using a first generation subunit; a second harvesting control parameter can be generated based on crop lodging data using a second generation subunit; and a third harvesting control parameter can be generated based on actual crop harvesting rate data using a third generation subunit. Then, the first, second, and third harvesting control parameters are superimposed and calculated by a comprehensive calculation subunit to optimize the harvesting control parameters for the ground agricultural machinery. These parameters are used to control the ground agricultural machinery to harvest along the harvesting path, ensuring successful harvesting while also allowing for adjustment of the harvesting effect. The harvesting control parameters can include parameters such as harvesting speed, so as to ensure that when the sparseness is large, the harvesting speed is controlled to speed up the harvesting and complete the harvest. When the crop is heavily lodged, the harvesting speed is further slowed down. In addition, when the actual harvesting rate data is low, it means that the current crop yield is low, and the harvesting can be appropriately accelerated. Through comprehensive regulation, the harvesting efficiency can be improved while ensuring the harvest yield.

[0043] Specifically, the optimization unit 114 further includes a sparsity calculation subunit, used to identify missing crop points in mature areas per unit length of the harvesting path, obtain the area of ​​the missing points, and obtain crop sparsity data based on the harvested area and the area of ​​the missing points per unit length of the harvesting path. The formula for calculating the crop sparsity data is as follows: , This represents the area of ​​each missing point. This represents the harvested area per unit length of the harvesting path; the lodging calculation subunit is used to identify the lodging points of crops in the mature area per unit length of the harvesting path, obtain the area of ​​each lodging point, identify the degree of lodging at each lodging point, obtain lodging degree data, and obtain crop lodging degree data based on the harvested area, lodging point area, and lodging degree data per unit length of the harvesting path. The formula for calculating crop lodging degree data is as follows: , This represents the area of ​​each dumping point. This represents the harvested area per unit length of the harvesting path. This represents the dumping degree data corresponding to each dumping point. This represents the first weight value corresponding to the area of ​​the dumping point. This represents the second weight value corresponding to the crop lodging data; and a harvest rate calculation subunit, used to compare crop images in mature areas per unit length of the harvesting path with historical harvest images. When a crop image reaches a set similarity, the actual crop harvest rate data is obtained based on the historical crop harvest rate corresponding to the historical harvest image. The formula for calculating the actual crop harvest rate data is as follows: , This represents the historical harvest rate of crops corresponding to historical harvest images. This represents the similarity value between historical harvest images and corresponding crop images in mature areas. This represents the similarity compensation coefficient.

[0044] Before calculating the first harvesting control parameters using the first generation subunit, the crop sparsity data can be calculated using the sparsity calculation subunit. Specifically, missing crop points can be identified in the mature areas within a unit length of the harvesting path to determine the missing points, and the corresponding missing point areas can be calculated. Then, based on the harvested area per unit length of the harvesting path (the harvesting width is the width of the harvesting path, and the harvesting length is the unit length) and the missing point areas, the crop sparsity data can be calculated. The calculation formula can be expressed as follows: .

[0045] Before calculating the second harvesting control parameters using the second generation subunit, the crop lodging data can be calculated using the lodging calculation subunit. Specifically, the lodging points in mature areas within a unit length of the harvesting path can be identified to determine the lodging locations. The corresponding lodging area is then calculated, and the degree of lodging at each location is identified to obtain lodging degree data. Finally, based on the harvested area per unit length of the harvesting path (harvesting width is the width of the harvesting path, and harvesting length is the unit length), the area of ​​missing points, and the lodging degree data, the crop lodging data is calculated. The calculation formula can be expressed as follows: .

[0046] Before calculating the third harvest control parameter using the third generation subunit, the actual crop harvest rate data can be calculated using the harvest rate calculation subunit. Specifically, this can be done by comparing crop images in mature areas per unit length of the harvesting path with historical harvest images. If no crop image reaches the set similarity threshold, the third harvest control parameter is not adjusted. However, if a crop image reaches the set similarity threshold, the historical harvest rate corresponding to the historical harvest image can be used, combined with the similarity value between the historical harvest image and the corresponding mature area crop image, to compensate for the harvest rate, thereby obtaining the actual crop harvest rate data. The calculation formula can be expressed as follows: .

[0047] The joint control system for UAVs and ground agricultural machinery provided by this invention further includes: a harvesting statistics unit, used to take images of harvested crops in mature areas within a unit length of the harvesting operation path as historical harvesting images, collect corresponding weight data, and calculate the historical harvesting rate of crops in the historical harvesting images based on the baseline weight and weight data. The formula for calculating the historical harvesting rate of crops is as follows: , Indicates weight data, Indicates the reference weight.

[0048] During the harvesting process, controlled by the control unit, the ground-based agricultural machinery continuously monitors weight data corresponding to historical harvesting images and uploads it to the harvesting statistics unit. The harvesting statistics unit then uses historical harvesting images of mature crops harvested per unit length of the harvesting path to calibrate the corresponding weight data against these images. Based on a pre-set baseline weight corresponding to the historical harvesting image, and combining this weight data, the historical harvesting rate of the crop in the historical harvesting image can be calculated, expressed by the formula: .

[0049] This invention also provides a method for joint control of unmanned aerial vehicles (UAVs) and ground agricultural machinery, comprising the following steps: Step S10: Acquire images of crops in the farm's harvesting area by the drone through the acquisition unit 111; Step S20: The crop image is divided into mature and immature areas by the detection unit 112 for region detection. Step S30: The analysis unit 113 analyzes the mature and immature areas, generates a harvesting operation path, and sends it to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path; Step S40: During the ground agricultural machinery harvesting operation, the optimization unit 114 optimizes the harvesting based on the crop data corresponding to the harvesting operation path to obtain the harvesting control parameters of the ground agricultural machinery. Step S50: The control unit 115 controls the ground agricultural machinery to continue the harvesting operation along the harvesting path based on the harvesting control parameters.

[0050] In summary, the joint control system for unmanned aerial vehicles (UAVs) and ground agricultural machinery disclosed in this invention controls the UAV to periodically collect crop images of the farm's harvesting area. Based on the crop images and their maturity status, the system accurately divides the crop images into mature and immature areas. Further analysis determines whether the current mature and immature areas are suitable for harvesting. When harvesting is suitable, a harvesting path that avoids immature areas is generated and sent to the ground agricultural machinery. This enables joint control of the ground agricultural machinery to perform harvesting operations according to the harvesting path. Through this joint control of harvesting operations by UAVs and ground agricultural machinery, the simultaneous harvesting of immature or diseased crops can be avoided, ensuring the uniformity of crop maturity and preventing a significant reduction in yield due to untimely harvesting. Furthermore, during the harvesting process, to ensure harvesting efficiency and yield, ground agricultural machinery can optimize its harvesting by utilizing data on the crops to be harvested along the harvesting path. This generates harvesting control parameters, which are continuously sent to the machinery. This ensures that the machinery harvests along the path while adhering to the control parameters, guaranteeing both harvesting efficiency and crop yield. Therefore, this invention effectively overcomes the shortcomings of existing technologies and possesses high industrial application value.

[0051] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A joint control system for unmanned aerial vehicles (UAVs) and ground agricultural machinery, characterized in that, include: The acquisition unit is used to acquire images of crops in the farm's harvesting area taken by the drone. The detection unit is used to perform region detection on the crop image and divide the crop image into mature areas and immature areas. The analysis unit is used to analyze the mature and immature areas, generate a harvesting operation path and send it to the ground agricultural machinery to control the ground agricultural machinery to carry out harvesting operations according to the harvesting operation path; An optimization unit is used to optimize the harvesting process based on crop data corresponding to the harvesting path during the ground agricultural machinery harvesting operation, and obtain the harvesting control parameters of the ground agricultural machinery. as well as The control unit is used to control the ground agricultural machinery to continue the harvesting operation along the harvesting path based on the harvesting control parameters.

2. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 1, characterized in that, The detection unit includes: The transformation subunit is used to transform the crop image into points corresponding to ground coordinates to obtain the actual coordinates of each point in the crop image; The cutting subunit is used to cut the crop image into segments to obtain multiple crop segments; The first judgment subunit is used to determine whether the similarity between the crop fragment and the crop maturity benchmark image reaches a preset similarity; if yes, the crop fragment is considered a mature fragment; if no, the crop fragment is considered an immature fragment, the immature fragment including immature crops and disaster-affected crops; and The sub-unit is used to stitch together the mature and immature segments according to their corresponding actual coordinates, thereby dividing the crop image into mature and immature regions. The mature regions are obtained by stitching together the mature segments, and the immature regions are obtained by stitching together the immature segments.

3. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 1, characterized in that, The analysis unit includes: The second judgment subunit is used to judge whether the mature and immature areas in the monitoring area meet the harvesting requirements and obtain a judgment result, wherein the monitoring area is an area selected from the farm's harvesting area; The first operation control subunit is configured to, when the judgment result indicates that the mature and immature areas in the monitored area meet the harvesting requirements, generate a harvesting operation path based on the mature and immature areas and send it to the ground agricultural machinery, so as to control the ground agricultural machinery to perform harvesting operations according to the harvesting operation path; and The second operation control subunit is used to continue controlling the drone to periodically collect crop images of the farm harvesting area when the judgment result is that the mature area and the immature area in the monitoring area do not meet the harvesting requirements.

4. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 3, characterized in that, The second judgment subunit includes: The ratio calculation module is used to calculate the ratio of the mature area based on the monitoring area to obtain the maturity ratio; The threshold determination module is used to determine whether the maturity ratio exceeds the ratio threshold; The first output module is configured to, when the maturity ratio exceeds the ratio threshold, predict the maturity time of immature crops in the immature areas of the monitoring area to obtain the predicted maturity time, and when the predicted maturity time exceeds a set time requirement, output that the mature areas and immature areas in the monitoring area meet the harvesting requirements as the judgment result; and The second output module is used to output that the mature area and the immature area in the monitoring area do not meet the harvesting requirements as the judgment result when the maturity ratio does not exceed the ratio threshold.

5. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 4, characterized in that, The ratio calculation module includes: A generation submodule is used to generate additional harvesting areas based on the disaster area of ​​the disaster-stricken crops in the immature areas. The additional harvesting areas consist of other mature areas outside the monitoring areas and adjacent to the monitoring areas. An additional submodule is used to add the increased harvesting area to the monitoring area to obtain an updated area; and The calculation submodule is used to calculate the maturity ratio of the mature areas based on the updated areas, wherein the calculation formula for the maturity ratio is: , This indicates the total area corresponding to the monitored area. This represents the first area corresponding to the mature zone. This represents the second area corresponding to the immature zone. This indicates the third area corresponding to the increased harvesting area. This indicates the total updated area of ​​the updated region.

6. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 4, characterized in that, The first output module, in the process of predicting the maturity time of immature crops in the immature areas of the monitored area and obtaining the predicted maturity time, includes: The region identification submodule is used to identify immature crops in the immature area to obtain the immature area; The comparison submodule is used to compare the similarity of the immature area with growth images in a crop growth image library, and obtain the growth image in the crop growth image library that is closest to the immature area as the target growth image; and The calculation submodule is used to calculate the maximum superposition time among all the immature areas as the predicted maturity time based on the image similarity between the immature areas and the target growth image, the time conversion factor, and the growth time required for the target growth image to grow into a mature crop. The formula for calculating the predicted maturity time is as follows: , Indicates image similarity. Indicates the time conversion factor. This indicates the growth time required for a target growth image to grow into a mature crop.

7. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 3, characterized in that, The process by which the first operation control subunit generates the harvesting operation path based on the mature and immature areas and sends it to the ground agricultural machinery to control the ground agricultural machinery to perform harvesting operations according to the harvesting operation path includes: The path generation module is used to define the mature areas in the monitored area that meet the harvesting requirements as harvesting operation areas, and to generate an optimal harvesting path that avoids the immature areas as the harvesting operation path based on the harvesting operation areas; and The sending module is used to send the harvesting operation path to the ground agricultural machinery so as to control the ground agricultural machinery to carry out the harvesting operation according to the harvesting operation path.

8. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 1, characterized in that, The crop data includes crop sparseness data, crop lodging data, and actual crop harvesting rate data per unit length of the harvesting operation path. The optimization unit includes: The first generation subunit is used to generate the first harvesting control parameters based on the crop sparseness data; The second generation subunit is used to generate the second harvesting control parameters based on the crop lodging data; The third generation subunit is used to generate second harvesting control parameters based on the actual crop harvesting rate data; and The integrated calculation subunit is used to optimize harvesting based on the first harvesting control parameter, the second harvesting control parameter, and the third harvesting control parameter to obtain the harvesting control parameters of the ground agricultural machinery. The calculation formula for the harvesting control parameters is as follows: ,in, Represents crop sparsity data. This represents data on crop dumping rate. This represents the actual harvest rate of crops. This represents the first transformation factor for crop sparsity data to harvest control parameters. This represents the second conversion factor for crop lodging data to harvest control parameters. This represents the third conversion factor for the actual crop harvest rate data to the harvest control parameters.

9. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 1, characterized in that, The crop data includes crop sparseness data, crop lodging data, and actual crop harvesting rate data per unit length of the harvesting operation path. The optimization unit further includes: The sparsity calculation subunit is used to identify missing crop points in the mature area per unit length of the harvesting path, obtain the area of ​​the missing points, and obtain the crop sparsity data based on the harvested area per unit length of the harvesting path and the area of ​​the missing points. The calculation formula for the crop sparsity data is as follows: , This represents the area of ​​each missing point. This indicates the harvested area per unit length of the harvesting path. The lodging degree calculation subunit is used to identify the lodging points of crops in the mature area per unit length of the harvesting operation path, obtain the area of ​​each lodging point, identify the degree of lodging at each lodging point, obtain lodging degree data, and obtain the crop lodging degree data based on the harvested area, the area of ​​each lodging point, and the lodging degree data per unit length of the harvesting operation path. The calculation formula for the crop lodging degree data is as follows: , This represents the area of ​​each dumping point. This represents the harvested area per unit length of the harvesting path. This represents the dumping degree data corresponding to each dumping point. This represents the first weight value corresponding to the area of ​​the dumping point. This represents the second weight value corresponding to the crop lodging data; and The harvest rate calculation subunit is used to compare crop images in the mature area per unit length of the harvesting operation path with historical harvest images. When a crop image reaches a set similarity, the actual crop harvest rate data is obtained based on the historical crop harvest rate corresponding to the historical harvest image. The calculation formula for the actual crop harvest rate data is as follows: , This represents the historical harvest rate of crops corresponding to historical harvest images. This represents the similarity value between historical harvest images and corresponding crop images in mature areas. This represents the similarity compensation coefficient.

10. The joint control system for unmanned aerial vehicles and ground agricultural machinery according to claim 1, characterized in that, Also includes: The harvesting statistics unit is used to take images of harvested crops in the mature area within a unit length of the harvesting operation path after harvesting as historical harvesting images, collect corresponding weight data, and calculate the historical harvesting rate of the crops in the historical harvesting images based on the baseline weight and the weight data. The formula for calculating the historical harvesting rate of crops is as follows: , Indicates weight data, Indicates the reference weight.

Images