Plant protection unmanned aerial vehicle spraying control method, device, equipment and medium

Abstract

The application relates to a plant protection unmanned aerial vehicle spraying control method, device, equipment and medium, the method comprising: acquiring pose transformation data of each trunk of a pose sensor in a fruit tree canopy in a preset time range, each flight parameter of a plant protection unmanned aerial vehicle and fruit tree canopy image data of a vision sensor in the plant protection unmanned aerial vehicle; determining a mist drop deposition amount in the fruit tree canopy under each flight parameter, constructing a plant protection unmanned aerial vehicle spraying control model according to the mist drop deposition amount and the canopy disturbance prediction model; inputting each flight parameter of the plant protection unmanned aerial vehicle into a pre-trained plant protection unmanned aerial vehicle spraying control model, determining a fruit tree canopy disturbance state corresponding to the flight parameter and a mist drop deposition amount under the fruit tree canopy disturbance state, so as to complete the control of the plant protection unmanned aerial vehicle spraying. The application can greatly avoid the great influence of the wind field generated by the unmanned aerial vehicle rotor on the deposition effect of the pesticide mist drop.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of plant protection operation, and in particular to a plant protection unmanned aerial vehicle spraying control method, a corresponding device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Under the trend of today's agricultural precision, unmanned aerial vehicle plant protection technology has a good development prospect in the field of domestic agricultural and forestry aviation plant protection due to its flexibility, universality, efficiency, environmental protection and many other advantages. Research has found that the wind field generated by the rotor of the unmanned aerial vehicle has a huge impact on the deposition effect of pesticide droplets, especially in complex fruit tree environment, the influence is more complex. When the rotor unmanned aerial vehicle is used for plant protection operation, the complex wind field generated by the flight of the rotor unmanned aerial vehicle will have a combined effect on the deformation of the crops. The influence of the related complex wind field on pesticide spraying is a non-negligible disturbance, which needs to consider factors such as the flight speed, flight attitude and height of the unmanned aerial vehicle.

[0003] At present, the droplet deposition amount prediction model is obtained by training sample unmanned aerial vehicle operation parameter data and sample environment information data using a convolutional neural network. This method is convenient for real-time remote detection of droplet deposition amount, but to some extent, it does not consider the combined effect of plant protection machine motion on the scene. The current part of the existing research combines the YOLO image detection model to evaluate the droplet deposition image, which cannot avoid the error caused by sensor imaging, nor can it avoid the influence caused by different environmental variables.

[0004] In summary, the droplet deposition amount prediction model in the prior art does not consider the combined effect of plant protection machine motion on the scene, and the evaluation of the droplet deposition image cannot avoid the error caused by sensor imaging, nor can it avoid the influence caused by different environmental variables. The present application makes corresponding exploration to solve this problem. SUMMARY

[0005] The present application aims to solve the above problems and provide a plant protection unmanned aerial vehicle spraying control method, a corresponding device, an electronic device and a computer readable storage medium.

[0006] To achieve the various purposes of the present application, the present application adopts the following technical solutions:

[0007] A plant protection unmanned aerial vehicle spraying control method is proposed to adapt to one of the purposes of the present application, comprising:

[0008] In response to the plant protection unmanned aerial vehicle spraying control instruction, the pose transformation data of each trunk of the pose sensor in the fruit tree canopy, the flight parameters of the plant protection unmanned aerial vehicle and the fruit tree canopy image data of the vision sensor in the plant protection unmanned aerial vehicle in a preset time range are obtained;

[0009] perform image segmentation on the fruit tree crown layer image data based on a preset disturbance region segmentation model, determine a fruit tree crown layer disturbance region corresponding to an air flow of the plant protection unmanned aerial vehicle under each flight parameter, match the pose transformation data with the fruit tree crown layer disturbance region, and determine a fruit tree crown layer disturbance region corresponding to the pose transformation data;

[0010] construct a crown layer disturbance prediction model based on the pose transformation data, each flight parameter, and the fruit tree crown layer disturbance region corresponding to the pose transformation data;

[0011] determine a mist drop deposition amount in the fruit tree crown layer under each flight parameter, and construct a plant protection unmanned aerial vehicle spraying control model according to the mist drop deposition amount and the crown layer disturbance prediction model;

[0012] input each flight parameter of the plant protection unmanned aerial vehicle into the pre-trained plant protection unmanned aerial vehicle spraying control model, determine a fruit tree crown layer disturbance state corresponding to the flight parameter and a mist drop deposition amount under the fruit tree crown layer disturbance state, and adjust the flight parameter of the plant protection unmanned aerial vehicle according to the mist drop deposition amount, so as to complete control of spraying of the plant protection unmanned aerial vehicle.

[0013] Optionally, after the steps of obtaining the pose transformation data of each trunk of the pose sensor in the fruit tree crown layer, each flight parameter of the plant protection unmanned aerial vehicle, and the fruit tree crown layer image data of the visual sensor in the plant protection unmanned aerial vehicle within a preset time range, the method further includes:

[0014] obtaining three-axis acceleration, three-axis angular velocity, and quaternion of the fruit tree crown layer trunk, and the pose transformation data including one or any multiple of the three-axis acceleration, the three-axis angular velocity, and the quaternion;

[0015] determining an instantaneous offset condition of the fruit tree crown layer trunk based on the three-axis acceleration and the quaternion, reflecting the instantaneous offset conditions of a plurality of the fruit tree crown layer trunks in the form of spatial composite transformation in a field model, and constructing a trunk instantaneous offset model;

[0016] constructing a crown layer disturbance prediction model based on the trunk instantaneous offset model, the flight parameter of the plant protection unmanned aerial vehicle, and the fruit tree crown layer disturbance region corresponding to the pose transformation data.

[0017] Optionally, after the step of determining a mist drop deposition amount in the fruit tree crown layer under each flight parameter, and constructing a plant protection unmanned aerial vehicle spraying control model according to the mist drop deposition amount and the crown layer disturbance prediction model, the method further includes:

[0018] The preset heuristic algorithm is used for multi-objective optimization of the plant protection unmanned aerial vehicle spraying control model, the target Pareto front is further obtained through iteration, and the plant protection unmanned aerial vehicle spraying control model is trained based on the target Pareto front, so as to complete the construction of the plant protection unmanned aerial vehicle spraying control model.

[0019] Optionally, the step of determining the mist drop deposition amount in the fruit tree canopy under each flight parameter, and constructing the plant protection unmanned aerial vehicle spraying control model according to the mist drop deposition amount and the canopy disturbance prediction model comprises:

[0020] The plant protection unmanned aerial vehicle spraying control model is:

[0021]

[0022]

[0023] Wherein, V is the visual disturbance amount returned by the visual sensor of the plant protection unmanned aerial vehicle, F a , F S is a specific convolutional neural network, ω is a problem weight parameter, a is a flight parameter weight amount, s is a pose sensor returned data weight amount, and b is a regression offset factor.

[0024] Optionally, the step of training the canopy disturbance prediction model comprises:

[0025] The optical flow method is used for frame-by-frame processing of the fruit tree canopy disturbance area, and the pose transformation data of each tree trunk corresponding to the fruit tree canopy disturbance area is determined.

[0026] A training set is constructed based on each flight parameter of the plant protection unmanned aerial vehicle, the pose transformation data of each tree trunk, and the fruit tree canopy disturbance area corresponding to the pose transformation data of each tree trunk.

[0027] The training set is input into the preset convolutional neural network for training, the weight of the canopy disturbance prediction model is updated when the precision rate does not reach the preset threshold, and other training sets are continuously called for iterative training until the model converges.

[0028] Optionally, the step of determining the fruit tree canopy disturbance area corresponding to the air flow of the plant protection unmanned aerial vehicle under each flight parameter based on the preset disturbance area segmentation model comprises:

[0029] The fruit tree canopy image data is input into the preset disturbance area segmentation model, and the first image feature of the fruit tree canopy image data is extracted by using the feature down-sampling module of the disturbance area segmentation model.

[0030] input the first image feature into a feature extraction module of the fruit tree canopy image data to obtain feature vectors with different receptive fields in the original image, input the feature vectors into a feature upsampling module of the disturbance region segmentation model for processing, restore a feature image consistent with the resolution of the input original image according to the feature image, and obtain a probability image of the fruit tree canopy image data according to the feature image;

[0031] segment the fruit tree canopy image data based on the probability image, and determine the fruit tree canopy disturbance region corresponding to the air flow of the plant protection unmanned aerial vehicle under each flight parameter.

[0032] Optionally, after the step of inputting each flight parameter of the plant protection unmanned aerial vehicle into a pre-trained plant protection unmanned aerial vehicle spraying control model to determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of droplet deposition under the fruit tree canopy disturbance state, the flight parameter includes one or any combination of flight direction, flight speed and flight height.

[0033] determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of droplet deposition under the fruit tree canopy disturbance state.

[0034] detect whether the amount of droplet deposition under the fruit tree canopy disturbance state in a preset spraying time range reaches a preset deposition threshold, and if not, adjust the flight parameter of the plant protection unmanned aerial vehicle to complete the control of the plant protection unmanned aerial vehicle spraying.

[0035] To achieve another purpose of the present application, a plant protection unmanned aerial vehicle spraying control device is provided, comprising:

[0036] a data acquisition module configured to acquire, in response to a plant protection unmanned aerial vehicle spraying control instruction, pose transformation data of each trunk of a pose sensor in a fruit tree canopy, each flight parameter of a plant protection unmanned aerial vehicle and fruit tree canopy image data of a vision sensor in the plant protection unmanned aerial vehicle within a preset time range;

[0037] a data matching module configured to perform image segmentation on the fruit tree canopy image data based on a preset disturbance region segmentation model, determine the fruit tree canopy disturbance region corresponding to the air flow of the plant protection unmanned aerial vehicle under each flight parameter, and match the pose transformation data with the fruit tree canopy disturbance region to determine the fruit tree canopy disturbance region corresponding to the pose transformation data.

[0038] a disturbance model construction module configured to construct a canopy disturbance prediction model based on the pose transformation data, each flight parameter and the fruit tree canopy disturbance region corresponding to the pose transformation data.

[0039] The spraying control model module is configured to determine the amount of mist deposition in the fruit tree canopy under each flight parameter, and to construct a plant protection unmanned aerial vehicle spraying control model according to the amount of mist deposition and the canopy disturbance prediction model.

[0040] The plant protection unmanned aerial vehicle spraying control module is configured to input each flight parameter of the plant protection unmanned aerial vehicle into the pre-trained plant protection unmanned aerial vehicle spraying control model, to determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of mist deposition under the fruit tree canopy disturbance state, so as to complete the control of the plant protection unmanned aerial vehicle spraying.

[0041] Another object of the present application is to provide an electronic device comprising a central processing unit and a memory, wherein the central processing unit is configured to invoke a computer program stored in the memory to execute the steps of the plant protection unmanned aerial vehicle spraying control method.

[0042] Another object of the present application is to provide a computer readable storage medium storing a computer program implemented according to the plant protection unmanned aerial vehicle spraying control method in the form of computer readable instructions, wherein the computer program is invoked and run by a computer to execute the steps included in the corresponding method.

[0043] Compared with the prior art, the present application does not consider the combined effect of the movement of the plant protection machine on the scene, and cannot avoid the errors caused by sensor imaging and the effects of different environmental variables.

[0044] Firstly, the plant protection unmanned aerial vehicle spraying control method of the present application designs a canopy disturbance prediction model, which can accurately obtain the crop disturbance synchronization corresponding to different flight parameters of the rotor unmanned aerial vehicle, and can be widely applied in multiple scenes.

[0045] Secondly, the plant protection unmanned aerial vehicle spraying control method of the present application considers the interaction effect of the rotor unmanned aerial vehicle wind field and the sprayed mist group with the fruit tree canopy, which not only optimizes the robustness of the evaluation model, but also avoids falling into local optimal decision through parameter optimization to achieve a better fitting strategy.

[0046] Thirdly, the plant protection unmanned aerial vehicle spraying control method of the present application can greatly avoid the huge influence of the wind field generated by the rotor of the unmanned aerial vehicle on the deposition effect of the pesticide mist, especially in complex fruit tree environments, the influence relationship is more complex. BRIEF DESCRIPTION OF DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0048] Figure 1 An exemplary architecture employed by the plant protection unmanned aerial vehicle spraying control system of the present application;

[0049] Figure 2 A flowchart of the plant protection unmanned aerial vehicle spraying control method in the embodiments of the present application;

[0050] Figure 3 A schematic diagram of the fruit tree canopy image returned by the vision sensor mounted on the plant protection unmanned aerial vehicle in the embodiments of the present application;

[0051] Figure 4 A schematic diagram of the multi-layer heterogeneous mounting array in the embodiments of the present application;

[0052] Figure 5 A schematic diagram of the fruit tree-aircraft relative spatial reference frame in the embodiments of the present application;

[0053] Figure 6 A principle block diagram of the plant protection unmanned aerial vehicle spraying control device in the embodiments of the present application;

[0054] Figure 7 A structural schematic diagram of the computer device in the embodiments of the present application. DETAILED DESCRIPTION

[0055] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which examples of the embodiments are shown, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as a limitation on the present application.

[0056] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the phrase "comprising" used in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0057] As those skilled in the art will appreciate, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further appreciated that terms, such as those defined in commonly used dictionaries, should be given their ordinary and customary meaning, unless explicitly defined otherwise herein and unless the context clearly dictates otherwise.

[0058] As those skilled in the art will appreciate, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further appreciated that terms, such as those defined in commonly used dictionaries, should be given their ordinary and customary meaning, unless explicitly defined otherwise herein and unless the context clearly dictates otherwise. As those skilled in the art will appreciate, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further appreciated that terms, such as those defined in commonly used dictionaries, should be given their ordinary and customary meaning, unless explicitly defined otherwise herein and unless the context clearly dictates otherwise. As those skilled in the art will appreciate, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further appreciated that terms, such as those defined in commonly used dictionaries, should be given their ordinary and customary meaning, unless explicitly defined otherwise herein and unless the context clearly dictates otherwise.

[0059] The hardware referred to by the names "server", "client", "service node" and the like in the present application is essentially an electronic device with the equivalent capabilities of a personal computer, and is a hardware device with the necessary components disclosed by the von Neumann principle, including a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in the memory, the central processing unit calls the program stored in the external memory into the memory for running, executes the instructions in the program, and interacts with the input and output devices to complete a specific function.

[0060] It should be noted that the concept of "server" in the present application can also be extended to the case of a server cluster. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided, and in physical space, these servers can be independent of each other but can be called through an interface, or can be integrated into a physical computer or a computer cluster. Those skilled in the art should understand this variation and should not be restricted by the implementation of the network deployment of the present application.

[0061] One or more technical features of the present application, unless explicitly specified, can be deployed on a server for implementation and accessed by a client remotely calling an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0062] The neural network model referred to or possibly referred to in the present application, unless explicitly specified, can be deployed on a remote server and remotely called by a client, or can be deployed on a client with sufficient device capability for direct calling. In some embodiments, when it is run on a client, its corresponding intelligence can be obtained through transfer learning to reduce the requirement for client hardware running resources and avoid excessive occupation of client hardware running resources.

[0063] The various data involved in the present application, unless explicitly specified, can be stored remotely on a server or stored locally on a terminal device, as long as it is suitable for being called by the technical solutions of the present application.

[0064] Those skilled in the art should know that the various methods of the present application, although based on the same concept and described to present commonality among them, are independently executable unless otherwise specified. Similarly, for each embodiment disclosed in the present application, it is based on the same inventive concept, and therefore, for the same concept of expression, and although the concept of expression is different, it is only for the convenience of appropriately transforming the concept, and should be understood as equivalent.

[0065] Unless otherwise indicated herein, the various disclosed embodiments can be combined in any and all permutations. It is intended that the following claims be construed to include all such embodiments.

[0066] Please refer to Figure 1 The plant protection unmanned aerial vehicle spraying control method of the present application can be implemented based on a plant protection unmanned aerial vehicle spraying control system, which mainly consists of a fruit tree disturbance sensing module 102, an unmanned aerial vehicle vision recovery processing module 101, and a comprehensive model processing module 103.

[0067] The unmanned aerial vehicle vision recovery processing module 101 mainly performs recovery processing on the fruit tree crown layer image data returned by the vision sensor mounted on the plant protection unmanned aerial vehicle. By combining the influence of the unmanned aerial vehicle downwash airflow on the fruit tree crown layer under different flight parameters, the disturbance feature of the fruit tree crown layer image data is extracted, and the parameter correlation and feature matching are performed in combination with deep learning.

[0068] The fruit tree disturbance sensing module 102 mainly applies attitude sensors and a number of sensing devices reflecting the motion of particles. The sensor array is arranged at different positions of the branches and trunks, and the vibration and offset of the branches and trunks are synchronously obtained. For the establishment of the instantaneous offset model, the obtained data is used for mathematical modeling.

[0069] The fruit tree disturbance sensing module 102 designs a multi-layer heterogeneous arrangement scheme for the fruit tree crown layer. This scheme uses attitude sensors in combination with the spatial distribution characteristics of the crown layer at different positions of the fruit tree. The sensors are arranged in a scattered, grid-like, and other ways. The appropriate transmission frequency band and synchronization rate are set. Based on the polling transmission mode, the target disturbance data is captured in real time and recorded.

[0070] Based on the above exemplary scenarios, please refer to Figure 2 The plant protection unmanned aerial vehicle spraying control method of the present application, in one embodiment, comprises:

[0071] Step S10, in response to the plant protection unmanned aerial vehicle spraying control instruction, obtaining the pose transformation data of each trunk of the pose sensor in the fruit tree crown layer, the flight parameters of the plant protection unmanned aerial vehicle, and the fruit tree crown layer image data of the vision sensor in the plant protection unmanned aerial vehicle within a preset time range;

[0072] The terminal device in the plant protection unmanned aerial vehicle can respond to the plant protection unmanned aerial vehicle spraying control instruction, obtain the pose transformation data of each trunk of the pose sensor in the fruit tree canopy within a preset time range, each flight parameter of the plant protection unmanned aerial vehicle, and the fruit tree canopy image data of the visual sensor in the plant protection unmanned aerial vehicle;

[0073] Specifically, when the plant protection unmanned aerial vehicle performs plant protection operation, the generated downwash airflow and other complex wind field disturb the fruit tree canopy, and the drift of the mist droplets is also affected by the wind field and each flight parameter of the plant protection unmanned aerial vehicle. The visual sensor mounted on the plant protection unmanned aerial vehicle can obtain the fruit tree canopy image data, the attitude sensor can obtain the pose transformation data of each trunk in the fruit tree canopy; the pose transformation data includes one or any number of items of three-axis acceleration, three-axis angular velocity and quaternion, and the flight parameters include one or any number of items of flight direction, flight speed and flight height

[0074] In some embodiments, the disturbed condition of each branch of the fruit tree canopy is mainly recorded by the attitude sensor in a polling mode, and a basic model is constructed by the recorded pose transformation data such as three-axis acceleration, three-axis angular velocity and quaternion.

[0075] Please refer to Figure 3 The fruit tree canopy image returned by the visual sensor mounted on the plant protection unmanned aerial vehicle is shown in the schematic diagram, such as the still state reference image 201 and the disturbed state feature frame interception 202, and different disturbance conditions correspond to related feature points of the canopy disturbance prediction model.

[0076] Please refer to Figure 4 In the layout of the sensing device, a multi-layer heterogeneous mounting array is designed, which is mainly designed based on the complex canopy of the fruit tree, and can reflect the multi-level synchronous feedback of the visual sensor, the attitude sensor and the deposition sensing device, and can also capture and process the detailed non-visual data at the bottom. This scheme mainly block-arranges the top canopy and scatter-arranges the bottom several canopies.

[0077] In some embodiments, please refer to Figure 5 A fruit tree-aircraft relative space reference system capture scheme is designed, and part of the flight parameters is described. The aerial vehicle involved in this scheme is arranged on the side and above the plant protection unmanned aerial vehicle, and keeps relative stillness with the fruit tree in the state of being able to completely face and look down at the fruit tree.

[0078] After the step of obtaining the pose transformation data of each trunk of the pose sensor in the fruit tree canopy within a preset time range, each flight parameter of the plant protection unmanned aerial vehicle, and the fruit tree canopy image data of the visual sensor in the plant protection unmanned aerial vehicle, the method further includes:

[0079] Step S101, acquiring three-axis acceleration, three-axis angular velocity and quaternion in the crown layer of the fruit tree, the pose transformation data including one or any multiple of the three-axis acceleration, three-axis angular velocity and quaternion;

[0080] Step S103, determining the instantaneous offset of the crown layer of the fruit tree based on the three-axis acceleration and the quaternion, reflecting the instantaneous offset of the crown layer of the fruit tree in the form of spatial composite transformation in the field model to construct a trunk instantaneous offset model;

[0081] Step S105, constructing a crown layer disturbance prediction model based on the trunk instantaneous offset model, the flight parameter of the plant protection unmanned aerial vehicle and the corresponding fruit tree crown layer disturbance region of the pose transformation data.

[0082] Step S20, performing image segmentation on the fruit tree crown layer image data based on a preset disturbance region segmentation model to determine the fruit tree crown layer disturbance region corresponding to the airflow of the plant protection unmanned aerial vehicle under each flight parameter, matching the pose transformation data with the fruit tree crown layer disturbance region to determine the fruit tree crown layer disturbance region corresponding to the pose transformation data;

[0083] After acquiring the fruit tree crown layer image data of the vision sensor in the plant protection unmanned aerial vehicle, performing image segmentation on the fruit tree crown layer image data based on a preset disturbance region segmentation model to determine the fruit tree crown layer disturbance region corresponding to the airflow of the plant protection unmanned aerial vehicle under each flight parameter, the basic network architecture of the disturbance region segmentation model can be a U-net model, etc., inputting the fruit tree crown layer image data into the preset disturbance region segmentation model, extracting the first image feature of the fruit tree crown layer image data by using the feature down-sampling module of the disturbance region segmentation model; inputting the first image feature into the feature extraction module of the fruit tree crown layer image data to obtain the feature vector with different receptive fields in the original image, inputting the feature vector into the feature up-sampling module of the disturbance region segmentation model for processing, restoring the feature image consistent with the resolution of the input original image, and obtaining the probability image of the fruit tree crown layer image data according to the feature image; segmenting the fruit tree crown layer image data based on the probability image to determine the fruit tree crown layer disturbance region corresponding to the airflow of the plant protection unmanned aerial vehicle under each flight parameter. Adopting the optical flow method to compare the fruit tree crown layer disturbance region with the still state reference image, matching the pose transformation data of each trunk of the pose sensor in the fruit tree crown layer with the fruit tree crown layer disturbance region to determine the fruit tree crown layer disturbance region corresponding to the pose transformation data.

[0084] Step S30, constructing a crown layer disturbance prediction model based on the pose transformation data, each flight parameter and the fruit tree crown layer disturbance region corresponding to the pose transformation data;

[0085] The pose transformation data of each tree trunk of the pose sensor in the fruit tree canopy is matched with the fruit tree canopy disturbance area, the fruit tree canopy disturbance area corresponding to the pose transformation data is determined, and a canopy disturbance prediction model is constructed based on the pose transformation data, each flight parameter, and the fruit tree canopy disturbance area corresponding to the pose transformation data.

[0086] The step of training the canopy disturbance prediction model comprises:

[0087] In step S201, the fruit tree canopy disturbance area is processed frame by frame using an optical flow method, and the fruit tree canopy disturbance area corresponding to the pose transformation data of each tree trunk is determined.

[0088] In step S203, a training set is constructed based on each flight parameter of the plant protection unmanned aerial vehicle, the pose transformation data of each tree trunk, and the fruit tree canopy disturbance area corresponding to the pose transformation data of each tree trunk.

[0089] In step S205, the training set is input to a preset convolutional neural network for training. When the precision rate does not reach a preset threshold, the weight of the canopy disturbance prediction model is updated, and other training sets are continuously called for iterative training until the model converges.

[0090] The multi-dimensional convolutional neural network comprises a plurality of layers of convolutional neural networks, a pooling layer, a fully connected layer, and an output layer, and adopts a RELU activation function for deposition condition prediction.

[0091] In step S40, the amount of fog droplet deposition in the fruit tree canopy under each flight parameter is determined, and a plant protection unmanned aerial vehicle spraying control model is constructed based on the amount of fog droplet deposition and the canopy disturbance prediction model.

[0092] After the canopy disturbance prediction model is constructed based on the pose transformation data, each flight parameter, and the fruit tree canopy disturbance area corresponding to the pose transformation data, the amount of fog droplet deposition in the fruit tree canopy under each flight parameter is determined, and a plant protection unmanned aerial vehicle spraying control model is constructed based on the amount of fog droplet deposition and the canopy disturbance prediction model.

[0093] In some embodiments, after the step of determining the amount of fog droplet deposition in the fruit tree canopy under each flight parameter and constructing the plant protection unmanned aerial vehicle spraying control model based on the amount of fog droplet deposition and the canopy disturbance prediction model, the step comprises:

[0094] A preset heuristic algorithm is used to perform multi-objective optimization on the plant protection unmanned aerial vehicle spraying control model, and a target Pareto front is further obtained through iteration. The plant protection unmanned aerial vehicle spraying control model is trained based on the target Pareto front to complete the construction of the plant protection unmanned aerial vehicle spraying control model.

[0095] In some embodiments, the step of determining the amount of droplet deposition in the fruit tree canopy under each flight parameter, and constructing a plant protection UAV spraying control model according to the amount of droplet deposition and the canopy disturbance prediction model comprises:

[0096] The plant protection UAV spraying control model is:

[0097]

[0098]

[0099] wherein V is the visual disturbance amount returned by the visual sensor of the plant protection UAV, F a , F S is a specific convolutional neural network, ω is a problem weight parameter, a is a flight parameter weight amount, s is a pose sensor returned data weight amount, and b is a regression offset factor.

[0100] Step S50, input each flight parameter of the plant protection UAV into the pre-trained plant protection UAV spraying control model, determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of droplet deposition under the fruit tree canopy disturbance state, and adjust the flight parameter of the plant protection UAV according to the amount of droplet deposition to complete the control of the plant protection UAV spraying.

[0101] After constructing the plant protection UAV spraying control model according to the amount of droplet deposition and the canopy disturbance prediction model, input each flight parameter of the plant protection UAV into the pre-trained plant protection UAV spraying control model, determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of droplet deposition under the fruit tree canopy disturbance state, and adjust the flight parameter of the plant protection UAV according to the amount of droplet deposition to complete the control of the plant protection UAV spraying.

[0102] In some embodiments, after the step of inputting each flight parameter of the plant protection UAV into the pre-trained plant protection UAV spraying control model to determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of droplet deposition under the fruit tree canopy disturbance state, the step comprises:

[0103] Step S501, determine the fruit tree canopy disturbance state corresponding to the flight parameter and the amount of droplet deposition under the fruit tree canopy disturbance state.

[0104] Step S503, detect whether the amount of droplet deposition under the fruit tree canopy disturbance state in the preset spraying time range reaches the preset deposition threshold, and if not, adjust the flight parameter of the plant protection UAV to complete the control of the plant protection UAV spraying.

[0105] It is not difficult to understand that the mist drop deposition amounts of different fruit trees are different, when the plant protection unmanned aerial vehicle sprays the fruit tree crown layer, whether the mist drop deposition amount of the fruit tree crown layer in the preset spraying time range under the disturbance state reaches the preset deposition threshold value is detected, if not, the corresponding deposition threshold value of different fruit trees can be set based on the mist drop deposition amount of different fruit trees, the corresponding deposition threshold value of different fruit trees is input into the plant protection unmanned aerial vehicle spraying control model, the flight direction, flight speed and flight height of the plant protection unmanned aerial vehicle and the disturbed area of the fruit tree crown layer are determined, the influence of the wind field generated by the unmanned aerial vehicle rotor on the deposition effect of the pesticide mist drop can be greatly avoided, especially in the complex fruit tree environment, the influence relationship is more complex, when the rotor unmanned aerial vehicle is used for plant protection, the complex wind field generated by the rotor unmanned aerial vehicle flight will have a combined effect on the crop deformation, so that the mist drop sprayed by the plant protection unmanned aerial vehicle can be accurately, effectively and uniformly sprayed in the crown of various fruit trees.

[0106] From the above embodiment, compared with the prior art, the present application does not consider the combined effect of the plant protection machine motion on the scene, and cannot avoid the error caused by the sensor imaging and the influence caused by different environmental variables. The present application includes but is not limited to the following beneficial effects:

[0107] Firstly, the plant protection unmanned aerial vehicle spraying control method of the present application designs a crown layer disturbance prediction model for specific operation environment, which can accurately obtain the crop disturbance synchronization corresponding to different flight parameters of the rotor unmanned aerial vehicle, and can be widely applied in multiple scenes;

[0108] Secondly, the plant protection unmanned aerial vehicle spraying control method of the present application considers the interaction effect of the rotor unmanned aerial vehicle wind field and the sprayed mist drop group with the fruit tree crown layer, which not only optimizes the robustness of the evaluation model, but also avoids falling into local optimal decision through parameter optimization to realize better fitting strategy;

[0109] Thirdly, the plant protection unmanned aerial vehicle spraying control method of the present application can greatly avoid the great influence of the wind field generated by the unmanned aerial vehicle rotor on the deposition effect of the pesticide mist drop, especially in the complex fruit tree environment, the influence relationship is more complex, when the rotor unmanned aerial vehicle is used for plant protection, the complex wind field generated by the rotor unmanned aerial vehicle flight will have a combined effect on the crop deformation, so that the mist drop sprayed by the plant protection unmanned aerial vehicle can be accurately, effectively and uniformly sprayed in the crown of various fruit trees.

[0110] Please refer to Figure 6, provided by one of the purposes of the application, a plant protection unmanned aerial vehicle spraying control device, comprising. Among them, the data acquisition module 1100 is set to respond to the plant protection unmanned aerial vehicle spraying control instruction, and the pose transformation data of each trunk of the pose sensor in the fruit tree canopy within the preset time range, each flight parameter of the plant protection unmanned aerial vehicle and the fruit tree canopy image data of the visual sensor in the plant protection unmanned aerial vehicle are acquired; the data matching module 1200 is set to image segmentation based on the preset disturbance region segmentation model to determine the fruit tree canopy disturbance region corresponding to the airflow of the plant protection unmanned aerial vehicle under each flight parameter, and the pose transformation data is matched with the fruit tree canopy disturbance region to determine the fruit tree canopy disturbance region corresponding to the pose transformation data; the disturbance model construction module 1300 is set to construct the canopy disturbance prediction model based on the pose transformation data, each flight parameter and the fruit tree canopy disturbance region corresponding to the pose transformation data; the spraying control model module 1400 is set to determine the mist drop deposition amount in the fruit tree canopy under each flight parameter, and the plant protection unmanned aerial vehicle spraying control model is constructed according to the mist drop deposition amount and the canopy disturbance prediction model; the plant protection unmanned aerial vehicle spraying control module 1500 is set to input each flight parameter of the plant protection unmanned aerial vehicle into the pre-trained plant protection unmanned aerial vehicle spraying control model, determine the fruit tree canopy disturbance state corresponding to the flight parameter and the mist drop deposition amount under the fruit tree canopy disturbance state, so as to complete the control of the plant protection unmanned aerial vehicle spraying.

[0111] On the basis of any embodiment of the present application, please refer to Figure 7 Another embodiment of the present application also provides an electronic device, which can be realized by a computer device, as shown in Figure 7 The internal structure diagram of the computer device. The computer device includes a processor, a computer readable storage medium, a memory and a network interface connected by a system bus. Among them, the computer readable storage medium of the computer device stores an operating system, a database and a computer readable instruction, the database can store control information sequence, and the computer readable instruction is executed by the processor, so that the processor realizes a plant protection unmanned aerial vehicle spraying control method. The processor of the computer device is used to provide computing and control ability to support the operation of the whole computer device. The memory of the computer device can store computer readable instructions, which are executed by the processor to make the processor execute the plant protection unmanned aerial vehicle spraying control method of the present application. The network interface of the computer device is used to connect with the terminal for communication. Those skilled in the art can understand that Figure 7The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] The processor in the embodiment is configured to execute the specific functions of each module and the sub-modules thereof in the above embodiment, and the memory stores the program codes and various data required for executing the above modules or sub-modules. The network interface is configured to transmit data between the user terminal or the server. The memory in the embodiment stores the program codes and data required for executing all modules / sub-modules in the plant protection unmanned aerial vehicle spraying control device of the present application, and the server can call the program codes and data of the server to execute the functions of all sub-modules. Figure 6

[0113] The present application also provides a storage medium storing computer readable instructions, which are executed by one or more processors to cause the one or more processors to perform the steps of the plant protection unmanned aerial vehicle spraying control method described in any embodiment of the present application.

[0114] The present application also provides a computer program product, which includes computer programs / instructions, and the computer programs / instructions are executed by one or more processors to implement the steps of the plant protection unmanned aerial vehicle spraying control method described in any embodiment of the present application.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of each method. The storage medium can be a computer readable storage medium such as a magnetic disc, an optical disc, a read-only memory (ROM), or a random access memory (RAM).

[0116] The above-mentioned is only some embodiments of the present application. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be regarded as the protection scope of the present application.

[0117] ​In summary, the plant protection unmanned aerial vehicle spraying control method can greatly avoid the great influence of the wind field generated by the unmanned aerial vehicle rotor on the deposition effect of the pesticide droplets, especially in the complex fruit tree environment, the influence relationship is more complex, and when the rotor unmanned aerial vehicle is used for plant protection, the complex wind field generated by the rotor unmanned aerial vehicle flight can have a combined effect on the crop deformation, so that the mist sprayed by the plant protection unmanned aerial vehicle can be accurately, effectively and uniformly sprayed in the crown of various fruit trees.

Claims

1. A plant protection unmanned aerial vehicle spraying control method for spraying fruit trees, characterized in that, The method comprises the steps of: in response to the plant protection unmanned aerial vehicle spraying control instruction, acquiring the pose transformation data of each trunk of the pose sensor in the fruit tree canopy within a preset time range, the flight parameters of the plant protection unmanned aerial vehicle, and the fruit tree canopy image data of the visual sensor in the plant protection unmanned aerial vehicle; wherein the pose transformation data comprises one or any number of items of three-axis acceleration, three-axis angular velocity, and quaternion; based on a preset disturbance region segmentation model, image segmentation is performed on the fruit tree canopy image data to determine the fruit tree canopy disturbance region corresponding to the airflow of the plant protection unmanned aerial vehicle under each flight parameter, and the pose transformation data is matched with the fruit tree canopy disturbance region to determine the fruit tree canopy disturbance region corresponding to the pose transformation data; based on the three-axis acceleration and the quaternion, the instantaneous offset condition of the fruit tree canopy trunk is determined, the instantaneous offset conditions of a plurality of fruit tree canopy trunks are reflected in the form of spatial composite transformation in the field model to construct a trunk instantaneous offset model, and based on the trunk instantaneous offset model, the flight parameters of the plant protection unmanned aerial vehicle, and the fruit tree canopy disturbance region corresponding to the pose transformation data, a canopy disturbance prediction model is constructed; determining the mist drop deposition amount in the fruit tree canopy under each flight parameter, and constructing a plant protection unmanned aerial vehicle spraying control model according to the mist drop deposition amount and the canopy disturbance prediction model, wherein the plant protection unmanned aerial vehicle spraying control model is: ， ， Wherein, V is the visual disturbance returned by the visual sensor of the plant protection unmanned aerial vehicle, F a 、 F S is a specific convolutional neural network, ω is a problem weight parameter, a is a flight parameter weight quantity, s is a pose sensor returned data weight quantity, and b is a regression offset factor. inputting the flight parameters of the plant protection unmanned aerial vehicle into the pre-trained plant protection unmanned aerial vehicle spraying control model to determine the fruit tree canopy disturbance state corresponding to the flight parameters and the mist drop deposition amount under the fruit tree canopy disturbance state, so as to complete the control of the plant protection unmanned aerial vehicle spraying. 2.The plant protection unmanned aerial vehicle spraying control method according to claim 1, characterized in that, After the step of determining the mist drop deposition amount in the fruit tree canopy under each flight parameter and constructing a plant protection unmanned aerial vehicle spraying control model according to the mist drop deposition amount and the canopy disturbance prediction model, the method comprises the steps of: using a preset heuristic algorithm to perform multi-objective optimization on the plant protection unmanned aerial vehicle spraying control model, further obtaining a target Pareto frontier through iteration, training the plant protection unmanned aerial vehicle spraying control model based on the target Pareto frontier, and completing the construction of the plant protection unmanned aerial vehicle spraying control model. 3.The plant-protection unmanned aerial vehicle spraying control method according to claim 1, characterized in that, The step of training the canopy disturbance prediction model comprises the steps of: using an optical flow method to perform frame-by-frame processing on the fruit tree canopy disturbance region to determine the fruit tree canopy disturbance region corresponding to the pose transformation data of each trunk; constructing a training set based on the flight parameters of the plant protection unmanned aerial vehicle, the pose transformation data of each trunk, and the fruit tree canopy disturbance region corresponding to the pose transformation data of each trunk; inputting the training set into a preset convolutional neural network for training, updating the weights of the canopy disturbance prediction model when the precision rate does not reach a preset threshold, and continuing to call other training sets for iterative training until the model converges. 4.The plant-protection unmanned aerial vehicle spraying control method according to claim 1, wherein, The step of performing image segmentation on the fruit tree canopy image data based on a preset disturbance region segmentation model to determine the fruit tree canopy disturbance region corresponding to the airflow of the plant protection unmanned aerial vehicle under each flight parameter comprises the steps of: The fruit tree crown layer image data is input into a preset disturbance area segmentation model, and a feature down-sampling module of the disturbance area segmentation model is used to extract first image features of the fruit tree crown layer image data; The first image features are input into a feature extraction module of the fruit tree crown layer image data to obtain feature vectors with different receptive fields in the original image, the feature vectors are input into a feature up-sampling module of the disturbance area segmentation model for processing, and a feature image consistent with the resolution of the input original image is restored, and a probability image of the fruit tree crown layer image data is obtained according to the feature image; The fruit tree crown layer image data is segmented based on the probability image to determine the fruit tree crown layer disturbance area corresponding to the air flow of the plant protection unmanned aerial vehicle under each flight parameter. 5.The method of claim 1 or 2 or 3 or 4, wherein, After the steps of inputting each flight parameter of the plant protection unmanned aerial vehicle into a pre-trained plant protection unmanned aerial vehicle spraying control model to determine the fruit tree crown layer disturbance state corresponding to the flight parameter and the amount of mist deposition under the fruit tree crown layer disturbance state, the method comprises: detecting whether the amount of mist deposition under the fruit tree crown layer disturbance state in a preset spraying time range reaches a preset deposition threshold, and if not, adjusting the flight parameter of the plant protection unmanned aerial vehicle to complete the control of the plant protection unmanned aerial vehicle spraying, wherein the flight parameter includes one or any combination of flight direction, flight speed and flight height.

6. An agricultural unmanned aerial vehicle spraying control device, characterized in that, The method comprises: a data acquisition module configured to acquire, in response to a plant protection unmanned aerial vehicle spraying control instruction, pose transformation data of each trunk of a pose sensor in a fruit tree crown layer, each flight parameter of a plant protection unmanned aerial vehicle and fruit tree crown layer image data of a vision sensor in the plant protection unmanned aerial vehicle within a preset time range; wherein the pose transformation data includes one or any combination of three-axis acceleration, three-axis angular velocity and quaternion; a data matching module configured to perform image segmentation on the fruit tree crown layer image data based on a preset disturbance area segmentation model to determine the fruit tree crown layer disturbance area corresponding to the air flow of the plant protection unmanned aerial vehicle under each flight parameter, and match the pose transformation data with the fruit tree crown layer disturbance area to determine the fruit tree crown layer disturbance area corresponding to the pose transformation data; a disturbance model construction module configured to determine the instantaneous offset of the fruit tree crown layer trunk based on the three-axis acceleration and the quaternion, reflect the instantaneous offset of a plurality of fruit tree crown layer trunks in the form of spatial composite transformation in a field model to construct a trunk instantaneous offset model, and construct a crown layer disturbance prediction model based on the trunk instantaneous offset model, each flight parameter of the plant protection unmanned aerial vehicle and the fruit tree crown layer disturbance area corresponding to the pose transformation data; a spraying control model module configured to determine the amount of mist deposition in the fruit tree crown layer under each flight parameter, and construct a plant protection unmanned aerial vehicle spraying control model according to the amount of mist deposition and the crown layer disturbance prediction model, wherein the plant protection unmanned aerial vehicle spraying control model is ， ， Wherein, V is the visual disturbance returned by the visual sensor of the plant protection unmanned aerial vehicle, F a 、 F S is a specific convolutional neural network, ω is a problem weight parameter, a is a flight parameter weight quantity, s is a pose sensor returned data weight quantity, and b is a regression offset factor. The plant protection unmanned plane spraying control module is configured to input each flight parameter of the plant protection unmanned plane into a pre-trained plant protection unmanned plane spraying control model, determine a fruit tree canopy disturbance state corresponding to the flight parameter and a mist drop deposition amount under the fruit tree canopy disturbance state, so as to complete control of the plant protection unmanned plane spraying.

7. An electronic device comprising a central processing unit and a memory, characterized in that The central processing unit is configured to call and run a computer program stored in the memory to perform the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer program is stored in the form of computer readable instructions and is implemented according to the method of any one of claims 1 to 5, and when the computer program is called and run by a computer, the steps included in the corresponding method are performed.

Images