Seed production corn missing tassel unmanned aerial vehicle detection and air-ground cooperative guidance system and method

Abstract

The application provides a kind of seed corn missing tassel unmanned aerial vehicle detection and air-ground cooperative guidance system and method, it is related to intelligent agriculture and unmanned aerial vehicle remote sensing detection, for guiding ground personnel in real time in flight process.The method comprises: obtaining the image data of target area and the position and attitude data of airborne flight equipment. Image data is subjected to target detection processing, and the pixel coordinate data of missing target object in image data is determined. According to the pixel coordinate data, the reference datum data of image data and the position and attitude data, the actual geographic coordinate data of missing target object is determined and sent. The actual geographic coordinate data is received and the current geographic position data of ground operation object is obtained. According to the actual geographic coordinate data and the current geographic position data, the navigation guidance path data for missing target object is generated.

Description

Technical Field

[0001] This invention relates to smart agriculture and drone remote sensing detection, and more particularly to a drone detection and air-ground collaborative guidance system and method for missing tassels in seed corn. Background Technology

[0002] In hybrid maize seed production, timely removal of the tassels from the female parent plant (i.e., detasseling) is a crucial step in ensuring seed purity. The detasseling period is typically only 7 to 15 days, a short window and a demanding task. Missing tassels can lead to self-pollination of the female parent, severely impacting hybrid seed quality. Therefore, quickly and accurately identifying and locating missing tassels is essential for guiding manual or mechanical detasseling.

[0003] Currently, remote sensing drones are typically used for aerial photography of farmland. After landing, the images are copied to a ground server for processing such as image stitching and cropping. Then, deep learning models are used to batch identify the cropped images, and the coordinates of the identified tassel images are converted into geographic coordinates. However, this process is time-consuming, resulting in a significant time delay between data acquisition and obtaining location information. Field workers still need to spend a considerable amount of time searching for targets in the farmland after the drone inspection is completed. Therefore, how to guide ground personnel in real time during flight has become a pressing technical problem to be solved. Summary of the Invention

[0004] This invention provides a drone detection and air-ground collaborative guidance system and method for missing tassels in seed corn, which solves the problem that in the prior art, a lot of time still needs to be spent searching for the target in the field after the drone inspection is completed, and realizes real-time guidance of ground personnel during flight.

[0005] This invention provides a drone detection and air-ground collaborative guidance system for missing tassels in seed maize, comprising the following units.

[0006] The airborne detection and positioning unit is used to acquire image data of the target area and position and attitude data of the airborne flight equipment; The airborne detection and positioning unit is also used to perform target detection processing on image data and determine the pixel coordinates of any missing target objects in the image data. The airborne detection and positioning unit is also used to determine the actual geographic coordinates of the missing target object based on pixel coordinate data, reference point data of image data, and position and attitude data, and to send the actual geographic coordinates. The ground-based collaborative guidance unit is used to receive actual geographic coordinate data and obtain the current geographic location data of the ground operation object; The ground-based collaborative guidance unit is also used to generate navigation and guidance path data for missed target objects based on actual geographic coordinate data and current geographic location data.

[0007] In one possible implementation, the airborne detection and positioning unit includes a coordinate matching processing module; The coordinate matching processing module is used to obtain reference point data for image data; The coordinate matching processing module is used to determine the pixel offset data of the missing target object in the image data based on the pixel coordinate data and the reference point data. The coordinate matching processing module is used to determine the actual distance offset data of the missing target object relative to the airborne flight equipment based on pixel offset data and preset camera parameter data. The coordinate matching processing module is used to determine the actual geographic coordinates of any missing target objects based on the actual distance offset data and position and orientation data.

[0008] In one possible implementation, the ground-based collaborative guidance unit includes a multi-target scheduling module; The multi-target scheduling module is used to obtain the current geographical location data of multiple ground operation objects when the actual geographic coordinate data of multiple missed target objects are received. The multi-target scheduling module is used to assign the corresponding missing target objects to the target ground operation objects based on the relative distance data between each actual geographic coordinate data and each current geographic location data. The multi-target scheduling module is used to generate sorted target sequence data based on the allocation results; The multi-target scheduling module is used to generate corresponding navigation and guidance path data based on the sorted target sequence data.

[0009] In one possible implementation, the airborne detection and positioning unit includes an edge detection module; The edge detection module is used to perform feature extraction processing on image data according to a preset target detection network to obtain candidate target region feature data; The edge detection module is used to classify and regress the feature data of candidate target regions to determine the bounding box data of missing target objects. The edge detection module is used to determine the geometric center pixel coordinates of any missing target objects based on the bounding box data, and then uses the geometric center pixel coordinates as the pixel coordinates.

[0010] In one possible implementation, the airborne detection and positioning unit includes a binocular depth processing module; The binocular depth processing module is used to extract disparity feature data between the first-view image data and the second-view image data when the image data includes first-view image data and second-view image data, and when there are missing target objects in both the first-view image data and the second-view image data. The binocular depth processing module is used to determine the depth distance data between the missed target object and the airborne flight equipment based on parallax feature data; The binocular depth processing module is used to correct the actual distance offset data based on the depth distance data, so as to obtain the corrected actual distance offset data.

[0011] In one possible implementation, the ground-based collaborative guidance unit includes an offline data interface module; The offline data interface module is used to obtain the set coordinate data of all missed target objects generated by the airborne detection and positioning unit after the airborne flight equipment completes the flight inspection task of the target area; The offline data interface module is used to perform offline storage processing on aggregate coordinate data and generate offline navigation task data. The offline data interface module is used to generate corresponding navigation guidance path data based on offline navigation task data and current geographical location data when no wireless communication connection is established.

[0012] In another possible implementation, the airborne detection and positioning unit includes a data acquisition module; The data acquisition module is used to acquire image data of the target area through airborne vision equipment; The data acquisition module is used to acquire position and attitude data that have a synchronized timestamp with the image data through airborne satellite positioning equipment; The data acquisition module is used to align image data and position / pose data in the time dimension and store the aligned data in a preset cache node.

[0013] This invention also provides a method for detecting missing tassels in seed maize using unmanned aerial vehicles (UAVs) and for air-to-ground collaborative guidance, comprising the following steps: Acquire image data of the target area and position and attitude data of the airborne flight equipment; Perform target detection processing on image data to determine the pixel coordinates of any missing target objects in the image data; Based on pixel coordinate data, reference point data of image data, and position and pose data, determine the actual geographic coordinate data of the missing target object and send the actual geographic coordinate data. Receive actual geographic coordinate data and obtain the current geographic location data of the ground operation object; Based on actual geographic coordinate data and current geographic location data, generate navigation and guidance path data for the missed target object.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described methods for detecting and guiding missing tassels in seed corn using unmanned aerial vehicles (UAVs).

[0015] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described methods for detecting missing tassels in seed maize using unmanned aerial vehicles (UAVs) and for air-to-ground collaborative guidance.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-described methods for detecting and guiding missing tassels in seed corn using unmanned aerial vehicles (UAVs).

[0017] The present invention provides a UAV detection and air-ground collaborative guidance system and method for missing tassels in seed maize. The airborne detection and positioning unit performs real-time image acquisition, target detection, and geographic coordinate determination during flight, and directly transmits the obtained actual geographic coordinate data to the ground collaborative guidance unit. This eliminates the need to wait for the UAV to land and subsequent offline image processing, thus shortening the time delay from target discovery to available location information. Simultaneously, the ground collaborative guidance unit generates a navigation path based on the received actual geographic coordinate data and its own real-time acquired current geographic location data. This guides ground-based operations to the target location accurately and quickly, avoiding wasted time on ineffective walking and searching due to reliance on manual memory or rough positioning, effectively improving the efficiency of target inspection and handling. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a system architecture diagram of the UAV detection and air-ground collaborative guidance system for missing tassels in seed maize provided by the present invention.

[0020] Figure 2 This is a flowchart illustrating the method for detecting missing tassels in seed maize using unmanned aerial vehicles and for air-to-ground collaborative guidance, provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] The terms “comprising” and “having”, and any variations thereof, used in the description of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include other steps or modules not listed, or may optionally include other steps or modules inherent to such processes, methods, products, or apparatus.

[0024] Furthermore, in this invention, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0025] In this invention, a missed target object refers to a target object that needs to be discovered, located, and subsequently processed (such as removed, marked, or supplemented) during inspection or testing tasks. In a typical application scenario, such as the emasculation period in a hybrid maize seed production field, a missed target object could refer to the female parent's tassels that have not been removed. Airborne flight equipment refers to a platform capable of carrying sensors and flying in the air, such as a multi-rotor drone or a fixed-wing drone. Ground operation objects refer to personnel or automated equipment performing operations on the ground.

[0026] To address the aforementioned technical problems, this invention provides a method for detecting missing tassels in seed maize using a drone and for air-to-ground collaborative guidance. In this method, an airborne detection and positioning unit performs real-time image acquisition, target detection, and geographic coordinate determination during flight, and directly transmits the obtained actual geographic coordinate data to a ground-based collaborative guidance unit. This eliminates the need to wait for the drone to land and for subsequent offline image processing, thus shortening the time delay between target discovery and available location information. Simultaneously, the ground-based collaborative guidance unit generates a navigation path based on the received actual geographic coordinate data and its own real-time acquired current geographic location data. This guides ground-based operations to the target location accurately and quickly, avoiding wasted time on ineffective walking and searching due to reliance on manual memory or rough positioning, effectively improving the efficiency of target inspection and handling.

[0027] The following is combined with Figures 1 to 3 This invention describes a drone-based detection and air-to-ground collaborative guidance system and method for missing tassels in seed maize.

[0028] The execution entity of the method for detecting and guiding missing tassels in seed maize using unmanned aerial vehicles (UAVs) provided by this invention can be a system for detecting and guiding missing tassels in seed maize using UAVs or a guidance device for air-to-ground coordination. This device can be an electronic device, or the system for detecting and guiding missing tassels in seed maize using UAVs can be deployed on an electronic device. Furthermore, this device can also be the central processing unit (CPU) of the electronic device, or a guidance module within the electronic device used for air-to-ground coordination guidance.

[0029] Figure 1 This is one of the structural schematic diagrams of the UAV detection and air-ground collaborative guidance system for missing tassels in seed maize provided by the present invention, as shown below. Figure 1 As shown, the system includes an airborne detection and positioning unit 101 and a ground-based collaborative guidance unit 102.

[0030] The airborne detection and positioning unit 101 is used to acquire image data of the target area and position and attitude data of the airborne flight equipment. The image data can be acquired by an optical camera or video sensor mounted on the airborne flight equipment to record the visual characteristics of the target area. The position and attitude data indicates the spatial position and orientation of the airborne flight equipment when acquiring the image data and can be acquired by a global navigation satellite system receiver and an inertial measurement unit.

[0031] The airborne detection and positioning unit 101 is also used to perform target detection processing on the image data to determine the pixel coordinates of any missed target objects in the image data. This process is executed by a computing module deployed on the airborne end, which analyzes each frame of the acquired image, identifies possible target objects, and calculates their coordinate positions on the two-dimensional plane of the image.

[0032] The airborne detection and positioning unit 101 is also used to determine the actual geographic coordinates of the missing target object based on pixel coordinate data, reference point data of the image data, and position and attitude data, and then send the actual geographic coordinates. Reference point data can be understood as a reference point with a known geographic location in the image, such as the center point of the image.

[0033] In this embodiment of the invention, the airborne detection and positioning unit 101 includes a coordinate matching processing module. This module is a component for performing precise target positioning.

[0034] The coordinate matching processing module is used to acquire reference point data for image data. This reference point data can be the position information of the geometric center point of each frame of the image, or the position information of a specific point corresponding to that image, determined by the position and attitude data of the airborne flight equipment.

[0035] The coordinate matching processing module is used to determine the pixel offset data of the missing target object in the image data based on pixel coordinate data and reference point data. This pixel offset data represents the positional deviation of the target relative to the reference point in the image plane, usually expressed in pixels. For example, it can calculate the horizontal and vertical pixel differences between the target's center point and the image center point.

[0036] The coordinate matching processing module is used to determine the actual distance offset of the missed target object relative to the airborne flight equipment based on pixel offset data and preset camera parameter data. The preset camera parameter data includes: camera focal length, principal point coordinates, and lens distortion parameters, etc.

[0037] It should be understood that the pinhole imaging model can convert pixel deviations in an image into physical size deviations on the camera's projection plane. Combined with the drone's altitude information above the ground, this physical size deviation can be converted into the actual distance deviation of the target relative to the drone on the ground.

[0038] In one possible implementation, the airborne detection and positioning unit 101 includes a binocular depth processing module. This module is used to extract disparity feature data between the first-view image data and the second-view image data, provided that the image data includes first-view image data and second-view image data, and that target objects are missing in both the first-view and second-view image data.

[0039] For example, the first-view image and the second-view image correspond to the images captured by the left and right eyes of the binocular camera, respectively. For the same target captured by both eyes simultaneously, its horizontal position in the two images will differ; this difference is the disparity feature data.

[0040] The binocular depth processing module is used to determine the depth distance between the missed target object and the airborne flight equipment based on parallax feature data.

[0041] For example, based on the triangulation principle of binocular vision, the depth of a target (i.e., its distance from the camera) is inversely proportional to the parallax. By calculating the parallax and combining it with the baseline length of the binocular cameras (the distance between the optical centers of the two cameras) and the focal length, the accurate distance from the target to the camera can be calculated.

[0042] The binocular depth processing module is used to correct the actual distance offset data based on the depth distance data, so as to obtain the corrected actual distance offset data.

[0043] It should be understood that when a target (such as a male ear of grain) grows at the top of a plant, its actual height may differ from the ground, or errors may occur in areas with significant terrain undulations. Depth distance measured directly through binocular vision can be corrected, making the conversion from pixel offset to actual distance offset more accurate.

[0044] The coordinate matching processing module is used to determine the actual geographic coordinates of any missing target objects based on the actual distance offset data and position and attitude data. The position and attitude data includes the UAV's current latitude and longitude, altitude, pitch angle, roll angle, and heading angle.

[0045] For example, the coordinate matching processing module performs coordinate rotation and projection transformation on the actual distance offset based on this attitude information, converts it into longitude and latitude offsets in the Earth coordinate system, and then adds them to the longitude and latitude of the UAV to finally obtain the target's accurate longitude and latitude coordinates.

[0046] In this embodiment, the binocular camera in the airborne detection and positioning unit 101 is fixedly connected to the airborne satellite positioning equipment via a rigid connector, and their relative spatial position relationship is pre-calibrated to obtain extrinsic parameter calibration data. Based on the extrinsic parameter calibration data, the coordinate matching processing module transforms the coordinates of the target box center point in the image coordinate system to the UAV body coordinate system, and then, combined with the UAV's current latitude, longitude, altitude, and attitude angle, calculates the absolute latitude and longitude coordinates of the ground target. The rigid connector fixing and pre-calibration of extrinsic parameters ensure that the spatial position relationship between the two remains constant during flight operations, thereby improving the accuracy and stability of coordinate transformation.

[0047] The ground-based collaborative guidance unit 102 is used to receive actual geographic coordinate data and obtain the current geographic location data of the ground operation object. This unit typically exists in the form of a portable terminal, capable of receiving data from the airborne detection and positioning unit 101 via wireless communication and obtaining its current location based on its own positioning module.

[0048] The ground-based collaborative guidance unit 102 is also used to generate navigation and guidance path data for the missed target object based on the actual geographic coordinate data and the current geographic location data. This navigation and guidance path data can be represented by information such as routes on an electronic map, directional arrows, or voice prompts, and is used to guide the ground operation object from its current location to the location of the missed target object.

[0049] In one possible implementation, the ground-based collaborative guidance unit 102 includes a multi-target scheduling module.

[0050] The multi-target scheduling module is used to obtain the current geographical location data of multiple ground operation objects when the actual geographical coordinate data of multiple missed target objects are received.

[0051] It should be understood that the system may detect multiple missed tassels at the same time, and there may be multiple workers in the field performing tassel removal. The multi-target scheduling module will first collect the real-time locations of all targets to be processed and all workers.

[0052] The multi-target scheduling module is used to assign the corresponding missing target objects to the target ground operation objects based on the relative distance data between each actual geographic coordinate data and each current geographic location data.

[0053] For example, the distance from each target to each worker can be calculated, and then the target can be assigned to the most suitable worker with the goal of minimizing the total travel distance or the shortest task completion time.

[0054] The multi-target scheduling module is used to generate sorted target sequence data based on the allocation results.

[0055] It should be understood that for the same ground operation object that has been assigned multiple targets, the module will sort all the targets it is responsible for according to their distance from the current location or other priority rules (such as target importance, urgency) to form an ordered task list.

[0056] The multi-target scheduling module is used to generate corresponding navigation and guidance path data based on the sorted target sequence data.

[0057] In this way, the ground-based collaborative guidance unit will generate navigation paths for the operators to each target sequentially based on this ordered sequence, and automatically switch to the next target after one target is completed. This effectively avoids situations where multiple operators repeatedly go to the same target, or where some areas are left unattended, thereby achieving a balanced distribution of tasks and optimizing human resources, further improving operational efficiency.

[0058] Based on the above technical solution, the airborne detection and positioning unit completes image acquisition, target detection, and geographic coordinate determination in real time during flight, and directly sends the obtained actual geographic coordinate data to the ground-based collaborative guidance unit. This eliminates the need to wait for the UAV to land and subsequent offline image processing, thus shortening the time delay between target discovery and available location information. Simultaneously, the ground-based collaborative guidance unit generates a navigation path based on the received actual geographic coordinate data and its own real-time acquired current geographic location data. This guides ground-based work objects to the target location accurately and quickly, avoiding wasted time on ineffective walking and searching due to reliance on manual memory or rough positioning, effectively improving the efficiency of target inspection and handling operations.

[0059] In some embodiments, the airborne detection and positioning unit 101 includes an edge detection module. The edge detection module is used to perform feature extraction processing on image data according to a preset target detection network to obtain candidate target region feature data.

[0060] For example, the preset object detection network can employ a lightweight convolutional neural network or a lightweight network based on the Transformer architecture. These network structures undergo compression before deployment, specifically through channel pruning and INT8 quantization. Channel pruning reduces the number of model parameters and computation by removing unimportant feature channels from the network. INT8 quantization converts floating-point weights and activation values ​​in the model into 8-bit integer representations, reducing model storage space and accelerating inference computation. After this compression process, the model can meet the real-time requirements of edge computing devices while maintaining detection accuracy. Feature extraction processing refers to the network using multi-layer convolutional operations to convert the original RGB image data into high-dimensional feature maps with semantic information. These feature maps contain clues about the location and category of potential targets.

[0061] The edge detection module is used to classify and regress the feature data of candidate target regions to determine the bounding box data of missing target objects.

[0062] For example, on the extracted feature map, the network can classify each region (determine whether it is a target or background) and perform location regression (precisely adjust the position and size of the bounding box) using preset anchor points or candidate regions. Then, the preset target detection network outputs bounding boxes containing the target, with each bounding box represented by its top-left and bottom-right corner coordinates in the image.

[0063] The edge detection module is used to determine the geometric center pixel coordinates of any missing target objects based on the bounding box data, and then uses the geometric center pixel coordinates as the pixel coordinates.

[0064] For example, for a given bounding box, the pixel coordinates of its geometric center point can be obtained by calculating the average of the coordinates of the four vertices of the bounding box. These center point coordinates represent the approximate location of the target in the image and are used as a reference for subsequent coordinate transformations.

[0065] Based on the above technical solution, the edge detection module performs feature extraction, classification, and location regression processing on image data using a pre-set target detection network on the airborne end. This enables it to automatically identify missed targets and determine their bounding boxes during flight. Thus, there is no need to transmit image data back to a ground server for processing, effectively reducing data transmission latency and improving the real-time performance of target detection. Simultaneously, determining the geometric center pixel coordinates of the target based on the bounding box data provides accurate location input for subsequent pixel offset calculations and geographic coordinate solutions.

[0066] In some embodiments, the ground-based cooperative guidance unit 102 includes an offline data interface module. This module ensures that the system can still function normally even when wireless communication conditions are poor or there is no need for real-time communication.

[0067] The offline data interface module is used to obtain the set coordinate data of all missed target objects generated by the airborne detection and positioning unit 101 after the airborne flight equipment completes the flight inspection task of the target area.

[0068] The offline data interface module is used to process and store the aggregated coordinate data offline, generating offline navigation task data. After receiving this batch of coordinate data, the ground cooperative guidance unit 102 saves it in its local storage, forming an offline task list.

[0069] The offline data interface module is used to generate corresponding navigation guidance path data based on offline navigation task data and current geographical location data when no wireless communication connection is established.

[0070] In some embodiments, the airborne detection and positioning unit 101 includes a data acquisition module. The data acquisition module is responsible for time synchronization and preprocessing of multi-source sensor data.

[0071] The data acquisition module is used to acquire image data of the target area through airborne vision equipment.

[0072] For example, the airborne vision device may be a high-resolution industrial camera or a binocular camera that continuously captures images of the target area at a fixed frame rate (such as 20 or 30 frames per second).

[0073] The data acquisition module is used to acquire position and attitude data that has a synchronized timestamp with the image data through airborne satellite positioning equipment.

[0074] Airborne satellite positioning equipment can be used for real-time dynamic carrier phase differential positioning, providing centimeter-level high-precision positioning information. The data acquisition module ensures that each position and attitude data point is recorded with an accurate time stamp.

[0075] The data acquisition module is used to align image data and position / pose data in the time dimension and store the aligned data in a preset cache node.

[0076] It should be understood that since image data and position / pose data come from different sensors, their data streams are independent and asynchronous. The data acquisition module matches and associates each frame of image with the set of position / pose data closest to it at the time of acquisition, based on the recorded timestamp, forming a unified time-aligned data packet. The processed data is then sent to a preset high-speed cache for real-time access by subsequent detection and localization modules.

[0077] Figure 2 The flowchart of the method for detecting missing tassels in seed maize using unmanned aerial vehicles and for air-to-ground coordinated guidance provided by this invention is shown below. Figure 2 As shown, it includes: Step 201: Acquire image data of the target area and position and attitude data of the airborne flight equipment.

[0078] In this step, the airborne detection and positioning unit 101, while the airborne flight equipment is flying along a preset route, captures images of the target area using its onboard vision equipment (such as a binocular camera), acquiring continuous image data. Simultaneously, through airborne satellite positioning equipment (such as a Global Navigation Satellite System receiver) and an inertial measurement unit, it synchronously records the position data (including longitude, latitude, and altitude) and attitude data (including pitch angle, roll angle, and heading angle) of the airborne flight equipment at the moment of each image acquisition. The data acquisition module timestamps and aligns the acquired image data and position / attitude data to ensure that the data used in subsequent processing is spatiotemporally matched.

[0079] Step 202: Perform target detection processing on the image data to determine the pixel coordinates of any missing target objects in the image data.

[0080] In this step, the edge computing module of the airborne detection and localization unit 101 calls a pre-deployed lightweight target detection model to perform real-time inference on each frame of image acquired in step 201. This model takes image data as input, performs feature extraction and classification regression, and outputs the detected target bounding box. The edge detection module further calculates the geometric center of this bounding box, whose coordinates in the image coordinate system are the pixel coordinate data.

[0081] In this step, if the image data includes first-view image data and second-view image data (i.e., the left and right views of the binocular camera), and the same missing target object is detected in both the left and right views, the edge detection module or the binocular depth processing module further extracts the disparity feature data between the left and right views, and calculates the depth distance data between the missing target object and the airborne flight equipment based on the triangulation principle of binocular vision.

[0082] Step 203: Based on pixel coordinate data, reference point data of image data, and position and pose data, determine the actual geographic coordinate data of the missing target object and send the actual geographic coordinate data.

[0083] In one possible implementation, step 203 includes steps 1-4.

[0084] Step 1: Obtain the reference point data for the image data.

[0085] In this step, the coordinate matching processing module reads each frame of image data, determines its geometric dimensions, and then calculates the pixel coordinates of its geometric center point. This center point is used as a fixed reference point for this frame of the image.

[0086] Step 2: Based on the pixel coordinate data and reference point data, determine the pixel offset data of the missing target object in the image data.

[0087] In this step, for the target detected in step 202, the pixel coordinates of its bounding box center point are: The coordinate matching processing module calculates the pixel offset of the target center point relative to the image center point. The calculation formula is as follows: in, and These are pixel offset data, representing the number of pixels the target is offset from in the horizontal and vertical directions of the image, respectively.

[0088] Step 3: Based on the pixel offset data and preset camera parameter data, determine the actual distance offset data of the missing target object relative to the airborne flight equipment.

[0089] In this step, the coordinate matching processing module converts the pixel offset into the actual distance offset on the ground based on the camera's intrinsic parameters and the drone's altitude relative to the ground. The calculation formula is as follows: in, This represents the altitude of the airborne flight equipment relative to the plane where the target is located. and These represent the normalized focal length of the camera in the horizontal and vertical directions, respectively. The normalized focal length is the ratio of the camera's physical focal length to the pixel size. The calculated... This refers to the actual distance offset of the target relative to the UAV on the ground. Preferably, this step also includes adjusting the target's position based on the current pitch and roll angles of the onboard flight equipment, using trigonometric relationships. Tilt correction is performed to eliminate projection errors caused by the tilt of the UAV, resulting in a more accurate ground distance offset.

[0090] Step 4: Based on the actual distance offset data and position and attitude data, determine the actual geographic coordinates of the missing target object.

[0091] In this step, the coordinate matching processing module combines the UAV's current location to convert the ground distance offset into a geographic coordinate offset. The calculation formula is as follows: in, The position and attitude data records the current longitude and latitude coordinates of the UAV. This is the average radius of the Earth. Divide the meridian by the Earth's radius to get the radian, then convert it to an angle to get the latitude offset. Along the parallels of latitude, it's necessary to consider the variation of the radius of the latitude circle with latitude, i.e., multiplying the Earth's radius by the cosine of the latitude, then converting it to angles to obtain the longitude offset. The final calculated... This means that the actual geographic coordinates of the target object were omitted.

[0092] Step 204: Receive actual geographic coordinate data and obtain the current geographic location data of the ground operation object.

[0093] Step 205: Generate navigation and guidance path data for the missing target object based on the actual geographic coordinate data and the current geographic location data.

[0094] It should be noted that for a detailed description of steps 201-205, please refer to the introduction of the ground-cooperative guidance system in the above embodiments, which will not be repeated here.

[0095] Based on the above technical solution, by integrating image target detection, airborne position and attitude data, and the real-time location of ground work objects, navigation and guidance paths can be automatically generated for missed targets discovered during aerial inspections. This avoids positioning errors caused by manual marking or visual judgment, improving the accuracy of geographic coordinate calculation for missed targets. Simultaneously, by sending the actual geographic coordinates to the ground work terminal in real time and combining them with the current geographic location to generate a navigation path, ground work objects can be guided accurately to the location of missed targets, thereby reducing repeated inspections caused by omissions and improving operational efficiency.

[0096] The air-to-ground collaborative guidance device provided by this invention is described below. This air-to-ground collaborative guidance device described below corresponds to the above-described method for detecting and guiding missing tassels in seed maize using a drone. It is understood that, in order to achieve the above functions, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the steps of the various examples of the method for detecting missing tassels in seed maize using a drone and the air-to-ground collaborative guidance method described in conjunction with the embodiments disclosed in this invention, this invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0097] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a method for detecting and guiding missing tassels in seed corn using a drone. This method includes: acquiring image data of the target area and position and attitude data of the airborne flight equipment; performing target detection processing on the image data to determine the pixel coordinates of the missing target object in the image data; determining and sending the actual geographic coordinates of the missing target object based on the pixel coordinates, reference point data of the image data, and position and attitude data; receiving the actual geographic coordinates and acquiring the current geographic location data of the ground operation object; and generating navigation and guidance path data for the missing target object based on the actual geographic coordinates and the current geographic location data.

[0098] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the UAV detection and air-to-ground cooperative guidance method for missing tassels in seed corn provided by the above methods. This method includes: acquiring image data of a target area and position and attitude data of an airborne flight device; performing target detection processing on the image data to determine the pixel coordinates of the missing target object in the image data; determining and sending the actual geographic coordinates of the missing target object based on the pixel coordinates, reference point data of the image data, and position and attitude data; receiving the actual geographic coordinates and acquiring the current geographic location data of the ground operation object; and generating navigation and guidance path data for the missing target object based on the actual geographic coordinates and the current geographic location data.

[0100] Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the method for detecting and guiding missing tassels in seed maize using a UAV, as described above. This method includes: acquiring image data of a target area and position and attitude data of an airborne flight device; performing target detection processing on the image data to determine the pixel coordinates of the missing target object in the image data; determining and transmitting the actual geographic coordinates of the missing target object based on the pixel coordinates, reference point data of the image data, and position and attitude data; receiving the actual geographic coordinates and acquiring the current geographic location data of the ground operation object; and generating navigation and guidance path data for the missing target object based on the actual geographic coordinates and the current geographic location data.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A drone-based detection and air-to-ground collaborative guidance system for missing tassels in seed maize, characterized in that, include: The airborne detection and positioning unit is used to acquire image data of the target area and position and attitude data of the airborne flight equipment; The airborne detection and positioning unit is also used to perform target detection processing on the image data to determine the pixel coordinate data of any missing target objects in the image data; The airborne detection and positioning unit is also used to determine the actual geographic coordinate data of the missed target object based on the pixel coordinate data, the reference reference point data of the image data, and the position and attitude data, and to send the actual geographic coordinate data. The ground-based collaborative guidance unit is used to receive the actual geographic coordinate data and obtain the current geographic location data of the ground operation object; The ground-based collaborative guidance unit is also used to generate navigation and guidance path data for the omitted target object based on the actual geographic coordinate data and the current geographic location data.

2. The system according to claim 1, characterized in that, The airborne detection and positioning unit includes a coordinate matching processing module; The coordinate matching processing module is used to obtain the reference point data of the image data; The coordinate matching processing module is used to determine the pixel offset data of the missing target object in the image data based on the pixel coordinate data and the reference point data. The coordinate matching processing module is used to determine the actual distance offset data of the missed target object relative to the airborne flight equipment based on the pixel offset data and the preset camera parameter data. The coordinate matching processing module is used to determine the actual geographic coordinates of the missed target object based on the actual distance offset data and the position and attitude data.

3. The system according to claim 1, characterized in that, The ground-based collaborative guidance unit includes a multi-target scheduling module; The multi-target scheduling module is used to obtain the current geographical location data of multiple ground operation objects when it receives the actual geographical coordinate data of multiple missing target objects; The multi-target scheduling module is used to assign the corresponding missing target object to the target ground operation object based on the relative distance data between each of the actual geographic coordinate data and each of the current geographic location data; The multi-objective scheduling module is used to generate sorted target sequence data based on the allocation results; The multi-target scheduling module is used to generate the corresponding navigation and guidance path data based on the sorted target sequence data.

4. The system according to claim 1, characterized in that, The airborne detection and positioning unit includes an edge detection module; The edge detection module is used to perform feature extraction processing on the image data using a preset target detection network to obtain candidate target region feature data; The edge detection module is used to classify and regress the feature data of the candidate target region to determine the bounding box data of the missing target object; The edge detection module is used to determine the geometric center pixel coordinates of the missing target object based on the bounding box data, and use the geometric center pixel coordinates as the pixel coordinates.

5. The system according to claim 2, characterized in that, The airborne detection and positioning unit includes a binocular depth processing module; The binocular depth processing module is used to extract disparity feature data between the first view image data and the second view image data when the image data includes first view image data and second view image data, and the missing target object exists in both the first view image data and the second view image data. The binocular depth processing module is used to determine the depth distance data between the missed target object and the airborne flight equipment based on the parallax feature data. The binocular depth processing module is used to correct the actual distance offset data based on the depth distance data to obtain corrected actual distance offset data.

6. The system according to claim 1, characterized in that, The ground-based collaborative guidance unit includes an offline data interface module; The offline data interface module is used to obtain the set coordinate data of all missed target objects generated by the airborne detection and positioning unit after the airborne flight equipment completes the flight inspection task of the target area. The offline data interface module is used to perform offline storage processing on the set of coordinate data to generate offline navigation task data; The offline data interface module is used to generate corresponding navigation guidance path data based on the offline navigation task data and the current geographical location data when no wireless communication connection is established.

7. The system according to claim 1, characterized in that, The airborne detection and positioning unit includes a data acquisition module; The data acquisition module is used to acquire image data of the target area through airborne vision equipment; The data acquisition module is used to acquire the position and attitude data that has a synchronized timestamp with the image data through an airborne satellite positioning device; The data acquisition module is used to perform time-dimensional alignment processing on the image data and the position and pose data, and store the aligned data in a preset cache node.

8. A method for detecting and guiding missing tassels in seed maize using unmanned aerial vehicles (UAVs), applied to the UAV detection and air-to-ground collaborative guidance system for detecting missing tassels in seed maize as described in any one of claims 1-7, comprising: Acquire image data of the target area and position and attitude data of the airborne flight equipment; The image data is subjected to target detection processing to determine the pixel coordinates of any missing target objects in the image data; Based on the pixel coordinate data, the reference point data of the image data, and the position and pose data, determine the actual geographic coordinate data of the missing target object and send the actual geographic coordinate data. Receive the actual geographic coordinate data and obtain the current geographic location data of the ground operation object; Based on the actual geographic coordinate data and the current geographic location data, a navigation and guidance path data is generated for the missing target object.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for detecting missing tassels in seed maize and the air-ground collaborative guidance method as described in claim 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for detecting missing tassels in seed maize and the air-ground collaborative guidance method as described in claim 8.

Images