Targeted drug delivery method, electronic device, storage medium, and system

Abstract

The application provides a target spraying method, comprising: obtaining position information of a marking box of a target output by a target recognition model; constructing a transverse actual spraying pixel line and a vertical area division pixel line in an image containing the target; controlling a spraying device to perform a spraying action according to the position information of the marking box of the target, the position information of the actual spraying pixel line and the position information of the area division pixel line; the target recognition model is obtained by training a target detection model according to sample images labeled with crop labels and weed labels; the target detection model is obtained by adding an NAM attention mechanism module in a residual feature learning C2f unit of a YOLOv8 model based on a YOLOv8 model framework, setting a classification loss of the YOLOv8 model as a VFL zoom loss, and setting a regression loss of the YOLOv8 model as a weighted loss of a DFL distribution focus loss and a WIoU regression loss.

Description

Technical Field

[0001] This invention relates to the field of precision pesticide application technology in agriculture, and in particular to a method, electronic device, storage medium and system for targeted pesticide application. Background Technology

[0002] Pesticide spraying is one of the main methods for controlling crop diseases and pests. Traditional, uniform, large-area spraying methods often result in large amounts of pesticides being accidentally sprayed onto the surface soil around the target crop, leading to low pesticide utilization and a series of problems such as high pesticide residues, high production costs, environmental pollution, and food safety concerns. Therefore, accurately identifying crops and weeds and targeting them with pesticides can improve pesticide utilization, effectively reduce costs, and minimize pollution to air, soil, and groundwater.

[0003] Currently, field target (crop or weed) identification methods are mainly based on traditional machine vision or ultrasonic sensing technology to identify targets. However, due to the complexity of the field environment and the similarity between weeds and crops, it is very difficult to identify weeds and crops at the same time. Summary of the Invention

[0004] To address the problems existing in the prior art, embodiments of the present invention provide a method for targeted drug delivery, an electronic device, a storage medium, and a system.

[0005] In a first aspect, the present invention provides a method for targeted drug delivery, comprising:

[0006] Acquire consecutive multi-frame images, which are obtained by image acquisition device from the target in the current spraying area, where the target is the object to be sprayed;

[0007] Each frame of the image is sequentially input into the target recognition model to obtain the position information of the target's bounding box output by the target recognition model;

[0008] Construct horizontal actual spray pixel lines and vertical region division pixel lines in the image;

[0009] The spraying device is controlled to perform the application action based on the position information of the target's marked box, the position information of the actual spray pixel line, and the position information of the region division pixel line.

[0010] The target recognition model is trained on a target detection model based on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss.

[0011] According to the target application method provided by the present invention, the step of controlling the spraying device to perform the application action based on the position information of the target's marker frame, the position information of the actual spray pixel line, and the position information of the region division pixel line includes:

[0012] Based on the position information of the pixel lines in the region division, the image is divided into several sub-regions to be sprayed;

[0013] If the actual spray pixel line is located between the upper and lower boundary lines of the target's marking frame, then the target sub-area to be sprayed is determined based on the position of the left and right boundary lines of the target's marking frame.

[0014] The spraying equipment of the spraying system is controlled to perform the application of pesticides according to the target sub-area to be sprayed.

[0015] According to the target application method provided by the present invention, the spraying device includes at least one nozzle, each nozzle being equipped with a corresponding solenoid valve. The step of controlling the spraying device of the spray system to perform the application action based on the target sub-area to be sprayed includes:

[0016] The location information of the target sub-area to be sprayed is sent to the spraying equipment, so that the spraying equipment opens the corresponding solenoid valve to perform the spraying action according to the location information of the target sub-area to be sprayed.

[0017] According to the targeted drug delivery method provided by the present invention, the sample image is obtained in the following manner:

[0018] Obtain raw sample images containing crops and weeds;

[0019] Determine the seed pixels in the original sample image, and segment the weed image from the original sample image based on the seed pixels;

[0020] Using the weed image as the object and the original sample image as the background, image synthesis is performed to obtain a sample image in which the number of weeds reaches a preset requirement.

[0021] According to the target-based drug delivery method provided by the present invention, the target recognition model includes an image input module, a feature extraction module, a feature fusion module, and a recognition module. The feature extraction module includes multiple cascaded first neural network units, the feature fusion module includes multiple cascaded second neural network units, and the recognition module includes a weighted loss function of VFL zoom loss function, DFL distributed focus loss function, and WIoU regression loss function.

[0022] The first neural network unit includes any one of the following: a residual feature learning C2f unit, a convolutional Conv unit, a residual feature learning C2f-N unit based on NAM attention mechanism, and a feature fusion SPPF unit.

[0023] The second neural network unit includes any one of the following: a convolutional Conv unit, a residual feature learning C2f-N unit based on the NAM attention mechanism, an image stitching Concat unit, and an image upsampling Upsample unit.

[0024] According to the target-based drug delivery method provided by the present invention, the step of sequentially inputting each frame of the image into a target recognition model and obtaining the coordinates of the target's bounding box output by the target recognition model includes:

[0025] The image is input to the feature extraction module through the image input module to obtain the first feature map output by the first C2f unit, the second feature map output by the second C2f unit, and the third feature map output by the SPPF unit in the feature extraction module. The second C2f unit is a subordinate unit of the first C2f unit.

[0026] The first feature map, the second feature map, and the third feature map are sequentially input into the first Concat unit, the second Concat unit, and the Upsample unit in the feature fusion module to obtain the first feature matrix output by the first C2f-N unit, the second feature matrix output by the second C2f-N unit, and the third feature matrix output by the third C2f-N unit in the feature fusion module. The second Concat unit is a subordinate unit of the first Concat unit, the third C2f-N unit is a subordinate unit of the second C2f-N unit, and the second C2f-N unit is a subordinate unit of the first C2f-N unit.

[0027] The first feature matrix, the second feature matrix, and the third feature matrix are output to the recognition module. Crops and weeds are identified by a weighted loss function of the VFL zoom loss function, the DFL distributed focus loss function, and the WIoU regression loss function in the recognition module. Targets are marked in the crops and weeds by a label box containing spatial coordinate information.

[0028] In a second aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target drug delivery method as described above.

[0029] Thirdly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target drug delivery method as described above.

[0030] Fourthly, the present invention also provides a target-based drug delivery system, comprising: a target identification and positioning processor and a program or instructions executable on the target identification and positioning processor, wherein the program or instructions, when executed by the target identification and positioning processor, implement the target-based drug delivery method as described above.

[0031] The targeted drug delivery system provided by the present invention further includes an image acquisition device and a spraying device; the image acquisition device and the spraying device are respectively connected to the target recognition and positioning processor;

[0032] The image acquisition device is used to acquire video streams of targets planted in the area to be sprayed in real time, and send the video streams to the target recognition and positioning processor so that the target recognition and positioning processor can acquire multiple consecutive frames of images based on the video streams;

[0033] The spraying device is used to perform pesticide application based on the control signal output by the target identification and positioning processor.

[0034] The present invention provides a method, electronic device, storage medium, and system for targeted pesticide application, comprising: acquiring multiple consecutive frames of images obtained by an image acquisition device from the current area to be sprayed; sequentially inputting each frame of images into a target recognition model to obtain the position information of the target's bounding box output by the target recognition model, wherein the target is the object to be sprayed; constructing horizontal actual spray pixel lines and vertical region division pixel lines in the images containing the target; and controlling the spraying device to perform pesticide application actions based on the position information of the target's bounding box, the position information of the actual spray pixel lines, and the position information of the region division pixel lines; wherein, the target... The target identification model is trained on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss. Thus, this invention can achieve accurate target identification and precise positioning in complex field environments, and achieve precise target application of pesticides. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is a schematic flowchart of the targeted drug delivery method provided by the present invention;

[0037] Figure 2 This is a schematic diagram of the target localization scenario provided by the present invention;

[0038] Figure 3 This is a schematic diagram of the spraying scenario of the spraying equipment provided by the present invention;

[0039] Figure 4 This is a schematic diagram of the target recognition model provided by the present invention;

[0040] Figure 5 This is a schematic diagram of the structure of the C2f-N unit provided by the present invention;

[0041] Figure 6 This is a schematic diagram of the target drug delivery device provided by the present invention;

[0042] Figure 7 This is a schematic diagram of the target delivery system provided by the present invention;

[0043] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] In the description of the invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0046] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is one of the flowcharts of the targeted drug delivery method provided by the present invention. The main body executing each step of the method can be a targeted drug delivery device, which can be implemented by software and / or hardware. The device can be integrated into an electronic device, which can be a terminal device (such as a smartphone, personal computer, learning machine, etc.), a server (such as a local server or cloud server, or a server cluster, etc.), a processor, or a chip, etc.

[0047] like Figure 1 As shown, the method may include the following steps:

[0048] Step 101: Acquire a series of consecutive frames of images. The series of consecutive frames of images are obtained by the image acquisition device acquiring images of the target in the current area to be sprayed. The target is the object to be sprayed.

[0049] In this embodiment, multiple consecutive frames of images are acquired via video streaming, which avoids the loss of target information due to missed shots.

[0050] Specifically, the image acquisition device can move along a preset route within the spraying area to acquire video streams of targets planted within the spraying area in real time, thereby obtaining images of each target from the video stream.

[0051] The area to be sprayed refers to the area where pesticides will be applied, which can be determined according to the actual situation. For example, it can be an open field or a greenhouse area. This embodiment of the invention does not make a specific limitation on this.

[0052] The preset route is the route for spraying pesticides onto targets planted in the area to be sprayed. The preset route can be determined based on prior knowledge and / or actual conditions. In this embodiment of the invention, the preset route is not specifically limited.

[0053] In this embodiment, the target is the object to be sprayed, which can be flexibly set according to the spraying needs. For example, when it is necessary to spray pesticides on crops, the target is the crops; when it is necessary to spray pesticides on weeds, the target is the weeds.

[0054] Step 102: Input the images of each frame into the target recognition model in sequence to obtain the position information of the target's bounding box output by the target recognition model;

[0055] The target recognition model is trained on a target detection model based on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss.

[0056] The target recognition model in this embodiment can identify crops and weeds around the crops in the current frame image, and mark the identified crops and weeds with a bounding box carrying coordinate labels, thereby outputting the coordinate information of crops and weeds in the current frame image.

[0057] In one example, when the target recognition model outputs the coordinate information of crops and weeds in the current frame image, it can choose to block the coordinate information of weeds and only output the coordinate information of the recognized crops, or it can choose to block the coordinate information of crops and only output the coordinate information of the recognized weeds, so that the crops or weeds can be treated separately in the future.

[0058] In this embodiment, since the image acquisition device can acquire images under different shooting conditions, the sample images in this embodiment can be acquired based on different shooting conditions. Shooting conditions may include, but are not limited to, lighting conditions, weather conditions, and weed density.

[0059] It should be noted that the crops and weeds in the sample images in this embodiment are the same types as those in the area to be sprayed, and the sample area corresponding to the sample image may be the same as or different from the area to be sprayed. The sample area is not specifically limited in this embodiment of the invention.

[0060] It should be noted that the YOLO (You Only Look Once) model is an object recognition and localization algorithm based on deep neural networks. YOLO is characterized by its fast running speed and suitability for real-time computing.

[0061] The NAM (Normalization-based Attention Module) is a normalization-based attention mechanism. Compared to other attention mechanisms, it does not require additional fully connected layers, convolutions, or other extra computations and parameters. It directly uses the scaling factor in Batch Normalization (BN) to calculate attention weights and further suppresses insignificant features by adding a regularization term. This approach applies sparse weight penalties to the attention module, making these weights more computationally efficient while maintaining the same performance.

[0062] VFL (VariFocal Loss) is a dynamically scaled binary cross-entropy loss. It employs an asymmetric training example weighting method. VFL reduces the weight of negative samples to address class imbalance during training, while increasing the weight of generating high-quality positive samples. This focuses training on high-quality positive samples, better addressing the class imbalance problem.

[0063] Distribution Focal Loss (DFL) adjusts the weights of samples by introducing a focal factor and a distribution parameter. The focal factor is mainly used to alleviate class imbalance by assigning higher weights to rarer classes, making the model focus more on the classification accuracy of rarer classes. The distribution parameter, on the other hand, controls the shape of the sample distribution and provides further weighting when dealing with more difficult samples.

[0064] WIoU (Weighted Intersection over Union Regression Loss) proposes a dynamic non-monotonic focusing mechanism that uses "outlier degree" instead of IoU to evaluate anchor box quality and provides a smart gradient gain allocation strategy. This strategy reduces the competitiveness of high-quality anchor boxes while also minimizing harmful gradients generated by low-quality examples. This allows WIoU to focus on anchor boxes of average quality, improving the overall performance of the detector and significantly addressing the problems of sample density and mutual occlusion.

[0065] Specifically, in this embodiment of the invention, the YOLOv8 model in the prior art is used as the basic framework. First, a NAM attention mechanism module is added to its residual feature learning C2f unit to increase the feature extraction capability of the backbone network and provide a reliable source of front-end information for the feature fusion stage. Second, the VFL classification loss function better solves the problem of imbalance between crop and weed sample numbers, enhancing the reliability of the network model. Finally, the use of WIoU regression loss + DFL distribution focal loss greatly solves the problem of difficulty in identifying and locating crops and weeds due to their dense growth and mutual occlusion.

[0066] In one example, model training and testing can be performed on a workstation equipped with an 11th Gen Intel i9 CPU and an NVIDIA GTX3080Ti GPU, using PyTorch version 1.10.2 for network model construction. Regarding parameter settings: the model training process preloads a weight file trained on the COCO dataset; the input crop and weed dataset images have a resolution of 640×640, the training run is 300 epochs, and the batch size is 8. The model weights are updated after each iteration, and the model performance is tested using the test set data. The weights of the model with the highest recognition accuracy are saved as the weights of the trained target recognition model.

[0067] Step 103: Construct horizontal actual spray pixel lines and vertical region division pixel lines in the image;

[0068] It should be noted that due to the inherent time delay in the information transmitted by the image acquisition device for the current frame, targets appearing at the current time point may not be sprayed in a timely manner. Secondly, the nozzle may not be able to accurately pinpoint the target's location. Therefore, in this embodiment, horizontal actual spray pixel lines and vertical region division pixel lines are constructed.

[0069] Understandably, the positions of multiple nozzles of a spraying device are usually set on the same horizontal line, and different nozzles correspond to spraying corresponding areas. Therefore, in this embodiment, the actual spray pixel lines (i.e., the horizontal lines corresponding to multiple nozzles) in the image acquired by the image acquisition device can be pre-configured according to the height set by the multiple nozzles of the spraying device and the viewing angle of the image acquired by the image acquisition device. The area division pixel lines in the image acquired by the image acquisition device can be pre-configured according to the sprayable range of each nozzle, so as to divide the image into spray control areas corresponding to each nozzle, thereby achieving precise spraying.

[0070] Step 104: Control the spraying device to perform the application action based on the position information of the target's marker box, the position information of the actual spray pixel line, and the position information of the region division pixel line;

[0071] In this embodiment, the position information of the target's marker frame and the position information of the actual spray pixel line are used to determine whether the target has reached the vicinity of the spraying area of ​​the spraying equipment. Based on the position information of the marker frame and the position information of the area division pixel line, it can be further determined which nozzle's spraying area the target has reached, thereby controlling the spraying equipment to perform the pesticide application action.

[0072] The target application method provided by this invention acquires multiple consecutive frames of images of the area to be sprayed using an image acquisition device; each frame is sequentially input into a target recognition model to obtain the position information of the target's bounding box output by the model, where the target is the object to be sprayed; horizontal actual spray pixel lines and vertical region division pixel lines are constructed in the image containing the target; the spraying device is controlled to perform the application action based on the position information of the target's bounding box, the actual spray pixel lines, and the region division pixel lines. This improves the real-time accuracy of application, avoids missed detections and spray response delays. Furthermore, the target recognition model in this invention is based on... The target detection model was trained using sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model framework. A NAM attention mechanism module was added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model was set as the VFL zoom loss, and the regression loss of the YOLOv8 model was constructed as a weighted loss of the DFL distributed focal loss and the WIoU regression loss. This improved the accuracy of the model in distinguishing between crops and weeds and improved the recognition accuracy of the target identification model. Therefore, this invention can achieve accurate target identification and precise positioning in complex field environments, and realize precise target application of pesticides.

[0073] In some embodiments, controlling the spraying device to perform the application action based on the position information of the target's marker frame, the position information of the actual spray pixel line, and the position information of the region division pixel line includes:

[0074] Based on the position information of the pixel lines in the region division, the image is divided into several sub-regions to be sprayed;

[0075] If the position of the actual spray pixel line is between the upper and lower boundary lines of the target's marker frame, then the target sub-area to be sprayed is determined based on the position of the left and right boundary lines of the target's marker frame.

[0076] The spraying equipment of the spraying system is controlled to perform the application of pesticides according to the target sub-area to be sprayed.

[0077] In this embodiment, as Figure 2 and Figure 3As shown, when the spraying equipment includes 4 nozzles, a horizontal actual spray pixel line and 4 vertical region division pixel lines are constructed in the image. The region division pixel lines divide the current frame image into 4 regions proportionally according to the pixel horizontal coordinate. Each region corresponds to a spray control area. Based on the vertical coordinates of the upper and lower boundary lines of the target's marker box, it is determined whether the target has reached the vicinity of the actual spray pixel line. When the vertical coordinate value of the upper boundary line is greater than the vertical coordinate value corresponding to the actual spray pixel line, and the vertical coordinate value of the lower boundary line is less than the vertical coordinate value corresponding to the actual spray pixel line, it can be considered that the target has moved to the designated position. By combining the horizontal coordinate values ​​of the left and right boundary lines of the target's marker box with the range of horizontal coordinate values ​​of the four spray control areas, the target sub-area to be sprayed can be determined.

[0078] This invention addresses the issues of target miss detection and response delay by dividing the area into several sub-areas to be sprayed based on the horizontal coordinate of the pixel line and determining whether the target has reached the vicinity of the actual spray pixel line based on the vertical coordinate, thereby improving the real-time accuracy of precise pesticide application.

[0079] In some embodiments, the spraying device includes at least one nozzle, each nozzle being equipped with a corresponding solenoid valve. Controlling the spraying device of the spraying system to perform pesticide application based on the target sub-area to be sprayed includes:

[0080] The location information of the target sub-area to be sprayed is sent to the spraying equipment, so that the spraying equipment opens the corresponding solenoid valve to perform the spraying action according to the location information of the target sub-area to be sprayed.

[0081] In this embodiment, serial communication is established between the host computer and the spraying equipment. Specifically, the host computer sends commands through serial communication, and the slave computer further controls the opening and closing state of the solenoid valve of the nozzle based on the target's regional location information.

[0082] In one example, to communicate with the host computer, the industrial control board pre-pairs the baud rate, data bits, parity bits, and stop bits of the UART serial port configuration module for information reception and parsing. The industrial control board determines the target area location based on the specific message information received. For example, when the industrial control board receives a hexadecimal flag bit 01, it considers the spraying mode to be activated; if it receives a flag bit 00, the spraying mode is not activated.

[0083] like Figure 3As shown, when the spraying equipment includes 4 nozzles, the pixel ranges of the 4 divided areas in the spraying mode are 0-160px, 160-320px, 320-480px, and 480-640px, respectively. If the target corresponding to each area reaches near the actual spray pixel line, the industrial control board will issue a control command to the spraying module according to the message information, and perform PWM (Pulse Width Modulation) adjustment on the four solenoid valves. The duty cycle of the PWM can be adjusted to control the opening and closing state of the solenoid valves, thereby realizing the spraying action.

[0084] In some embodiments, the sample image is obtained in the following manner:

[0085] Obtain raw sample images containing crops and weeds;

[0086] Determine the seed pixels in the original sample image, and segment the weed image from the original sample image based on the seed pixels;

[0087] Using the weed image as the object and the original sample image as the background, image synthesis is performed to obtain a sample image in which the number of weeds reaches a preset requirement.

[0088] It should be noted that, since the number of crops and weeds in the original sample images acquired by the image acquisition device may be unbalanced, this embodiment can use a region information-based segmentation method to segment the soil background and weeds in the image, and expand the segmented weed image to ensure the balance of the number of crops and weeds.

[0089] Specifically, a pixel is randomly selected from the image as a seed pixel, and pixels near the seed pixel are retrieved. Similar regions are aggregated, and the image segmentation is completed by traversing all pixels in the image using the region information segmentation method. Then, the segmented weed image is used as the "object," and other original sample images are used as the "background" to synthesize the image. This increases the number of weed samples in the image, resulting in a sample image with a preset number of weeds.

[0090] After obtaining the sample images, the LabelImg annotation software can be used to add bounding boxes to the crops and weeds in the sample images to obtain annotated sample images. The annotated sample images are then divided into training, testing, and validation sets according to a preset ratio to train the target detection model, thereby obtaining a trained target recognition model.

[0091] The embodiments of the present invention effectively solve the problem of imbalance in the sample ratio of crops and weeds by performing pixel-level data augmentation on the original sample images. This can improve the robustness of the target recognition model for crops and weeds and expand the applicability of the target recognition model for crops and weeds.

[0092] In some embodiments, the target recognition model includes an image input module, a feature extraction module, a feature fusion module, and a recognition module. The feature extraction module includes multiple cascaded first neural network units, the feature fusion module includes multiple cascaded second neural network units, and the recognition module includes a weighted loss function of VFL zoom loss function, DFL distributed focus loss function, and WIoU regression loss function.

[0093] The first neural network unit includes any one of the following: a residual feature learning C2f unit, a convolutional Conv unit, a residual feature learning C2f-N unit based on NAM attention mechanism, and a feature fusion SPPF unit.

[0094] The second neural network unit includes any one of the following: a convolutional Conv unit, a residual feature learning C2f-N unit based on the NAM attention mechanism, an image stitching Concat unit, and an image upsampling Upsample unit.

[0095] In this embodiment, reference Figure 4 The target recognition model includes an input image input module 201, a backbone feature extraction module 202, a neck feature fusion module 203, and a prediction recognition module 204.

[0096] The input image input module 201 is used to input RGB images with a resolution of 640×640. The backbone feature extraction module 202 includes multiple cascaded first neural network units. The neck feature fusion module 203 is built based on the PANet network and includes multiple cascaded second neural network units. The input of the current neural network unit is the output of the previous neural network unit, or the output of the previous neural network unit and the output of the previous N neural network units, where N is a positive integer greater than 1. The prediction module 204 includes a weighted loss function of VFL zoom loss function, DFL distributed focus loss function and WIoU regression loss function.

[0097] The residual feature learning (C2f) unit is used to learn residual features; the convolution (Conv) unit is used to perform convolution, normalization, and activation function calculation on the input image; the feature fusion (SPPF) unit is used to fuse features of input images of different sizes, enriching the expressive power of the output image and facilitating the detection of targets with large size differences at different growth stages; the image upsampling (Upsample) unit is used to upsample the input image; and the image stitching (Concat) unit is used to calculate the Concat function on the input image.

[0098] In this embodiment, to improve the target recognition accuracy of the target identification model, reduce model parameters to some extent, and improve the real-time detection speed, a residual feature learning C2f-N unit based on the NAM attention mechanism is proposed. This increases the feature extraction capability of the backbone network and provides a reliable source of front-end information for the feature fusion stage. The residual feature learning C2f-N unit based on the NAM attention mechanism is constructed based on the NAM attention mechanism module, which is used to suppress insignificant features. Specifically, refer to... Figure 5 The C2f-N unit consists of a ConvBNSiLU network layer 301, a Split network layer 302, n BottleNeck network layers 303, a NAM network layer 304, and a ConvBNSiLU network layer 305. In this embodiment, the number of BottleNeck network layers 303 can be flexibly adjusted according to the actual situation, and there is no limitation on this.

[0099] Optionally, in this embodiment, the feature extraction module 202 may be composed of at least three Conv units, at least three C2f units or C2f-N units and at least one SPFF unit cascaded together, and any two C2f units or C2f-N units are not adjacent, and the SPFF unit is the last unit in the feature extraction module 202.

[0100] It should be noted that, in this embodiment of the invention, the number and cascading method of Conv units, C2f units or C2f-N units and SPFF units in the Backbone feature extraction module 202 can be determined based on prior knowledge. In this embodiment of the invention, no specific limitation is made on the number and cascading method of Conv units, C2f units or C2f-N units and SPFF units in the Backbone feature extraction module 202.

[0101] Optionally, the Neck feature fusion module 203 may be composed of at least three Conv units, at least three C2f units or C2f-N units, multiple Concat units and Upsample units cascaded together, and any two C2f units or C2f-N units are not adjacent.

[0102] In this embodiment of the invention, the number and cascading method of Conv units, C2f units or C2f-N units, concat units and Upsample units in the Neck feature fusion module 203 can be determined based on prior knowledge. In this embodiment of the invention, no specific limitation is made on the number and cascading method of Conv units, C2f units or C2f-N units, Concat units and Upsample units in the Neck feature fusion module 203.

[0103] To facilitate understanding of the target recognition model in this embodiment of the invention, an example is provided below to illustrate the target recognition model in this embodiment. The connection relationships of each neural network unit in the target recognition model are shown in Table 1 and... Figure 4 As shown in Table 1, the model parameters of the target recognition model are shown in Table 1.

[0104] In Table 1, the note “n=2” for C2f unit 210 indicates that the number of BottleNeck modules in the internal structure of C2f unit is 2; the note “n=1” for C2f-N unit 214 in Table 1 indicates that the number of BottleNeck modules in the internal structure of C2f-N unit is 1.

[0105] Table 1. Structural Relationships and Model Parameters of the Target Recognition Model

[0106]

[0107] The notes “210, 217” for Concat unit 216 in Table 1 indicate that the input of Concat unit 216 is the output of Cf2 unit 210 and Upsample unit 217; the notes “212, 220” for Concat unit 219 in Table 1 indicate that the input of Concat unit 219 is the output of Cf2 unit 212 and Upsample unit 220.

[0108] Specifically, in this embodiment, inputting each frame of the image sequentially into the target recognition model to obtain the coordinates of the target's bounding box output by the target recognition model includes:

[0109] The image is input to the feature extraction module through the image input module to obtain the first feature map output by the first C2f unit, the second feature map output by the second C2f unit, and the third feature map output by the SPPF unit in the feature extraction module. The second C2f unit is a subordinate unit of the first C2f unit.

[0110] The first feature map, the second feature map, and the third feature map are sequentially input into the first Concat unit, the second Concat unit, and the Upsample unit in the feature fusion module to obtain the first feature matrix output by the first C2f-N unit, the second feature matrix output by the second C2f-N unit, and the third feature matrix output by the third C2f-N unit in the feature fusion module. The second Concat unit is a subordinate unit of the first Concat unit, the third C2f-N unit is a subordinate unit of the second C2f-N unit, and the second C2f-N unit is a subordinate unit of the first C2f-N unit.

[0111] The first feature matrix, the second feature matrix, and the third feature matrix are output to the recognition module. Crops and weeds are identified by a weighted loss function of the VFL zoom loss function, the DFL distributed focus loss function, and the WIoU regression loss function in the recognition module. Targets are marked in the crops and weeds by a label box containing spatial coordinate information.

[0112] In this embodiment, after the input image input module 201 inputs the current frame image into the Backbone feature extraction module 202, the Backbone feature extraction module 202 can extract crop and weed features from the current frame image. The Cf2 unit 210 (i.e., the first C2f unit) in the Backbone feature extraction module 202 can output the first feature map to the Concat unit 216 in the feature fusion module 203; the Cf2 unit 212 (i.e., the second C2f unit) in the feature extraction module 202 can output the second feature map to the Concat unit 219 in the feature fusion module 203; and the SPFF unit 215 in the feature extraction module 202 can output the third feature map to the Upsample unit 220 and the Concat unit 226 in the feature fusion module 203.

[0113] The C2f-N unit 221 (i.e., the first C2f-N unit) in the Neck feature fusion module 203 can output a first feature matrix to the loss function module 205 in the Prediction module 204. The C2f-N unit 224 (i.e., the second C2f-N unit) in the Neck feature fusion module 203 can output a second feature matrix to the loss function module 205. The C2f-N unit 227 (i.e., the third C2f-N unit) in the Neck feature fusion module 203 can output a third feature matrix to the loss function module 205. The dimensions of the first feature matrix are 80×80×45, the dimensions of the second feature matrix are 40×40×45, and the dimensions of the third feature matrix are 20×20×45.

[0114] The Loss function module 205 can identify target crops and weeds based on the first feature matrix, the second feature matrix, and the third feature matrix, using the WIoU+DFL regression loss function 228 and the VFL classification loss function 229. It can also mark the identified target crops and weeds in the current frame image using bounding boxes, and then output the spatial coordinate information of the marked crops and weeds.

[0115] The target drug delivery device provided by the present invention is described below. The target drug delivery device described below and the target drug delivery method described above can be referred to in correspondence.

[0116] Figure 6 This is a schematic diagram of the target drug delivery device provided by the present invention, as shown below. Figure 6As shown, the device includes: an image acquisition module 610, used to acquire multiple consecutive frames of images, which are obtained by an image acquisition device capturing images of a target within the current spraying area, the target being the object to be sprayed; a target recognition module 620, used to sequentially input each frame of the images into a target recognition model to obtain the position information of the target's bounding box output by the target recognition model; a target positioning module 630, used to construct horizontal actual spray pixel lines and vertical region division pixel lines in the images; and a spray control module 640, used to control the spray based on the position information of the target's bounding box and the actual spray... The position information of the pixel lines and the position information of the pixel lines in the region division control the spraying equipment to perform the spraying action; wherein, the target recognition model is obtained by training the target detection model on sample images labeled with crop tags and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss.

[0117] The target application device provided by this invention acquires multiple consecutive frames of images of the area to be sprayed by an image acquisition device; each frame is sequentially input into a target recognition model to obtain the position information of the target's bounding box output by the target recognition model, where the target is the object to be sprayed; horizontal actual spray pixel lines and vertical region division pixel lines are constructed in the image containing the target; the spraying device is controlled to perform the application action based on the position information of the target's bounding box, the position information of the actual spray pixel lines, and the position information of the region division pixel lines; wherein, the target recognition model is trained on a target detection model based on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model, with a NAM attention mechanism module added to the residual feature learning C2f unit of the YOLOv8 model, and the classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed as a weighted loss of the DFL distributed focal loss and the WIoU regression loss, thereby achieving accurate target identification and precise positioning in complex field environments, and realizing precise target application.

[0118] The targeted drug delivery system provided by this invention is described below, such as... Figure 7 As shown, a precision target delivery system based on an improved YOLOv8 includes a target identification and positioning processor 701 and a program or instruction that can run on the target identification and positioning processor.

[0119] It should be noted that the specific process by which the program or instructions running on the target identification and positioning processor are executed by the target identification and positioning processor using the above target identification and positioning method can be found in the contents of the above embodiments, and will not be repeated in the embodiments of the present invention.

[0120] Optionally, the target identification and localization processor described above can be an NVIDIA Jeston Xavier NX development board with a power consumption of 15W. The target identification and localization processor can execute the target identification and localization method described above based on the PyTorch 1.10.2 framework.

[0121] Based on the above embodiments, a precision target application system based on an improved YOLOv8 further includes: a power supply 702, an image acquisition device 703, a display device 704, and a spraying device 705; the power supply 702, image acquisition device 703, display device 703, and spraying device 705 are respectively connected to a target recognition and positioning processor 701. The spraying device 705 includes: a medicine tank 706, a pipeline 707, a nozzle 708, a spray bar 709, a pump 710, and a solenoid valve 711.

[0122] The power supply 702 is connected to the target identification and positioning processor 701 and the spraying device 705, thereby providing power to the target identification and positioning processor 701 and the spraying device 705.

[0123] The image acquisition device 703 is used to acquire video streams of targets planted in the area to be sprayed in real time, and send the video streams to the target recognition and positioning processor so that the target recognition and positioning processor can obtain the current frame image based on the video stream;

[0124] Display device 704 is used to receive and display the target area location information sent by target identification and positioning processor 701.

[0125] The spraying equipment 705 consists of a medicine tank 706, a pipeline 707, a nozzle 708, a spray bar 709, a pump 710, and a solenoid valve 711. After receiving the control signal output by the target identification and positioning processor 701, it controls the opening and closing state of the solenoid valve 711 to perform the spraying action.

[0126] The targeted application system of this invention improves the robustness of crop and weed identification, increases the accuracy of simultaneous crop and weed identification, improves the response speed of application actions, and enables precise targeted application.

[0127] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a target drug delivery method, which includes:

[0128] Acquire consecutive multi-frame images, which are obtained by image acquisition device from the target in the current spraying area, where the target is the object to be sprayed;

[0129] Each frame of the image is sequentially input into the target recognition model to obtain the position information of the target's bounding box output by the target recognition model;

[0130] Construct horizontal actual spray pixel lines and vertical region division pixel lines in the image;

[0131] The spraying device is controlled to perform the application action based on the position information of the target's marked box, the position information of the actual spray pixel line, and the position information of the region division pixel line.

[0132] The target recognition model is trained on a target detection model based on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss.

[0133] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the target drug delivery method provided by the methods described above, the method comprising:

[0135] Acquire consecutive multi-frame images, which are obtained by image acquisition device from the target in the current spraying area, where the target is the object to be sprayed;

[0136] Each frame of the image is sequentially input into the target recognition model to obtain the position information of the target's bounding box output by the target recognition model;

[0137] Construct horizontal actual spray pixel lines and vertical region division pixel lines in the image;

[0138] The spraying device is controlled to perform the application action based on the position information of the target's marked box, the position information of the actual spray pixel line, and the position information of the region division pixel line.

[0139] The target recognition model is trained on a target detection model based on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for targeted drug delivery, characterized in that, include: Acquire consecutive multi-frame images, which are obtained by image acquisition device from the target in the current spraying area, where the target is the object to be sprayed; Each frame of image is sequentially input into the target recognition model to obtain the position information of the target's bounding box output by the target recognition model; Based on the height set by the multiple nozzles of the spraying equipment and the viewing angle of the image acquired by the image acquisition device, horizontal actual spray pixel lines are pre-configured in the image, and the actual spray pixel lines are the horizontal lines corresponding to the multiple nozzles; vertical area division pixel lines are pre-configured in the image according to the sprayable range of each nozzle, so as to divide the image into spray control areas corresponding to each nozzle. Based on the position information of the pixel lines in the region division, the image is divided into several sub-regions to be sprayed; if the position of the actual spray pixel line is between the upper and lower boundary lines of the target's marker frame, then the target sub-region to be sprayed is determined according to the position of the left and right boundary lines of the target's marker frame; the spraying equipment of the spraying system is controlled to perform the spraying action according to the target sub-region to be sprayed. The target recognition model is trained on a target detection model based on sample images labeled with crop and weed tags. The target detection model is based on the YOLOv8 model. A NAM attention mechanism module is added to the residual feature learning C2f unit of the YOLOv8 model. The classification loss of the YOLOv8 model is set as the VFL zoom loss, and the regression loss of the YOLOv8 model is constructed by weighting the DFL distributed focal loss and the WIoU regression loss. The sample images were obtained in the following way: Obtain raw sample images containing crops and weeds; A pixel in the original sample image is randomly selected as a seed pixel, and pixels near the seed pixel are retrieved. Similar regions are aggregated, and all pixels in the original sample image are traversed using the region information segmentation method to segment the weed image from the original sample image. Using the weed image as the object and the original sample image as the background, image synthesis is performed to obtain a sample image in which the number of weeds reaches a preset requirement.

2. The method for targeted drug delivery according to claim 1, characterized in that, The spraying equipment includes at least one nozzle, and each nozzle is equipped with a corresponding solenoid valve. Controlling the spraying equipment of the spraying system to perform pesticide application based on the target sub-area to be sprayed includes: The location information of the target sub-area to be sprayed is sent to the spraying equipment, so that the spraying equipment opens the valve of the corresponding solenoid valve to perform the spraying action according to the location information of the target sub-area to be sprayed.

3. The method for targeted drug administration according to any one of claims 1 to 2, characterized in that, The target recognition model includes an image input module, a feature extraction module, a feature fusion module, and a recognition module. The feature extraction module includes multiple cascaded first neural network units, the feature fusion module includes multiple cascaded second neural network units, and the recognition module includes a weighted loss function of VFL zoom loss function, DFL distributed focus loss function, and WIoU regression loss function. The first neural network unit includes any one of the following: a residual feature learning C2f unit, a convolutional Conv unit, a residual feature learning C2f-N unit based on NAM attention mechanism, and a feature fusion SPPF unit. The second neural network unit includes any one of the following: a convolutional Conv unit, a residual feature learning C2f-N unit based on the NAM attention mechanism, an image stitching Concat unit, and an image upsampling Upsample unit.

4. The method for targeted drug delivery according to claim 3, characterized in that, The step of sequentially inputting each frame of the image into the target recognition model to obtain the coordinates of the target's bounding box output by the target recognition model includes: The image is input to the feature extraction module through the image input module to obtain the first feature map output by the first C2f unit, the second feature map output by the second C2f unit, and the third feature map output by the SPPF unit in the feature extraction module. The second C2f unit is a subordinate unit of the first C2f unit. The first feature map, the second feature map, and the third feature map are sequentially input into the first Concat unit, the second Concat unit, and the Upsample unit in the feature fusion module to obtain the first feature matrix output by the first C2f-N unit, the second feature matrix output by the second C2f-N unit, and the third feature matrix output by the third C2f-N unit in the feature fusion module. The second Concat unit is a subordinate unit of the first Concat unit, the third C2f-N unit is a subordinate unit of the second C2f-N unit, and the second C2f-N unit is a subordinate unit of the first C2f-N unit. The first feature matrix, the second feature matrix, and the third feature matrix are output to the recognition module. Crops and weeds are identified by a weighted loss function of the VFL zoom loss function, the DFL distributed focus loss function, and the WIoU regression loss function in the recognition module. Targets are marked in the crops and weeds by a label box containing spatial coordinate information.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the targeted drug delivery method as described in any one of claims 1 to 4.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the targeted drug delivery method as described in any one of claims 1 to 4.

7. A targeted drug delivery system, comprising: A target identification and positioning processor and a program or instructions that can run on the target identification and positioning processor, characterized in that, when the program or instructions are executed by the target identification and positioning processor, they implement the drug delivery method for the target as described in any one of claims 1 to 4.

8. The targeted drug delivery system according to claim 7, characterized in that, It also includes an image acquisition device and a spraying device; the image acquisition device and the spraying device are respectively connected to the target recognition and positioning processor; The image acquisition device is used to acquire video streams of targets planted in the area to be sprayed in real time, and send the video streams to the target recognition and positioning processor so that the target recognition and positioning processor can acquire multiple consecutive frames of images based on the video streams; The spraying device is used to perform the application of pesticides based on the control signals output by the target identification and positioning processor.

Images