Lightweight throwing tool automatic unloading method and system based on computer vision

Abstract

The invention discloses an automatic unloading method and system for a lightweight throwing tool based on computer vision, and relates to the field of agricultural machinery automation. Agricultural harvesting site videos are collected through imaging equipment, and video data are obtained; receiving the video data through an image processing module, and analyzing the video data by using a pre-trained deep learning model to obtain position data of the transport vehicle; calculating adjustment pulse information of a material throwing tool according to the position data of the transport vehicle; and according to the adjusting pulse information of the material throwing tool, the position and posture of the material throwing tool are adjusted in real time through the discharging controller, and automatic discharging is achieved. The position of the material throwing tool can be automatically adjusted in real time to achieve accurate discharging, meanwhile, the computing power cost is reduced, the working efficiency of the harvester can be improved, dependence on manpower is reduced, and a solution is provided for intelligent transformation of small and medium-sized enterprises.

Description

Technical Field

[0001] The present invention relates to the field of agricultural machinery automation, and in particular to a method and system for automatically unloading a lightweight throwing tool based on computer vision. Background Art

[0002] In recent years, unmanned agricultural equipment technology has made significant progress, and the automatic unloading of harvesters is also moving towards unmanned operation. At present, the harvesting and unloading process mainly relies on the traditional manual control unloading mode, but some studies have begun to explore automatic unloading technology. At present, the research on automatic unloading mainly focuses on the fixed-point automatic unloading mode and the automatic unloading mode while harvesting.

[0003] The fixed-point unloading mode mainly relies on the harvester to set a fixed unloading point in advance. After reaching the given unloading point, the harvester automatically unloads, and then the harvester returns to harvest. Different unloading points must be set for different work locations, which limits the flexibility of the operation and affects the overall efficiency.

[0004] The automatic unloading while harvesting mode is that the transport vehicle always maintains a relative position with the harvester in operation. When the grain bin of the transport vehicle is full, it returns to the storage point, and the new transport vehicle continues to maintain a relative position with the harvester, and continues to unload while harvesting. The advantage of this mode is that the harvester can harvest without stopping, which improves the operation efficiency. However, most of the existing research relies on unmanned systems and various detection equipment, and cannot be promoted on a large scale.

[0005] Combining the traditional manual unloading mode with the automatic unloading mode while harvesting may be a new solution. With the development of deep learning and the use of image recognition technology, it has become a feasible solution to fix the image acquisition device to the thrower of the harvester to identify the transport vehicle in real time and adjust the position of the thrower according to the recognition results.

[0006] However, existing deep learning-based image recognition methods ensure algorithm accuracy by constructing complex models, which have the problems of high training costs and high model computing power requirements. Small and medium-sized enterprises usually lack high-end GPUs or dedicated AI acceleration hardware, and mostly use embedded devices for inference training. Under the condition of limited cost and computing power, existing methods are difficult to apply effectively. Therefore, it is of great practical significance and application value to design an image recognition technology that can still operate efficiently in an environment with limited cost and computing power, which does not require the use of unmanned systems to control harvesters and transport vehicles and can ensure automatic unloading while harvesting. Summary of the invention

[0007] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and system for automatic unloading of a lightweight throwing tool based on computer vision, which solves the problems of high cost, low real-time performance and high dependence on manual labor in the prior art.

[0008] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a method for automatically unloading a lightweight throwing tool based on computer vision, comprising the following steps: S1. Collect agricultural harvesting scene videos through imaging equipment to obtain video data; S2. Receive video data through an image processing module and analyze the video data using a pre-trained deep learning model to obtain position data of the transport vehicle; calculate adjustment pulse information of the throwing tool according to the position data of the transport vehicle; S3. According to the adjustment pulse information of the throwing tool, the posture of the throwing tool is adjusted in real time through the unloading controller to realize automatic unloading.

[0009] The present invention also provides a system for realizing the above-mentioned computer vision-based lightweight throwing tool automatic unloading method, comprising: Imaging equipment, used to collect agricultural harvesting site videos and obtain video data; An image processing module is used to receive video data and analyze the video data using a pre-trained deep learning model to obtain the position data of the transport vehicle; and calculate the adjustment pulse information of the throwing tool according to the position data of the transport vehicle; The unloading controller is used to adjust the posture of the throwing tool in real time according to the adjustment pulse information of the throwing tool to realize automatic unloading; Throwing tool, used for unloading.

[0010] The beneficial effects of the present invention are: 1. This invention proposes a deep learning model by reconstructing the YOLOV8 model, which not only significantly enhances the feature extraction capability and adaptability of the system, but also reduces the computational cost of the system. The alternately stacked ShuffleBlock and AugShuffleBlock structures in the backbone part make full use of the information in the middle layer of the feature map, avoid information waste, and promote cross-layer information mixing to improve model performance. At the same time, it reduces the optimization path length of the module part, can fully alleviate the risk of gradient disappearance and explosion inside the module, and can identify the transport vehicle in the agricultural harvesting scene faster and more accurately.

[0011] 2. All AugShuffleBlocks in the backbone of the deep learning model proposed in this invention use XBN (XBN-free batch normalization) technology to replace traditional batch normalization (BN), which effectively prevents the accumulation of estimation offsets in the network and reduces the performance degradation caused by distribution offsets during testing. This allows the model to maintain stable performance in different application scenarios, which is particularly important in highly variable environments such as agricultural harvesting scenarios.

[0012] 3. The neck of the deep learning model proposed in the present invention adopts a full-dimensional dynamic convolution layer (ODConv), which realizes multi-dimensional differentiated convolution operations by assigning different attention values ​​to the position of the convolution kernel, input channel, output channel and the overall convolution kernel, thereby greatly improving the feature capture capability of the convolution, which not only enriches the contextual information, but also improves the model's ability to handle complex backgrounds and target diversity.

[0013] 4. The present invention uses the video data provided by the imaging device, and the image processing module identifies the position of the transport vehicle and calculates the target position of the required throwing tool in real time. The unloading controller is used to accurately adjust the posture of the throwing tool according to the adjustment pulse, thereby realizing accurate and real-time throwing and improving the automation and accuracy of the entire operation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flow chart of the method proposed by the present invention; Figure 2 This is a schematic diagram of the deep learning model structure obtained after reconstructing YOLOV8; Figure 3 This is a schematic diagram of the ShuffleBlock (SFB) structure; Figure 4 This is a schematic diagram of the AugShuffleBlock (AugSFB) structure; Figure 5 It is the operation flow chart of the system proposed by the present invention; Figure 6 Schematic diagram of the system in the embodiment.

[0015] Among them, 1. transport vehicle; 2. imaging equipment; 3. throwing device; 4. image processing module; 5. display; 6. unloading controller; 7. y Axis control motor; 8. x Axis control motor. DETAILED DESCRIPTION

[0016] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0017] In one embodiment of the present invention, Figure 1 As shown, the present invention provides a method for automatically unloading a lightweight throwing tool based on computer vision, comprising the following steps: S1, collecting agricultural harvesting scene video through imaging device 2 to obtain video data; S2, receiving the video data through the image processing module 4 and analyzing the video data using the pre-trained deep learning model to obtain the position data of the transport vehicle 1; calculating the adjustment pulse information of the throwing tool 3 according to the position data of the transport vehicle 1; S3. According to the adjustment pulse information of the throwing tool 3, the posture of the throwing tool 3 is adjusted in real time through the unloading controller 6 to realize automatic unloading.

[0018] The deep learning model is a model obtained by reconstructing YOLOV8; in order to reduce the parameters of the model itself and ensure a certain feature extraction capability, the Darknet53 in the backbone of the YOLOV8 model is replaced with ShuffleBlock (shuffle block) and AugShuffleBlock (enhanced shuffle block) stacked three times alternately; and the BN in AugShuffleBlock is replaced with XBN (no batch normalization) to avoid the accuracy loss caused by the cumulative error when training BN; the convolution layer before the feature fusion in the neck of the YOLOV8 model is replaced with a 3×3 ODConv (full-dimensional dynamic convolution layer) to enhance the detail extraction capability before feature fusion and capture finer spatial features; at the same time, the C2f in the neck of the YOLOV8 model is replaced with a DWConv (deep separable convolution layer), which effectively reduces the amount of calculation and the number of parameters while retaining the feature extraction capability; the last two Concats in the neck of the YOLOV8 model are replaced with ADD (feature addition enhancement layer), which can increase the amount of information in each dimension without increasing the dimension of the image.

[0019] like Figure 2 As shown, the backbone of the deep learning model obtained after reconstructing YOLOV8 includes: a convolutional layer, a first shuffle block, a first enhanced shuffle block, a second shuffle block, a second enhanced shuffle block, a third shuffle block, and a third enhanced shuffle block connected in sequence; the input of the convolutional layer is the input of the entire deep learning model.

[0020] The neck of the deep learning model includes: a first full-dimensional dynamic convolution layer, a first upsampling layer, a first splicing layer, a first depth-separable convolution layer, a second full-dimensional dynamic convolution layer, a second upsampling layer, a second splicing layer, a second depth-separable convolution layer, a third depth-separable convolution layer, a first feature-added enhancement layer, a fourth depth-separable convolution layer, a fifth depth-separable convolution layer, a second feature-added enhancement layer, and a sixth depth-separable convolution layer, which are sequentially connected; Among them, the input end of the first full-dimensional dynamic convolution layer is connected to the output end of the third enhancement shuffle block; the other input end of the first splicing layer is connected to the output end of the second enhancement shuffle block; the other input end of the second splicing layer is connected to the output end of the first enhancement shuffle block; the other input end of the first feature incremental enhancement layer is connected to the output end of the second full-dimensional dynamic convolution layer; the other input end of the second feature incremental enhancement layer is connected to the output end of the first full-dimensional dynamic convolution layer; the outputs of the second depth-separable convolution layer, the fourth depth-separable convolution layer, and the sixth depth-separable convolution layer are respectively used as inputs of the deep learning model head.

[0021] Through the above reconstruction of the YOLOV8 model, the model can be made more lightweight while ensuring a certain accuracy, suitable for deployment on low-cost computing devices, and reducing the cost pressure of the enterprise.

[0022] like Figure 3 , Figure 4 As shown in the figure, the specific steps of extracting image features using the backbone of the deep learning model for the data to be analyzed are: B11, through the Shuffleblock (shuffle block) input feature map , Divided along the channel dimension Two parts, the number of channels in each part is ; B12, Yes Perform 1×1 point-by-point convolution to obtain , whose expression is ;right Perform a 3×3 depth-wise separable convolution with a step size of 2, and we get , whose expression is ;right Use 1×1 point-by-point convolution and adjust the number of channels to get , whose expression is ; B13, Yes Perform a 3×3 depth-separable convolution to obtain , whose expression is ;right Perform 1×1 point-by-point convolution and adjust the number of channels to obtain , whose expression is ; B14. and Put together and get , whose expression is ;right Shuffle the channels so that the channels from different parts are staggered to obtain a new feature map , whose expression is ; B21, through AugShuffleBlock (enhanced shuffle block) , Further feature extraction is performed; the feature map Divide into according to the set split ratio r (less than 0.5) Two parts; r is an adjustable parameter used to control the computational overhead and redundancy of the module; B22, yes Perform 1×1 point-by-point convolution to obtain , whose expression is , The number of channels is still ;right Perform a 3×3 depth convolution with a step size of 2 to get , whose expression is ; B23, yes Perform channel crossover to obtain , whose expression is ; B24, yes Perform channel crossover to obtain , whose expression is ; B25, will and Splice and get , whose expression is ; B26, Yes Perform 1×1 point-by-point convolution and adjust the number of channels to obtain , whose expression is ; B27, will and Splice and get ; Its expression is ; B28, will and Splice and get , whose expression is ;right Shuffle the channels and get , whose expression is .

[0023] In the neck structure of the deep learning model, each time a feature map passes through ODConv, the following operations are performed on the input feature map:

[0024] in, represents the input of ODConv, represents the output of ODConv, is the convolution kernel The attention scalar, is the newly introduced attention mechanism, along the convolution kernel Calculation of spatial dimensions and input channel dimensions; Represents the attention mechanism along the output channel dimension. The deep learning model enhances the representation ability of the backbone network by fusing the features of the bottom-up and top-down paths, avoiding the rigid matching of the target size and network depth.

[0025] In this embodiment, the specific steps of training the deep learning model include: A1. Use imaging equipment to collect agricultural harvesting site videos, determine whether the data collected by the imaging equipment meets the requirements (whether the object is captured, whether the positive and negative samples are balanced, whether there is severe blur or occlusion, etc.), and use the data that meets the requirements as the original data; A2. Set the size of each frame image in the original data to 320×320 to obtain data in a unified format; A3. After manually annotating the data obtained in the unified format, data enhancement techniques such as random rotation, changing saturation, changing brightness, horizontal flipping, adding Gaussian noise, and adding salt and pepper noise are used to increase the diversity of images and expand the number of image samples to obtain a training set. A4. Input the training set into the deep learning model to obtain the target location data, i.e., the location data of transport vehicle 1; A5. Calculate the final detection loss; its calculation expression is:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] in is the final detection loss, is the CIoU loss of the bounding box, is the cross entropy loss of the object category, Binary cross entropy loss for object scores; , , is the weight factor assigned to each loss term; the center coordinates and size of the real box are , represents the central horizontal coordinate of the real frame, Represents the central ordinate of the real frame, Indicates the width of the real box, Represents the height of the real box; the center coordinates and size of the predicted box are , Represents the central horizontal coordinate of the prediction box, Represents the central ordinate of the prediction box, Indicates the width of the prediction box, Indicates the height of the prediction box; Represents the sum of the widths of the real boxes; Represents the sum of the heights of the prediction boxes; is the balance parameter; is a parameter used to measure the consistency of aspect ratio; c is the number of categories, is the category in the true label c The indicator variable, For Category c The predicted probability of is the true label, is the predicted score; Used to measure the overlap between the predicted box and the true box. is the bounding box regression loss function; is the area of ​​the overlap between the real box and the predicted box; is the ratio of pi; tan is the tangent trigonometric function; A6. Use the calculated final detection loss for back propagation and use the Adam optimizer to update the parameters of the deep learning model to reduce the value of the loss function. A7. Determine whether the loss function no longer decreases or whether the accuracy of the deep learning model has not been significantly improved. If so, end the training; otherwise, return to step A4.

[0033] In order to further verify the advantages of the deep learning model described in the present invention compared with other existing mainstream models, comparative experiments were conducted, including YOLOv3-tiny, YOLOv5-n, YOLOv6-n, YOLOv7-tiny, YOLOv8-n and SSD300. The experimental results are shown in Table 1: Table 1

[0034] Experimental results show that the number of parameters and the amount of calculation required for the deep learning model proposed in this invention are smaller than those of other mainstream models, which greatly reduces the demand for computing resources and enables the solution to be carried out smoothly under low computing power and low cost conditions.

[0035] The present invention also provides a system for realizing the above-mentioned computer vision-based lightweight throwing tool 3 automatic unloading method, comprising an imaging device 2, which is mostly a high-definition camera, used to collect agricultural harvesting scene videos to obtain video data; an image processing module 4, used to receive video data and analyze the video data using a pre-trained deep learning model to obtain position data of the transport vehicle 1; calculate the adjustment pulse information of the throwing tool 3 according to the position data of the transport vehicle 1; a unloading controller 6, used to adjust the posture of the throwing tool 3 in real time according to the adjustment pulse information of the throwing tool 3 to realize automatic unloading; the throwing tool 3 is used for unloading.

[0036] In this embodiment, if Figure 6 As shown, the imaging device 2 is installed 1m away from the discharge port of the throwing tool 3 to ensure that its field of view can cover all areas that the transport vehicle 1 may reach, and the angle between the imaging device 2 and the throwing tool 3 is set between 30° and 45°. In other embodiments, it can be adjusted according to the specifications of different throwing tools 3. The imaging device 2 is calibrated before installation to ensure the consistency and accuracy of the image quality. In this embodiment, a display 5 is also included for displaying real-time video and warning information. After the imaging device 2 collects the agricultural field operation video, it transmits the video to the image processing module 4. The image processing module 4 is mainly composed of an MPU chip, a control board, a high-speed storage unit, a RAM and other devices. It is mainly responsible for identifying the position of the transport vehicle 1 in the video data, inferring the displacement of the throwing tool 3 and transmitting the processed video to the display 5. After the lightweight deep learning model in the image processing module 4 recognizes the position of the transport vehicle 1, it immediately calculates the target position of the throwing tool 3 and then uses the PID algorithm to calculate x Axis motor 8 and y The adjustment pulse information of the axis motor 7, when the pulse information meets the mechanical requirements, sends a discharge signal to the discharge controller 6; the discharge controller 6 receives the discharge request from the image processing module 4 according to x Axis motor 8 and y The adjustment information of the shaft motor 7 adjusts the posture of the throwing device 3, and the unloading operation is performed after the adjustment is completed. If the transport vehicle 1 is not recognized for a period of time, a stop unloading signal is sent to the unloading controller 6, and a warning message is displayed on the display 5, and the system stops working. When the system is in operation, it will continue to receive new image inputs from the imaging device 2 and respond quickly.

[0037] The specific working process of the system proposed by the present invention is as follows: Figure 5 As shown, including: C1. Start and initialize system settings; C2. After initialization is completed, check whether the imaging device 2 is available. If yes, use the imaging device 2 to collect raw data and proceed to step C4. Otherwise, proceed to step C3. C3, output the reason for unavailability and shut down the system; C4, loading the trained deep learning model, checking whether the deep learning model is loaded, if yes, proceed to step C5, otherwise proceed to step C3; C5, check whether the original data collected by the imaging device is available, if yes, go to step C6, otherwise return to step C1; C6. Using the raw data collected by the imaging device as the input of the image processing module 4, and using the deep learning model to perform target recognition on the raw data to obtain a recognition result; C7, check the recognition result: if the transport vehicle 1 is not recognized, go to step C8; if the transport vehicle 1 is recognized, identify the center point of the transport vehicle 1; in order to facilitate the uniform unloading of the throwing device 3 to the transport vehicle 1, the transport vehicle 1 is divided into M Areas, and set the limit time for each area to , record the current time and go to step C9; C8, determine whether n identifications have been performed and the transport vehicle 1 has not been identified, if yes, send a stop unloading request, output a warning message and shut down the system; otherwise, return to step C6; C9. Calculate the target position of the throwing device 3 according to the current position of the transport vehicle 1. , and determine whether the area where the target center point of the throwing tool 3 is located meets the time limit, if yes, then go to step C10; otherwise, update the unloading area of ​​the transport vehicle, and calculate the target position of the throwing tool according to the current position of the transport vehicle; The time limits are:

[0038] The constraints are:

[0039] ,

[0040] in, N It means that the total time T of the whole unloading process is divided into N The duration of each period is ; M Indicates the number of divided areas for the transport vehicle, ; Indicated in i During the time period, transport vehicle 1 j The actual length of stay in each area; Indicates that transport vehicle 1 is in j Expected duration of stay in each area; Indicates minimization.

[0041] C10, use PID algorithm to calculate the time required for the throwing tool 3 to adjust from the current position to the target position x Axis motor and y The adjustment pulse of the axis motor; its expression is:

[0042]

[0043]

[0044] in, Indicates the current position coordinates of the throwing tool. Indicates the target position coordinates of the throwing tool; Indicates time, express The target position and current position at the moment x Axis error, express The target position and current position at the moment y Axis error; for time x Axis adjustment information, for time y Axis adjustment information; , They are x axis, y Proportional gain of the axis; , They are x axis, y The integral gain of the axis; , They are x axis, y The differential gain of the axis; Indicates time, .

[0045] C11. Determine whether the calculated adjustment pulse meets the mechanical condition limit of the motor (the rotation angle of the throwing tool 3 is different for different types of transport vehicles 1, that is, x Axis motor and y The maximum and minimum pulses of the axis motor are different), if yes, enter C12; otherwise, output a warning message and shut down the system; C12, outputting a unloading request to the unloading controller 6. When the unloading controller 6 receives the unloading request, it immediately adjusts the posture of the throwing device 3 according to the received pulse information and performs the throwing operation; repeating steps C6-C7.

[0046] In summary, the present invention has simple deployment, low computing power requirements, low learning costs and good detection effect.

Claims

1. A method for automatic unloading of lightweight throwing tools based on computer vision, characterized in that: The following steps are involved: S1. Collect agricultural harvesting scene videos through imaging equipment to obtain video data; S2. Receive video data through an image processing module and analyze the video data using a pre-trained deep learning model to obtain position data of the transport vehicle; calculate adjustment pulse information of the throwing tool according to the position data of the transport vehicle; S3. According to the adjustment pulse information of the throwing tool, the posture of the throwing tool is adjusted in real time through the unloading controller to realize automatic unloading.

2. According to the computer vision-based lightweight throwing tool automatic unloading method of claim 1, it is characterized in that: The deep learning model is an optimized YOLOV8 model; the Darknet53 of the backbone part of the YOLOV8 model is replaced with alternately stacked ShuffleBlock and AugShuffleBlock; the BN in AugShuffleBlock is replaced with XBN; the convolution layer before the neck feature fusion of the YOLOV8 model is replaced with 3×3 ODConv, C2f is replaced with DWConv, and the last two Concats are replaced with ADD.

3. The method for automatically unloading a lightweight throwing tool based on computer vision according to claim 2 is characterized in that: The backbone of the deep learning model obtained after reconstructing YOLOV8 includes: a convolutional layer, a first shuffle block, a first enhanced shuffle block, a second shuffle block, a second enhanced shuffle block, a third shuffle block, and a third enhanced shuffle block connected in sequence; the input of the convolutional layer is the input of the entire deep learning model.

4. The method for automatically unloading a lightweight throwing tool based on computer vision according to claim 3 is characterized in that: The neck of the deep learning model obtained after reconstructing YOLOV8 includes: a first full-dimensional dynamic convolution layer, a first upsampling layer, a first splicing layer, a first depth-separable convolution layer, a second full-dimensional dynamic convolution layer, a second upsampling layer, a second splicing layer, a second depth-separable convolution layer, a third depth-separable convolution layer, a first feature-added enhancement layer, a fourth depth-separable convolution layer, a fifth depth-separable convolution layer, a second feature-added enhancement layer, and a sixth depth-separable convolution layer, which are sequentially connected; Wherein, the input end of the first full-dimensional dynamic convolutional layer is connected to the output end of the third enhanced shuffle block; Another input end of the first splicing layer is connected to the output end of the second enhanced shuffle block; Another input end of the second splicing layer is connected to the output end of the first enhanced shuffle block; Another input end of the first feature gradual enhancement layer is connected to the output end of the second full-dimensional dynamic convolution layer; Another input end of the second feature gradual enhancement layer is connected to the output end of the first full-dimensional dynamic convolution layer; The outputs of the second depth-wise separable convolutional layer, the fourth depth-wise separable convolutional layer, and the sixth depth-wise separable convolutional layer are respectively used as inputs to the head of the deep learning model.

5. The method for automatically unloading a lightweight throwing tool based on computer vision according to claim 4 is characterized in that: The specific steps for training a deep learning model include: A1. Use imaging equipment to collect agricultural harvesting site videos and obtain raw data; A2. Set the size of each frame of the original data to a×a to obtain data in a unified format; A3. After manually labeling the data in a unified format, the number of image samples is expanded through data enhancement technology to obtain a training set; A4. Input the training set into the deep learning model to obtain the target location data, i.e., the transport vehicle location data; A5. Calculate the final detection loss; its calculation expression is: in is the final detection loss, is the CIoU loss of the bounding box, is the cross entropy loss of the object category, Binary cross entropy loss for object scores; , , is the weight factor assigned to each loss term; the center coordinates and size of the real box are , represents the central horizontal coordinate of the real frame, Represents the central ordinate of the real frame, Indicates the width of the real box, Represents the height of the real box; the center coordinates and size of the predicted box are , Represents the central horizontal coordinate of the prediction box, Represents the central ordinate of the prediction box, Indicates the width of the prediction box, Indicates the height of the prediction box; Represents the sum of the widths of the real boxes; Represents the sum of the heights of the prediction boxes; is the balance parameter; is a parameter used to measure the consistency of aspect ratio; c is the number of categories, is the category in the true label c The indicator variable, For Category c The predicted probability of is the true label, is the predicted score; Used to measure the overlap between the predicted box and the true box. is the bounding box regression loss function; is the area of ​​the overlap between the real box and the predicted box; is the ratio of pi; tan is the tangent trigonometric function; A6. Use the calculated final detection loss for back propagation and use the Adam optimizer to update the parameters of the deep learning model to reduce the value of the loss function. A7. Determine whether the loss function no longer decreases or whether the accuracy of the deep learning model has not been significantly improved. If so, end the training; otherwise, return to step A4.

6. The method for automatically unloading a lightweight throwing tool based on computer vision according to claim 2, characterized in that: The specific steps to extract image features using the backbone of the deep learning model are: B11, through the Shuffleblock feature map Divided along the channel dimension Two parts; B12, Yes Perform 1×1 point-by-point convolution to obtain , whose expression is ;right Perform a 3×3 depth-wise separable convolution with a step size of 2, and we get , whose expression is ;right Using 1×1 point-wise convolution, we get , whose expression is ; B13, Yes Perform a 3×3 depth-separable convolution to obtain , whose expression is ;right Perform 1×1 point-by-point convolution to obtain , whose expression is ; B14. and Put together and get , whose expression is ;right Shuffle the channels to get a new feature map , whose expression is ; B21, through AugShuffleBlock Further feature extraction is performed; the feature map According to the set split ratio r, Two parts, where r is an adjustable parameter; B22, yes Perform 1×1 point-by-point convolution and adjust the number of channels to obtain , whose expression is ;right Perform a 3×3 depth convolution with a step size of 2 to get , whose expression is ; B23, yes Perform channel crossover to obtain , whose expression is ; B24, yes Perform channel crossover to obtain , whose expression is ; B25, will and Splice and get , whose expression is ; B26, Yes Perform 1×1 point-by-point convolution and adjust the number of channels to obtain , whose expression is ; B27, will and Splice and get ; Its expression is ; B28, will and Splice and get , whose expression is ;right Shuffle the channels and get , whose expression is .

7. A system for automatic unloading of lightweight throwing tools based on computer vision according to any one of claims 1 to 6, characterized in that: include: Imaging equipment, used to collect agricultural harvesting site videos and obtain video data; An image processing module is used to receive video data and analyze the video data using a pre-trained deep learning model to obtain the position data of the transport vehicle; and calculate the adjustment pulse information of the throwing tool according to the position data of the transport vehicle; The unloading controller is used to adjust the posture of the throwing tool in real time according to the adjustment pulse information of the throwing tool to realize automatic unloading; Throwing tool, used for unloading.

8. The computer vision-based lightweight throwing tool automatic unloading system according to claim 7, characterized in that: The specific working steps of the system are: C1. Start and initialize system settings; C2. After initialization is completed, check whether the imaging device is available. If yes, use the imaging device to collect raw data and proceed to step C4. Otherwise, proceed to step C3. C3, output the reason for unavailability and shut down the system; C4, loading the trained deep learning model, checking whether the deep learning model is loaded, if yes, proceed to step C5, otherwise proceed to step C3; C5, check whether the original data collected by the imaging device is available, if yes, go to step C6, otherwise return to step C1; C6. Use the raw data collected by the imaging device as the input of the image processing module, use the deep learning model to perform target recognition on the raw data, and obtain the recognition result; C7, check the recognition result: if the transport vehicle is not recognized, go to step C8; if the transport vehicle is recognized, classify the transport vehicle into M area, record the current time and go to step C9; C8, determine whether n identifications have been performed and no transport vehicle has been identified, if yes, send a stop unloading request, output a warning message and shut down the system; otherwise, return to step C6; C9. Calculate the target position of the throwing device according to the current position of the transport vehicle , and determine whether the area where the target center point of the throwing tool is located meets the time limit, if yes, proceed to step C10; otherwise, update the unloading area of ​​the transport vehicle, and calculate the target position of the throwing tool according to the current position of the transport vehicle; C10, using PID algorithm to calculate the adjustment pulse required for the throwing tool to adjust from the current position to the target position; C11, determine whether the calculated adjustment pulse meets the mechanical condition limit, if yes, enter C12; otherwise, output a warning message and shut down the system; C12, output the unloading request to the unloading controller, and the unloading controller controls the throwing device to automatically adjust the posture and perform the throwing operation; repeat steps C6-C7.

9. The computer vision-based lightweight throwing tool automatic unloading system according to claim 8, characterized in that: The expression of the time limit described in step C9 is: The constraints are: ， in, N It means that the total time T of the whole unloading process is divided into N The duration of each period is ; M Indicates the number of divided areas for the transport vehicle; Indicated in i During the period, the transport vehicle j The actual length of stay in each area; Indicates that the transport vehicle is j Expected duration of stay in the area; Indicates minimization.

10. The computer vision-based lightweight throwing tool automatic unloading system according to claim 8, characterized in that: The PID algorithm is used to calculate the adjustment pulse required for the throwing tool to adjust from the current position to the target position. The expression is: in, Indicates the current position coordinates of the throwing tool. Indicates the target position coordinates of the throwing tool; Indicates time, express The target position and current position at the moment x Axis error, express The target position and current position at the moment y Axis error; for time x Axis adjustment information, for time y Axis adjustment information; , They are x axis, y Proportional gain of the axis; , They are x axis, y The integral gain of the axis; , They are x axis, y The differential gain of the axis; Indicates time, .

Images