Method and system for detecting smudginess and weeds of agricultural mechanical lens, pesticide spraying machine and medium

Abstract

The invention relates to the technical field of agricultural automation, and discloses an agricultural machinery lens smudginess and weed detection method and system, a pesticide spraying machine and a medium, and the method comprises the steps: obtaining a video shot by a camera device on an agricultural machinery in real time; weed detection is carried out on the video in the main thread, and shot smudginess judgment is carried out on the video in branches of the main thread; wherein the lens smudginess is judged to be an independent thread, and the thread and the main thread run at the same time; pesticide spraying work of the agricultural machinery is carried out according to a detection result of weed detection, and smudginess early warning is carried out according to a judgment result of lens smudginess judgment. According to the method, weed detection and lens smudginess judgment are carried out at the same time in a multi-branch asynchronous execution mode, and the spraying accuracy is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural automation, and in particular to a method and system for detecting dirt and weeds on agricultural machinery lenses, a pesticide sprayer, and a medium. Background Art

[0002] In agriculture, agricultural machinery is usually used to spray pesticides on weeds to carry out weed control. In order to avoid the sprayed pesticides affecting crops, cameras are installed to detect weeds. Pesticides are only sprayed when weeds are detected.

[0003] Agricultural automation operations usually take place outdoors. During operation, the lenses of agricultural machinery may be blocked due to weather and other reasons. However, existing agricultural machinery only considers weed detection and cannot cope with complex outdoor environments. Once the lens is blocked by dirt, weed detection will fail, resulting in missed spraying or continuous spraying. Summary of the Invention

[0004] In order to overcome the problem that the lens is blocked by dirt, which may lead to weed detection failure, spraying omissions or continuous spraying, the present invention provides a method, system, sprayer and medium for detecting dirt and weeds on agricultural machinery lenses.

[0005] In a first aspect, in order to solve the above technical problems, the present invention provides a method for detecting dirt and weeds on a lens of agricultural machinery, comprising:

[0006] Real-time acquisition of videos captured by cameras on agricultural machinery;

[0007] The main thread detects weeds in the video, and a branch of the main thread determines if the lens is dirty. The lens dirtiness detection process is a separate thread that runs concurrently with the main thread.

[0008] Agricultural machinery sprays pesticides based on the weed detection results, and lens contamination warnings are issued based on the lens contamination judgment results.

[0009] In a second aspect, the present invention provides a system for detecting dirt and weeds on a lens of agricultural machinery, comprising:

[0010] Video acquisition module, used to acquire videos shot by cameras on agricultural machinery in real time;

[0011] A detection module is used to detect weeds in the video in the main thread and to determine if the lens is dirty in the video in a branch of the main thread; the lens dirtiness determination is a separate thread and runs simultaneously with the main thread;

[0012] The spraying module is used to carry out the spraying work of agricultural machinery according to the detection results of weed detection, and to issue a contamination warning according to the judgment results of lens contamination.

[0013] In a third aspect, the present invention provides a system for detecting dirt and weeds on the lens of agricultural machinery, comprising: a camera device, a controller, a nozzle, an alarm and a communication module; wherein the controller is connected to the camera, and the controller is connected to the nozzle and the alarm respectively through the communication module, and the controller is used to execute the above-mentioned method for detecting dirt and weeds on the lens of agricultural machinery.

[0014] In a fourth aspect, the present invention provides a pesticide sprayer comprising the agricultural machinery lens dirt and weed detection system as described above.

[0015] In a fifth aspect, the present invention provides a computing device comprising a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the above-mentioned method for detecting dirt and weeds on the agricultural machinery lens are implemented.

[0016] The beneficial effects of the present invention are: acquiring video captured by a camera on agricultural machinery, performing weed detection and lens contamination determination in a main thread and a branch of the main thread, respectively; finally, using the weed detection results to initiate spraying, and using the lens contamination determination results to issue a contamination warning, thereby enabling the agricultural machinery to spray. This application employs a multi-branch asynchronous execution method to simultaneously perform weed detection and lens contamination determination, ensuring spraying accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention is further described below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 Schematic diagram of the flow of a method for detecting dirt and weeds on a lens of agricultural machinery according to an embodiment of the present invention;

[0019] Figure 2 This is a flow chart of a method for detecting dirt and weeds on a lens of agricultural machinery according to another embodiment of the present invention;

[0020] Figure 3 This is a schematic structural diagram of a system for detecting dirt and weeds on a lens of agricultural machinery according to an embodiment of the present invention;

[0021] Figure 4 Schematic diagram of the structure of a system for detecting dirt and weeds on agricultural machinery lenses according to another embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following examples are provided to further explain and supplement the present invention and do not constitute any limitation to the present invention.

[0023] The following describes, with reference to the accompanying drawings, a method, system, sprayer, and medium for detecting dirt and weeds on a lens of agricultural machinery according to embodiments of the present invention.

[0024] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting dirt and weeds on a lens of agricultural machinery, comprising:

[0025] S1. Real-time acquisition of video captured by a camera device on agricultural machinery.

[0026] S2. Detect weeds in the video in the main thread, and determine whether the lens is dirty in the video in a branch of the main thread; wherein, the lens dirtiness determination is a separate thread and runs simultaneously with the main thread.

[0027] S3. The agricultural machinery sprays pesticides based on the weed detection results, and issues a contamination warning based on the lens contamination judgment results.

[0028] In this embodiment, video captured by a camera on agricultural machinery is captured. Weed detection and lens contamination determination are performed in the main thread and branches of the main thread, respectively. Finally, the weed detection results are used to initiate spraying, while the lens contamination determination results are used to issue a contamination warning and initiate spraying. This application utilizes a multi-branch asynchronous execution method to simultaneously perform weed detection and lens contamination determination, ensuring accurate spraying.

[0029] Optionally, weed detection is performed on the video in the main thread, and lens dirtiness is determined in a branch of the main thread, including:

[0030] Parse the video to get image frames;

[0031] Extracting common features of the image frame to obtain a multi-dimensional feature map; wherein the multi-dimensional feature map includes a first feature map and a second feature map;

[0032] Perform weed detection on the first feature map in the main thread;

[0033] In the branch of the main thread, the second feature map is used to determine whether the lens is dirty.

[0034] In this embodiment, the main thread and branches are used to execute different tasks simultaneously, and weed detection and lens dirtiness judgment are processed asynchronously, thereby achieving efficient and low-computing-power weeding.

[0035] In this embodiment, the image frame is input into a shared feature layer (such as a shared convolutional neural network) to extract common features and obtain a multi-dimensional feature map. After the shared convolutional neural network extracts the common features of the image frame, two completely identical feature maps can be directly obtained using the common features. The two feature maps can be used by the main thread and the branch of the main thread respectively.

[0036] Optionally, extracting common features of the image frame to obtain a multi-dimensional feature map further includes:

[0037] Perform cropping, scaling, color space conversion, and image enhancement on the image frame to obtain a preprocessed image;

[0038] The common features are extracted from the preprocessed image to obtain a multi-dimensional feature map.

[0039] In this embodiment, the image frame is preprocessed, including cropping, scaling, color space conversion, and image enhancement, so as to enhance the features of the image frame, thereby making the subsequently extracted general features more accurate, thereby making the lens dirt judgment and weed detection more precise.

[0040] Optionally, performing lens dirtiness determination on the second feature map in a branch of the main thread includes:

[0041] Performing a dirtiness judgment on the second feature image to determine whether the lens is clean;

[0042] If it is judged to be dirty, the dirty area is extracted and the proportion of the dirty area to the total area of the second feature map is calculated.

[0043] In this embodiment, a fully connected layer (FCN Fully Connected Network) may be used to perform binary classification processing on the second feature map for dirtiness judgment. If the output is 1, it indicates clean, and if the output is 0, it indicates dirty.

[0044] In this embodiment, a threshold segmentation algorithm may be used to extract the dirty area and calculate the ratio of the dirty area to the total area of the second feature map, so as to determine whether the dirty area will affect the determination of weeds, thereby causing the spraying to fail.

[0045] Optionally, agricultural machinery can be sprayed based on the weed detection results, and a contamination warning can be issued based on the lens contamination judgment results, including:

[0046] If weeds are present, turn on the sprinkler;

[0047] If there are no weeds, turn off the sprinkler;

[0048] If the proportion of the dirty area is greater than or equal to a preset threshold, an alarm message is triggered, the agricultural machinery stops working, and waits for the lens to be cleaned before resetting and continuing to work.

[0049] In this embodiment, the preset threshold can be set according to actual conditions, such as 30% or 50%. That is, if the proportion of the dirty area to the total area of the second feature map reaches 30% or 50%, it indicates that the lens is too dirty and needs to be cleaned.

[0050] In this embodiment, the alarm information can be a buzzer alarm, thereby promptly reminding relevant personnel to perform lens cleaning work.

[0051] Optionally, weed detection is performed on the video in the main thread, and lens dirtiness is determined in a branch of the main thread, including:

[0052] The main thread performs weed detection on the video in real time, and the branch of the main thread performs a lens dirtiness judgment on the video every preset time or preset frame.

[0053] In this embodiment, a target detection model (such as YOLO) is used in the main thread to detect weeds in the image frame. If weeds are detected and the lens is clean or the ratio is lower than a preset threshold, the drug can be sprayed.

[0054] In this embodiment, lens dirt detection is executed periodically as an independent thread, while weed detection is performed in real time as the main thread. The two tasks are executed in parallel without interfering with each other. Furthermore, in relatively clean environments (such as sunny, windless, and muddy environments), the frequency of lens dirt detection can be reduced by adjusting the preset time or frame, thereby reducing power consumption.

[0055] In this embodiment, the preset time and the preset frame can be set according to actual conditions, for example, the lens dirtiness determination is performed every 30 minutes or every 300 frames.

[0056] like Figure 2 As shown, another embodiment of the method for detecting dirt and weeds on the lens of agricultural machinery is described as follows:

[0057] The video captured by the camera device on the agricultural machinery is obtained in real time, the video is parsed to obtain image frames, and the image frames are input as input images into the shared convolutional neural network.

[0058] Shared features (common features) are extracted in a shared convolutional neural network to obtain a multidimensional feature map, which includes a first feature map and a second feature map.

[0059] In the main thread, weed detection is performed on the first feature map to determine whether there are weeds.

[0060] If there are no weeds, the sprinkler is turned off.

[0061] If there are weeds, turn on the sprinkler and spray.

[0062] In the branch of the main thread, the second feature map is used to determine whether the lens is dirty or not.

[0063] If there is no dirt, continue working.

[0064] If there is dirt, the dirty area is extracted and the proportion of the dirty area to the total area of the second feature map is calculated (dirty area calculation).

[0065] If the ratio does not exceed the threshold, continue working.

[0066] If the ratio exceeds the threshold, a cleaning alarm will be issued, the machine will stop working, and will reset and continue working after the lens is cleaned.

[0067] like Figure 3 As shown, the present invention provides a system for detecting dirt and weeds on a lens of agricultural machinery, comprising:

[0068] Video acquisition module, used to acquire videos shot by cameras on agricultural machinery in real time;

[0069] A detection module is used to detect weeds in the video in the main thread and to determine if the lens is dirty in the video in a branch of the main thread; the lens dirtiness determination is a separate thread and runs simultaneously with the main thread;

[0070] The spraying module is used to carry out the spraying work of agricultural machinery according to the detection results of weed detection, and to issue a contamination warning according to the judgment results of lens contamination.

[0071] Optionally, the detection module is specifically used to: parse the video to obtain image frames; extract common features of the image frames to obtain a multidimensional feature map; wherein the multidimensional feature map includes a first feature map and a second feature map; perform weed detection on the first feature map in the main thread; and perform lens dirtiness judgment on the second feature map in a branch of the main thread.

[0072] Optionally, the detection module is specifically used to: perform cropping, scaling, color space conversion and image enhancement on the image frame to obtain a preprocessed image; and extract common features from the preprocessed image to obtain a multidimensional feature map.

[0073] Optionally, the detection module is specifically configured to: perform a dirtiness judgment on the second feature map to determine whether the lens is clean; if judged to be dirty, extract a dirty area and calculate a ratio of the dirty area to the total area of the second feature map.

[0074] Optionally, the spraying module is specifically used to:

[0075] If weeds are present, turn on the sprinkler;

[0076] If there are no weeds, turn off the sprinkler;

[0077] If the proportion of the dirty area is greater than or equal to a preset threshold, an alarm message is triggered, the agricultural machinery stops working, and waits for the lens to be cleaned before resetting and continuing to work.

[0078] Optionally, the detection module is specifically configured to: perform weed detection on the video in real time in the main thread, and perform lens dirtiness determination on the video at every preset time or preset frame in a branch of the main thread.

[0079] The present invention provides a system for detecting dirt and weeds on agricultural machinery lenses, comprising: a camera device, a controller, a nozzle, an alarm, and a communication module; wherein the controller is connected to the camera, and the controller is connected to the nozzle and the alarm respectively through the communication module, and the controller is used to execute the above-mentioned method for detecting dirt and weeds on agricultural machinery lenses. Figure 4 As shown, a spray boom is installed in front of the sprayer head, and a spray head is provided on the spray boom. A camera device is installed above the spray boom to capture picture frames and enter the controller through an interface for calculation and processing.

[0080] In this embodiment, the communication module may be a CAN bus, and the CAN signal is used to control the nozzle and the alarm.

[0081] In this embodiment, the alarm may be a buzzer.

[0082] In this embodiment, the imaging device may be a camera.

[0083] An embodiment of the present invention further provides a pesticide sprayer, comprising the agricultural machinery lens dirt and weed detection system as described above.

[0084] An embodiment of the present invention further provides a computer-readable storage medium storing instructions. When the instructions are executed on a terminal device, the terminal device executes the steps of the above-mentioned method for detecting dirt and weeds on agricultural machinery lenses.

[0085] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: in the form of complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may also be implemented in the form of a computer program product in one or more computer-readable media, the computer-readable media containing computer-readable program code. Computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof.

[0086] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0087] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for detecting dirt and weeds on agricultural machinery lenses, characterized in that: include: Real-time acquisition of videos captured by cameras on agricultural machinery; Performing weed detection on the video in the main thread, and performing lens dirtiness determination on the video in a branch of the main thread; wherein the lens dirtiness determination is a separate thread and runs simultaneously with the main thread; Agricultural machinery sprays pesticides based on the weed detection results, and lens contamination warnings are issued based on the lens contamination judgment results.

2. The method according to claim 1, characterized in that The main thread detects weeds on the video, and the branch of the main thread determines whether the lens is dirty on the video, including: Parsing the video to obtain image frames; Extracting common features of the image frame to obtain a multi-dimensional feature map; wherein the multi-dimensional feature map includes a first feature map and a second feature map; Perform weed detection on the first feature map in the main thread; In the branch of the main thread, the second feature map is used to determine whether the lens is dirty.

3. The method according to claim 2, characterized in that The step of extracting the common features of the image frame to obtain a multi-dimensional feature map further includes: Perform cropping, scaling, color space conversion, and image enhancement on the image frame to obtain a preprocessed image; The common features are extracted from the preprocessed image to obtain a multi-dimensional feature map.

4. The method according to claim 2, characterized in that The branch of the main thread performs lens dirtiness determination on the second feature map, including: Performing a dirtiness judgment on the second characteristic image to determine whether the lens is clean; If it is judged to be dirty, the dirty area is extracted, and the proportion of the dirty area to the total area of the second feature map is calculated.

5. The method according to claim 4, characterized in that The agricultural machinery spraying operation is performed based on the weed detection result, and the lens contamination warning is issued based on the lens contamination judgment result, including: If weeds are present, turn on the sprinkler; If there are no weeds, turn off the sprinkler; If the proportion of the dirty area is greater than or equal to a preset threshold, an alarm message is triggered, the agricultural machinery stops working, and waits for the lens to be cleaned before resetting and continuing to work.

6. The method according to any one of claims 1 to 5, characterized in that The weed detection on the video is performed in the main thread, and the lens dirtiness judgment on the video is performed in the branch of the main thread, including: The main thread performs weed detection on the video in real time, and the branch of the main thread performs lens dirtiness judgment on the video every preset time or preset frame.

7. Agricultural machinery lens dirt and weed detection system, characterized by: For implementing the method for detecting dirt and weeds on agricultural machinery lenses according to any one of claims 1 to 6, the system comprises: Video acquisition module, used to acquire videos shot by agricultural machinery in real time; a detection module, configured to perform weed detection on the video in a main thread and to perform lens dirtiness determination on the video in a branch of the main thread; wherein the lens dirtiness determination is a separate thread and runs concurrently with the main thread; The spraying module is used to carry out the spraying work of agricultural machinery according to the detection results of weed detection, and to issue a contamination warning according to the judgment results of lens contamination.

8. Agricultural machinery lens dirt and weed detection system, characterized by: include: A camera device, a controller, a nozzle, an alarm and a communication module; wherein the controller is connected to the camera, and the controller is connected to the nozzle and the alarm respectively through the communication module, and the controller is used to execute the agricultural machinery lens dirt and weed detection method according to any one of claims 1 to 6.

9. A pesticide sprayer comprising the agricultural machinery lens dirt and weed detection system according to claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a terminal device, cause the terminal device to execute the steps of the method for detecting dirt and weeds on an agricultural machinery lens as described in any one of claims 1 to 6.

Images