Weed identification method and device based on combined network

Abstract

The invention discloses a weed identification method and device based on a combined network. The method comprises the following steps: acquiring an original image of a farmland area; identifying a crop region from the original image by adopting a semantic segmentation model of a pre-trained combined model; processing the original image by adopting a preset image processing mode corresponding to the weeding mode to obtain a to-be-processed image; and identifying a weed region in the to-be-processed image by adopting an image classification model in the combined model. The technical problem of relatively low weed recognition accuracy in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of weed identification, and in particular to a weed identification method and device based on a combined network. Background Art

[0002] Weeds compete with crops for water, nutrients and light in the field environment, breed pests and diseases, and lead to crop yield reduction or even crop failure. There are many common weed species in the field, more than 140 species, which brings great challenges to agricultural production. Traditional weed control methods include manual weeding, chemical weeding, biological weeding and mechanical weeding. Among them, manual weeding is labor-intensive and inefficient, and with the shortage of rural labor and the increase in labor costs, it can no longer meet the needs of modern agriculture. Although chemical weeding is effective quickly, excessive use will cause environmental pollution and ecological damage. Although biological weeding is environmentally friendly, it is not suitable for sudden weed damage. Mechanical weeding, as a green and pollution-free weeding method, has received widespread attention and application. However, the effect of mechanical weeding is highly dependent on the accurate identification of weeds, otherwise it is easy to accidentally damage crops and affect yield. In practical applications, weed recognition faces the following challenges: 1) There are many types of weeds: It is extremely challenging, costly and time-consuming to establish an image dataset containing all weed species. 2) High similarity between weeds and crops: Weeds and crops are often similar in shape and color, and traditional recognition algorithms are difficult to accurately distinguish, resulting in limited recognition accuracy. 3) Small targets and high-density weeds: For densely distributed small weeds, existing deep learning models (such as semantic segmentation networks or object detection networks) have low recognition accuracy.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a weed identification method and device based on a combined network, so as to at least solve the technical problem of low weed identification accuracy in the related art.

[0005] According to one aspect of an embodiment of the present application, a weed identification method based on a combined network is provided, comprising: collecting an original image of a farmland area; identifying a crop area from the original image using a semantic segmentation model of a pre-trained combined model; processing the original image using a preset image processing method corresponding to a weeding method to obtain an image to be processed; and identifying a weed area in the image to be processed using an image classification model in the combined model.

[0006] Optionally, a semantic segmentation model of a pre-trained combined model is used to identify crop areas from the original image, including: using the semantic segmentation model to identify the original image to obtain a segmentation result, wherein the segmentation result is used to indicate the crop area and non-crop area in the original image; generating a binary mask based on the segmentation result, wherein the area marked as a valid value in the binary mask is the crop area, and the area marked as an invalid value in the binary mask is the non-crop area.

[0007] Optionally, the original image is processed using a preset image processing method corresponding to the weeding method to obtain an image to be processed, including: dividing the original image into multiple grid areas, and the area covered by each grid area is determined according to the working area of ​​the weeding equipment; processing the grid areas in the original image based on the weeding method to obtain the image to be processed; and determining the weed area from the image to be processed.

[0008] Optionally, determining the weed area from the image to be processed includes: determining whether each grid area contains the crop area based on coordinate information of the crop area and the non-crop area; when the weeding method is mechanical weeding, removing the grid area containing the crop area in the original image to obtain the image to be processed; using the image classification model to identify the image to be processed, determining the area containing plants as the weed area, and performing a weeding operation.

[0009] Optionally, determining the weed area from the image to be processed includes: determining whether each grid area contains the crop area based on coordinate information of the crop area and the non-crop area; when the weeding method is chemical weeding, setting the pixel value of the grid area containing the crop area in the original image to a preset value to obtain the image to be processed; using the image classification model to identify the image to be processed, determining the area containing plants as the weed area, and performing a weeding operation.

[0010] Optionally, the semantic segmentation model is determined in the following manner, including: acquiring a first data set, the first data set including: crop images in different environments and under different lighting conditions, the crop images all having their contours annotated; using the first data set to train an initial semantic segmentation model until the initial semantic segmentation model converges to obtain a converged semantic segmentation model, and using a first verification data set to verify the performance of the converged semantic segmentation model; if the converged semantic segmentation model passes the verification, determining the converged semantic segmentation model as the semantic segmentation model.

[0011] Optionally, the image classification model is determined in the following manner, including: acquiring a second data set, the second data set including: positive samples and negative samples, wherein the positive samples include a variety of plant images, and the negative samples are background images; using the second data set to train the initial image classification model until the initial image classification model converges to obtain a converged image classification model, and using a second verification data set to verify the classification accuracy of the converged image classification model; if the converged image classification model passes the verification, determining the converged image classification model as the image classification model.

[0012] According to another aspect of an embodiment of the present application, a weed identification device based on a combined network is also provided, including: an acquisition module, used to acquire original images of a farmland area; a first identification module, used to identify a crop area from the original image using a semantic segmentation model of a pre-trained combined model; a processing module, used to process the original image using a preset image processing method corresponding to a weeding method to obtain an image to be processed; and a second identification module, used to identify a weed area in the image to be processed using an image classification model in the combined model.

[0013] According to another aspect of the embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the above-mentioned weed identification method based on the combined network.

[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, wherein the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned weed identification method based on the combined network by running the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned weed identification method based on the combination network when executed by a processor.

[0016] In the embodiment of the present application, an original image of a farmland area is collected; a semantic segmentation model of a pre-trained combined model is used to identify a crop area from the original image; the original image is processed by a preset image processing method corresponding to a weeding method to obtain an image to be processed; an image classification model in the combined model is used to identify a weed area in the image to be processed. The crop area is first identified by the combined model, and then weeds are identified in non-crop areas, thereby avoiding the direct identification of a wide variety of weeds, thereby achieving the technical effect of high weed identification accuracy, and further solving the technical problem of low weed identification accuracy in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a weed identification method based on a combined network according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a weed identification method based on a combined network according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an original image according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of dividing an original image into multiple grid areas according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of a weed area according to an embodiment of the present application;

[0023] Figure 6 is another schematic diagram of a weed area according to an embodiment of the present application;

[0024] Figure 7 It is a schematic diagram of the structure of a weed identification device based on a combined network according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In the related art, weeds are directly identified, and there are many types of weeds. The traditional weed recognition algorithm needs to directly distinguish between the two, which is difficult and has a technical problem of low weed recognition accuracy. In order to solve the problems existing in the related art, the embodiment of the present application provides a weed recognition method based on a combined network, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0028] The weed identification method embodiment based on the combined network provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a weed identification method based on a combined network. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0029] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the combined network-based weed identification method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the combined network-based weed identification method described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0033] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0034] In the above operating environment, the embodiment of the present application provides an embodiment of a weed identification method based on a combined network. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that herein.

[0035] Figure 2 is a weed identification method based on a combined network according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0036] Step S202, collecting original images of the farmland area;

[0037] In step S202, a camera or a visual system of an intelligent weeding equipment may be used to collect original images of the agricultural area.

[0038] Step S204, using a pre-trained semantic segmentation model of the combined model to identify a crop area from the original image;

[0039] In step S204, after the crop area is identified from the original image using the semantic segmentation model, other areas in the original image except the crop area may be regarded as non-crop areas, and the coordinate information of the crop area and the non-crop area may be recorded.

[0040] Step S206, processing the original image using a preset image processing method corresponding to the weeding method to obtain an image to be processed;

[0041] In step S206, the weeding methods include, but are not limited to, mechanical weeding and chemical weeding, wherein mechanical weeding is to remove weeds by physical means, usually using mechanical equipment such as a weeder, a rotary tiller, a grass cutter or a mechanical device carried by an unmanned aerial vehicle, etc., and its working principle is to use the physical force of the mechanical device (such as cutting, pulling, turning over the soil, etc.) to destroy the growth structure of the weeds, thereby achieving the purpose of weeding. However, mechanical weeding is less efficient in removing small or densely growing weeds, and it is easy to accidentally damage the crops when weeds and crops grow together in the same area.

[0042] Chemical weed control is the process of suppressing or killing weeds by spraying chemical herbicides (also called herbicides). It is easy to operate and can avoid accidental damage to crops.

[0043] Step S208: using the image classification model in the combined model to identify the weed area in the image to be processed.

[0044] It can be understood that the semantic segmentation model is used to distinguish between crops and weeds, and the image classification model is used to identify weed areas from the image to be processed.

[0045] Through the above steps, it is possible to achieve the following steps: collecting an original image of a farmland area; using a semantic segmentation model of a pre-trained combined model to identify a crop area from the original image; using a preset image processing method corresponding to a weeding method to process the original image to obtain an image to be processed; using an image classification model in the combined model to identify a weed area in the image to be processed, first identifying the crop area through the combined model, and then identifying the weeds in the non-crop area, thereby achieving the purpose of avoiding directly identifying a wide variety of weeds, thereby achieving the technical effect of high weed identification accuracy, and further solving the technical problem of low weed identification accuracy in related technologies.

[0046] In some embodiments of the present application, a semantic segmentation model of a pre-trained combined model is used to identify crop areas from the original image, including: using the semantic segmentation model to identify the original image to obtain a segmentation result, and the segmentation result is used to indicate the crop area and non-crop area in the original image; generating a binary mask based on the segmentation result, the area marked as a valid value in the binary mask is the crop area, and the area marked as an invalid value in the binary mask is the non-crop area.

[0047] Taking the valid value as 1 and the invalid value as 0 as an example, the specific operation of segmenting the crop area from the original image is as follows: input the original image into the trained semantic segmentation model, obtain the crop segmentation result, generate a binary mask (the crop area is 1 and other areas are 0); record the coordinate information of the crop area in the image for subsequent processing. Figure 3 A schematic diagram of an original image is shown, such as Figure 3 As shown, the original image contains crops and weeds.

[0048] The specific steps of processing the original image by using a preset image processing method corresponding to the weeding method to obtain the image to be processed are as follows: dividing the original image into multiple grid areas, and the area covered by each grid area is determined according to the working area of ​​the weeding equipment; processing the grid areas in the original image based on the weeding method to obtain the image to be processed; and determining the weed area from the image to be processed.

[0049] Specifically, according to the operating range of the weeding actuator, the original image is evenly divided into several grids, such as Figure 4As shown in the figure, the original image is divided into 12 grids. The size of the grid should match the actual working range of the actuator. For example, if the working range of the actuator is 10cm×10cm, the corresponding image area should be divided into grids of corresponding proportions.

[0050] It should be noted that the size of the grid should match the actual operating range of the actuator, which means that the difference between the area of ​​the grid and the operating area of ​​the weeding actuator (weeding equipment) is less than a preset threshold.

[0051] Since mechanical weeding and chemical weeding cause different accidental damage to crops, when the weeding method is mechanical weeding, based on the coordinate information of the crop area and the non-crop area, it is determined whether each grid area contains the crop area; when the weeding method is mechanical weeding, the grid area containing the crop area in the original image is removed to obtain the image to be processed; the image to be processed is identified using the image classification model, the area containing plants is determined as the weed area, and the weeding operation is performed, such as Figure 5 As shown, grids 1, 5, 6, 7, 8, 9, 10, and 11 are directly eliminated, and grids 2, 3, 4, and 12 are input into the image classification network as the images to be processed for plant recognition. Since grids 2 and 3 contain plants, they are identified as areas containing weeds.

[0052] Based on the coordinate information of the crop area and the non-crop area, determine whether each grid area contains the crop area; when the weeding method is chemical weeding, set the pixel value of the grid area containing the crop area in the original image to a preset value to obtain an image to be processed; use the image classification model to identify the image to be processed, determine the area containing plants as the weed area, and perform a weeding operation, then Figure 5 After the pixel values ​​of the crop areas in grids 1, 5, 6, 7, 8, 9, 10, and 11 are set to the preset values, the crop areas are eliminated to obtain the image to be processed, and the image to be processed is input into the image classification network for plant recognition. Grids 2, 3, 5, 7, and 11 contain plants and are thus identified as areas containing weeds. Taking the preset value as 0 as an example, Figure 6 is the image to be processed after setting the pixel value of the crop area to 0, such as Figure 6 As shown, the crop areas in the image to be processed are all eliminated.

[0053] Optionally, the semantic segmentation model is determined in the following manner, including: acquiring a first data set, the first data set including: crop images in different environments and under different lighting conditions, the crop images all having their contours annotated; using the first data set to train an initial semantic segmentation model until the initial semantic segmentation model converges to obtain a converged semantic segmentation model, and using a first verification data set to verify the performance of the converged semantic segmentation model; if the converged semantic segmentation model passes the verification, determining the converged semantic segmentation model as the semantic segmentation model.

[0054] Specifically, step 1, the first dataset is constructed: collect crop images in different field environments and lighting conditions, manually annotate the pixel-level contours of the crops, and establish a semantic segmentation dataset of the crops. Step 2, model training: select a suitable semantic segmentation network (such as U-Net, DeepLab, etc.), and train it with the semantic segmentation dataset of the crops until the model converges or reaches the preset accuracy requirements. Step 3, model verification: use an independent verification set to evaluate the performance of the model to ensure that the model can accurately segment the crop area.

[0055] Since the crop types are single or limited, this application establishes a crop image dataset (first dataset) and trains a semantic segmentation network to accurately identify crop areas. Compared with identifying a wide variety of weeds, the model training cost for identifying crops is lower and the accuracy is higher.

[0056] Optionally, the image classification model is determined in the following manner, including: obtaining a second data set, the second data set including: positive samples and negative samples, wherein the positive samples include a variety of plant images, and the negative samples are background images; using the second data set to train the initial image classification model until the initial image classification model converges to obtain a converged image classification model, and using a second verification data set to verify the classification accuracy of the converged image classification model; if the converged image classification model passes the verification, determining the converged image classification model as the image classification model.

[0057] Specifically, step 1, the second data set is constructed: Positive samples: collect images containing various plants (crops and weeds), without distinguishing the types of plants. Negative samples: collect images containing only background (such as soil, stones, dead leaves, etc.). Step 2, model training: select a suitable image classification network (such as ResNet, VGG, etc.) and train it with the above data set. The positive sample label is "plant" and the negative sample label is "background". Step 3, model verification: use an independent verification set to evaluate the classification accuracy of the model to ensure that the model can effectively distinguish between plants and background.

[0058] It should be noted that an image dataset (second dataset) containing various plants (crops and weeds) and backgrounds is established to train the image classification network to distinguish between plant and non-plant areas. Positive samples are images containing plants, and negative samples are images containing only backgrounds (such as soil, stones, etc.).

[0059] The present application also proposes another weed identification method based on a combined network, including: Step 1: For the original image collected in the field, first use the trained semantic segmentation model to identify and mark the crop area and record its location information. Step 2: Divide the original image into several grids according to the operating range of the weeding actuator (mechanical weeding device or herbicide nozzle). The physical size of each grid should be equal to or slightly smaller than the actual operating range of the actuator. Step 3: Determine whether each grid contains a crop area: Mechanical weeding: Remove the grids containing crops (to avoid accidental damage to crops), and the remaining grids are used as areas to be processed. Chemical weeding: After the crop area is removed from the original image (the pixel value of the crop area is set to 0), the remaining part is used as the area to be processed. Step 4: Input the grid to be processed into the trained image classification model to determine whether there are plants (i.e., weeds) in the grid. The grid identified as a plant is the area where the weeds are located.

[0060] It can be seen that the weed recognition method based on the combined network proposed in the embodiment of the present application improves the adaptability and robustness of illumination. Traditional image processing methods usually rely on the segmentation of green pixels, but this method is very sensitive to illumination changes and is easily affected by ambient light, resulting in unstable recognition effect. The present application adopts a semantic segmentation and image classification model of deep learning, and the image data collected under various illumination conditions are included in the training process. In this way, the model has stronger robustness to light changes and can stably identify crops and weeds in different illumination environments. It avoids directly distinguishing weeds from crops and improves recognition accuracy. Weeds and crops are often very similar in morphology and color. Traditional weed recognition algorithms need to directly distinguish between the two, which is difficult and easy to reduce recognition accuracy. The two models of the present application, the semantic segmentation model and the image classification model, do not need to directly distinguish between weeds and crops. The semantic segmentation model focuses on identifying crop areas, while the image classification model only needs to distinguish between plants and backgrounds. This method simplifies the recognition task and improves the accuracy and reliability of the model. Enhance the recognition ability of small targets and high-density weeds. Traditional semantic segmentation networks or target detection networks often have high error rates when identifying weeds of small size or high density distribution. The present application divides the image into multiple grid areas and uses an image classification model to identify each grid. This method has a higher recognition rate for small targets and high-density weeds, is more robust, and can effectively detect small and dense weeds in the field. Reduce the cost of training set construction. Simplified data set for semantic segmentation model: Since the types of crops are single or limited, the present application only needs to collect and annotate images of a single crop when training the semantic segmentation model. This greatly reduces the workload of data collection and annotation, reduces the cost of training set construction, and ensures the training quality of the model. Simplified data set for image classification model: When training the image classification model, there is no need to collect and annotate all types of weed images, nor is there any need to distinguish whether the plants in the training set are weeds or crops. Because plants have obvious characteristics relative to the soil background, the model only needs to learn to distinguish between plants and background. This further reduces the workload of data collection and annotation, reduces costs, and improves the generalization ability and recognition rate of the model.

[0061] According to another aspect of the embodiment of the present application, a weed identification device based on a combined network is also provided. Figure 7 As shown, including:

[0062] The acquisition module 70 is used to acquire the original image of the farmland area;

[0063] A first recognition module 72, configured to recognize a crop region from the original image using a semantic segmentation model of a pre-trained combined model;

[0064] A processing module 74, configured to process the original image using a preset image processing method corresponding to the weeding method to obtain an image to be processed;

[0065] The second identification module 76 is used to identify the weed area in the image to be processed by using the image classification model in the combined model.

[0066] The weed identification method based on the combined network executed by the weed identification device based on the combined network adopts the method of collecting original images of the farmland area; using the semantic segmentation model of the pre-trained combined model to identify the crop area from the original image; using the preset image processing method corresponding to the weeding method to process the original image to obtain the image to be processed; using the image classification model in the combined model to identify the weed area in the image to be processed, the crop area is first identified by the combined model, and then the weeds are identified in the non-crop area, so as to avoid directly identifying a wide variety of weeds, thereby achieving the technical effect of high weed identification accuracy, and further solving the technical problem of low weed identification accuracy in the related art.

[0067] The first recognition module 72 includes: a recognition submodule, which is used to use a semantic segmentation model of a pre-trained combined model to identify the crop area from the original image, including: using the semantic segmentation model to recognize the original image to obtain a segmentation result, and the segmentation result is used to indicate the crop area and non-crop area in the original image; generating a binary mask based on the segmentation result, the area marked as a valid value in the binary mask is the crop area, and the area marked as an invalid value in the binary mask is the non-crop area.

[0068] The identification submodule includes: a processing unit, which is used to process the original image using a preset image processing method corresponding to the weeding method to obtain an image to be processed, including: dividing the original image into multiple grid areas, and the area covered by each grid area is determined according to the working area of ​​the weeding equipment; processing the grid areas in the original image based on the weeding method to obtain the image to be processed; and determining the weed area from the image to be processed.

[0069] The processing unit includes: a first determining subunit and a second determining subunit, wherein the first determining subunit is used to determine the weed area from the image to be processed, including: based on the coordinate information of the crop area and the non-crop area, determining whether each grid area contains the crop area; when the weeding method is mechanical weeding, removing the grid area containing the crop area in the original image to obtain the image to be processed; using the image classification model to identify the image to be processed, determining the area containing plants as the weed area, and performing a weeding operation.

[0070] The second determination subunit is used to determine the weed area from the image to be processed, including: based on the coordinate information of the crop area and the non-crop area, determining whether each grid area contains the crop area; when the weeding method is chemical weeding, setting the pixel value of the grid area containing the crop area in the original image to a preset value to obtain the image to be processed; using the image classification model to identify the image to be processed, determining the area containing plants as the weed area, and performing a weeding operation.

[0071] The weed identification device based on the combined network also includes: a first training submodule and a second training submodule, wherein the first training submodule is used to train a semantic segmentation model, including: obtaining a first data set, wherein the first data set includes: crop images under different environments and different lighting conditions, and the crop images have been marked with contours; using the first data set to train an initial semantic segmentation model until the initial semantic segmentation model converges to obtain a converged semantic segmentation model, and using a first verification data set to verify the performance of the converged semantic segmentation model; when the converged semantic segmentation model passes the verification, determining the converged semantic segmentation model as the semantic segmentation model.

[0072] The second training submodule is used to train the image classification model, including: acquiring a second data set, the second data set including: positive samples and negative samples, wherein the positive samples include a variety of plant images, and the negative samples are background images; using the second data set to train the initial image classification model until the initial image classification model converges to obtain a converged image classification model, and using a second verification data set to verify the classification accuracy of the converged image classification model; if the converged image classification model passes the verification, determining the converged image classification model as the image classification model.

[0073] According to another aspect of the embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the above-mentioned weed identification method based on the combined network.

[0074] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, wherein the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned weed identification method based on the combined network by running the computer program.

[0075] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned weed identification method based on the combination network when executed by a processor.

[0076] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0077] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0079] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0080] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0081] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0082] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A weed identification method based on a combined network, characterized in that: include: Collecting original images of farmland areas; Using a pre-trained semantic segmentation model of the combined model to identify a crop region from the original image; The original image is processed by using a preset image processing method corresponding to the weeding method to obtain an image to be processed; The image classification model in the combined model is used to identify the weed area in the image to be processed.

2. The method according to claim 1, characterized in that A semantic segmentation model of a pre-trained combined model is used to identify a crop region from the original image, including: Using the semantic segmentation model to identify the original image, to obtain a segmentation result, wherein the segmentation result is used to indicate a crop area and a non-crop area in the original image; A binary mask is generated based on the segmentation result, wherein the area marked as a valid value in the binary mask is the crop area, and the area marked as an invalid value in the binary mask is the non-crop area.

3. The method according to claim 2, characterized in that The original image is processed using a preset image processing method corresponding to the weeding method to obtain an image to be processed, including: Dividing the original image into a plurality of grid areas, the area covered by each grid area is determined according to the operation area of ​​the weeding equipment; Processing the grid area in the original image based on the weeding method to obtain the image to be processed; The weed area is determined from the image to be processed.

4. The method according to claim 3, characterized in that: Determining the weed area from the image to be processed includes: Based on the coordinate information of the crop area and the non-crop area, determining whether each grid area includes the crop area; In the case where the weeding method is mechanical weeding, the grid area containing the crop area in the original image is removed to obtain an image to be processed; The image to be processed is identified using the image classification model, the area containing plants is determined as the weed area, and a weeding operation is performed.

5. The method according to claim 3, characterized in that: Determining the weed area from the image to be processed includes: Based on the coordinate information of the crop area and the non-crop area, determining whether each grid area includes the crop area; In the case where the weeding method is chemical weeding, the pixel value of the grid area containing the crop area in the original image is set to a preset value to obtain an image to be processed; The image to be processed is identified using the image classification model, the area containing plants is determined as the weed area, and a weeding operation is performed.

6. The method according to claim 1, characterized in that The semantic segmentation model is determined by: Acquire a first data set, wherein the first data set includes: crop images under different environments and different lighting conditions, wherein the crop images are all marked with contours; Using the first data set to train an initial semantic segmentation model until the initial semantic segmentation model converges to obtain a converged semantic segmentation model, and using a first verification data set to verify the performance of the converged semantic segmentation model; When the converged semantic segmentation model passes the verification, the converged semantic segmentation model is determined as the semantic segmentation model.

7. The method according to claim 1, characterized in that The image classification model is determined by: Acquire a second data set, wherein the second data set includes: positive samples and negative samples, wherein the positive samples include multiple plant images, and the negative samples are background images; Using the second data set to train the initial image classification model until the initial image classification model converges to obtain a converged image classification model, and using the second verification data set to verify the classification accuracy of the converged image classification model; In a case where the converged image classification model passes the verification, the converged image classification model is determined as the image classification model.

8. A weed identification device based on a combined network, characterized in that: include: A collection module, used to collect original images of farmland areas; A first recognition module, configured to recognize a crop area from the original image using a semantic segmentation model of a pre-trained combined model; A processing module, used for processing the original image by using a preset image processing method corresponding to the weeding method to obtain an image to be processed; The second recognition module is used to recognize the weed area in the image to be processed by using the image classification model in the combined model.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory, and is used to execute the weed identification method based on a combination network according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the weed identification method based on a combination network according to any one of claims 1 to 7 is implemented.

Images