Method for identifying insect pests in facility based on SANet model

Abstract

The invention provides a method for identifying insect pests in a facility based on a SANet model. The method comprises the following steps that S1, a crop growth insect pest identification platform obtains image data shot by a walking shooting device; s2, after the crop growth insect pest recognition platform obtains image data shot by the walking shooting device, the crop growth insect pest recognition platform recognizes insect pests.

Description

Technical Field

[0001] The present invention relates to the technical field of crops, and in particular to a method for identifying insect pests in facilities based on a SANet model. Background Art

[0002] Pest problems are a major factor that plagues the growth of crops in facilities. It is particularly important to detect pests and deal with them in a timely manner. At present, pest identification is mainly divided into methods based on traditional visual recognition (such as image processing, binary classification, SVM, etc.) and deep learning (neural network). The traditional learning-based methods require the setting of a large number of complex parameters and are easily affected by the surrounding environment and imaging quality in actual use, resulting in fluctuations in accuracy and even algorithm failure. In recent years, methods based on deep learning have developed rapidly in fields such as image recognition. They not only have strong adaptability to environmental changes and high accuracy, but the model can also be improved through continuous iterative optimization. At present, the artificial intelligence smart agriculture technology that combines deep learning and agricultural production is also constantly developing, and it will also be a trend in future science and technology and efficient smart agriculture.

[0003] At present, crop pests in facilities have the characteristics of rapid onset, rapid spread, serious harm, and difficulty in prevention and control. To effectively implement pest control in facilities, "early detection and early elimination" are the most critical factors. Summary of the invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a method for identifying pests in facilities based on the SANet model.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a method for identifying pests in facilities based on the SANet model, comprising the following steps:

[0006] S1, the crop growth pest identification platform obtains image data captured by the walking camera device;

[0007] S2, after the crop growth pest identification platform obtains the image data captured by the walking shooting device, the crop growth pest identification platform identifies the pests.

[0008] In a preferred embodiment of the present invention, the method for the crop growth pest identification platform to obtain image data captured by the walking camera in step S1 includes the following steps:

[0009] S11, the walking shooting device walks to a shooting point, which is inside a crop growth facility, where a yellow board for attracting sticky pests is provided;

[0010] S12, after the walking shooting device walks to the shooting point, the binocular camera shoots the yellow plate image data;

[0011] S13, uploading all yellow board image data captured by the binocular camera to the crop growth pest identification platform.

[0012] In a preferred embodiment of the present invention, the yellow board is a yellow square-shaped cardboard, and one side of the cardboard is smeared with transparent mucus that can attract pests.

[0013] In a preferred embodiment of the present invention, the method for identifying pests by the crop growth pest identification platform in step S2 comprises the following steps:

[0014] S21, the crop growth pest identification platform performs image processing on the images taken at the same shooting point;

[0015] S22, after performing image processing on all images taken at the same shooting point, performing fusion processing on the processed images;

[0016] S23, judge p η The size relationship between v1 and v2:

[0017] If p η <v1,p η represents the pest density threshold of the image captured at the shooting point η, η=1, 2, 3, ..., B, v1 represents the preset first pest density threshold; the shooting point here is the early stage of pest infestation;

[0018] If v1≤p η ≤v2, v1 represents the preset first pest density threshold, p η represents the pest density threshold of the image taken at the shooting point η, the preset first pest density threshold is greater than the preset second pest density threshold, and v2 represents the preset second pest density threshold; the shooting point here is the mid-term pest period;

[0019] If p η >v2,p η represents the pest density threshold of the image captured at the shooting point η, v2 represents the preset second pest density threshold, and the shooting point here is in the late stage of pest.

[0020] In a preferred embodiment of the present invention, in step S23, η The calculation method includes the following steps:

[0021] S231, let η=1, V1=V2=V3=…=V B =0, x=1, y=1;

[0022] S232, judge U ηThe relationship between (x,y) and u1 and u2:

[0023] If u1≤U η (x,y)≤u2,U η (x, y) represents the A captured at the shooting point η η The fusion value of the first image at the pixel point (x, y) of the first image, u1 represents the preset first image value, u2 represents the preset second image value, and the preset second image value is greater than the preset second image value, then V η =V η +1; x=x+1; execute the next step; V η represents the pest count threshold for images taken at shooting point η, V1 represents the pest count threshold for images taken at shooting point 1, V2 represents the pest count threshold for images taken at shooting point 2, V3 represents the pest count threshold for images taken at shooting point 3, ..., V B represents the threshold value of the number of pests in the image taken at the shooting point B;

[0024] If U η (x,y)>u2 or U η (x,y)<u1,U η (x, y) represents the A captured at the shooting point η η The fusion value of the first image at the pixel point (x, y) of the first image, u1 represents the preset first image value, u2 represents the preset second image value, and the preset second image value is greater than the preset second image value, then V η =V η +0; x=x+1; execute the next step;

[0025] S233, determine the size relationship between x and X:

[0026] If x>X, X represents the number of horizontal pixels of the image, then x=1, y=y+1; proceed to the next step;

[0027] If x≤X, where X represents the number of horizontal pixels of the image, then execute step S232;

[0028] S234, determine the size relationship between y and Y:

[0029] If y>Y, Y represents the number of vertical pixels of the image, then η=η+1, x=1, y=1; proceed to the next step;

[0030] If y≤Y, where Y represents the number of vertical pixels of the image, then execute step S232;

[0031] S235, determine the relationship between η and B:

[0032] If η>B, where B represents the number of shooting points set in the crop growth facility, the next step is executed;

[0033] If η≤B, where B represents the number of shooting points set in the crop growth facility, then step S232 is executed;

[0034] S235, p η =V η / XY,p η represents the pest density threshold of the image taken at the shooting point η, η=1, 2, 3, ..., B, V η represents the pest number threshold of the image taken at the shooting point η, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image.

[0035] In a preferred embodiment of the present invention, the method of uploading all yellow plate image data captured by the binocular camera to the crop growth pest identification platform in step S13 comprises the following steps:

[0036] S131, counting the number of yellow plate images taken, recorded as A b , A b represents the number of photos taken at the shooting point b, b=1, 2, 3, ..., B, B represents the number of shooting points set in the crop growth facility;

[0037] S132, establish B yellow board image folders, namely the 1st yellow board image folder, the 2nd yellow board image folder, the 3rd yellow board image folder, ..., the Bth yellow board image folder, and put the yellow board images taken at the same shooting point into the same yellow board image folder; that is, put A1 yellow board image taken at shooting point 1 into the 1st yellow board image folder, put A2 yellow board image taken at shooting point 2 into the 2nd yellow board image folder, put A3 yellow board image taken at shooting point 3 into the 3rd yellow board image folder, ..., put A4 yellow board image taken at shooting point B into the 4th yellow board image folder, ..., put A5 yellow board image taken at shooting point C into the 5th yellow board image folder, ..., put A6 yellow board image taken at shooting point D into the 6th yellow board image folder, ..., put A7 yellow board image taken at shooting point E into the 7th yellow board image folder, ..., put A8 yellow board image taken at shooting point F into the 8th yellow board image folder, ..., put A9 yellow board image taken at shooting point F into the 9th yellow board image folder, ..., put A1 yellow board image taken at shooting point G into the 10th yellow board image folder, ..., put A2 yellow board image taken at shooting point G into the 11th yellow board image folder, ..., put A3 yellow board image taken at shooting point G into the 12th yellow board image folder, ..., put A4 yellow board image taken at shooting point G into the 13th yellow board image folder, ..., put A5 yellow board image taken at shooting point G into the 14th yellow board image folder, ..., put A6 yellow board image taken at shooting point G into the 15th yellow board image folder, ..., put A7 yellow board image taken at shooting point G into the 16th yellow board image folder, B Put the yellow plate image into the B yellow plate image folder;

[0038] S133, compressing B yellow board image folders to obtain B yellow board image compressed file packages; that is, compressing the first yellow board image folder to obtain the first yellow board image compressed file package; compressing the second yellow board image folder to obtain the second yellow board image compressed file package; compressing the third yellow board image folder to obtain the third yellow board image compressed file package; ...; compressing the Bth yellow board image folder to obtain the Bth yellow board image compressed file package;

[0039] S134, determining whether the number of images in each yellow board image compression file package is consistent with the number of images in the corresponding yellow board image folder:

[0040] If the number of images in the yellow board image compressed file package is inconsistent with the number of images in the corresponding yellow board image folder, then delete the redundant files in the yellow board image compressed file package; that is, if the number of images in the first yellow board image compressed file package is inconsistent with the number of images in the first yellow board image folder, then delete the files in the first yellow board image compressed file package that are different from those in the first yellow board image folder; if the number of images in the second yellow board image compressed file package is inconsistent with the number of images in the second yellow board image folder, then delete the files in the second yellow board image compressed file package that are different from those in the second yellow board image folder; if the number of images in the third yellow board image compressed file package is inconsistent with the number of images in the third yellow board image folder, then delete the files in the third yellow board image compressed file package that are different from those in the third yellow board image folder; ...; if the number of images in the Bth yellow board image compressed file package is inconsistent with the number of images in the Bth yellow board image folder, then delete the files in the Bth yellow board image compressed file package that are different from those in the Bth yellow board image folder;

[0041] If the number of images in all the yellow board image compressed file packages is consistent with the number of images in the corresponding yellow board image folder, proceed to the next step;

[0042] S135, calculating the upload codes of B yellow board image compressed file packages to obtain B upload codes, that is, calculating the upload code of the first yellow board image compressed file package to obtain the first upload code; calculating the upload code of the second yellow board image compressed file package to obtain the second upload code; calculating the upload code of the third yellow board image compressed file package to obtain the third upload code; ...; calculating the upload code of the Bth yellow board image compressed file package to obtain the Bth upload code; the calculation method of the upload code is:

[0043] Noupload j =uploadcalculateway(Imagepackage j ),

[0044] Noupload j represents the jth uploaded code; j = 1, 2, 3, ..., B;

[0045] uploadcalculateway() indicates the upload code calculation method, preferably using the MD5 digest function;

[0046] Imagepackage j represents the j-th yellow plate image compressed file package;

[0047] S136, uploading B upload codes to the crop growth pest identification platform. After receiving the B upload codes, the crop growth pest identification platform forms an identification set with the received B upload codes, and then the crop growth pest identification platform requests the walking camera device to send B yellow plate image compressed file packages;

[0048] S137, after receiving the request from the crop growth pest identification platform, the walking shooting device uploads B yellow plate image compressed file packages to the crop growth pest identification platform;

[0049] S138, after receiving B yellow board image compressed file packages, the crop growth pest identification platform calculates identification codes of the received B yellow board image compressed file packages to obtain B identification codes; that is, the identification code of the received first yellow board image compressed file package is calculated to obtain the first identification code; the identification code of the received second yellow board image compressed file package is calculated to obtain the second identification code; the identification code of the received third yellow board image compressed file package is calculated to obtain the third identification code; ...; the identification code of the received Bth yellow board image compressed file package is calculated to obtain the Bth identification code; the calculation method of the identification code is:

[0050] NoJudge i =Judgecalculateway(Receivepackage i ),

[0051] NoJudge i represents the i-th identification code; i=1, 2, 3, ..., B;

[0052] Judgecalculateway() represents the identification code calculation method, preferably using the MD5 digest function;

[0053] Receivepackage i represents the i-th yellow plate image compressed file package received by the crop growth pest identification platform;

[0054] S139, the crop growth pest identification platform determines whether the identification code exists and the identification is concentrated:

[0055] If the identification code exists in the identification set, the received yellow board image compressed file package corresponding to the identification code is decompressed to obtain a yellow board image decompression folder and store it; and the received yellow board image compressed file package corresponding to the identification code is deleted and the identification code is deleted from the identification set;

[0056] If the identification code does not exist in the identification set, the received yellow plate image compressed file package corresponding to the identification code is deleted; after the judgment is completed, it is determined whether the identification set is an empty set:

[0057] If the identification set is an empty set, the crop growth pest identification platform stores B yellow plate image decompression folders;

[0058] If the identification set is not an empty set, the crop growth pest identification platform requests the walking camera device to send the yellow plate image compressed file package corresponding to the element in the identification set. That is, to determine whether the first identification code exists in the identification set:

[0059] If the first identification code exists in the identification set, decompress the received first yellow board image compressed file package to obtain the first yellow board image decompressed folder and store the first yellow board image decompressed folder; delete the received first yellow board image compressed file package and delete the first identification code from the identification set;

[0060] If the first identification code does not exist in the identification set, the first yellow plate image compressed file package received is deleted, and the next identification code is determined;

[0061] Determine whether the second identification code exists and is in the identification set:

[0062] If the second identification code exists in the identification set, decompress the received second yellow board image compressed file package to obtain a second yellow board image decompression folder and store the second yellow board image decompression folder; delete the received second yellow board image compressed file package and delete the second identification code from the identification set;

[0063] If the second identification code does not exist in the identification set, the received second yellow plate image compressed file package is deleted, and the next identification code is determined;

[0064] Determine whether the third identification code exists and is in the identification set:

[0065] If the third identification code exists in the identification set, decompress the received third yellow board image compressed file package to obtain the third yellow board image decompression folder and store the third yellow board image decompression folder; delete the received third yellow board image compressed file package and delete the third identification code from the identification set;

[0066] If the third identification code does not exist in the identification set, the received third yellow plate image compressed file package is deleted, and the next identification code is determined; ...;

[0067] Determine whether the Bth identification code exists and is in the identification set:

[0068] If the Bth identification code exists in the identification set, decompress the received Bth yellow plate image compressed file package to obtain the Bth yellow plate image decompression folder and store the Bth yellow plate image decompression folder; delete the received Bth yellow plate image compressed file package and delete the Bth identification code from the identification set;

[0069] If the Bth identification code does not exist in the identification set, the received Bth yellow plate image compressed file package is deleted, and it is determined whether the identification set is an empty set:

[0070] If the identification set is an empty set, the crop growth pest identification platform stores B yellow plate image decompression folders;

[0071] If the identification set is not an empty set, the crop growth pest identification platform requests the walking camera device to send a yellow plate image compressed file package corresponding to the elements in the identification set.

[0072] In a preferred embodiment of the present invention, in step S21, the method for the crop growth pest identification platform to process images taken at the same shooting point is:

[0073] Gray(x,y)=red(x,y)*ε1+green(x,y)*ε2+bule(x,y)*ε3,

[0074] Where Gray(x,y) represents the image value at the pixel point (x,y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0075] red(x,y) represents the red value at the pixel point (x,y);

[0076] green(x,y) represents the green value at the pixel point (x,y);

[0077] bule(x,y) represents the blue value at the pixel point (x,y);

[0078] ε1, ε2, ε3 are adjustment coefficients of red value, green value and blue value respectively, ε1+ε2+ε3=1, ε1, ε2, ε3 are all natural numbers not greater than 1.

[0079] 8. The method for identifying pests in facilities based on the SANet model according to claim 1, characterized in that the fusion processing method in step S22 is:

[0080]

[0081] Among them, U b (x,y) represents A taken at shooting point b b The fusion value of the image at the pixel point (x, y); b = 1, 2, 3, ..., B, B represents the number of shooting points set in the crop growth facility; x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0082] A b Indicates the number of photos taken at shooting point b;

[0083] Gray(x,y) b,h represents the image value of the hth image captured at the shooting point b at the pixel point (x, y);

[0084] That is

[0085] Among them, U1(x, y) represents the fusion value of the A1 image taken at the shooting point 1 at the pixel point (x, y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0086] A1 indicates the number of shots taken at shooting point 1;

[0087] Gray(x,y) 1,h represents the image value of the hth image captured at the shooting point 1 at the pixel point (x, y);

[0088]

[0089] Among them, U2(x,y) represents the fusion value of the A2 image taken at the shooting point 2 at the pixel point (x,y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0090] A2 indicates the number of photos taken at shooting point 2;

[0091] Gray(x,y) 2,h represents the image value of the hth image captured at the shooting point 2 at the pixel point (x, y);

[0092]

[0093] Among them, U3(x,y) represents the fusion value of the A3 image taken at the shooting point 3 at the pixel point (x,y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0094] A3 indicates the number of shots taken at shooting point 3;

[0095] Gray(x,y) 3,h represents the image value of the hth image captured at the capturing point 3 at the pixel point (x, y); ...;

[0096]

[0097] Among them, U B (x,y) represents A taken at shooting point B B The fusion value of the image at the pixel point (x, y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0098] A B Indicates the number of photos taken at shooting point B;

[0099] Gray(x,y) B,h Represents the image value of the hth image captured at the shooting point B at the pixel point (x, y).

[0100] In a preferred embodiment of the present invention, S21, obtaining yellow board image data to be identified in the facility;

[0101] S22, input the data in the training group in step S21 into the first 13 convolutional layers of VGG16 for encoding;

[0102] S23, dividing the code obtained in step S22 into five feature levels, with downsampling steps of 1, 2, 4, 8, and 16 respectively, and the corresponding feature maps are denoted as V1 to V5;

[0103] S24, P1~P5 are multi-scale feature map representations in the decoding stage, each P is generated by Vi and P(i+1); in order to make the sizes of Vi and P(i+1) match, upsampling is required, and the upsampling method is the nearest neighbor interpolation method;

[0104] S25, input P1~P5 into the density branch Density Head to obtain five density maps D1~D5, and then splice these five density maps in the channel dimension to obtain Density Maps

[0105] S26, input P1 to P5 into the confidence branch Confidence Head to obtain five confidence maps C1 to C5, and then splice these five confidence maps in the channel dimension to obtain Confidence Maps;

[0106] S27, perform Softmax operation on Confidence Maps in the channel dimension, so that the sum of the values ​​of the five two-dimensional confidence maps at each corresponding position pixel is 1;

[0107] S28, multiply the corresponding positions of Confidence Maps and Density Maps pixel by pixel, and then sum them in the channel dimension to get the final density map prediction:

[0108]

[0109] The final density prediction map represents the final density prediction map corresponding to the position. The main network is generated by a five-layer pyramid structure, and each layer is connected to two output ends to output the predicted density map and the confidence map respectively.

[0110] The method for identifying pests in facilities based on the SANet model according to claim 1 is characterized in that in steps S23 to S28, the loss function is defined as a Euclidean Distence loss function:

[0111]

[0112] Used to learn the features corresponding to each pixel in the image.

[0113] In summary, due to the adoption of the above technical solution, the present invention can realize the identification of insect pests.

[0114] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0116] Figure 1 It is a schematic block diagram of the process of the present invention.

[0117] Figure 2 It is a schematic diagram of the yellow plate image data captured by the present invention.

[0118] Figure 3 It is a schematic diagram of the present invention after image processing is performed on the photographed yellow plate image data. DETAILED DESCRIPTION

[0119] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0120] The present invention discloses a method for identifying pests in facilities based on the SANet model, such as Figure 1 As shown, the following steps are included:

[0121] S1, the crop growth pest identification platform obtains image data captured by the walking camera device;

[0122] S2, after the crop growth pest identification platform obtains the image data captured by the walking shooting device, the crop growth pest identification platform identifies the pests.

[0123] In a preferred embodiment of the present invention, the method for the crop growth pest identification platform to obtain image data captured by the walking camera in step S1 includes the following steps:

[0124] S11, the walking shooting device walks to a shooting point, which is inside a crop growth facility, where a yellow board for attracting sticky pests is provided;

[0125] S12, after the walking shooting device walks to the shooting point, the binocular camera shoots the yellow plate image data;

[0126] S13, uploading all yellow board image data captured by the binocular camera to the crop growth pest identification platform.

[0127] In a preferred embodiment of the present invention, the yellow board is a yellow square-shaped cardboard, and one side of the cardboard is smeared with transparent mucus that can attract pests.

[0128] In a preferred embodiment of the present invention, the method for identifying pests by the crop growth pest identification platform in step S2 comprises the following steps:

[0129] S21, the crop growth pest identification platform performs image processing on the images taken at the same shooting point;

[0130] S22, after performing image processing on all images taken at the same shooting point, performing fusion processing on the processed images;

[0131] S23, judge p η The size relationship between v1 and v2:

[0132] If p η <v1,p η represents the pest density threshold of the image captured at the shooting point η, η=1, 2, 3, ..., B, v1 represents the preset first pest density threshold; the shooting point here is the early stage of pest infestation;

[0133] If v1≤p η ≤v2, v1 represents the preset first pest density threshold, p η represents the pest density threshold of the image taken at the shooting point η, the preset first pest density threshold is greater than the preset second pest density threshold, and v2 represents the preset second pest density threshold; the shooting point here is the mid-term pest period;

[0134] If p η >v2,p ηrepresents the pest density threshold of the image captured at the shooting point η, v2 represents the preset second pest density threshold, and the shooting point here is in the late stage of pest.

[0135] In a preferred embodiment of the present invention, in step S23, η The calculation method includes the following steps:

[0136] S231, let η=1, V1=V2=V3=…=V B =0, x=1, y=1;

[0137] S232, judge U η The relationship between (x,y) and u1 and u2:

[0138] If u1≤U η (x,y)≤u2,U η (x, y) represents the A captured at the shooting point η η The fusion value of the first image at the pixel point (x, y) of the first image, u1 represents the preset first image value, u2 represents the preset second image value, and the preset second image value is greater than the preset second image value, then V η =V η +1; x=x+1; execute the next step; V η represents the pest count threshold for images taken at shooting point η, V1 represents the pest count threshold for images taken at shooting point 1, V2 represents the pest count threshold for images taken at shooting point 2, V3 represents the pest count threshold for images taken at shooting point 3, ..., V B represents the threshold value of the number of pests in the image taken at the shooting point B;

[0139] If U η (x,y)>u2 or U η (x,y)<u1,U η (x, y) represents the A captured at the shooting point η η The fusion value of the first image at the pixel point (x, y) of the first image, u1 represents the preset first image value, u2 represents the preset second image value, and the preset second image value is greater than the preset second image value, then V η =V η +0; x=x+1; execute the next step;

[0140] S233, determine the size relationship between x and X:

[0141] If x>X, X represents the number of horizontal pixels of the image, then x=1, y=y+1; proceed to the next step;

[0142] If x≤X, where X represents the number of horizontal pixels of the image, then execute step S232;

[0143] S234, determine the size relationship between y and Y:

[0144] If y>Y, Y represents the number of vertical pixels of the image, then η=η+1, x=1, y=1; proceed to the next step;

[0145] If y≤Y, where Y represents the number of vertical pixels of the image, then execute step S232;

[0146] S235, determine the relationship between η and B:

[0147] If η>B, where B represents the number of shooting points set in the crop growth facility, the next step is executed;

[0148] If η≤B, where B represents the number of shooting points set in the crop growth facility, then step S232 is executed;

[0149] S235, p η =V η / XY,p η represents the pest density threshold of the image taken at the shooting point η, η=1, 2, 3, ..., B, V η represents the pest number threshold of the image taken at the shooting point η, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image.

[0150] In a preferred embodiment of the present invention, the method of uploading all yellow plate image data captured by the binocular camera to the crop growth pest identification platform in step S13 comprises the following steps:

[0151] S131, counting the number of yellow plate images taken, recorded as A b , A b represents the number of photos taken at the shooting point b, b=1, 2, 3, ..., B, B represents the number of shooting points set in the crop growth facility;

[0152] S132, establish B yellow board image folders, namely the 1st yellow board image folder, the 2nd yellow board image folder, the 3rd yellow board image folder, ..., the Bth yellow board image folder, and put the yellow board images taken at the same shooting point into the same yellow board image folder; that is, put A1 yellow board image taken at shooting point 1 into the 1st yellow board image folder, put A2 yellow board image taken at shooting point 2 into the 2nd yellow board image folder, put A3 yellow board image taken at shooting point 3 into the 3rd yellow board image folder, ..., put A4 yellow board image taken at shooting point B into the 4th yellow board image folder, ..., put A5 yellow board image taken at shooting point C into the 5th yellow board image folder, ..., put A6 yellow board image taken at shooting point D into the 6th yellow board image folder, ..., put A7 yellow board image taken at shooting point E into the 7th yellow board image folder, ..., put A8 yellow board image taken at shooting point F into the 8th yellow board image folder, ..., put A9 yellow board image taken at shooting point F into the 9th yellow board image folder, ..., put A1 yellow board image taken at shooting point G into the 10th yellow board image folder, ..., put A2 yellow board image taken at shooting point G into the 11th yellow board image folder, ..., put A3 yellow board image taken at shooting point G into the 12th yellow board image folder, ..., put A4 yellow board image taken at shooting point G into the 13th yellow board image folder, ..., put A5 yellow board image taken at shooting point G into the 14th yellow board image folder, ..., put A6 yellow board image taken at shooting point G into the 15th yellow board image folder, ..., put A7 yellow board image taken at shooting point G into the 16th yellow board image folder, B Put the yellow plate image into the B yellow plate image folder;

[0153] S133, compressing B yellow board image folders to obtain B yellow board image compressed file packages; that is, compressing the first yellow board image folder to obtain the first yellow board image compressed file package; compressing the second yellow board image folder to obtain the second yellow board image compressed file package; compressing the third yellow board image folder to obtain the third yellow board image compressed file package; ...; compressing the Bth yellow board image folder to obtain the Bth yellow board image compressed file package;

[0154] S134, determining whether the number of images in each yellow board image compression file package is consistent with the number of images in the corresponding yellow board image folder:

[0155] If the number of images in the yellow board image compressed file package is inconsistent with the number of images in the corresponding yellow board image folder, then delete the redundant files in the yellow board image compressed file package; that is, if the number of images in the first yellow board image compressed file package is inconsistent with the number of images in the first yellow board image folder, then delete the files in the first yellow board image compressed file package that are different from those in the first yellow board image folder; if the number of images in the second yellow board image compressed file package is inconsistent with the number of images in the second yellow board image folder, then delete the files in the second yellow board image compressed file package that are different from those in the second yellow board image folder; if the number of images in the third yellow board image compressed file package is inconsistent with the number of images in the third yellow board image folder, then delete the files in the third yellow board image compressed file package that are different from those in the third yellow board image folder; ...; if the number of images in the Bth yellow board image compressed file package is inconsistent with the number of images in the Bth yellow board image folder, then delete the files in the Bth yellow board image compressed file package that are different from those in the Bth yellow board image folder;

[0156] If the number of images in all the yellow board image compressed file packages is consistent with the number of images in the corresponding yellow board image folder, proceed to the next step;

[0157] S135, calculating the upload codes of B yellow board image compressed file packages to obtain B upload codes, that is, calculating the upload code of the first yellow board image compressed file package to obtain the first upload code; calculating the upload code of the second yellow board image compressed file package to obtain the second upload code; calculating the upload code of the third yellow board image compressed file package to obtain the third upload code; ...; calculating the upload code of the Bth yellow board image compressed file package to obtain the Bth upload code; the calculation method of the upload code is:

[0158] Noupload j =uploadcalculateway(Imagepackage j ),

[0159] Nouploadj represents the jth uploaded code; j = 1, 2, 3, ..., B;

[0160] uploadcalculateway() indicates the upload code calculation method, preferably using the MD5 digest function;

[0161] Imagepackage j represents the j-th yellow plate image compressed file package;

[0162] S136, uploading B upload codes to the crop growth pest identification platform. After receiving the B upload codes, the crop growth pest identification platform forms an identification set with the received B upload codes, and then the crop growth pest identification platform requests the walking camera device to send B yellow plate image compressed file packages;

[0163] S137, after receiving the request from the crop growth pest identification platform, the walking shooting device uploads B yellow plate image compressed file packages to the crop growth pest identification platform;

[0164] S138, after receiving B yellow board image compressed file packages, the crop growth pest identification platform calculates identification codes of the received B yellow board image compressed file packages to obtain B identification codes; that is, the identification code of the received first yellow board image compressed file package is calculated to obtain the first identification code; the identification code of the received second yellow board image compressed file package is calculated to obtain the second identification code; the identification code of the received third yellow board image compressed file package is calculated to obtain the third identification code; ...; the identification code of the received Bth yellow board image compressed file package is calculated to obtain the Bth identification code; the calculation method of the identification code is:

[0165] NoJudge i =Judgecalculateway(Receivepackage i ),

[0166] NoJudge i represents the i-th identification code; i=1, 2, 3, ..., B;

[0167] Judgecalculateway() represents the identification code calculation method, preferably using the MD5 digest function;

[0168] Receivepackage i represents the i-th yellow plate image compressed file package received by the crop growth pest identification platform;

[0169] S139, the crop growth pest identification platform determines whether the identification code exists and the identification is concentrated:

[0170] If the identification code exists in the identification set, the received yellow board image compressed file package corresponding to the identification code is decompressed to obtain a yellow board image decompression folder and store it; and the received yellow board image compressed file package corresponding to the identification code is deleted and the identification code is deleted from the identification set;

[0171] If the identification code does not exist in the identification set, the received yellow plate image compressed file package corresponding to the identification code is deleted; after the judgment is completed, it is determined whether the identification set is an empty set:

[0172] If the identification set is an empty set, the crop growth pest identification platform stores B yellow plate image decompression folders;

[0173] If the identification set is not an empty set, the crop growth pest identification platform requests the walking camera device to send the yellow plate image compressed file package corresponding to the element in the identification set. That is, to determine whether the first identification code exists in the identification set:

[0174] If the first identification code exists in the identification set, decompress the received first yellow board image compressed file package to obtain the first yellow board image decompressed folder and store the first yellow board image decompressed folder; delete the received first yellow board image compressed file package and delete the first identification code from the identification set;

[0175] If the first identification code does not exist in the identification set, the first yellow plate image compressed file package received is deleted, and the next identification code is determined;

[0176] Determine whether the second identification code exists and is in the identification set:

[0177] If the second identification code exists in the identification set, decompress the received second yellow board image compressed file package to obtain a second yellow board image decompression folder and store the second yellow board image decompression folder; delete the received second yellow board image compressed file package and delete the second identification code from the identification set;

[0178] If the second identification code does not exist in the identification set, the received second yellow plate image compressed file package is deleted, and the next identification code is determined;

[0179] Determine whether the third identification code exists and is in the identification set:

[0180] If the third identification code exists in the identification set, decompress the received third yellow board image compressed file package to obtain the third yellow board image decompression folder and store the third yellow board image decompression folder; delete the received third yellow board image compressed file package and delete the third identification code from the identification set;

[0181] If the third identification code does not exist in the identification set, the received third yellow plate image compressed file package is deleted, and the next identification code is determined; ...;

[0182] Determine whether the Bth identification code exists and is in the identification set:

[0183] If the Bth identification code exists in the identification set, decompress the received Bth yellow plate image compressed file package to obtain the Bth yellow plate image decompression folder and store the Bth yellow plate image decompression folder; delete the received Bth yellow plate image compressed file package and delete the Bth identification code from the identification set;

[0184] If the Bth identification code does not exist in the identification set, the received Bth yellow plate image compressed file package is deleted, and it is determined whether the identification set is an empty set:

[0185] If the identification set is an empty set, the crop growth pest identification platform stores B yellow plate image decompression folders;

[0186] If the identification set is not an empty set, the crop growth pest identification platform requests the walking camera device to send a yellow plate image compressed file package corresponding to the elements in the identification set.

[0187] In a preferred embodiment of the present invention, in step S21, the method for the crop growth pest identification platform to process images taken at the same shooting point is:

[0188] Gray(x,y)=red(x,y)*ε1+green(x,y)*ε2+bule(x,y)*ε3,

[0189] Where Gray(x,y) represents the image value at the pixel point (x,y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0190] red(x,y) represents the red value at the pixel point (x,y);

[0191] green(x,y) represents the green value at the pixel point (x,y);

[0192] bule(x,y) represents the blue value at the pixel point (x,y);

[0193] ε1, ε2, ε3 are the adjustment coefficients of red value, green value and blue value respectively, ε1+ε2+ε3=1, ε1, ε2, ε3 are all natural numbers not greater than 1. Figure 2 After image processing of the yellow plate image data, we can obtain Figure 3 The image data in .

[0194] In a preferred embodiment of the present invention, the method of fusion processing in step S22 is:

[0195]

[0196] Among them, U b (x,y) represents A taken at shooting point b b The fusion value of the image at the pixel point (x, y); b = 1, 2, 3, ..., B, B represents the number of shooting points set in the crop growth facility; x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0197] A b Indicates the number of photos taken at shooting point b;

[0198] Gray(x,y) b,h represents the image value of the hth image captured at the shooting point b at the pixel point (x, y);

[0199] That is

[0200] Among them, U1(x, y) represents the fusion value of the A1 image taken at the shooting point 1 at the pixel point (x, y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0201] A1 indicates the number of shots taken at shooting point 1;

[0202] Gray(x,y) 1,h represents the image value of the hth image captured at the shooting point 1 at the pixel point (x, y);

[0203]

[0204] Among them, U2(x,y) represents the fusion value of the A2 image taken at the shooting point 2 at the pixel point (x,y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0205] A2 indicates the number of photos taken at shooting point 2;

[0206] Gray(x,y) 2,h represents the image value of the hth image captured at the shooting point 2 at the pixel point (x, y);

[0207]

[0208] Among them, U3(x,y) represents the fusion value of the A3 image taken at the shooting point 3 at the pixel point (x,y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0209] A3 indicates the number of shots taken at shooting point 3;

[0210] Gray(x,y) 3,h represents the image value of the hth image captured at the capturing point 3 at the pixel point (x, y); ...;

[0211]

[0212] Among them, U B (x,y) represents A taken at shooting point B B The fusion value of the image at the pixel point (x, y); x = 1, 2, 3, ..., X, y = 1, 2, 3, ..., Y, X represents the number of horizontal pixels in the image, and Y represents the number of vertical pixels in the image;

[0213] A B Indicates the number of photos taken at shooting point B;

[0214] Gray(x,y) B,h Represents the image value of the hth image captured at the shooting point B at the pixel point (x, y).

[0215] In a preferred embodiment of the present invention, S21, obtaining yellow board image data to be identified in the facility;

[0216] S22, input the data in the training group in step S21 into the first 13 convolutional layers of VGG16 for encoding;

[0217] S23, dividing the code obtained in step S22 into five feature levels, with downsampling steps of 1, 2, 4, 8, and 16 respectively, and the corresponding feature maps are denoted as V1 to V5;

[0218] S24, P1~P5 are multi-scale feature map representations in the decoding stage, each P is generated by Vi and P(i+1); in order to make the sizes of Vi and P(i+1) match, upsampling is required, and the upsampling method is the nearest neighbor interpolation method;

[0219] S25, input P1~P5 into the density branch Density Head to obtain five density maps D1~D5, and then splice these five density maps in the channel dimension to obtain Density Maps

[0220] S26, input P1 to P5 into the confidence branch Confidence Head to obtain five confidence maps C1 to C5, and then splice these five confidence maps in the channel dimension to obtain Confidence Maps;

[0221] S27, perform Softmax operation on Confidence Maps in the channel dimension, so that the sum of the values ​​of the five two-dimensional confidence maps at each corresponding position pixel is 1;

[0222] S28, multiply the corresponding positions of Confidence Maps and Density Maps pixel by pixel, and then sum them in the channel dimension to get the final density map prediction:

[0223]

[0224] represents the final density prediction map corresponding to the position of the final density prediction map;

[0225] C i,j,k ′ represents the value of Confidence Maps at layer i;

[0226] D i,j,k Represents the value of Density Maps of layer i;

[0227] The main network is generated by a five-layer pyramid structure, with two output terminals connected to each layer to output the predicted density map and confidence map respectively.

[0228] The method for identifying pests in facilities based on the SANet model according to claim 1 is characterized in that in steps S23 to S28, the loss function is defined as a Euclidean Distence loss function:

[0229]

[0230] L den Represents the cumulative value of the L2 norm loss function of the 5-layer predicted density map and the ground truth density map;

[0231] D i Density Maps value representing the predicted density map;

[0232] D gt Density Maps value representing the ground truth (actual) density map;

[0233] Represents L2 norm loss, i.e., least squares error (LSE), which is the square of L2 norm;

[0234] Used to learn the features corresponding to each pixel in the image.

[0235] The present invention also discloses a system for identifying pests growing on crops in facilities, including a walking shooting device for walking in the crop facilities, and also including a crop pest identification platform. The walking shooting device transmits image data shot in the crop facilities to the crop pest identification platform, and the crop pest identification platform is used to identify whether there are pests on the crops.

[0236] In a preferred embodiment of the present invention, the walking shooting device comprises a disc-shaped upper support plate and a disc-shaped lower support plate, and the disc-shaped upper support plate is supported above the disc-shaped lower support plate by M support columns of the same length, where M is a positive integer greater than or equal to 3;

[0237] An omnidirectional wheat wheel group fixed mounting seat for fixedly installing the omnidirectional wheat wheel group is arranged at the bottom of the disc-shaped lower support plate, and the omnidirectional wheat wheel group is fixedly installed on the omnidirectional wheat wheel group fixed mounting seat, a power supply battery fixed mounting seat for fixedly installing the power supply battery and a power supply module fixed mounting seat for fixedly installing the power supply module are arranged on the top of the disc-shaped lower support plate, the power supply battery is fixedly installed on the power supply battery fixed mounting seat, and the power module is fixedly installed on the power supply module fixed mounting seat, a main control board fixed mounting seat for fixedly installing the main control board, a laser radar fixed mounting seat for fixedly installing the laser radar and a binocular camera fixed mounting seat for installing the binocular camera are arranged on the top of the disc-shaped upper support plate, and the binocular camera is fixedly installed on the binocular camera fixed mounting seat;

[0238] The power output end of the power supply battery is connected to the power input end of the power module, the first power output end of the power module is connected to the power input end of the omnidirectional wheat wheel group, the second power output end of the power module is connected to the power input end of the main control board, the third power output end of the power module is connected to the power input end of the laser radar, and the fourth power output end of the power module is connected to the power input end of the binocular camera, providing adaptive power for the omnidirectional wheat wheel group, the main control board, the laser radar and the binocular camera respectively;

[0239] The walking data control end of the main control board is connected to the walking data control end of the omnidirectional wheat wheel group, the ranging data end of the main control board is connected to the ranging data end of the laser radar, and the image data end of the main control board is connected to the image data end of the binocular camera. The distance data and image data collected by the laser radar and the binocular camera are used to control the omnidirectional wheat wheel to achieve walking in different directions.

[0240] In a preferred embodiment of the present invention, the omnidirectional Mecklenburg wheels in the omnidirectional Mecklenburg wheel set are 75 mm omnidirectional metal Mecklenburg wheels.

[0241] In a preferred embodiment of the present invention, the main control board is an STM32 main control board C30D or / and a Jetson OrinNX1 main control board.

[0242] In a preferred embodiment of the present invention, the laser radar is the laser radar N10P.

[0243] In a preferred embodiment of the present invention, the binocular camera is a binocular camera Astra.

[0244] In a preferred embodiment of the present invention, the power supply battery is a 12V 9800mah power supply battery.

[0245] In a preferred embodiment of the present invention, when M is 6, 6 through holes are respectively arranged on the circumferential edges of the disc-shaped upper support plate and the disc-shaped lower support plate, the 6 through holes are evenly distributed on the disc-shaped upper support plate and the disc-shaped lower support plate, and internal threaded holes are respectively arranged at both ends of the 6 support columns, and the 6 support columns are arranged between the disc-shaped upper support plate and the disc-shaped lower support plate by 12 external threaded bolts matched with the internal threaded holes.

[0246] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for identifying pests in facilities based on the SANet model, characterized in that: The following steps are involved: S1, the crop growth pest identification platform obtains image data captured by the walking camera device; S2, after the crop growth pest identification platform obtains the image data captured by the walking shooting device, the crop growth pest identification platform identifies the pests.

2. The method for identifying pests in facilities based on the SANet model according to claim 1, characterized in that: In step S1, the method for the crop growth pest identification platform to obtain image data captured by the walking camera device includes the following steps: S11, the walking shooting device walks to a shooting point, which is inside a crop growth facility, where a yellow board for attracting sticky pests is provided; S12, after the walking shooting device walks to the shooting point, the binocular camera shoots the yellow plate image data; S13, uploading all yellow board image data captured by the binocular camera to the crop growth pest identification platform.

3. The method for identifying pests in facilities based on the SANet model according to claim 1, characterized in that: The yellow board is a yellow square cardboard with one side smeared with transparent mucus that can attract pests.

4. The method for identifying pests in facilities based on the SANet model according to claim 1, characterized in that: In step S2, the method for identifying pests by the crop growth pest identification platform includes the following steps: S21, the crop growth pest identification platform performs image processing on the images taken at the same shooting point; S22, after performing image processing on all images taken at the same shooting point, performing fusion processing on the processed images; S23, judge p η The size relationship between v1 and v2: If p η <v1,p η represents the pest density threshold of the image captured at the shooting point η, η=1, 2, 3, ..., B, v1 represents the preset first pest density threshold; the shooting point here is the early stage of pest infestation; If v1≤p η ≤v2, v1 represents the preset first pest density threshold, p η represents the pest density threshold of the image taken at the shooting point η, the preset first pest density threshold is greater than the preset second pest density threshold, and v2 represents the preset second pest density threshold; the shooting point here is the mid-term pest period; If p η >v2,p η represents the pest density threshold of the image captured at the shooting point η, v2 represents the preset second pest density threshold, and the shooting point here is in the late stage of pest.

Images