Multi-view intelligent observation method, device and equipment for corn yield

Abstract

The present disclosure provides a multi-perspective intelligent observation method, device, equipment, medium and program product for corn yield, which belongs to the technical field of grain yield prediction. The multi-perspective intelligent observation method for corn yield includes: obtaining a two-dimensional image and a three-dimensional point cloud of a target corn field; obtaining the area of ​​the target corn field, the total number of corn ears in the target corn field and the geometric parameters of each single ear of multiple corn ears in a target corn ear group in the target corn field according to the two-dimensional image and the three-dimensional point cloud; obtaining a visible light image and an infrared image of exposed corn ears in the target corn ear group; obtaining the average single ear grain weight of corn ears in the target corn field according to the visible light image, the infrared image and the multiple single ear geometric parameters; and calculating the corn yield based on the area, the total number of ears and the average single ear grain weight of the target corn field. The present disclosure also provides a multi-perspective intelligent observation device, equipment, storage medium and program product for corn yield.

Description

Technical Field

[0001] The present disclosure relates to the technical field of grain yield prediction, and more specifically, to a multi-perspective intelligent observation method, device, equipment, medium and program product for corn yield. Background Art

[0002] Corn is one of the main food crops, as well as an important feed and industrial raw material. The frequency and intensity of meteorological disasters such as high temperatures, droughts and floods in agricultural planting areas around the world have increased, and the yield of corn is more susceptible to fluctuations due to global climate change. The traditional method of observing corn yield requires harvesting, drying, measuring moisture and weighing corn ears, which is time-consuming and labor-intensive.

[0003] There are methods of observing yields using non-destructive technology in related technologies. However, due to the temporal and spatial differences in corn planting, corn in different regions is affected by various factors such as climate, water resources, and topography, and the growth state and growth level will vary. In addition, biophysical stress at different growth stages will also have different degrees of impact on the final yield. Therefore, in related technologies, the method of observing yields using a single non-destructive technology is difficult to effectively reflect and represent the inter-annual and spatial growth differences of crops, and the estimation results are less accurate. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a multi-perspective intelligent observation method, device, equipment, medium and program product for corn yield.

[0005] A first aspect of the present disclosure provides a multi-view intelligent observation method for corn yield, comprising:

[0006] Obtain a 2D image and 3D point cloud of the target cornfield;

[0007] According to the two-dimensional image and the three-dimensional point cloud, the area of ​​the target corn field, the total number of corn ears in the target corn field, and the geometric parameters of each single corn ear of a plurality of corn ears in a target corn ear group in the target corn field are obtained;

[0008] Obtaining visible light images and infrared images of naked corn ears in a target corn ear group;

[0009] Obtaining an average single-ear grain weight of corn ears in a target corn field according to the visible light image, the infrared image and a plurality of single-ear geometric parameters; and

[0010] The corn yield per unit area is calculated based on the area, total number of ears and average grain weight per ear of the target corn field, wherein the corn yield per unit area is used to characterize the corn yield per unit area in the target corn field.

[0011] According to an embodiment of the present disclosure, obtaining the total number of corn ears in a target corn field based on a two-dimensional image and a three-dimensional point cloud includes:

[0012] According to the three-dimensional point cloud, the total number of plants in the target corn field and the local number of plants in the target corn ear group are obtained;

[0013] According to the three-dimensional point cloud and the two-dimensional image, the number of corn ears in the target corn ear group is obtained;

[0014] The total number of corn ears in the target corn field is calculated based on the number of local plants, the number of corn ears in the target corn ear group and the total number of plants.

[0015] According to an embodiment of the present disclosure, obtaining the number of corn ears in a target corn ear group according to the three-dimensional point cloud and the two-dimensional image includes:

[0016] According to the three-dimensional point cloud and the two-dimensional image, the detection frames of the multiple corn ears in the target corn ear group are obtained;

[0017] The number of corn ears in the detection frame of each of the multiple corn ears is counted to obtain the number of corn ears in the target corn ear group.

[0018] According to an embodiment of the present disclosure, obtaining the geometric parameters of a single ear of corn in a target corn ear group in a target corn field based on a two-dimensional image and a three-dimensional point cloud includes:

[0019] According to the three-dimensional point cloud and the two-dimensional image, the detection frames of the multiple corn ears in the target corn ear group are obtained;

[0020] Mapping the detection frames of the multiple corn ears in the target corn ear group to the three-dimensional point cloud to obtain a first corn ear three-dimensional point cloud;

[0021] The farthest point sampling algorithm is used to perform step-by-step downsampling of the non-uniform point cloud and upsampling of the point cloud based on inverse distance weighted interpolation on the first corn ear 3D point cloud to obtain multiple point cloud features;

[0022] A cross-level skip link hierarchical propagation strategy is used to splice multiple point cloud features to obtain complete point cloud features;

[0023] Perform point cloud segmentation on the complete point cloud features to obtain global point cloud features;

[0024] The global point cloud features are input into a fully connected network to perform point cloud classification, thereby obtaining a second corn ear three-dimensional point cloud classified as a corn ear;

[0025] According to the second corn cob three-dimensional point cloud, the geometric parameters of each single corn cob of a plurality of corn cobs in the target corn cob group in the target corn field are obtained.

[0026] According to an embodiment of the present disclosure, obtaining detection frames of multiple corn ears in a target corn ear group according to the three-dimensional point cloud and the two-dimensional image includes:

[0027] The three-dimensional point cloud of the target corn ear group is transformed in the vertical direction to obtain a depth image;

[0028] Perform feature extraction on the two-dimensional image and the depth image respectively to obtain the extracted multi-layer features;

[0029] Based on multi-layer features, a path aggregation network is used to construct a multi-layer feature pyramid and decouple it to obtain the detection frames of multiple corn ears in the target corn ear group.

[0030] According to an embodiment of the present disclosure, obtaining the average single ear grain weight of corn ears in a target corn field based on a visible light image, an infrared image and a plurality of single ear geometric parameters includes:

[0031] The moisture content of each kernel of a plurality of exposed corn ears is obtained according to the infrared image and the visible light image;

[0032] According to the visible light image, the sizes of kernels of multiple exposed corn ears are obtained;

[0033] Obtaining a single-ear kernel weight of each of the plurality of naked corn ears according to the moisture content of each kernel of the naked corn ears and the size of each kernel of the naked corn ears;

[0034] Based on the geometric parameters of each single ear of corn and the grain weight of each single ear of corn, a relationship model between the geometric parameters of each single ear of corn and the grain weight of each single ear of corn was established.

[0035] Based on the relationship model and the geometric parameters of each single ear of corn in the target corn ear group, the single ear grain weight of each of the multiple corn ears in the target corn ear group is obtained;

[0036] According to the kernel weights of each of the multiple corn ears in the target corn ear group, the average kernel weight of each corn ear in the target corn field is obtained.

[0037] According to an embodiment of the present disclosure, obtaining the size of each of the multiple exposed corn cob kernels according to the visible light image includes:

[0038] The visible light image is subjected to background removal, color space transformation, edge enhancement and adaptive threshold segmentation in sequence to obtain the sizes of kernels of multiple exposed corn ears.

[0039] According to an embodiment of the present disclosure, obtaining the area of ​​a target cornfield according to a two-dimensional image and a three-dimensional point cloud includes:

[0040] According to the three-dimensional point cloud, determine the planting range of the target corn field in the two-dimensional image, wherein the two-dimensional image is an orthophoto image;

[0041] Determine the area of ​​the target corn field based on the planting range of the target corn field.

[0042] A second aspect of the present disclosure provides a multi-view intelligent observation device for corn yield, comprising:

[0043] A first acquisition module is used to acquire a two-dimensional image and a three-dimensional point cloud of a target cornfield;

[0044] A first acquisition module is used to obtain the area of ​​the target corn field, the total number of corn ears in the target corn field, and the geometric parameters of each single corn ear of a plurality of corn ears in a target corn ear group in the target corn field according to the two-dimensional image and the three-dimensional point cloud;

[0045] The second acquisition module is used to acquire visible light images and infrared images of exposed corn ears in the target corn ear group;

[0046] A second obtaining module is used to obtain an average single ear grain weight of corn ears in a target corn field according to the visible light image, the infrared image and a plurality of single ear geometric parameters; and

[0047] The calculation module is used to calculate the corn yield per unit area based on the area, total number of ears and average single ear grain weight of the target corn field, wherein the corn yield per unit area is used to characterize the corn yield per unit area in the target corn field.

[0048] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned multi-perspective intelligent observation method of corn yield.

[0049] The fourth aspect of the present disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the multi-perspective intelligent observation method for corn yield.

[0050] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-perspective intelligent observation method for corn yield.

[0051] According to the embodiments of the present disclosure, the area, total number of ears and geometric parameters of a single ear of the target corn field are obtained based on two-dimensional images and three-dimensional point clouds; the average grain weight of a single ear is obtained based on visible light images, infrared images and geometric parameters of a single ear; the corn yield per unit area is calculated based on the area, total number of ears and average grain weight of a single ear of the target corn field; the corn yield per unit area is calculated through data from on-site multi-view non-destructive observations, thereby improving the efficiency and accuracy of corn yield observations. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0053] Figure 1 A schematic diagram showing an application scenario of a multi-view intelligent observation method, device, equipment, medium, and program product for corn yield according to an embodiment of the present disclosure;

[0054] Figure 2 A flowchart of a multi-view intelligent observation method for corn yield according to an embodiment of the present disclosure is schematically shown;

[0055] Figure 3 A flowchart of obtaining the total number of corn ears in a target corn field according to an embodiment of the present disclosure is schematically shown;

[0056] Figure 4 A flowchart of obtaining the number of corn ears in a target corn ear group according to an embodiment of the present disclosure is schematically shown;

[0057] Figure 5 A flowchart of obtaining geometric parameters of a single ear of corn for each of a plurality of corn ears in a target corn ear group in a target corn field according to an embodiment of the present disclosure is schematically shown;

[0058] Figure 6 A flowchart of obtaining detection frames of multiple corn ears within a target corn ear group according to an embodiment of the present disclosure is schematically shown;

[0059] Figure 7 A flowchart of obtaining the average single-ear kernel weight of corn ears in a target corn field according to an embodiment of the present disclosure is schematically shown;

[0060] Figure 8 A flow chart for obtaining the number of kernels per ear of corn and the size of kernels of each ear of corn according to an embodiment of the present disclosure is schematically shown;

[0061] Fig. 9 A structural block diagram schematically shows a multi-view intelligent observation device for corn yield according to an embodiment of the present disclosure; and

[0062] Fig.10 A block diagram of an electronic device suitable for implementing a multi-view intelligent observation method for corn yield according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0063] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0064] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0065] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0066] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0067] In actual work, it is found that there are temporal and spatial differences in corn planting. Corn in different regions is affected by climate, water resources, terrain and other factors, and the growth state and growth level will be different. Therefore, in related technologies, the method of using a single non-destructive technology to observe yield is difficult to effectively reflect and represent the inter-annual and spatial growth differences of crops, and the estimation results are less accurate.

[0068] In response to the above problems, an embodiment of the present disclosure provides a multi-perspective intelligent observation method for corn yield, including: acquiring a two-dimensional image and a three-dimensional point cloud of a target corn field; obtaining the area of ​​the target corn field, the total number of corn ears in the target corn field, and the geometric parameters of each single ear of multiple corn ears in a target corn ear group in the target corn field based on the two-dimensional image and the three-dimensional point cloud; acquiring a visible light image and an infrared image of exposed corn ears in the target corn ear group; obtaining the average single ear grain weight of corn ears in the target corn field based on the visible light image, the infrared image and the multiple single ear grain weights; and calculating the corn yield based on the area, the total number of corn ears and the average single ear grain weight of the target corn field, wherein the corn yield is used to characterize the corn yield per unit area in the target corn field.

[0069] Figure 1 The application scenario diagram of the multi-perspective intelligent observation method of corn yield according to an embodiment of the present disclosure is schematically shown.

[0070] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a fourth terminal device 104, a network 105, and a server 106. The network 105 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, the fourth terminal device 104, and the server 106. The network 105 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0071] The user can use the first terminal device 101, the second terminal device 102, the third terminal device 103, and the fourth terminal device 104 to interact with the server 106 through the network 105 to receive or send messages, etc. The first terminal device 101 is used to collect a two-dimensional image of the target corn field, the second terminal device 102 is used to collect a three-dimensional point cloud of the target corn field, the third terminal device 103 is used to collect a visible light image of the exposed corn ears in the target corn ear group, and the fourth terminal device 104 is used to collect an infrared image of the exposed corn ears in the target corn ear group. Specifically, the first terminal device 101 can be a camera, a video camera, a mobile phone, etc., the second terminal device 102 can be a laser radar, a laser scanner, a depth camera, a 3D scanner, etc., the third terminal device 103 can be a camera, a video camera, a mobile phone, etc., and the fourth terminal device 104 can be an infrared camera, an infrared imager, an infrared camera, etc. The second terminal device 101 and the third terminal device 103 can be the same device or different devices. It should be noted that in the embodiment of the present disclosure, the first terminal device 101, the second terminal device 102, the third terminal device 103, and the fourth terminal device 104 can be integrated into the same device or distributed on different devices.

[0072] The server 106 may be a server that provides various services, such as a background management server (only as an example) that provides support for the data collected by the user using the first terminal device 101, the second terminal device 102, the third terminal device 103, and the fourth terminal device 104. The background management server may analyze and process the received two-dimensional image, three-dimensional point cloud, visible light image, and infrared image data, and feed back the processing results to the terminal device.

[0073] It should be noted that the multi-perspective intelligent observation method for corn yield provided in the embodiment of the present disclosure can generally be executed by the server 106. Correspondingly, the multi-perspective intelligent observation device for corn yield provided in the embodiment of the present disclosure can generally be arranged in the server 106. The multi-perspective intelligent observation method for corn yield provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 106 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 106. Correspondingly, the multi-perspective intelligent observation device for corn yield provided in the embodiment of the present disclosure can also be arranged in a server or server cluster that is different from the server 106 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 106. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0074] The following will be based on Figure 1 The scene described by Figure 2 to Figure 8 The multi-perspective intelligent observation method of corn yield in the disclosed embodiment is described in detail.

[0075] Figure 2 A flowchart of a multi-perspective intelligent observation method for corn yield according to an embodiment of the present disclosure is schematically shown.

[0076] like Figure 2 As shown, the multi-view intelligent observation method 200 of corn yield in this embodiment includes operations S210 to S250.

[0077] In operation S210 , a two-dimensional image and a three-dimensional point cloud of a target cornfield are acquired.

[0078] In operation S220, the area of ​​the target corn field, the total number of corn ears in the target corn field, and geometric parameters of each single corn ear of a target corn ear group in the target corn field are obtained based on the two-dimensional image and the three-dimensional point cloud.

[0079] In operation S230, a visible light image and an infrared image of exposed corn ears in the target corn ear group are acquired.

[0080] In operation S240, an average single-ear kernel weight of corn ears in the target corn field is obtained based on the visible light image, the infrared image, and a plurality of single-ear geometric parameters.

[0081] In operation S250, the corn yield per unit area is calculated based on the area, the total number of ears, and the average weight of grains per ear of the target corn field, wherein the corn yield per unit area is used to represent the corn yield per unit area in the target corn field.

[0082] According to an embodiment of the present disclosure, the target corn field may be any corn field for which the corn yield is to be calculated.

[0083] According to an embodiment of the present disclosure, in operation S210, a two-dimensional image of the target cornfield can be collected by a camera, a camcorder, a mobile phone and other devices; a three-dimensional point cloud of the target cornfield can be collected by a lidar, a laser scanner, a depth camera, a 3D scanner and other devices.

[0084] According to an embodiment of the present disclosure, in operation S210, a two-dimensional image and a three-dimensional point cloud of a target corn field at different corn growth stages may be acquired.

[0085] According to an embodiment of the present disclosure, in operation S220, the total number of corn ears in the target corn field is the sum of the number of corn ears in the target corn field. It should be noted that, since some corn plants have no corn ears, while some corn plants have double or multiple ears, the total number of corn ears does not correspond to the number of corn plants.

[0086] According to an embodiment of the present disclosure, in operation S220, the target corn ear group in the target corn field may be a corn ear group in a local area in the target corn field, for example, a corn ear group in the outermost row of the target corn field. Single ear geometric parameters may include ear length, ear diameter, ear surface area, columnarity, ear volume, etc., wherein the columnarity is the quotient obtained by dividing the ear surface area by the minimum circumscribed cylindrical area of ​​the ear.

[0087] According to an embodiment of the present disclosure, in operation S220, the geometric parameters of a single ear of corn of a target corn group in the target corn field can be obtained based on the three-dimensional point cloud and the two-dimensional image of the target corn group in the target corn field. For example, the geometric parameters of a single ear of corn of a target corn group in the outermost row can be obtained based on the three-dimensional point cloud and the two-dimensional image of the outermost row of the target corn field.

[0088] According to an embodiment of the present disclosure, in operation S230, the naked corn cobs may be peeled corn cobs. The naked corn cobs in the target corn cob group may be peeled corn cobs in a local area in the target cornfield, and the peeled corn cobs may be a certain proportion of corn cobs in the outermost row. For example, the naked corn cobs in the target corn cob group may be peeled corn cobs of 30% of the corn cobs randomly selected from the outermost row of the target cornfield.

[0089] According to an embodiment of the present disclosure, in operation S230, visible light images may be collected by a camera, a video camera, a mobile phone, or the like; infrared images may be collected by an infrared camera, an infrared imager, an infrared video camera, or the like.

[0090] According to an embodiment of the present disclosure, in operation S240, the average single-ear kernel weight of corn ears in the target corn field may be the average single-ear kernel weight of corn ears in the target corn field. It should be noted that the average single-ear kernel weight of corn ears is the weight of single-ear kernels after removing water, which is the dry weight of single-ear kernels.

[0091] According to an embodiment of the present disclosure, in operation S250, the corn yield per unit area may be calculated according to formula (1).

[0092]

[0093] According to the embodiments of the present disclosure, the area, total number of ears and geometric parameters of a single ear of the target corn field are obtained based on two-dimensional images and three-dimensional point clouds; the average grain weight of a single ear is obtained based on visible light images, infrared images and geometric parameters of a single ear; the corn yield per unit area is calculated based on the area, total number of ears and average grain weight of a single ear of the target corn field; the corn yield per unit area is calculated through data from on-site multi-view non-destructive observations, thereby improving the efficiency and accuracy of corn yield observations.

[0094] Figure 3 A flowchart for obtaining the total number of corn ears in a target corn field according to an embodiment of the present disclosure is schematically shown.

[0095] like Figure 3 As shown, the method for obtaining the total number of corn ears in the target corn field in this embodiment includes operations S310 to S330.

[0096] In operation S310, the total number of plants in the target corn field and the local number of plants in the target corn ear group are obtained according to the three-dimensional point cloud.

[0097] In operation S320, the number of corn ears in the target corn ear group is obtained according to the three-dimensional point cloud and the two-dimensional image.

[0098] In operation S330, the total number of corn ears in the target corn field is calculated based on the number of local plants, the number of corn ears in the target corn ear group, and the total number of plants.

[0099] According to an embodiment of the present disclosure, in operation S310, the total number of plants in the target corn field and the local number of plants in the target corn ear group can be obtained based on the three-dimensional point cloud of the target corn field at any corn growth stage. For example, the total number of plants in the target corn field and the local number of plants in the target corn ear group can be obtained based on the three-dimensional point cloud of the target corn field at the corn seedling stage, ear stage or grain stage. Specifically, for example, the three-dimensional point cloud of the seedling stage can be clustered in the horizontal direction to perform point cloud density analysis, and then the local maximum filtering of the three-dimensional point cloud can be performed to obtain the center position of each plant, and the three-dimensional point cloud can be segmented according to the center point of each plant, and the local number of plants and the total number of plants can be counted.

[0100] According to an embodiment of the present disclosure, in operation S310, the local plant number of the target corn ear group may be the number of corn plants in the outermost row of the target corn field, or the number of corn plants in other areas of the target corn field.

[0101] According to an embodiment of the present disclosure, in operation S320, the number of corn ears in the target corn ear group may be obtained based on the three-dimensional point cloud and the two-dimensional image of the target corn field at the corn maturity stage.

[0102] According to an embodiment of the present disclosure, in operation S330, the double ear rate of the target corn ear group can be calculated by the local plant number and the number of corn ears in the target corn ear group, and then the total number of corn ears in the target corn field can be calculated according to the double ear rate and the total number of plants. Specifically, the total number of corn ears in the target corn field can be calculated by the following formula (2).

[0103]

[0104] According to the embodiments of the present disclosure, the three-dimensional point cloud and the two-dimensional image are combined to obtain the number of corn ears in the target corn ear group, and then the total number of ears is calculated based on the number of local plants, the number of corn ears in the target corn ear group and the total number of plants. The calculation result is more accurate, which improves the accuracy of corn yield observation.

[0105] Figure 4 The flowchart of obtaining the number of corn ears in a target corn ear group according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 4 As shown, the method for obtaining the number of corn ears in a target corn ear group in this embodiment includes operation S410 and operation SS420.

[0107] In operation S410, detection frames of respective ears of corn within the target group of ears of corn are obtained according to the three-dimensional point cloud and the two-dimensional image.

[0108] In operation S420, the number of corn ears in the detection frame of each of the plurality of corn ears is counted to obtain the number of corn ears in the target corn ear group.

[0109] According to the embodiments of the present disclosure, the number of corn ears in the target corn ear group is obtained by counting the number of corn ears in the detection frame of each of the multiple corn ears. The operation is more convenient and the counting result is more accurate.

[0110] Figure 5 A flowchart for obtaining geometric parameters of individual ears of corn for each of a plurality of ears of corn in a target group of ears of corn in a target corn field according to an embodiment of the present disclosure is schematically shown.

[0111] like Figure 5As shown, the method of obtaining the geometric parameters of a single ear of corn of each of a plurality of corn ears in a target corn ear group in a target corn field in this embodiment includes operations S510 to S570.

[0112] In operation S510, detection frames of respective ears of corn within the target group of ears of corn are obtained according to the three-dimensional point cloud and the two-dimensional image.

[0113] In operation S520, the detection frames of the plurality of corn ears in the target corn ear group are mapped to a three-dimensional point cloud to obtain a first corn ear three-dimensional point cloud.

[0114] In operation S530, the farthest point sampling algorithm is used to sequentially perform non-uniform point cloud downsampling and point cloud upsampling based on inverse distance weighted interpolation on the first corn ear three-dimensional point cloud to obtain a plurality of point cloud features.

[0115] In operation S540 , a cross-level skip link hierarchical propagation strategy is used to concatenate multiple point cloud features to obtain a complete point cloud feature.

[0116] In operation S550, the complete point cloud features are segmented to obtain global point cloud features.

[0117] In operation S560, the global point cloud features are input into a fully connected network to perform point cloud classification, thereby obtaining a second corn ear three-dimensional point cloud classified as a corn ear.

[0118] In operation S570, geometric parameters of individual ears of corn are obtained for each of the plurality of ears of corn in the target group of ears of corn in the target corn field according to the second three-dimensional point cloud of the ear of corn.

[0119] According to an embodiment of the present disclosure, in operation S570, obtaining the geometric parameters of each single ear of corn in a target corn ear group in a target corn field according to the second corn ear three-dimensional point cloud may include: performing abnormal point filtering on the second corn ear three-dimensional point cloud; obtaining the circumscribed cylinder of the corn ear point cloud, and calculating the length and diameter of the cylinder. The corn ear point cloud is evenly divided into several layers from bottom to top, and the ear diameter of each layer is calculated; the ear volume of each layer is calculated according to the geometric shape and parameters of each layer, and the volume of the corn ear is obtained after the cumulative sum.

[0120] According to the embodiments of the present disclosure, the three-dimensional point cloud of the corn ear in the detection frame is subjected to step-by-step downsampling of the non-uniform point cloud and upsampling of the point cloud based on the inverse distance weighted interpolation in turn using the farthest point sampling algorithm, which can reduce the amount of calculation and obtain more and more accurate point cloud features; the point cloud features are spliced ​​using a cross-horizontal jump link hierarchical propagation strategy, and then the discriminative features of multiple point clouds are obtained for point cloud segmentation, which can further improve the accuracy of the geometric parameters of a single ear.

[0121] Figure 6A flowchart for obtaining detection frames of respective multiple ears of corn within a target group of ears of corn according to an embodiment of the present disclosure is schematically shown.

[0122] like Figure 6 As shown, the method for obtaining the detection frames of the plurality of corn ears in the target corn ear group in this embodiment includes operations S610 to S630.

[0123] In operation S610, the three-dimensional point cloud of the target corn ear group is transformed in a vertical direction to obtain a depth image.

[0124] In operation S620, feature extraction is performed on the two-dimensional image and the depth image to obtain extracted multi-layer features.

[0125] In operation S630, based on the multi-layer features, a multi-layer feature pyramid is constructed using a path aggregation network and decoupled to obtain respective detection frames of a plurality of corn ears in the target corn ear group.

[0126] According to an embodiment of the present disclosure, in operation S610, a three-dimensional point cloud of a target corn field at a corn maturity stage may be converted in a vertical direction to obtain a depth image.

[0127] According to an embodiment of the present disclosure, in operation S620, a convolutional neural network based on a combination of residual connection and dense connection may be used to perform feature extraction on the two-dimensional image and the depth image respectively to obtain extracted multi-layer features.

[0128] According to an embodiment of the present disclosure, in operation S630, the detection frames of the plurality of corn ears may be the detection frames of the plurality of corn ears in the depth image, or may be the detection frames of the plurality of corn ears in the two-dimensional image.

[0129] According to an embodiment of the present disclosure, a decoupling head can be used to decouple the feature pyramid to obtain the detection frames of multiple corn ears in the target corn ear group. The decoupling head may include an independent classification prediction head, a positioning prediction head, and an angle prediction head. The classification prediction head can be used to predict the target type of the feature pyramid to obtain the detection frame type. The positioning prediction head can be used to predict the target position and size of the feature pyramid to obtain the center position and size of the detection frame. The angle prediction head can be used to predict the target angle of the feature pyramid to obtain the detection frame angle, wherein the angle prediction head is composed of a channel attention layer and a convolution layer.

[0130] According to an embodiment of the present disclosure, a three-dimensional point cloud is converted into a depth image, and corn ear detection is performed by combining the depth image with the two-dimensional image, thereby improving the detection accuracy.

[0131] According to an embodiment of the present disclosure, obtaining the area of ​​the target corn field based on the two-dimensional image and the three-dimensional point cloud includes: determining the planting range of the target corn field in the two-dimensional image based on the three-dimensional point cloud, wherein the two-dimensional image is an orthophoto image; and determining the area of ​​the target corn field based on the planting range of the target corn field.

[0132] According to an embodiment of the present disclosure, the method for obtaining the area of ​​the target corn field may also include: setting the boundary of the target corn field to be observed, and planning the observation route, position and angle; synchronously collecting visible light images and laser point clouds of the target corn field at different positions and angles along the planned route until the entire target corn field is observed; processing the positioning data, inertial navigation data, visible light images and laser point clouds to obtain a three-dimensional point cloud and orthophoto image of the target corn field; denoising and ground segmenting the three-dimensional point cloud to obtain a normalized three-dimensional point cloud.

[0133] According to an embodiment of the present disclosure, by combining a two-dimensional image and a three-dimensional point cloud, the target cornfield area is obtained more accurately.

[0134] Figure 7 A flow chart for obtaining the average kernel weight per ear of corn ears in a target corn field according to an embodiment of the present disclosure is schematically shown.

[0135] like Figure 7 As shown, the method for obtaining the average kernel weight per ear of corn ears in a target corn field in this embodiment includes operations S710 to S760.

[0136] In operation S710, the moisture content of each of the plurality of exposed corn ears is obtained according to the infrared image and the visible light image;

[0137] In operation S720, the size of each of the plurality of exposed corn ears kernels is obtained according to the visible light image;

[0138] In operation S730, the kernel weight of each bare ear of corn is obtained according to the moisture content of each kernel of the bare ear of corn and the size of each kernel of the bare ear of corn.

[0139] In operation 740, a relationship model between the geometric parameters of a single ear of corn and the grain weight of a single ear of corn is established based on the geometric parameters of a single ear of corn and the grain weight of a single ear of corn.

[0140] In operation S750, the kernel weight of each of the plurality of corn ears in the target corn ear group is obtained based on the relationship model and the geometric parameters of each of the plurality of corn ears in the target corn ear group.

[0141] In operation S760, an average kernel weight per ear of corn ears in the target corn field is obtained according to the kernel weight per ear of each of the plurality of corn ears in the target corn ear group.

[0142] According to an embodiment of the present disclosure, in operation S710, the infrared image and the visible light image may be registered, the infrared image may be segmented using the result of the visible light image ear-grain segmentation, and the average moisture content of each ear-grain may be calculated using the average reflectivity of each ear-grain in the infrared image and the relationship model between moisture content and reflectivity. The infrared image may be a short-wave infrared image.

[0143] According to an embodiment of the present disclosure, in operation S730, the weight of a single ear of kernels may be calculated by adding the weights of all kernels of a corn ear. For example, the weight of a single ear of kernels may be calculated by the following formula (3), where n is the number of kernels of each corn ear.

[0144]

[0145] According to an embodiment of the present disclosure, in operation S740, three most correlated single ear geometric parameters and single ear kernel weights of each naked corn ear may be obtained, and a relationship model between single ear geometric parameters and single ear kernel weights may be established.

[0146] According to the embodiments of the present disclosure, the kernel weight of each of the multiple corn ears in the target corn ear group can be obtained based on the relationship model and the geometric parameters of a single ear; then, the average kernel weight of the multiple corn ears in the target corn ear group can be calculated based on the kernel weight of each of the multiple corn ears in the target corn ear group; the average kernel weight of the multiple corn ears in the target corn ear group is used to characterize the average kernel weight of the corn ears in the target corn field, thereby further improving the calculation efficiency and accuracy of the average kernel weight.

[0147] According to an embodiment of the present disclosure, obtaining the size of each of the multiple exposed corn cob kernels according to the visible light image includes:

[0148] The visible light image is subjected to background removal, color space transformation, edge enhancement and adaptive threshold segmentation in sequence to obtain the sizes of kernels of multiple exposed corn ears.

[0149] According to another embodiment of the present disclosure, obtaining the number of kernels in a single ear of corn and the size of each kernel of the corn ear includes: performing background removal, color space transformation, edge enhancement and adaptive threshold segmentation on a visible light image in sequence to obtain an image after the kernels are separated and the size of each kernel; performing corrosion operation on the image after the kernels are separated, calculating the centroid of the kernels after corrosion, counting the centroids of the kernels, and obtaining the number of kernels in a single ear.

[0150] Figure 8 The flowchart of obtaining the number of kernels per ear of corn and the size of each kernel of corn ear according to an embodiment of the present disclosure is schematically shown.

[0151] like Figure 8As shown, the method of obtaining the number of kernels in a single ear of corn and the size of each kernel in the ear of corn in this embodiment includes operations S810 to S850.

[0152] In operation S810, the visible light image is processed using a moving average algorithm, and the processing result is subjected to threshold segmentation, and the segmented background portion becomes black.

[0153] In operation S820, color transformation is performed on the corn ear picture with a black background, from the RGB space to the Lab space.

[0154] In operation S830, a color deconvolution algorithm is used to expand the color gradient of the ear of corn to enhance the edge of the corn kernel.

[0155] In operation S840, an adaptive threshold method is used to compensate for the non-uniformity of illumination or reflection to separate the corn kernels, and an image of the separated kernels and the sizes of the kernels of the corn ear are obtained.

[0156] In operation S850, a erosion operation is performed on the corn kernels, the centroid of the kernels after erosion is calculated, and the centroids are counted to obtain the number of kernels per ear.

[0157] Based on the multi-view intelligent observation method of corn yield, the present disclosure also provides a multi-view intelligent observation device for corn yield. Fig. 9 The device is described in detail.

[0158] Fig. 9 The structural block diagram of the multi-view intelligent observation device for corn yield according to an embodiment of the present disclosure is schematically shown.

[0159] like Fig. 9 As shown, the multi-view intelligent observation device 900 for corn yield of this embodiment includes a first acquisition module 910 , a first acquisition module 920 , a second acquisition module 930 , a second acquisition module 940 and a calculation module 950 .

[0160] The first acquisition module 910 is used to acquire a two-dimensional image and a three-dimensional point cloud of the target cornfield. In one embodiment, the first acquisition module 910 can be used to perform the operation S210 described above, which will not be described in detail here.

[0161] The first acquisition module 920 is used to obtain the area of ​​the target corn field, the total number of corn ears in the target corn field, and the geometric parameters of the single ear of corn of the target corn ear group in the target corn field according to the two-dimensional image and the three-dimensional point cloud. In one embodiment, the first acquisition module 920 can be used to perform the operation S220 described above, which will not be repeated here.

[0162] The second acquisition module 930 is used to acquire the visible light image and the infrared image of the exposed corn ears in the target corn ear group. In one embodiment, the second acquisition module 930 can be used to perform the operation S230 described above, which will not be described in detail here.

[0163] The second obtaining module 940 is used to obtain the average single ear kernel weight of corn ears in the target corn field according to the visible light image, the infrared image and multiple single ear geometric parameters. In one embodiment, the second obtaining module 940 can be used to perform the operation S240 described above, which will not be repeated here.

[0164] The calculation module 950 is used to calculate the corn yield per unit area based on the area, total number of ears and average single ear grain weight of the target corn field, wherein the corn yield per unit area is used to characterize the corn yield per unit area in the target corn field. In one embodiment, the calculation module 950 can be used to perform the operation S250 described above, which will not be repeated here.

[0165] According to an embodiment of the present disclosure, the first acquisition module 920 includes a first acquisition submodule, a second acquisition submodule and a third acquisition submodule. The first acquisition submodule is used to obtain the total number of corn ears in the target corn field according to the two-dimensional image and the three-dimensional point cloud; the second acquisition submodule is used to obtain the geometric parameters of each single ear of corn in the target corn ear group in the target corn field according to the two-dimensional image and the three-dimensional point cloud; the third acquisition submodule is used to obtain the area of ​​the target corn field according to the two-dimensional image and the three-dimensional point cloud.

[0166] According to an embodiment of the present disclosure, the first acquisition submodule includes a first acquisition unit, a second acquisition unit and a calculation unit. The first acquisition unit is used to obtain the total number of plants in the target corn field and the number of local plants in the target corn cob group according to the three-dimensional point cloud; the second acquisition unit is used to obtain the number of corn cobs in the target corn cob group according to the three-dimensional point cloud and the two-dimensional image; the calculation unit is used to calculate the total number of corn cobs in the target corn field according to the number of local plants, the number of corn cobs in the target corn cob group and the total number of plants.

[0167] According to an embodiment of the present disclosure, the second obtaining unit includes an obtaining subunit and a counting subunit. The obtaining subunit is used to obtain the detection frames of the multiple corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image; the counting subunit is used to count the number of corn ears in the detection frames of the multiple corn ears, and obtain the number of corn ears in the target corn ear group.

[0168] According to an embodiment of the present disclosure, the second acquisition submodule includes a third acquisition unit, a mapping unit, a sampling unit, a splicing unit, a segmentation unit, a classification unit and a fourth acquisition unit. Among them, the third acquisition unit is used to obtain the detection frames of multiple corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image; the mapping unit is used to map the detection frames of multiple corn ears in the target corn ear group to the three-dimensional point cloud to obtain the first corn ear three-dimensional point cloud; the sampling unit is used to sequentially use the farthest point sampling algorithm to perform non-uniform point cloud step-by-step downsampling and point cloud upsampling based on inverse distance weighted interpolation on the first corn ear three-dimensional point cloud to obtain multiple point cloud features; the splicing unit is used to splice multiple point cloud features using a cross-level jump link hierarchical propagation strategy to obtain a complete point cloud feature; the segmentation unit is used to perform point cloud segmentation on the complete point cloud feature to obtain a global point cloud feature; the classification unit is used to input the global point cloud feature into a fully connected network to perform point cloud classification to obtain a second corn ear three-dimensional point cloud of the category of corn ear; the fourth acquisition unit is used to obtain the geometric parameters of a single ear of each of the multiple corn ears in the target corn ear group in the target cornfield according to the second corn ear three-dimensional point cloud.

[0169] According to an embodiment of the present disclosure, the third acquisition unit or acquisition subunit is also used to convert the three-dimensional point cloud of the target corn cob group in the vertical direction to obtain a depth image; perform feature extraction on the two-dimensional image and the depth image respectively to obtain the extracted multi-layer features; based on the multi-layer features, a path aggregation network is used to construct a multi-layer feature pyramid and decouple it to obtain the detection frames of multiple corn cobs in the target corn cob group.

[0170] According to an embodiment of the present disclosure, the third obtaining submodule includes a first determining unit and a second determining unit. The first determining unit is used to determine the planting range of the target corn field in the two-dimensional image according to the three-dimensional point cloud, wherein the two-dimensional image is an orthophoto image; and the second determining unit is used to determine the area of ​​the target corn field according to the planting range of the target corn field.

[0171] According to an embodiment of the present disclosure, the second acquisition module 940 includes a fourth acquisition submodule, a fifth acquisition submodule, a sixth acquisition submodule, an establishment submodule, a seventh acquisition submodule, and an eighth acquisition submodule. Among them, the fourth acquisition submodule obtains the moisture content of each of the kernels of multiple naked corn ears according to the infrared image and the visible light image; the fifth acquisition submodule is used to obtain the size of each of the kernels of multiple naked corn ears according to the visible light image; the sixth acquisition submodule is used to obtain the single-ear weight of each of the multiple naked corn ears according to the moisture content of each kernel and the size of each kernel; the establishment submodule is used to establish a relationship model between the single-ear geometric parameters and the single-ear weight based on the single-ear geometric parameters of each of the multiple naked corn ears and the single-ear weight of each of the multiple naked corn ears; the seventh acquisition submodule is used to obtain the single-ear weight of each of the multiple corn ears of the target corn ear group based on the relationship model and the single-ear geometric parameters of each of the multiple corn ears of the target corn ear group; the eighth acquisition submodule is used to obtain the average single-ear weight of the corn ears in the target corn field according to the single-ear weight of each of the multiple corn ears of the target corn ear group.

[0172] According to an embodiment of the present disclosure, the fifth acquisition submodule includes a fifth acquisition unit, wherein the fifth acquisition unit is used to sequentially perform background removal, color space transformation, edge enhancement, and adaptive threshold segmentation on the visible light image to obtain the size of each kernel of multiple exposed corn ears.

[0173] According to an embodiment of the present disclosure, any multiple modules of the first acquisition module 910, the first acquisition module 920, the second acquisition module 930, the second acquisition module 940 and the calculation module 950 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first acquisition module 910, the first acquisition module 920, the second acquisition module 930, the second acquisition module 940 and the calculation module 950 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the first acquisition module 910 , the first acquisition module 920 , the second acquisition module 930 , the second acquisition module 940 and the calculation module 950 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0174] Fig.10 A block diagram of an electronic device suitable for implementing a multi-view intelligent observation method for corn yield according to an embodiment of the present disclosure is schematically shown.

[0175] like Fig.10 As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0176] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in ROM 1002 and / or RAM 1003. It should be noted that the above-mentioned programs can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the above-mentioned one or more memories.

[0177] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0178] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0179] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0180] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the multi-view intelligent observation method for corn yield provided by the embodiment of the present disclosure.

[0181] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0182] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0183] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0184] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0185] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0186] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0187] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A multi-view intelligent observation method for corn yield, comprising: Obtain a 2D image and 3D point cloud of the target cornfield; According to the two-dimensional image and the three-dimensional point cloud, the area of ​​the target corn field, the total number of corn ears in the target corn field, and the geometric parameters of each single corn ear of a plurality of corn ears in a target corn ear group in the target corn field are obtained; Acquire visible light images and infrared images of exposed corn ears in the target corn ear group; Obtaining an average single-ear grain weight of corn ears in the target corn field according to the visible light image, the infrared image and the plurality of single-ear geometric parameters; as well as The corn yield per unit area is calculated based on the area of ​​the target corn field, the total number of ears and the average single ear kernel weight, wherein the corn yield per unit area is used to characterize the corn yield per unit area in the target corn field; Wherein, obtaining the geometric parameters of a single ear of corn in a target corn ear group in the target corn field according to the two-dimensional image and the three-dimensional point cloud includes: Obtaining detection frames of respective multiple corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image; Mapping the detection frames of the plurality of corn ears in the target corn ear group to the three-dimensional point cloud to obtain a first corn ear three-dimensional point cloud; The first corn ear three-dimensional point cloud is sequentially downsampled by the farthest point sampling algorithm and upsampled by the point cloud based on inverse distance weighted interpolation to obtain multiple point cloud features; The multiple point cloud features are spliced ​​by adopting a cross-level skip link hierarchical propagation strategy to obtain a complete point cloud feature; Performing point cloud segmentation on the complete point cloud features to obtain global point cloud features; Inputting the global point cloud features into a fully connected network to perform point cloud classification to obtain a second corn ear three-dimensional point cloud classified as a corn ear; According to the second corn ear three-dimensional point cloud, obtaining geometric parameters of a single corn ear of each of a plurality of corn ears in a target corn ear group in the target corn field; The step of obtaining the average single-ear grain weight of corn ears in the target corn field according to the visible light image, the infrared image and the plurality of single-ear geometric parameters comprises: Obtaining the moisture content of each of the plurality of exposed corn ears according to the infrared image and the visible light image; According to the visible light image, obtaining the size of each of the plurality of exposed corn ears; Obtaining the single-ear kernel weight of each of the plurality of naked corn ears according to the moisture content of each of the naked corn ears kernels and the size of each of the naked corn ears kernels; Based on the geometric parameters of each single ear of corn and the grain weight of each single ear of corn, a relationship model between the geometric parameters of each single ear of corn and the grain weight of each single ear of corn is established; Based on the relationship model and the geometric parameters of each single ear of corn of the target corn ear group, obtaining the single ear grain weight of each of the multiple corn ears of the target corn ear group; According to the single-ear kernel weights of the plurality of corn ears in the target corn ear group, the average single-ear kernel weight of the corn ears in the target corn field is obtained.

2. The method according to claim 1, wherein: The obtaining the total number of corn ears in the target corn field according to the two-dimensional image and the three-dimensional point cloud comprises: According to the three-dimensional point cloud, the total number of plants in the target corn field and the local number of plants in the target corn ear group are obtained; Obtaining the number of corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image; The total number of corn ears in the target corn field is calculated based on the local plant number, the number of corn ears in the target corn ear group and the total plant number.

3. The method according to claim 2, wherein: The step of obtaining the number of corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image comprises: Obtaining detection frames of respective multiple corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image; The number of corn ears in the detection frame of each of the plurality of corn ears is counted to obtain the number of corn ears in the target corn ear group.

4. The method according to claim 3, wherein: The step of obtaining the detection frames of the plurality of corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image comprises: Transforming the three-dimensional point cloud of the target corn ear group in the vertical direction to obtain a depth image; Performing feature extraction on the two-dimensional image and the depth image respectively to obtain extracted multi-layer features; Based on the multi-layer features, a multi-layer feature pyramid is constructed using a path aggregation network and decoupled to obtain respective detection frames of multiple corn ears in the target corn ear group.

5. The method according to claim 1, wherein: The obtaining of the sizes of the multiple exposed corn ears and kernels according to the visible light image comprises: The visible light image is subjected to background removal, color space transformation, edge enhancement and adaptive threshold segmentation in sequence to obtain the sizes of the kernels of the plurality of exposed corn ears.

6. The method according to claim 1, wherein: According to the two-dimensional image and the three-dimensional point cloud, the area of ​​the target cornfield is obtained including: Determine the planting range of the target corn field in the two-dimensional image according to the three-dimensional point cloud, wherein the two-dimensional image is an orthophoto image; The area of ​​the target corn field is determined according to the planting range of the target corn field.

7. A multi-view intelligent observation device for corn yield, comprising: A first acquisition module is used to acquire a two-dimensional image and a three-dimensional point cloud of a target cornfield; A first obtaining module is used to obtain the area of ​​the target corn field, the total number of corn ears in the target corn field, and the geometric parameters of each single corn ear of a plurality of corn ears in a target corn ear group in the target corn field according to the two-dimensional image and the three-dimensional point cloud; A second acquisition module is used to acquire visible light images and infrared images of exposed corn ears in the target corn ear group; A second obtaining module is used to obtain an average single ear grain weight of corn ears in the target corn field according to the visible light image, the infrared image and the plurality of single ear geometric parameters; as well as A calculation module, used for calculating the corn yield per unit area based on the area of ​​the target corn field, the total number of ears and the average single ear kernel weight, wherein the corn yield per unit area is used to characterize the corn yield per unit area in the target corn field; Wherein, the first acquisition module includes a second acquisition submodule, and the second acquisition submodule is used for the two-dimensional image and the three-dimensional point cloud to obtain the geometric parameters of each single ear of corn in the target corn ear group in the target corn field; The second obtaining submodule comprises: A third obtaining unit is used to obtain a detection frame of each of the plurality of corn ears in the target corn ear group according to the three-dimensional point cloud and the two-dimensional image; A mapping unit, used for mapping the detection frames of the plurality of corn ears in the target corn ear group to the three-dimensional point cloud to obtain a first corn ear three-dimensional point cloud; A sampling unit is used to sequentially downsample the non-uniform point cloud by using a farthest point sampling algorithm and upsample the point cloud based on inverse distance weighted interpolation to obtain a plurality of point cloud features. A splicing unit, used for splicing the plurality of point cloud features by adopting a cross-level skip link hierarchical propagation strategy to obtain a complete point cloud feature; A segmentation unit, used for performing point cloud segmentation on the complete point cloud feature to obtain a global point cloud feature; a classification unit, configured to input the global point cloud features into a fully connected network, perform point cloud classification, and obtain a second corn ear three-dimensional point cloud classified as a corn ear; A fourth obtaining unit is used to obtain, according to the second corn ear three-dimensional point cloud, a single corn ear geometric parameter of each of a plurality of corn ears in a target corn ear group in the target corn field; The second obtaining module comprises: A fourth obtaining submodule is used to obtain the moisture content of each of the plurality of exposed corn ears according to the infrared image and the visible light image; A fifth obtaining submodule, used for obtaining the size of each kernel of a plurality of exposed corn ears according to the visible light image; A sixth obtaining submodule is used to obtain the single-ear kernel weight of each of the plurality of naked corn ears according to the moisture content of each kernel of the naked corn ear and the size of each kernel of the naked corn ear; Establishing a submodule for establishing a relationship model between the geometric parameters of a single ear of corn and the grain weight of a single ear of corn based on the geometric parameters of a single ear of corn and the grain weight of a single ear of corn; A seventh obtaining submodule, for obtaining the single ear grain weight of each of the multiple corn ears of the target corn ear group based on the relationship model and the single ear geometric parameters of each of the multiple corn ears of the target corn ear group; The eighth obtaining submodule is used to obtain the average single-ear kernel weight of corn ears in the target corn field according to the single-ear kernel weight of each of the multiple corn ears in the target corn ear group.

8. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

Images