Plant growth state detection method, system, program and electronic terminal

Abstract

According to the plant growth state detection method and system, the program and the electronic terminal provided by the invention, by setting a multi-task learning method, a model can process a plurality of targets (density, seedling line and coverage rate) at the same time and share low-level features, so that the training process is more efficient, redundant calculation during traditional independent processing of each task is avoided, and the training efficiency is improved. The comprehensive capability of the model is improved, information of different tasks can be integrated under a unified framework, and then the overall performance is optimized.

Description

Technical Field

[0001] The present application relates to the field of plant detection, and in particular to a method, system, program and electronic terminal for detecting plant growth status. Background Art

[0002] With the rapid development of modern agriculture, precision agriculture technology has become an important means to increase crop yields, reduce resource waste, and address climate change. Plant monitoring technology can provide real-time perception of crop growth, providing a scientific basis for precision agriculture.

[0003] Traditional monitoring methods rely on manual labor, are inefficient, and are costly. In recent years, computer vision technology based on deep learning has made significant progress in image analysis and can automatically identify plant features. However, existing studies often treat plant density estimation, seedling line identification, and coverage separately, lacking a joint optimization solution. This results in incomplete monitoring results and fails to fully utilize the advantages of multi-task learning. Therefore, the present invention aims to develop a joint output network based on deep learning to comprehensively monitor plant density, seedling lines, and coverage, and use seedling line identification to support visual navigation, thereby improving the level of agricultural automation. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of this application is to provide a method for solving the above-mentioned problems.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a plant growth status detection method, comprising: obtaining a first initial image for training; preprocessing the first initial image to generate a first preprocessed image, wherein the preprocessing includes cropping and downsampling the first initial image according to preset parameters; performing feature extraction on the first preprocessed image to generate a plurality of first feature images, wherein the feature extraction includes using a multi-level convolutional layer to perform feature extraction on the first preprocessed image from local to global level, retaining the feature image output by each convolutional layer, and extracting the global feature of the feature image corresponding to the last level of convolutional layer as the first feature; mapping the first feature to the density recognition space, the seedling line recognition space and the coverage recognition space respectively, and outputting the corresponding density information, seedling line information and coverage information for training respectively; verifying the density information, seedling line information and coverage information for training and generating a corresponding loss function; obtaining a second initial image for reasoning, and repeatedly performing the above-mentioned preprocessing, feature extraction and mapping steps performed on the first initial image on the second initial image, and outputting the corresponding density information, seedling line information and coverage information for reasoning respectively.

[0006] In an embodiment of the first aspect of the present application, the method for calculating the loss function includes: Ltotal =L density +L lane +L cover , L lane =L cls +αL str +βL seg , wherein the L total is the total loss function, L density is the density loss function, L cover is the coverage loss function, L lane is the seedling line loss function, L cls is the classification loss of each point on each line, L str is the loss of shape of the seedling line, L seg is the seedling line segmentation loss, α and β are the str and L seg The scaling factor.

[0007] In one embodiment of the first aspect of the present application, the L density and L cover Use the cross entropy algorithm function.

[0008] In an embodiment of the first aspect of the present application, the steps of verifying the density information, seedling line information and coverage information used for training and generating a corresponding loss function include: after performing feature extraction on the first preprocessed image to generate several first feature images, using a semantic segmentation branch to aggregate global information and local information of the several first feature images, and verifying the output data of the semantic segmentation branch with the density information, seedling line information and coverage information used for training to obtain a loss function.

[0009] In an embodiment of the first aspect of the present application, the steps of mapping the first features to the seedling line recognition space and then outputting the corresponding seedling line information for training include: when each ridge of plants is arranged vertically, setting m equally spaced second horizontal dividing lines from top to bottom in the vertical direction of the first initial image, and setting n equally spaced second vertical dividing lines from left to right in the horizontal direction, each second horizontal dividing line corresponds to a second vertical coordinate value, and each second vertical dividing line corresponds to a second horizontal coordinate value; the probability value of the seedling line feature appearing in the rectangle enclosed by the second vertical dividing line corresponding to the second vertical coordinate value and the second vertical dividing line adjacent above or below it is added to the second vertical coordinate value to obtain the second vertical coordinate parameter; the seedling line information includes the second vertical coordinate parameter, the second horizontal coordinate value and the second horizontal dividing line corresponding to each second vertical coordinate value and the second horizontal dividing line adjacent above or below it. the number of seedling lines in the rectangle enclosed by the second horizontal dividing lines; when each ridge of plants is arranged horizontally, x third vertical dividing lines with equal spacing are set from left to right in the horizontal direction of the first initial image, and y third horizontal dividing lines 4 with equal spacing are set from top to bottom in the vertical direction, each third horizontal dividing line 4 corresponds to a third horizontal coordinate value, and each third horizontal dividing line 4 corresponds to a third vertical coordinate value; the probability value of the seedling line feature appearing in the rectangle enclosed by the third horizontal dividing line 4 corresponding to the third horizontal dividing line 4 and the third horizontal dividing line 4 adjacent above or below is added to the third horizontal coordinate value to obtain the third horizontal coordinate parameter; the seedling line information includes the third horizontal coordinate parameter, the third vertical coordinate value and the number of seedling lines in the rectangle enclosed by the third vertical dividing line corresponding to each third horizontal coordinate value and the third vertical dividing line adjacent to the left or right side.

[0010] In an embodiment of the first aspect of the present application, the step of mapping the first features to the density recognition space and then outputting the corresponding density information for training includes: dividing the plant density into k levels; when the plants in each ridge are arranged vertically, setting p equally spaced first horizontal dividing lines from top to bottom in the first initial image along the vertical direction, and each first horizontal dividing line corresponds to a first vertical coordinate value; the density information includes the first vertical coordinate value, the level of plant density, and the number of seedling lines in the rectangle enclosed by the first horizontal dividing line corresponding to each first vertical coordinate and the first horizontal dividing line adjacent to it above or below; when the plants in each ridge are arranged horizontally, setting q equally spaced first vertical dividing lines from left to right in the first initial image along the horizontal direction, and each first vertical dividing line corresponds to a first horizontal coordinate value; the density information includes the first horizontal coordinate value, the level of plant density, and the number of seedling lines in the rectangle enclosed by the first vertical dividing line corresponding to each first horizontal coordinate and the first vertical dividing line adjacent to it on the left or right.

[0011] In an embodiment of the first aspect of the present application, the coverage information includes a third horizontal coordinate value, a third vertical coordinate value, and the number of seedling lines in a rectangle enclosed by the third longitudinal dividing line corresponding to each third horizontal coordinate value and the third longitudinal dividing line adjacent to its left or right side.

[0012] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a plant growth status detection system, comprising: an image acquisition module, for acquiring a first initial image for training and a second initial image for reasoning; a preprocessing module, for preprocessing the first initial image to generate a first preprocessed image, and preprocessing the second initial image to generate a second preprocessed image, wherein the preprocessing includes cropping and downsampling the first initial image according to preset parameters; a feature extraction module, for performing feature extraction on the first preprocessed image to generate a plurality of first feature images, and performing feature extraction on the second preprocessed image to generate a plurality of second feature images, wherein the feature extraction includes using multiple The first convolution layer extracts features of the first preprocessed image from local to global level by level, retains the feature image output by each convolution layer, and extracts the global feature of the feature image corresponding to the last convolution layer as the first feature; the calculation output module is used to map the first feature to the density recognition space, the seedling line recognition space and the coverage recognition space respectively, and output the corresponding density information, seedling line information and coverage information for training respectively, and map the second feature to the density recognition space, the seedling line recognition space and the coverage recognition space respectively, and output the corresponding density information, seedling line information and coverage information for reasoning respectively; the compensation module is used to verify the density, seedling line and coverage information for training and generate a corresponding loss function.

[0013] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer program product, which includes computer program code. When the computer program code is run on a computer, the computer implements any of the methods described above.

[0014] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the aforementioned methods.

[0015] As described above, the present application has the following beneficial effects:

[0016] The present invention sets a multi-task learning method so that the model can process multiple targets (density, seedling line, coverage) at the same time and share the underlying features, making the training process more efficient, avoiding the redundant calculations when traditionally processing each task separately, improving the comprehensive ability of the model, and helping to integrate information from different tasks under a unified framework, thereby optimizing overall performance. In addition, the present invention also provides a loss function calculation method in multi-task learning, as well as a specific model operation method. For example, by setting uniform dividing lines along the vertical or horizontal direction of the image, the position and form of the seedling line can be accurately captured under different arrangements. Such a design helps to improve the accuracy of seedling line recognition and avoids inaccurate recognition caused by resolution or illumination problems in traditional methods. That is, the technical solution proposed in this application can achieve joint optimization and accurate recognition of plant density, seedling line, and coverage, adapt to the needs of automated monitoring in complex agricultural scenarios, and provide effective technical support for agricultural management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Shown is a schematic diagram of segmentation of a first initial image when plants in each ridge are arranged longitudinally in one embodiment of the present application.

[0018] Figure 2 Shown is a schematic diagram of segmentation of a first initial image when plants in each ridge are arranged horizontally in one embodiment of the present application.

[0019] Figure 3 Shown is a schematic diagram of segmentation of a first initial image when plants in each ridge are arranged longitudinally in one embodiment of the present application.

[0020] Figure 4 Shown is a schematic diagram of segmentation of a first initial image when plants in each ridge are arranged horizontally in one embodiment of the present application.

[0021] Figure 5 The output shape of the seedling line recognition when the plants in each ridge are arranged longitudinally in one embodiment of the present application is shown.

[0022] Figure 6 The output shape of density recognition when plants in each ridge are arranged longitudinally in one embodiment of the present application is shown.

[0023] Figure 7 The display shows the output shape of coverage recognition when plants in each ridge are arranged longitudinally in one embodiment of the present application.

[0024] Figure 8 Shown is a structural schematic diagram of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0026] As shown in the figure, the first aspect of the present application provides a method for detecting plant growth status, comprising:

[0027] S1: Get the first initial image for training.

[0028] S2: Preprocessing the first initial image to generate a first preprocessed image, wherein the preprocessing includes cropping and downsampling the first initial image according to preset parameters.

[0029] Specifically, the first initial image is cropped, and the cropped area is determined according to preset parameters. Usually, the area with low clarity is cropped to improve the accuracy of the output result, and it can reduce the amount of calculation, concentrate on processing the image area related to the target, and ensure that subsequent processing is more efficient. Secondly, downsampling is performed. Downsampling is to reduce the size and amount of data of the image by lowering the image resolution. Usually, a certain ratio (such as 2:1, 4:1, etc.) is used to reduce the image to reduce the amount of calculation. Larger image size will increase the complexity of training and memory consumption. Through downsampling, subsequent feature extraction can be performed more quickly, and overfitting can also be prevented, because the lower resolution can make the model pay more attention to the global features of the image rather than local details.

[0030] S3: Perform feature extraction on the first preprocessed image to generate a plurality of first feature images, wherein the feature extraction includes using a multi-level convolutional layer to perform feature extraction on the first preprocessed image step by step from local to global, and retaining the feature image output by each convolutional layer.

[0031] In this phase, the convolutional neural network (CNN) first uses the convolutional layers of the preprocessed image to extract features. This feature extraction process proceeds layer by layer through multiple convolutional layers, typically starting with low-level image features (such as edges, textures, and color variations) and gradually transitioning to higher-level features that contain global information. Each convolutional layer produces a feature map that contains different levels of image information. Local feature extraction refers to the fact that in the initial layers of the CNN, convolution operations primarily focus on extracting features within local regions. These features are typically simple, low-level information such as edges, corners, and textures. By sliding the filters (convolution kernels), the model can identify basic morphology and structure in the image. Layer-by-layer progression refers to the fact that as the network progresses, the convolutional layers gradually extract more complex features, gradually moving from local to global features. Intermediate layers may focus on more complex local structures (such as shapes and color combinations), while higher-level convolutional layers begin to learn global information and abstract object features within the image.

[0032] During this process, each convolutional layer generates a feature map, and the feature maps output by each convolutional layer are retained. During the training phase, by inputting the feature maps of each layer into the semantic segmentation branch, the model can be optimized simultaneously at multiple levels. The feature maps of each convolutional layer contribute differently to the model. Especially during the calculation of the loss function, layer-by-layer optimization can help the network gradually improve the segmentation results from low-level to high-level features, ensuring a more refined and comprehensive calculation of the loss function.

[0033] S4: Extract the global features of the feature image corresponding to the last level of convolutional layer as the first feature.

[0034] Because the primary convolutional layers of convolutional neural networks usually focus on local features in the image, such as edges, corners, and textures, as the convolutional layers go deeper, the network begins to capture more complex global structures composed of local features. Therefore, in the last convolutional layer of the network, the feature map will integrate the global information of the image to represent more abstract features such as the overall structure of the image, the morphology of objects, and semantics. In the last convolutional layer of the network, the feature map contains depth information from multiple areas of the image. By performing operations such as pooling or global average pooling on the feature map, the global features of the image can be further extracted. Therefore, the first feature is the most representative and comprehensive feature extracted by the network through multi-level convolutional layers. By extracting this feature, the network can better cope with changes in different scenes and backgrounds, thereby improving its stability and adaptability in various practical applications.

[0035] S5: Map the first feature to the density recognition space, seedling line recognition space and coverage recognition space respectively, and then output the corresponding density information, seedling line information and coverage information for training respectively.

[0036] like Figure 1-2 As shown, in an embodiment of the first aspect of the present application, the steps of mapping the first features to the seedling line recognition space and outputting the corresponding seedling line information for training include: Figure 1 As shown, when the plants in each ridge are arranged vertically, m equally spaced second horizontal dividing lines 1 are set in the vertical direction from top to bottom of the first initial image, and n equally spaced second vertical dividing lines 2 are set in the horizontal direction from left to right, each second horizontal dividing line 1 corresponds to a second vertical coordinate value, and each second vertical dividing line 2 corresponds to a second horizontal coordinate value; the probability value of the seedling line feature appearing in the rectangle enclosed by the second vertical dividing line 2 corresponding to the second vertical coordinate value and the second vertical dividing line 2 adjacent above or below it is added to the second vertical coordinate value to obtain the second vertical coordinate parameter; the seedling line information includes the second vertical coordinate parameter, the second horizontal coordinate value and the number of seedling lines in the rectangle enclosed by the second horizontal dividing line 1 corresponding to each second vertical coordinate value and the second horizontal dividing line 1 adjacent above or below it. number; when the plants in each ridge are arranged horizontally, x third longitudinal dividing lines 3 with equal spacing are set in the first initial image from left to right in the horizontal direction, and y third transverse dividing lines 4 with equal spacing are set in the longitudinal direction from top to bottom, each third transverse dividing line 4 corresponds to a third horizontal coordinate value, and each third transverse dividing line 4 corresponds to a third vertical coordinate value; the probability value of the seedling line feature appearing in the rectangle enclosed by the third transverse dividing line 4 corresponding to the third transverse coordinate value and the third transverse dividing line 4 adjacent above or below it is added to the third transverse coordinate value to obtain the third transverse coordinate parameter; the seedling line information includes the third transverse coordinate parameter, the third vertical coordinate value and the number of seedling lines in the rectangle enclosed by the third longitudinal dividing line 3 corresponding to each third transverse coordinate value and the third longitudinal dividing line 3 adjacent to its left or right side.

[0037] For example, when the plants in each ridge are arranged vertically, assume that the first initial image used for training is an RGB image with a size of 640*480 and 3 channels; the image obtained after cropping is 448*320 with 3 channels; the first pre-processed image obtained after downsampling by 4 times is 80*112 with 64 channels. Assume that there are three convolutional layers to extract features from the first pre-processed image, and the feature sizes of the images obtained from the bottom layer to the top layer are: [128, 40, 56], [256, 20, 28] and [512, 10, 14], respectively named feature1, feature2, and feature3. Use Global Average Pooling (GAP) to extract the global features of feature3 (the feature image corresponding to the last level of convolutional layer), and use the fully connected layer (FC) to map the features to the seedling line recognition space. The output shape is as follows: Figure 5 The value is [113, 31, 6], where 113 is the second vertical coordinate parameter, which is obtained by adding the second vertical coordinate value 112 and the probability value 1, 31 is the second horizontal coordinate value, and 6 represents the number of seedling lines.

[0038] like Figure 3-4 As shown, in an embodiment of the first aspect of the present application, the step of mapping the first features to the density recognition space and then outputting the corresponding density information for training includes: dividing the plant density into k levels; Figure 3 As shown, when each ridge of plants is arranged vertically, p equal-spaced first transverse dividing lines 5 are set along the vertical direction from top to bottom in the first initial image, and each first transverse dividing line 5 corresponds to a first ordinate value; the density information includes the first ordinate value, the level of plant density, and the number of seedling lines within the rectangle enclosed by each first transverse dividing line 5 corresponding to the first ordinate and the first transverse dividing line 5 adjacent above or below it. Figure 4 As shown, when the plants in each ridge are arranged horizontally, q equally spaced first longitudinal dividing lines 6 are set in the first initial image from left to right along the horizontal direction, and each of the first longitudinal dividing lines 6 corresponds to a first horizontal coordinate value; the density information includes the first horizontal coordinate value, the level of plant density and the number of seedling lines in the rectangle enclosed by the first longitudinal dividing line 6 corresponding to each first horizontal coordinate and the first longitudinal dividing line 6 adjacent to its left or right side.

[0039] For example, when the plants in each ridge are arranged vertically, assume that the first initial image used for training is an RGB image with a size of 640*480 and 3 channels; the image obtained after cropping is 448*320 with 3 channels; the first preprocessed image obtained after downsampling by 4 times is 80*112 with 64 channels. Assume that there are three convolutional layers to extract features from the first preprocessed image, and the feature sizes of the images obtained from the bottom layer to the top layer are: [128, 40, 56], [256, 20, 28] and [512, 10, 14], respectively named feature1, feature2, feature3. Use Global Average Pooling (GAP) to extract the global features of feature3 (the feature image corresponding to the last convolutional layer), and use the fully connected layer (FC) to map the features to the density recognition space. The output shape is as follows: Figure 6 The value is [11, 14, 6], where 11 is the plant density level, 14 is the first vertical coordinate value, and 6 represents the number of seedling lines.

[0040] It should be understood that the goal of seedling line recognition is to accurately identify and locate the positions of plant lines. A seedling line typically refers to the arrangement of plants within a row. Because the distribution of plants within a row is highly detailed, seedling line recognition requires more refined segmentation to capture the morphology, position, and arrangement of each plant or plant group. Therefore, seedling line recognition requires the image to be divided into smaller regions so that the number and position of seedling lines can be accurately calculated within each small region. This requires the use of more segmentation lines to refine each detection region, thereby more accurately locating the seedling lines within each row of plants. Density recognition, on the other hand, aims to assess the density of plants within a region. Density recognition typically requires assessment at a larger scale, focusing on the overall distribution of plants within the image. Unlike seedling line recognition, density recognition does not require the precise location of each plant row; it only calculates the plant density level within a specific area. Therefore, the segmentation lines for density recognition can be set more sparsely, with each segmentation region typically covering a larger area. The number and density of seedling lines in each region can be assessed using larger segmentation regions, eliminating the need for the denser segmentation lines required for seedling line recognition. Therefore, two sets of dividing line coordinate systems, m, n and p, and x, y and q, were established, where m is greater than p and x is greater than q.

[0041] In an embodiment of the first aspect of the present application, the coverage information includes a third horizontal coordinate value, a third vertical coordinate value, and the number of seedling lines in a rectangle enclosed by the third longitudinal dividing line 3 corresponding to each third horizontal coordinate value and the third longitudinal dividing line 3 adjacent to its left or right side.

[0042] S6: Verify the density information, seedling line information and coverage information used for training and generate a corresponding loss function.

[0043] In an embodiment of the first aspect of the present application, the steps of verifying the density information, seedling line information and coverage information used for training and generating a corresponding loss function include: after performing feature extraction on the first preprocessed image to generate several first feature images, using a semantic segmentation branch to aggregate global information and local information of the several first feature images, and verifying the output data of the semantic segmentation branch with the density information, seedling line information and coverage information used for training to obtain a loss function.

[0044] For example, when the plants in each ridge are arranged vertically, assume that the first initial image used for training is an RGB image with a size of 640*480 and 3 channels; the image obtained after cropping is 448*320 with 3 channels; the first preprocessed image obtained after downsampling by 4 times is 80*112 with 64 channels. Assume that there are three convolutional layers to extract features from the first preprocessed image, and the feature sizes of the images obtained from the bottom layer to the top layer are: [128, 40, 56], [256, 20, 28] and [512, 10, 14], respectively named feature1, feature2, feature3. Use Global Average Pooling (GAP) to extract the global features of feature3 (the feature image corresponding to the last convolutional layer), and use the fully connected layer (FC) to map the features to the coverage recognition space. The output shape is as follows: Figure 7 It is shown as [40,14,3], where 3 is the number of ridges (each ridge has two seedling lines on the left and right), 40 represents the number of plants in a ridge, that is, the plant coverage rate, and 14 is the first vertical coordinate value.

[0045] In an embodiment of the first aspect of the present application, the method for calculating the loss function includes: L total =L density +L lane +L cover , L lane =L cls +αL str +βL seg , wherein the L total is the total loss function, L density is the density loss function, L cover is the coverage loss function, L lane is the seedling line loss function, L cls is the classification loss of each point on each line, L str is the loss of shape of the seedling line, L seg is the seedling line segmentation loss, α and β are the str and L seg The scaling factor.

[0046] The loss function (LossFunction) is a mathematical expression used to measure the difference between the prediction result and the target label during the model training process. During training, the system first obtains the predicted output through the network through the input data, and then the loss function compares the output with the true label to calculate a real value, which is used as a quantitative evaluation of the current model performance. The optimizer then automatically adjusts the model parameters based on the derivative of the loss function, so that the next round of predictions is closer to the target, thereby promoting the gradual convergence of the model. In other words, the loss function can be regarded as a mechanism of "indirect compensation output", which indirectly and progressively compensates the model output by continuously correcting the internal parameters until the error bill approaches zero. The above content of the present invention mentions the loss function L lane =L cls +αL str +βL seg The traditional method is to use a single Cross-Entropy or DiceLoss to process the seedling line, while the present invention is to split the seedling line into L cls , L str and L seg The three parts are constrained, so that the common "breakage" and "sawtooth" problems of slender seedling lines are suppressed, and the F1 value can be increased by 6%-9%, and the continuity (average line segment length) is increased by >12%.

[0047] In one embodiment of the first aspect of the present application, the L density and L cover Use the cross entropy algorithm function.

[0048] It should be understood that the gradient of the cross-entropy with respect to the probability output is monotonic and continuous in the range of 0-1. This avoids the problem of the mean square error gradient approaching zero once it approaches saturation, thus allowing the model to still obtain effective learning signals in the later stages. Furthermore, cross-entropy facilitates the superposition of category weights / focal mechanisms, significantly reducing the negative impact of long-tail distributions on the model. This improves the overall accuracy of the system.

[0049] S7: Acquire a second initial image for inference.

[0050] S8: After repeatedly performing the above-mentioned preprocessing, feature extraction and mapping steps performed on the first initial image on the second initial image, the corresponding density information, seedling line information and coverage information for reasoning are output respectively.

[0051] It should be understood that steps S7-S8 are relevant steps in the formal reasoning stage, and the processing process of the second initial image is basically the same as that of the first initial image. They must all undergo preprocessing, feature extraction and mapping before outputting the corresponding density information, seedling line information and coverage information. The only difference is: after using the first initial image to obtain the corresponding training density information, seedling line information and coverage information, the training density information, seedling line information and coverage information are verified and the corresponding loss function is generated, while after using the second initial image to obtain the corresponding reasoning density information, seedling line information and coverage information, it is not necessary to verify and generate the corresponding loss function.

[0052] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a plant growth status detection system, comprising: an image acquisition module, for acquiring a first initial image for training and a second initial image for reasoning; a preprocessing module, for preprocessing the first initial image to generate a first preprocessed image, and preprocessing the second initial image to generate a second preprocessed image, wherein the preprocessing includes cropping and downsampling the first initial image according to preset parameters; a feature extraction module, for performing feature extraction on the first preprocessed image to generate a plurality of first feature images, and performing feature extraction on the second preprocessed image to generate a plurality of second feature images, wherein the feature extraction includes using multiple The first convolution layer extracts features of the first preprocessed image from local to global level by level, retains the feature image output by each convolution layer, and extracts the global feature of the feature image corresponding to the last convolution layer as the first feature; the calculation output module is used to map the first feature to the density recognition space, the seedling line recognition space and the coverage recognition space respectively, and output the corresponding density information, seedling line information and coverage information for training respectively, and map the second feature to the density recognition space, the seedling line recognition space and the coverage recognition space respectively, and output the corresponding density information, seedling line information and coverage information for reasoning respectively; the compensation module is used to verify the density, seedling line and coverage information for training and generate a corresponding loss function.

[0053] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0054] It should also be understood that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0055] like Figure 8 8 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. The electronic terminal includes: at least one processor 801, a memory 802, at least one network interface 803, and a user interface 805. The various components in the device are coupled together via a bus system 804. It will be understood that the bus system 804 is used to enable connection and communication between these components. In addition to including a data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus.

[0056] The user interface 805 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0057] It will be appreciated that the memory 802 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0058] The memory 802 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 800. Examples of such data include: any executable program for operating on the electronic terminal 800, such as an operating system 8021 and an application 8022; the operating system 8021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 8022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The method provided by the embodiment of the present invention can be included in the application 8022.

[0059] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 801 or by software instructions. The above processor 801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 801 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 801 can be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0060] In an exemplary embodiment, the electronic terminal 800 may be configured to execute the aforementioned method using one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs).

[0061] A third aspect of the present application provides a computer program product, which includes computer program code. When the computer program code is run on a computer, the computer is enabled to implement any of the aforementioned methods.

[0062] A fourth aspect of the present application provides an electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the aforementioned methods.

[0063] As used in this specification, the terms "component," "module," "system," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on one computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component across a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0064] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0065] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0066] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0067] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0068] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0069] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0070] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0071] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0072] In summary, the present invention effectively overcomes various shortcomings of the prior art and has high industrial utilization value.

[0073] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A method for detecting plant growth status, characterized in that: include: Obtaining a first initial image for training; Preprocessing the first initial image to generate a first preprocessed image, wherein the preprocessing includes cropping and downsampling the first initial image according to preset parameters; Performing feature extraction on the first preprocessed image to generate a plurality of first feature images, wherein the feature extraction includes using a multi-level convolutional layer to perform feature extraction on the first preprocessed image from local to global levels, retaining the feature image output by each convolutional layer, and extracting the global features of the feature image corresponding to the last convolutional layer as the first features; Mapping the first feature to the density recognition space, the seedling line recognition space, and the coverage recognition space respectively, and then outputting the corresponding density information, seedling line information, and coverage information for training respectively; Verifying the density information, seedling line information, and coverage information used for training and generating a corresponding loss function; Obtain a second initial image for inference, and repeat the above-mentioned preprocessing, feature extraction and mapping steps performed on the first initial image on the second initial image, and output the corresponding density information, seedling line information and coverage information for inference respectively.

2. A method for detecting plant growth status according to claim 1, characterized in that: The calculation method of the loss function includes: L total =L density +L lane +L cover L lane =L cls +αL str +βL seg Among them, the L total is the total loss function, L density is the density loss function, L cover is the coverage loss function, L lane is the seedling line loss function, L cls is the classification loss of each point on each line, L str is the loss of shape of the seedling line, L seg is the seedling line segmentation loss, α and β are the str and L seg The scaling factor.

3. A method for detecting plant growth status according to claim 2, characterized in that: The L density and L cover Use the cross entropy algorithm function.

4. A method for detecting plant growth status according to claim 1, characterized in that: The steps of verifying the density information, seedling line information and coverage information used for training and generating a corresponding loss function include: After feature extraction is performed on the first preprocessed image to generate several first feature images, a semantic segmentation branch is used to aggregate global information and local information of the several first feature images, and the output data of the semantic segmentation branch is verified with the density information, seedling line information and coverage information used for training to obtain a loss function.

5. A method for detecting plant growth status according to claim 1, characterized in that: The steps of mapping the first features to the seedling line recognition space and outputting corresponding seedling line information for training include: When the plants in each ridge are arranged vertically, m second horizontal dividing lines with equal spacing are set in the vertical direction from top to bottom of the first initial image, and n second vertical dividing lines with equal spacing are set in the horizontal direction from left to right, each second horizontal dividing line corresponds to a second vertical coordinate value, and each second vertical dividing line corresponds to a second horizontal coordinate value; a second vertical coordinate parameter is obtained by adding a probability value of the seedling line feature appearing in a rectangle enclosed by the second vertical dividing line corresponding to the second vertical coordinate value and the second vertical dividing line adjacent above or below it, and the second vertical coordinate value; the seedling line information includes the second vertical coordinate parameter, the second horizontal coordinate value and the number of seedling lines in the rectangle enclosed by the second horizontal dividing line corresponding to each second vertical coordinate value and the second horizontal dividing line adjacent above or below it; When the plants in each ridge are arranged horizontally, x equally spaced third longitudinal dividing lines are set in the first initial image from left to right in the horizontal direction, and y equally spaced third transverse dividing lines 4 are set in the longitudinal direction from top to bottom, each third transverse dividing line 4 corresponds to a third horizontal coordinate value, and each third transverse dividing line 4 corresponds to a third vertical coordinate value; the probability value of the seedling line feature appearing in the rectangle enclosed by the third transverse dividing line 4 corresponding to the third horizontal coordinate value and the third transverse dividing line 4 adjacent above or below it is added to the third transverse coordinate value to obtain the third transverse coordinate parameter; the seedling line information includes the third transverse coordinate parameter, the third vertical coordinate value and the number of seedling lines in the rectangle enclosed between the third longitudinal dividing line corresponding to each third transverse coordinate value and the third longitudinal dividing line adjacent to its left or right side.

6. A method for detecting plant growth status according to claim 1, characterized in that: The steps of mapping the first features to density recognition spaces and outputting corresponding density information for training include: Divide plant density into k levels; When the plants in each ridge are arranged longitudinally, p first horizontal dividing lines with equal spacing are set in the first initial image from top to bottom along the longitudinal direction, each first horizontal dividing line corresponding to a first vertical coordinate value; the density information includes the first vertical coordinate value, the level of plant density, and the number of seedling lines within a rectangle enclosed by the first horizontal dividing line corresponding to each first vertical coordinate and the first horizontal dividing line adjacent to it above or below; When the plants in each ridge are arranged horizontally, q equally spaced first longitudinal dividing lines are set in the first initial image from left to right along the horizontal direction, and each of the first longitudinal dividing lines corresponds to a first horizontal coordinate value; the density information includes the first horizontal coordinate value, the level of plant density and the number of seedling lines in the rectangle enclosed by the first longitudinal dividing line corresponding to each first horizontal coordinate and the first longitudinal dividing line adjacent to its left or right side.

7. A method for detecting plant growth status according to claim 6, characterized in that: The coverage information includes the third horizontal coordinate value, the third vertical coordinate value and the number of seedling lines in the rectangle enclosed by the third longitudinal dividing line corresponding to each third horizontal coordinate value and the third longitudinal dividing line adjacent to its left or right side.

8. A plant growth status detection system, characterized in that: include: An image acquisition module, configured to acquire a first initial image for training and a second initial image for reasoning; a preprocessing module, configured to preprocess the first initial image to generate a first preprocessed image, and preprocess the second initial image to generate a second preprocessed image, wherein the preprocessing includes cropping and downsampling the first initial image according to preset parameters; a feature extraction module, configured to perform feature extraction on the first preprocessed image to generate a plurality of first feature images, and perform feature extraction on the second preprocessed image to generate a plurality of second feature images, wherein the feature extraction includes using a multi-level convolutional layer to perform feature extraction on the first preprocessed image from local to global levels, retaining the feature image output by each convolutional layer, and extracting the global feature of the feature image corresponding to the last convolutional layer as the first feature; a calculation output module for mapping the first feature to a density recognition space, a seedling line recognition space, and a coverage recognition space, and outputting the corresponding density information, seedling line information, and coverage information for training, respectively; and for mapping the second feature to a density recognition space, a seedling line recognition space, and a coverage recognition space, and outputting the corresponding density information, seedling line information, and coverage information for reasoning, respectively; The compensation module is used to verify the density, seedling line and coverage information used for training and generate a corresponding loss function.

9. A computer program product, characterized in that The computer program product includes computer program code, which, when executed on a computer, enables the computer to implement the method according to any one of claims 1 to 7.

10. An electronic terminal comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images