A deep learning-based rice nutrient state recognition method

Abstract

The present application relates to the technical field of agricultural management, in particular to a rice nutrient state recognition method based on deep learning, comprising the following steps: calling image data set, recognizing leaf green pixel change trend, measuring contour structure and eliminating abnormality, adjusting feature channel proportion, outputting label and confidence, and analyzing regional nutrient evolution trend and aggregation. In the present application, the change trend of the green area pixel density in the continuous time series rice leaf image is dynamically analyzed, the change node of the rice growth state is accurately judged, the sensitivity problem of the existing unified input model structure to the stage transition is eliminated, the abnormal structure image is real-time eliminated combined with the leaf edge closure and contour coherence characteristics, the capture ability of the model to the subtle structure difference is strengthened, the accurate classification and dynamic aggregation of the rice nutrient state label are realized, the recognition ability of the regional scale rice nutrient evolution trend is obviously improved, and the data basis is provided for the regional agricultural decision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural management, and in particular to a rice nutrient state recognition method based on deep learning. BACKGROUND

[0002] The technical field of agricultural management includes monitoring, control, analysis and decision support of each link in the whole process of agricultural production, aiming to improve the intelligent, precise and sustainable level of agricultural production, including crop planting management, soil and nutrient monitoring, pest prediction and control, irrigation and fertilization scheduling, crop growth state evaluation, etc. Through the integration of agricultural sensors, geographic information systems, remote sensing technology and data analysis models, dynamic perception and comprehensive utilization of agricultural resources and environmental information are realized. The technical field of agricultural management forms an intelligent decision-making mechanism based on data driving, which promotes the transformation of agriculture from traditional experience planting to scientific management and precision operation. Among them, the rice nutrient state recognition method based on deep learning refers to constructing an image recognition model based on convolutional neural network to extract and classify the features of visible light images of rice at different growth stages, to determine whether the nutrient element level is normal. It covers image acquisition and processing, feature parameter labeling, rice sample data set construction, neural network model training and optimization, classification label output and other technical matters. The method is based on the acquired rice leaf image, predefines the image size and standard format, extracts color, texture, edge and other features through convolutional neural network, performs supervised learning training on the existing sample set, and outputs the nutrient state label corresponding to the leaf, including nitrogen deficiency, phosphorus deficiency, potassium deficiency and other categories. The recognition process uses a unified image input channel and a fixed model structure to ensure the consistency of data processing flow and discrimination basis.

[0003] Traditional rice nutrient state recognition technology relies on fixed image input channel and model structure to uniformly process all stage rice samples, ignoring the continuous change characteristics of rice leaf features with growth process. When the growth state of rice changes periodically, the feature extraction may be inaccurate, which makes it difficult to finely distinguish the nutrient state classification label, and the image processing of abnormal structure or local damage of leaf is insufficient, which may cause sample error and reduce the model classification precision, limiting the reliability of classification effect in actual application scenarios. SUMMARY

[0004] The purpose of the present application is to solve the problems existing in the prior art and to provide a rice nutrient state recognition method based on deep learning.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme: a rice nutrient state recognition method based on deep learning, comprising the following steps:

[0006] S1: call the rice image dataset, analyze the change trend of the green area pixel density in the leaf image in the continuous time period, identify the trend difference between adjacent images, judge the growth state transformation node, mark the current image with the mutation node, and generate the sequence node division parameter;

[0007] S2: according to the sequence node division parameter, measure the leaf edge and contour distribution characteristics in the image, analyze the boundary closure and contour continuity, judge the abnormal structure image and eliminate it, and generate the training sample screening result;

[0008] S3: call the training sample screening result, analyze the expansion ratio of the leaf shape boundary in the horizontal and vertical directions, match the rice type label, and adjust the channel response ratio between the texture information processing area and the edge contour processing area, train the rice nutrient state recognition model, and generate the processing channel matching result;

[0009] S4: based on the processing channel matching result, using the rice nutrient state recognition model, obtaining the label response score of each image, extracting the main label and the secondary label, comparing the response score, identifying the difference degree and combining the identification label, and evaluating the confidence of the model recognition result, outputting the rice state dataset.

[0010] As a further scheme of the present application, the sequence node division parameter includes the green area pixel density direction, the green pixel change trajectory trend and the image mutation position index, the training sample screening result includes the contour boundary distribution range, the edge continuity index and the image elimination position information, the processing channel matching result is specifically the texture channel weight ratio, the edge channel weight ratio and the type label classification item, and the rice state dataset is specifically the main and secondary label pair, the response order structure and the identification confidence score.

[0011] As a further scheme of the present application, the sequence node division parameter acquisition step is specifically:

[0012] S111: call the rice image dataset, collect the leaf images in the continuous time period, extract the green channel pixel value for each image, calculate the green pixel density in each sampling unit, combine the arrangement order of the image sequence, detect the green pixel density change trend of each frame, and generate the green change trend sequence;

[0013] S112: according to the green change trend sequence, analyze the green pixel density of adjacent image frames, identify the change direction and trend inflection point, calculate the trend mutation index, and extract the extreme value point, obtain the growth state transformation identification point set;

[0014] S113: Call the growth state transformation recognition point set, judge the growth state transformation node of the rice, mark the mutation node of the current image, and generate sequence node division parameters.

[0015] As a further scheme of the present application, the acquisition step of the training sample screening result is specifically:

[0016] S211: According to the sequence node division parameters, the leaf edge range and contour extension value of each image in the image frame sequence are obtained, the edge width, edge symmetry ratio and extension deformation amplitude of the current frame are calculated, and the boundary shape abnormal value is obtained;

[0017] S212: According to the boundary shape abnormal value, the boundary distribution direction change, leaf transverse width change ratio and area coverage ratio of two consecutive frames in the image frame are extracted, the region alignment angle and boundary matching degree are analyzed, and the structure coherence trend value is calculated;

[0018] S213: According to the structure coherence trend value, the abnormal structure image is judged and removed, the model training data set is screened, and the training sample screening result is established.

[0019] As a further scheme of the present application, the acquisition step of the processing channel matching result is specifically:

[0020] S311: Call the training sample screening result, collect the leaf boundary information of each frame image, measure the transverse contour length and longitudinal contour length, analyze the length proportion structure of boundary expansion, construct the boundary space distribution index according to the transverse-longitudinal ratio and the edge closure degree of leaf contour, and generate the leaf structure expansion coefficient;

[0021] S312: According to the leaf structure expansion coefficient, by comparing with the preset shape classification section, the morphological type to which the image belongs is judged, the image label is marked, and the feature processing path requirement corresponding to each type is identified, and the rice type structure label group is obtained;

[0022] S313: Based on the rice type structure label group, according to the type label, the proportion adjustment of the channel response between the texture information processing area and the edge contour processing area is carried out, the processing combination of the image input channel is established, the rice nutrient state recognition model is trained, and the image input channel matching result is generated.

[0023] As a further scheme of the present application, the acquisition step of the rice state data set is specifically:

[0024] S411: Call the image input channel matching result, obtain the label response score group output by the recognition model for each image, extract the first two labels according to the response score, obtain the primary label and the secondary label, and generate the primary and secondary label extraction result;

[0025] S412: According to the main and secondary label extraction result, the difference value between the main label and the secondary label and the sorting position interval value are calculated, the difference intensity index structure is established, and the label difference sorting distance value is obtained.

[0026] S413: The label difference sorting distance value is called, compared with the set recognition confidence benchmark interval, the recognition confidence level is recognized, the classification reliability level is obtained, and the rice state data set is established combined with the main label response value.

[0027] As a further scheme of the application, the method further comprises:

[0028] S5: Using the rice state data set, the rice nutrient state label is extracted, the label time distribution sequence is constructed according to the plot and collection batch information, the evolution trend consistent plots are identified and aggregated by analyzing the main label evolution trajectory direction and frequency between plots, and the regional state analysis result is generated.

[0029] The regional state analysis result includes plot label change direction code, label frequency aggregation grouping, and trend clustering boundary demarcation.

[0030] As a further scheme of the application, the acquisition step of the regional state analysis result is specifically:

[0031] S511: The rice state data set is called, the plot number and collection batch number corresponding to each image are located, the main label information of the corresponding image is identified, the time sequence arrangement structure is constructed according to the plot number, the label time change trajectory vector corresponding to each plot is constructed based on the sampling sequence of the label on the time axis, and the label time distribution sequence is obtained.

[0032] S512: According to the label time distribution sequence, the label sequence difference value of adjacent plots in each batch is extracted, the direction change index and step change value of the main label of each plot in the same batch are calculated, the trajectory difference combination is constructed, the trend difference index between plots is calculated, and the plot evolution trend deviation value is obtained.

[0033] S513: According to the plot evolution trend deviation value, the trend distance between each pair of plots is calculated, the evolution trend consistent plots are identified and aggregated, and the regional state analysis result is generated.

[0034] Compared with the prior art, the application has the advantages and positive effects that:

[0035] In the present application, the change trend of the pixel density of the green area in the continuous time sequence rice leaf image is analyzed dynamically, the change node of the rice growth state is accurately judged, the sensitivity problem of the existing unified input model structure to the stage transition is eliminated, the abnormal structure image is removed in real time by combining the edge closure and contour coherence characteristics of the leaf, the capture ability of the model to the subtle structural differences is strengthened, the accurate classification and dynamic aggregation of the rice nutrient state label are realized, the identification ability of the regional scale rice nutrient evolution trend is obviously improved, and a data basis is provided for regional agricultural decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is a schematic diagram of the main steps of the present application.

[0037] Figure 2 It is a sequence node division parameter acquisition flowchart of the present application.

[0038] Figure 3 It is a training sample screening result acquisition flowchart of the present application.

[0039] Figure 4 It is a processing channel matching result acquisition flowchart of the present application.

[0040] Figure 5 It is a rice state data set acquisition flowchart of the present application.

[0041] Figure 6 It is a regional state analysis result acquisition flowchart of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0043] In the description of the present application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore it cannot be understood as a limitation of the present application. In addition, in the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0044] Please refer to Figure 1 The present application provides a technical scheme: a rice nutrient state recognition method based on deep learning, comprising the following steps:

[0045] S1: Call the rice image dataset, analyze the change trend of the green area pixel density in the leaf image of the continuous time period, identify the trend difference between adjacent images, judge the growth state transformation node, mark the mutation node of the current image, and generate sequence node division parameters;

[0046] S2: According to the sequence node division parameters, measure the leaf edge and contour distribution characteristics in the image, analyze the boundary closure and contour continuity, judge the abnormal structure image and eliminate it, and generate training sample screening results;

[0047] S3: Call the training sample screening result, analyze the expansion ratio of the leaf shape boundary in the horizontal and vertical directions, match the rice type label, and adjust the channel response ratio between the texture information processing area and the edge contour processing area, train the rice nutrient state recognition model, and generate processing channel matching results;

[0048] S4: Based on the processing channel matching result, use the rice nutrient state recognition model to obtain the label response score of each image, extract the main label and secondary label, compare the response scores, identify the difference degree and combine the recognition label, and evaluate the confidence of the model recognition result, and output the rice state dataset;

[0049] S5: Use the rice state dataset to extract the rice nutrient state label, construct the label time distribution sequence according to the plot and collection batch information, analyze the main label evolution trajectory direction and frequency between plots, identify the plots with consistent evolution trend and aggregate them, and generate regional state analysis results.

[0050] The sequence node division parameters include the green area pixel density direction, the green pixel change trajectory trend, and the image mutation position index. The training sample screening result includes the contour boundary distribution range, the edge continuity index, and the image elimination position information. The processing channel matching result is specifically the texture channel weight ratio, the edge channel weight ratio, and the type label classification item. The rice state dataset specifically refers to the main and secondary label pair, the response ordering structure, and the recognition confidence score. The regional state analysis result includes the plot label change direction code, the label frequency aggregation grouping, and the trend clustering boundary.

[0051] Please refer to Figure 2 , the acquisition steps of the sequence node division parameters are as follows:

[0052] S111: Call the rice image dataset, collect leaf images in a continuous time period, extract green channel pixel values for each image, calculate the green pixel density in each sampling unit, combine the arrangement order of the image sequence, detect the change trend of the green pixel density of each frame, and generate a green change trend sequence;

[0053] First, a fixed-point high-definition camera device is erected in the agricultural demonstration area field, and the rice leaves are continuously photographed at a frequency of once per hour, 24 frames of images are obtained per day, 7 days are taken as an analysis period, a total of 168 frames of images are obtained as the initial data set, the resolution of the photographed images is 2048*1536 pixels, and the image format is JPEG; secondly, the pixel value of the green channel is extracted from each frame of original image, and the gray value of the green channel is recorded by traversing each pixel point of the whole frame image, for example, for the RGB channel value (65, 150, 48) of a certain pixel position (1200, 850), the green channel value 150 is extracted, and all the pixel values of the green channel are extracted frame by frame to form a green channel gray scale image; thirdly, each frame of image is binarized by threshold setting method, wherein the threshold setting adopts gray scale histogram statistical method, and the specific operation is to analyze the green channel gray scale distribution histogram of the first frame of image, to initially set the threshold value near the gray scale value 120, to calculate the pixel frequency and to adjust according to the distribution of the statistical graph, and the determined threshold value is 125, which is higher than the threshold value 125 and is defined as a green pixel, and which is lower than 125 and is defined as a non-green pixel; then, the pixel density calculation area is defined as a sampling unit with an image center area of 600*600 pixels, the number of green pixels is recorded, and assuming that the number of green pixels in the area of the first frame of image is 245000, the maximum possible number of pixels is 360000, and then the green pixel density is 245000 / 360000≈0.6806, the above operation is performed on the sampling unit of 168 frames of images to obtain a green pixel density sequence; finally, the green pixel density data of 168 frames of images are arranged in time sequence to generate a complete green change trend sequence, and for example, the density data of the first to third frames is 0.6806, 0.6952 and 0.7015, which represents the actual change trend of the green pixels in the leaf area within the monitoring period.

[0054] S112: According to the green change trend sequence, the green pixel density of adjacent image frames is analyzed, the change direction and trend inflection point are identified, and the formula is used:

[0055]

[0056] The trend mutation index is calculated, and the extreme value point is extracted to obtain a growth state conversion recognition point set;

[0057] Wherein, G i is the normalized value of the green pixel density of the i-th frame of image, which is obtained by dividing the green pixel area of each frame by the maximum green pixel area in the same period, and ∈ is a very small positive constant to prevent the denominator from being zero, is the normalized value of the leaf shape change of the i-th frame of image, which is obtained by dividing the leaf shape area or contour length of each frame by the maximum shape value in the period, is the time normalization value of the i-th frame image, which is obtained by subtracting the starting time of the cycle from the actual time and then dividing by the total cycle length, ω is a dimensionless adjustment coefficient, is obtained by analyzing the influence weight of the morphological change rate in the overall index through cross-validation, λ i is the i-th frame trend mutation index, which represents the comprehensive sensitivity of the current frame green pixel intensive change trend and the dynamic change of leaf structure, i is the position index of the current image frame in the sequence, which is obtained by analyzing the sequence number of the collection batch;

[0058] The green pixel intensity sequence G i is normalized, and the operation method is to divide the intensity value of each frame image by the maximum intensity value in the cycle. Taking the second frame as an example, the green pixel intensity is 0.6952, and the maximum green pixel intensity in 168 frames is assumed to be 0.9200, so the normalized value is 0.6952 / 0.9200≈0.7557, that is, the normalized green intensity sequence is obtained; for the leaf morphological change normalization value is obtained by measuring the contour length of the leaf, such as the contour length of the second frame leaf being 520 pixels, and the maximum length in the cycle being 640 pixels, then the morphological normalization value of the second frame is 520 / 640≈0.8125; the collection time normalization value is obtained by subtracting the starting time of the cycle from the actual time and then dividing by the total cycle length. Assuming that the second frame is taken at 2 hours on the first day, the collection time is 2 hours, and the total cycle length is 168 hours, so the time normalization value is 2 / 168≈0.0119; the setting of the adjustment coefficient ω is determined by calculating the influence proportion of the morphological change in the overall index through cross-validation, and through pre-experiment, 0.2, 0.4, 0.6, 0.8 are taken for comparison, and it is found that the best effect is obtained when ω=0.6, so ω=0.6 is selected, and the specific experimental data is shown in Table 1:

[0059] Table 1 Experimental comparison table of adjustment coefficient ω

[0060] Adjustment coefficient Mutation index average sensitivity 0.2 0.253 0.4 0.312 0.6 0.395 0.8 0.365

[0061] As shown in Table 1, ω=0.6 with the maximum sensitivity (0.395) is selected as the parameter; then the formula is used:

[0062]

[0063] The above example data is calculated, taking the first frame data as an example, assuming that G1=0.7402,

[0064] =0.0119, ∈ takes the minimum positive value 0.0001, then the calculation steps are as follows:

[0065]

[0066] The calculation of each part: the first term is 0.0209, the second term is 0.0086, and the third term is 1.2605, and finally:

[0067] λ1=0.0209+0.0086+1.2605=1.2900;

[0068] The trend mutation index is a normalized numerical parameter used to describe the comprehensive change amplitude, short-term fluctuation inflection point, and structural dynamic variation intensity of rice leaves in a time sequence image sequence. It directly quantifies the "state mutation" significance of a leaf corresponding to a frame image. Through this index, it can intuitively reflect which moments in the image sequence have synchronous abnormality of leaf color and structure, changes in nutritional response, or key states such as growth period turning points, and thus serve as numerical basis for automatic segmentation, sequence labeling, and key frame selection for model training. The larger the index value, the more likely it is that the frame is a node where the state of rice has undergone a significant turning point or mutation, which helps to accurately lock the phase change of the sample and improve the structural integrity and scientificity of the recognition results of the whole process. By calculating the value of each frame in the sequence, a mutation index sequence is formed; by extracting local extreme points in the sequence, the reference value for determining local extreme points is defined as the mutation index value of the sequence that exceeds the adjacent two frames before and after it, for example, the 20th frame λ i =1.502, which is greater than the 19th frame 1.232 and the 21st frame 1.309, respectively. The 20th frame is considered as a trend inflection point, and a complete set of growth state transformation recognition points is obtained. 20

[0069] S113: Call the growth state transformation recognition point set, judge the growth state transformation node of the rice, and perform mutation node labeling processing on the current image to generate sequence node division parameters;

[0070] First, the obtained mutation index sequence is traversed one by one, and the mutation index value λ i of each position i is compared with the mutation index values of adjacent positions i-1 and i+1 one by one. When the mutation index value of position i is higher than the index values of the adjacent two positions, then this position is marked as a trend inflection point. Taking the 20th frame data in the monitoring period as an example, the λ 20 value calculation result is 1.502, which is higher than λ 19 =1.232 and λ 21

[0071] ​= 1.309, the 20th frame data is determined as a growth state transition node, and is recorded in the state transition identification point set; subsequently, all nodes in the image data sequence that meet the above conditions are recorded completely, such as assuming that four inflection point positions of the 20th, 45th, 92nd and 138th frames are obtained through one-by-one comparison and calculation, and the corresponding mutation index values are 1.502, 1.643, 1.721 and 1.568 respectively, then the growth state transition identification point set is recorded in an array form: [20, 45, 92, 138]; next, the image sequence is divided by using a section-by-section marking method, the image data sequence from the initial position to the first trend inflection point (1st to 19th frames) is defined as the first node section, and the 20th to 44th frames are defined as the second node section, the 45th to 91st frames are defined as the third node section, the 92nd to 137th frames are defined as the fourth node section, and the 138th to 168th frames are defined as the fifth node section, so that the sequence node division sections are obtained, each node section corresponds to a specific image frame index range, and the start and end indexes of each node section are recorded respectively, and finally the sequence node division parameters with the node section start and end positions as parameters are generated.

[0072] Referring to Figure 3 , the acquisition step of the training sample screening result is specifically:

[0073] S211: According to the sequence node division parameters, the leaf edge range and contour extension value of each image in the image frame sequence are obtained, the edge width, edge symmetry ratio and extension deformation amplitude of the current frame are calculated, and the boundary morphology abnormal value is obtained;

[0074] According to the aforementioned sequence node division parameters, taking the second node section (20th to 44th frame) as an example, the following operations are performed frame by frame: first, the leaf edge contour pixel of the current frame image, for example, the 21st frame, is extracted, the boundary pixel set is obtained by using the boundary pixel tracking technology, and the boundary contour length is counted, for example, the actual measurement value of the boundary length of the 21st frame is 650 pixels, which is recorded as the leaf edge range of the frame; second, the leaf edge width is quantitatively processed, and the widest horizontal intercept of the leaf contour circumscribed rectangle is taken, for example, the intercept of the 21st frame is measured to be 260 pixels, which is the edge width of the frame; third, the edge symmetry ratio is determined by calculating the left and right area ratio of the leaf edge contour, and the leaf is divided into left and right areas by the vertical line of the boundary center, and the edge symmetry ratio is set to be equal to the smaller area divided by the larger area of the left and right area, for example, the left and right areas of the 21st frame are 30000 pixels and 33000 pixels respectively, and the edge symmetry ratio is 30000 / 33000≈0.9091; then, the extension deformation amplitude is calculated, that is, the absolute value of the difference between the leaf edge contour length of the current frame and the previous frame divided by the length of the previous frame, for example, the leaf boundary length of the 20th frame is 635 pixels, and the extension deformation amplitude of the 21st frame is |650-635| / 635≈0.0236; finally, the boundary morphology abnormal value is set to be the sum of the extension deformation amplitude and the edge symmetry ratio, that is, 0.0236+0.9091=0.9327, which is used as the basis data for identifying abnormal images in the subsequent steps.

[0075] S212: According to the boundary morphology abnormal value, the boundary distribution direction change, the leaf transverse width change ratio and the area coverage ratio of the two consecutive frames in the image frame are extracted, the area alignment angle and the boundary matching degree are analyzed, and the formula is used:

[0076]

[0077] The structure coherence trend value is calculated;

[0078] Wherein, Δθ k is the boundary distribution direction angle difference value of frame k and frame k+1, which is obtained by extracting the image contour line direction vector and calculating the included angle, is the edge range normalization value of frame k, which is obtained by measuring the leaf boundary length in the image and dividing by the diagonal length of the image, is the edge range normalization value of frame k+1, is the area coverage ratio normalization value of frame k, which is obtained by extracting the leaf pixel area in the image and dividing by the total pixel area of the image, is the area coverage ratio normalization value of frame k+1, k is the time index number in the image frame sequence, which identifies the position of the image in the sequence, A k is the structure coherence trend value, which represents the structure consistency degree of frame k under the change of boundary direction angle;

[0079] According to the obtained boundary shape abnormal value, the following operations are performed on the two consecutive image frames (21st frame and 22nd frame): first, the direction vector of the two frame leaf boundary contour is extracted, and the change of the boundary distribution direction is quantized, that is, the included angle difference Δθ of the two frame main directions is calculated k , for example, the main direction of the 21st frame is 35.0° with the horizontal axis, and the 22nd frame is 37.5°, then the direction difference of the two frames is 37.5°-35.0°=2.5°; second, the edge range normalized value is calculated , which is obtained by dividing the current frame boundary contour length by the image diagonal length (the image diagonal length is 2560 pixels), the normalized value of the 21st frame is 650 / 2560≈0.2539, and the boundary length of the 22nd frame is 660 pixels, and the normalized value is 660 / 2560≈0.2578; third, the area ratio normalized value is calculated k :

[0080]

[0081] , wherein the numerator Δθ k represents the change of the boundary direction included angle of the two consecutive frames, and the denominator contains the square root of the sum of the square of the edge range change amount and the absolute value of the area ratio change , which aims to comprehensively evaluate the influence of leaf shape and area change on structure continuity, and takes the absolute value to prevent negative values from affecting the accuracy of trend judgment. The above example data is brought into the formula:

[0082]

[0083] , wherein the structure continuity trend value represents the consistency between the boundary direction change and the shape structure change between the two consecutive image frames, which specifically measures whether the boundary trend change matches the geometric change of the edge range and the area ratio. This index can be used as a quantitative basis for screening image frame structure continuity. The larger the value, the more abrupt the boundary direction change and the weaker the structure support, which may represent abnormal images such as noise interference, shooting blur or incomplete contour; on the contrary, the smaller the value, the more natural and stable the structure continuity. This parameter directly serves as an important threshold for removing samples, controlling the consistency and stability of the input training samples, and effectively avoiding the interference of structure noise on the training accuracy of the model. The structure continuity trend value A is obtained21 ≈122.55, which indicates that the leaf structure between the 21st frame and the 22nd frame has high consistency.

[0084] S213: According to the structure consistency trend value, judge the abnormal structure image and remove it, filter the model training data set, and establish the training sample filtering result;

[0085] According to the calculated structure consistency trend value A k , the specific process of judging and removing abnormal structure images is as follows: first, determine the judgment threshold of the structure consistency trend value, determine it by using the statistical method of the structure consistency trend value in the early experimental data, obtain the average value and standard deviation of the structure consistency trend value by analyzing 100 continuous frames of normal leaf image data, and the specific experimental data is shown in Table 2:

[0086] Table 2 Structure consistency trend value statistics

[0087] Data set Trend value average Standard deviation Sample 1 125.4 8.6 Sample 2 123.9 9.1 Sample 3 124.8 8.8

[0088] As shown in Table 2, take the mean value 124.7 and the maximum standard deviation 9.1, set the judgment threshold to be the mean value minus twice the standard deviation 124.7-2x9.1=106.5, that is, when the structure consistency trend value A k <106.5, it is defined as an abnormal structure image; second, compare the structure consistency trend values of consecutive frames one by one, for example, calculate the trend value of the 35th frame as 102.3, which is less than the threshold value 106.5, then judge the 35th frame image as an abnormal structure image and mark it; Finally, execute the above judgment operation on the entire data set and mark all abnormal structure images, for example, the trend values of the 35th, 68th and 102nd frames are 102.3, 95.8 and 100.2 respectively, which are all less than the threshold value 106.5, then they are removed from the data set, the remaining normal images are used as the model training data set, and the establishment of the training sample filtering result is completed.

[0089] Please refer to Figure 4 , the acquisition step of processing channel matching result is as follows:

[0090] S311: Call the training sample filtering result, collect the leaf boundary information of each frame image, measure the horizontal profile length and the vertical profile length, analyze the length proportion structure of boundary expansion, construct the boundary space distribution index according to the horizontal and vertical ratio and the edge closure degree of leaf profile, and generate the leaf structure expansion coefficient;

[0091] First, the screened images are extracted frame by frame, for example, the 50th screened leaf image, the leaf boundary contour information of this frame is obtained, the boundary pixel point position is marked point by point, and the leaf transverse contour length and longitudinal contour length are obtained by recording the pixel point coordinates; secondly, the transverse contour length is measured, that is, the leftmost pixel point coordinate (such as (420, 850)) and the rightmost pixel point coordinate (such as (980, 850)) of the leaf contour are extracted respectively, and the transverse contour length is calculated as the pixel horizontal coordinate difference 980-420 = 560 pixels; thirdly, the longitudinal contour length is measured, that is, the uppermost pixel point coordinate (such as (700, 300)) and the lowermost pixel point coordinate (such as (700, 1200)) of the leaf contour are extracted respectively, and the longitudinal contour length is calculated as the pixel vertical coordinate difference 1200-300 = 900 pixels; then the boundary expansion length proportion structure is analyzed, and the specific operation is to divide the transverse contour length by the longitudinal contour length, for example, the transverse length of the 50th image is 560 pixels, and the longitudinal length is 900 pixels, so the transverse and longitudinal length ratio is 560 / 900 ≈ 0.6222; then the boundary space distribution index is constructed according to the leaf contour edge closure degree, which is specifically defined as the ratio of the actual length of the leaf contour to the ideal closed elliptical contour length, for example, the actual length of the leaf contour is 2200 pixels, and the ideal closed elliptical contour length is 2500 pixels, so the boundary space distribution index is 2200 / 2500 = 0.8800; finally, the leaf structure expansion coefficient is generated, which is the product of the boundary expansion length proportion and the boundary space distribution index, that is, 0.6222 x 0.8800 ≈

[0092] 0.5475, which is used as the leaf structure expansion coefficient data for subsequent processing.

[0093] S312: According to the leaf structure expansion coefficient, by comparing with the preset shape classification section, the shape type to which the image belongs is judged, the image label is marked, and the feature processing path requirement corresponding to each type is identified, and a rice type structure label group is obtained.

[0094] First, the leaf structure expansion coefficient is compared with the preset shape classification section, and the preset classification section is set according to the actual situation of the rice leaf structure, and the specific experimental section value is shown in Table 3:

[0095] Table 3: Rice leaf shape classification section table

[0096] Shape type Leaf structure expansion coefficient interval Japonica type [0.40,0.60] Nong tiao type (0.60,0.85]

[0097] As shown in Table 3, the specific type of structure expansion coefficient falling into the corresponding interval is determined by using the comparative operation; for example, the structure expansion coefficient of the 50th frame image is 0.5475, which is in the japonica rice type leaf structure expansion coefficient interval [0.40, 0.60], so the 50th frame image is determined to belong to the japonica rice type structure and is marked as japonica rice type; secondly, the above operation is performed on all images to obtain an image label set, for example, among the total of 200 screening images, 160 images are marked as japonica rice type, 40 images are indica rice type, and a rice type structure label group is formed; finally, the feature processing path requirement corresponding to each type is identified, for example, for the japonica rice type image, the texture information processing area is marked as 60% of the overall leaf area, and the edge contour processing area is marked as 40% of the overall leaf area, while for the indica rice type image, the texture information processing area is marked as 50%, and the edge contour processing area is marked as 50%, and the marking of the feature processing path is completed to obtain a complete rice type structure label group.

[0098] S313: Based on the rice type structure label group, the channel response between the texture information processing area and the edge contour processing area is proportionally adjusted according to the type label, the processing combination of the image input channel is established, the water rice nutrient state recognition model is trained, and the image input channel matching result is generated;

[0099] Firstly, the channel response proportion between the image texture information processing area and the edge contour processing area under different type labels is adjusted according to the specific value of the type label, specifically, the channel response proportion adjustment parameter is called, the texture information area channel response proportion of the japonica rice type image is set to 0.6, the edge contour area channel response proportion is set to 0.4, while the texture information area channel response proportion of the indica rice type image is set to 0.5, and the edge contour area channel response proportion is set to 0.5; secondly, the processing combination of the image input channel is established, the input channel data matrix is defined, the texture area data weight of the japonica rice type image is set to 0.6 times, and the contour area data weight is set to 0.4 times, taking the 50th frame japonica rice type image data as an example, the texture area channel data value is assumed to be 0.85, and the edge contour area channel data value is 0.70, then the combined input channel data value is 0.85x0.6+0.70x0.4=0.79, and the processing combination data of the japonica rice type image input channel is determined; thirdly, similar operations are performed on the indica rice type image, for example, the texture area channel data value of the 70th frame indica rice type image is 0.80, and the edge contour area channel data value is 0.75, then the combined input channel data value is 0.80x0.5+0.75x0.5=0.775, and the processing combination data of the indica rice type image input channel is determined; subsequently, all labeled images are called for combined channel processing data operation to obtain a complete image input channel combination data matrix for input to the water rice nutrient state recognition model training; finally, the model training process is performed to generate the image input channel matching result, and the complete data result is output for subsequent step calling.

[0100] Referring to Figure 5 , the obtaining step of the rice state data set is specifically as follows:

[0101] S411: Call the image input channel matching result, obtain the label response score set output by each image in the recognition model, extract the top two labels according to the response score, obtain the primary label and the secondary label, and generate the primary and secondary label extraction result;

[0102] First, the label response score set output by each image in the recognition model is obtained. For a specific image (such as the 80th frame), the response score values of all labels output by the model for the rice nutrient state are extracted, respectively. It is assumed that the 80th frame image outputs five labels (normal, slight nitrogen deficiency, moderate nitrogen deficiency, slight phosphorus deficiency, and moderate phosphorus deficiency), and the corresponding response scores are 0.78, 0.56, 0.42, 0.30, and 0.25, respectively. Then, the extracted label response score values are sorted. The specific sorting action is to compare and arrange the score values from high to low one by one. After sorting, the label order from high to low is obtained: normal (0.78), slight nitrogen deficiency (0.56), moderate nitrogen deficiency (0.42), slight phosphorus deficiency (0.30), and moderate phosphorus deficiency (0.25). Then, the top two labels of the sorted label sequence are extracted, i.e., the top two labels “normal” and “slight nitrogen deficiency” are extracted as the primary label and the secondary label, respectively, and their corresponding response scores 0.78 and 0.56 are stored in an array, which is marked as the primary and secondary label extraction result for subsequent use.

[0103] S412: According to the primary and secondary label extraction result, calculate the score difference between the primary label and the secondary label and the order interval value, establish the difference intensity index structure, and obtain the label difference order distance value;

[0104] First, the score difference between the primary label and the secondary label is calculated, and the specific calculation action is to subtract the secondary label score value from the primary label score value. Taking the 80th frame of image data as an example, the response score value of the primary label "normal" is 0.78, and the response score value of the secondary label "slight nitrogen deficiency" is 0.56. Then, the response score difference is calculated, and the primary-secondary label score difference is 0.78-0.56=0.22. Then, the interval value of the primary-secondary label ranking position is calculated, which is specifically calculated by subtracting the primary label ranking position index from the secondary label ranking position index. The ranking position is marked in integer form. The primary label "normal" ranking position index is marked as 1, and the secondary label "slight nitrogen deficiency" ranking position index is marked as 2. Then, the ranking position interval is calculated, and the ranking position interval value is 2-1=1. Then, the difference intensity index structure is established based on the above calculation results. The specific operation is to add the primary-secondary label score difference and the ranking position interval value to obtain the difference intensity index structure. For example, the primary-secondary label score difference calculated in the 80th frame of image data is 0.22, and the ranking position interval value is 1. Then, the difference intensity index structure is 0.22+1=1.22. This value is used as the label difference ranking distance value for subsequent steps.

[0105] S413: Call the label difference ranking distance value, compare it with the set recognition confidence benchmark interval, identify the confidence level and obtain the classification reliability level, combine the primary label response value, and establish the rice state data set;

[0106] First, set the recognition confidence benchmark interval. This interval is obtained by statistically analyzing the relationship between the primary-secondary label difference ranking distance value and the actual recognition accuracy through a large number of label recognition experiments on actual rice leaf images in the early stage. The experimental data is shown in Table 4:

[0107] Table 4: Setting of recognition confidence benchmark interval

[0108] Classification reliability level Label difference ordering distance value interval High reliability [1.50,2.00] Medium reliability [1.00,1.50) Low reliability [0.00,1.00)

[0109] As shown in Table 4, the label difference ranking distance value is compared based on the above interval. For example, the label difference ranking distance value of the 80th frame of image data is 1.22. Through interval-by-interval comparison, it is determined that 1.22 falls into the medium reliability interval [1.00, 1.50), so the classification reliability level is determined to be "medium reliability". Then, the rice state data set is established by combining the primary label response value. The specific action is to store the primary label response value (e.g., 0.78 for the 80th frame of image data) and the classification reliability level mark together in the data structure to establish the corresponding data record unit. Assuming that the data set number is set as R80, the storage content is: [data set number: R80, primary label: "normal", response value: 0.78, classification reliability level: "medium reliability"]. Finally, the rice state data set is established.

[0110] Please refer toFigure 6 The step of obtaining the regional state analysis result is specifically:

[0111] S511: Call the rice state dataset, locate the plot number and collection batch number corresponding to each image, identify the main label information of the corresponding image, construct a time sequence arrangement structure according to the plot number, construct a label time change trajectory vector corresponding to each plot based on the sampling sequence of the label on the time axis, and obtain a label time distribution sequence;

[0112] First, locate the plot number and collection batch number corresponding to each image in the rice state dataset, and the specific execution action is to extract the plot number (for example, the plot numbers are D 01 , D 02 , and D 03 ) and collection batch number (for example, the batch numbers are B1, B2, B3, and B4) of the images in the dataset frame by frame, secondly, identify and extract the main label information in the corresponding image, for example, for the image of plot D 01 , the main labels corresponding to batches B1 to B4 are extracted in turn, such as normal (label index is 1), slight nitrogen deficiency (index is 2), normal (index is 1), and slight nitrogen deficiency (index is 2), then, sort and arrange the main labels of each plot according to the batch number on the time axis, and arrange them in the order of batch number, for example, the label time sequence of plot D 01 is [1(B1), 2(B2), 1(B3), 2(B4)], similarly, plot D 02 is [1(B1), 1(B2), 2(B3), 2(B4)], and plot D 03 is [2(B1), 2(B2), 2(B3), 1(B4)], then, based on the sampling sequence of the label of the plot on the time axis, construct a label time change trajectory vector corresponding to each plot, and the specific action is to directly convert the label index values of each batch into a digital vector, for example, the trajectory vector of plot D 01 is represented as the vector [1, 2, 1, 2], complete the label vector construction action of all plots, and finally obtain the label time distribution sequence corresponding to each plot as the basic data for subsequent operations.

[0113] S512: According to the label time distribution sequence, extract the label sequence difference value of adjacent plots in each batch, calculate the direction change index and step change value of the main label of each plot in the same batch, and construct a trajectory difference combination, using the formula:

[0114]

[0115] Calculate the trend difference index between plots to obtain the plot evolution trend deviation value;

[0116] wherein, is the trend difference index of plot m and plot n, T is the total number of collection batches, obtained by counting the number of collection time periods in the rice state data set, j is the current batch index, indicating the time number from the 1st batch to the Tth batch, ΔL' m,j is the label change amplitude normalized value of plot m in batch j, calculated and standardized by the variation range and change rate of the main label index of adjacent batches, ΔL' n,j is the label change amplitude normalized value of plot n in batch j, S j is the disturbance intensity normalized value of the jth batch, indicating the dispersion degree of the sampling label distribution in the region in this batch, calculated and standardized by variance analysis combined with clustering tightness, R' m,j is the stability normalized value of plot m in batch j, indicating the time span or proportion that the main label remains consistent, obtained by statistical analysis of the time period that the main label remains unchanged between the previous and subsequent batches and normalized processing, R' n,j is the stability normalized value of plot n in batch j, γ is the stability adjustment coefficient, controlling the weight proportion of disturbance and stability in the deviation calculation, selected by model parameter adjustment and experience setting;

[0117] First, the change amplitude normalized value ΔL' m,j of the main label index of adjacent plots is calculated for each batch, which is normalized after subtracting the absolute value of the label index of the previous batch from the current batch label index, with the maximum index difference (for example, set to 2) as the standard. For example, the batch B2 of plot D 01 is |2-1| / 2=0.5, the batch B3 is |1-2| / 2=0.5, and the batch B4 is |2-1| / 2=0.5, and the batch B2 of plot D 02 is |1-1| / 2=0, the batch B3 is |2-1| / 2=0.5, and the batch B4 is |2-2| / 2=0; secondly, the disturbance intensity normalized value S j of each batch is calculated by normalizing the variance of the batch label index, and the calculation results are shown in Table 5:

[0118] Table 5 Batch disturbance intensity normalized value table

[0119]

[0120] Thirdly, the stability normalized value R' m,j of the plot is calculated, which is 1 if the main label index of the current batch is the same as that of the previous batch, and 0 if it is different. For example, the stability of plot D 01 is batch B2=0, B3=0, B4=0, and the stability of plot D 02For batch B2 = 1, B3 = 0, B4 = 1; then all batch data is substituted into the formula: Where, let stability adjustment coefficient γ = 0.8, total batch number T = 4 of the plot, and plot D 01 and D 02 The complete calculation steps of the trend difference index are as follows:

[0121] The first batch (B1):

[0122] The second batch (B2):

[0123] The third batch (B3):

[0124] The fourth batch (B4):

[0125] Subsequently, the calculation results of the four batches are summed:

[0126] 0 + 0.196 + 0 + 0.1643 = 0.3603;

[0127] Finally, the sum is divided by the batch number T = 4 to obtain the final trend difference index value:

[0128]

[0129] Where, the trend difference index represents the overall difference in the change pattern of the main label in the time dimension between two different plots, and the larger the value, the more obvious the difference between the two plots in the change direction, amplitude and label evolution volatility in each collection batch; the smaller the value, the more similar the stage trajectory in the state evolution of the two plots. Through this index, multiple plots can be divided into trend consistent group and trend discrete group, providing decision basis and input support for subsequent regional aggregation, nutrient space-time prediction and state synchronization control. The results show that the trend difference index between plot D 01 and D 02 in the overall batch is 0.0901, and similarly, the trend difference index between each pair of plots (such as D 01 and D 03 , D 02 and D 03 ) is calculated to obtain the evolution trend deviation value between all plots.

[0130] S513: According to the plot evolution trend deviation value, calculate the trend distance between each pair of plots, identify the plots with consistent evolution trend and aggregate them, and generate the regional state analysis result;

[0131] First, the trend difference index between each pair of plots is obtained by the previous step, for example, plot D 01 and plot D 02 has a trend difference index value of plot D 01 and plot D 03 has a trend difference index of plot D 02 and plot D 03 has a trend difference index of Subsequently, a preset trend distance judgment threshold is called, which is set by preliminary experimental statistics, and the experimental data results are shown in Table 6:

[0132] Table 6 Trend distance judgment threshold experiment table

[0133] Trend category Trend difference index interval Trend consistent [0,0.300] Trend significantly deviates (0.300,1.000]

[0134] As shown in Table 6, the trend difference index values between plots are compared one by one with the threshold interval, and the judgment operation is performed, for example, plot D 01 and plot D 02 has a trend difference index of 0.0901 in the interval [0, 0.300], so it is judged that plot D 01 and plot D 02 have the same trend, and the difference index of plot D 01 and plot D 03 is 0.375 in the interval (0.300, 1.000], which is judged to be significantly deviated from the trend, and the trend difference index of plot D 02 and plot D 03 is 0.298 in the interval [0, 0.300], which is judged to be consistent, and then all the plots with the same trend are aggregated, and the specific aggregation action is to mark the plots with the same trend as the same category, for example, plot D 01 and plot D 02 are marked as the first category area, and plot D 03 is marked as the second category area alone, the classification action of region aggregation is completed, the region state analysis result data is generated, and is recorded in the form of classification as the region state analysis result.

[0135] The above is only a preferred embodiment of the present application, and does not limit the form of the present application, and any skilled person in the art can use the above disclosed technical content to make changes or modifications to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made to the above embodiments without departing from the technical solution content of the present application, according to the technical essence of the present application, still belongs to the protection scope of the technical solution of the present application.

Claims

1. A deep learning-based rice nutrient status recognition method, characterized in that, The method comprises the following steps: S1: call the rice image dataset, analyze the change trend of the green area pixel density in the leaf image in the continuous time period, identify the trend difference between adjacent images, judge the growth state transformation node, mark the mutation node of the current image, and generate the sequence node division parameter; The sequence node division parameter acquisition step is specifically: S111: call the rice image dataset, collect the leaf images in the continuous time period, extract the green channel pixel value for each image, calculate the green pixel density in each sampling unit, detect the green pixel density change trend of each frame in combination with the arrangement order of the image sequence, and generate a green change trend sequence; S112: according to the green change trend sequence, analyze the green pixel density of adjacent image frames, identify the change direction and trend inflection point, and calculate the trend mutation index by using the formula: ； extract the extreme value point to obtain the growth state transformation recognition point set; in, For the first The normalized value of the green pixel density in the frame image. It is a very small positive constant. For the first Normalized values ​​of leaf morphology changes in frame images. For the first Frame image acquisition time normalized value, This is a dimensionless adjustment coefficient. For the first Frame trend abrupt change index, This is the index of the current image frame's position in the sequence; S113: call the growth state transformation recognition point set, judge the growth state transformation node of the rice, mark the mutation node of the current image, and generate the sequence node division parameter; S2: according to the sequence node division parameter, measure the leaf edge and contour distribution characteristics in the image, analyze the boundary closure and contour continuity, judge the abnormal structure image and eliminate it, and generate a training sample screening result; S3: call the training sample screening result, analyze the expansion ratio of the leaf shape boundary in the horizontal and vertical directions, match the rice type label, and adjust the channel response ratio between the texture information processing area and the edge contour processing area, train the rice nutrient state recognition model, and generate a processing channel matching result; S4: based on the processing channel matching result, using the rice nutrient state recognition model, obtaining the label response score of each image, extracting the main label and the secondary label, comparing the response scores, identifying the difference degree and combining the recognition label, and evaluating the confidence of the model recognition result, and outputting a rice state dataset. 2.The deep learning-based rice nutrient status recognition method according to claim 1, characterized in that, The sequence node division parameter includes the green area pixel density direction, the green pixel change trajectory trend and the image mutation position index. The training sample screening result includes the contour boundary distribution range, the edge continuity index and the image elimination position information. The processing channel matching result is specifically the texture channel weight ratio, the edge channel weight ratio and the type label classification item. The rice state dataset is specifically the main and secondary label pair, the response ordering structure and the recognition confidence score. 3.The deep learning-based rice nutrient status recognition method of claim 1, wherein, The training sample screening result acquisition step is specifically: S211: according to the sequence node division parameter, obtain the leaf edge range and contour extension value of each image in the image frame sequence, calculate the edge width, edge symmetry ratio and extension deformation amplitude of the current frame, and obtain the boundary shape abnormal value; S212: according to the boundary shape abnormal value, extract the boundary distribution direction change, leaf horizontal width change ratio and area coverage ratio of the two consecutive frames in the image frame, analyze the area alignment angle and boundary matching degree, and calculate the structure coherence trend value; S213: According to the structure coherent trend value, judge the abnormal structure image and perform rejection, filter the model training data set, and establish the training sample filtering result. 4.The deep learning-based rice nutrient status recognition method according to claim 3, characterized in that, The processing channel matching result acquisition step is specifically: S311: Call the training sample filtering result, collect the leaf boundary information of each frame of image, measure the transverse profile length and longitudinal profile length, analyze the length proportion structure of boundary expansion, construct the boundary space distribution index according to the transverse-longitudinal ratio and the edge closure degree of leaf profile, generate the leaf structure expansion coefficient; S312: According to the leaf structure expansion coefficient, compare with the preset shape classification section, judge the morphological type of the image, mark the image label, identify the feature processing path requirement corresponding to each type, obtain the rice type structure label group; S313: Based on the rice type structure label group, according to the type label, the proportion adjustment of the channel response between the texture information processing area and the edge contour processing area is carried out, the processing combination of the image input channel is established, the rice nutrient state recognition model is trained, and the image input channel matching result is generated. 5.The deep learning-based rice nutrient status recognition method according to claim 4, characterized in that, The rice state data set acquisition step is specifically: S411: Call the image input channel matching result, obtain the label response score group output by the recognition model for each image, extract the first two labels according to the response score, obtain the primary label and the secondary label, and generate the primary and secondary label extraction result; S412: According to the primary and secondary label extraction result, calculate the score difference and the sorting distance value between the primary label and the secondary label, establish the difference intensity index structure, and obtain the label difference sorting distance value; S413: Call the label difference sorting distance value, compare with the set recognition confidence benchmark interval, identify the confidence level and obtain the classification reliability level, combine the primary label response value, and establish the rice state data set. 6.The deep learning-based rice nutrient status recognition method according to claim 1, characterized in that, The method further comprises: S5: Use the rice state data set to extract the rice nutrient state label, construct the label time distribution sequence according to the plot and collection batch information, analyze the direction and frequency of the evolution trend of the primary label between plots, identify the plots with consistent evolution trend and aggregate them, and generate the regional state analysis result; The regional state analysis result includes plot label change direction code, label frequency aggregation grouping, and trend clustering boundary demarcation. 7.The deep learning-based method for identifying the nutrient status of rice according to claim 6, characterized in that, The acquisition step of the regional state analysis result is specifically: S511: Call the rice state data set, locate the plot number and collection batch number corresponding to each image, identify the primary label information of the corresponding image, construct the time sequence arrangement structure according to the plot number, construct the label time change trajectory vector corresponding to each plot based on the sampling sequence of the label on the time axis, and obtain the label time distribution sequence; S512: According to the label time distribution sequence, extract the label sequence difference value of adjacent plots in each batch, calculate the direction change index and step change value of the primary label of each plot in the same batch, construct the trajectory difference combination, calculate the trend difference index between plots, and obtain the plot evolution trend deviation value; S513: According to the land evolution trend deviation value, the trend distance between each pair of lands is calculated, the lands with consistent evolution trend are identified and aggregated, and the regional state analysis result is generated.

Images