A method and system for training a crop phenology recognition model

Abstract

This invention discloses a training method and system for a crop phenological recognition model, relating to the field of agricultural computer vision technology. The method generates a bioenergy map based on the first and second biological responses and adaptive weights of phenological stages in crop training images labeled with phenological stages; determines the number of global scaling regions and local cropping regions based on the phenological stage of the anchor image, and determines the cropping position based on the bioenergy map; performs global scaling and local cropping on the anchor image and supplementary image, and stitches them together to form a hybrid training image; determines the label weights based on the bioenergy integrals of each image region, generating hybrid labels; trains the model using the hybrid training image and hybrid labels, and combines ordinal constraints to reduce cross-stage misclassification. This invention can adapt to plant tissue changes in different phenological stages, improving the accuracy and stability of phenological recognition in difficult stages, adjacent stages, and across environments.

Description

Technical Field

[0001] This invention belongs to the fields of agricultural information technology, computer vision and artificial intelligence technology, and in particular relates to a training method and system for a crop phenology recognition model. Background Technology

[0002] Crop phenological stages reflect the growth and development status of crops and are an important basis for planting management and agricultural production decisions. Current technologies typically use deep learning models to classify field images to identify the phenological stage of crops. However, the appearance differences between adjacent phenological stages are small, and the plant tissues that play a distinguishing role in different growth stages are not the same, which can easily lead to confusion between adjacent stages.

[0003] To expand the training samples, existing techniques typically involve randomly cropping, scaling, and stitching multiple images, generating mixed labels based on the area ratio of each image region. However, field images contain a large amount of bare soil, shadows, and other background areas, and the area of ​​these regions cannot accurately reflect the amount of effective crop information contained within them. Random cropping may also miss key areas related to phenological stages, such as seedlings, ears, or mature tissues, leading to a mismatch between the mixed images and the mixed labels.

[0004] In addition, fixed biological indices mainly respond strongly to green leaves and canopy, making it difficult to take into account mature, senescent, and withered biological tissues. Existing crop image enhancement methods lack the ability to perceive the dynamic changes in phenological stages, and fixed data enhancement strategies cannot adaptively switch between different stages, which restricts the recognition performance of the entire phenological cycle. Summary of the Invention

[0005] This invention provides a training method and system for a crop phenology identification model, aiming to solve the above-mentioned technical problems.

[0006] This invention is implemented as follows: This invention provides a training method for a crop phenology recognition model, comprising: Multiple crop training images labeled with phenological stages are acquired. Based on the first biological response, the second biological response, and the corresponding phenological stage adaptive weights of the crop training images, the two biological responses of the crop training images are fused to obtain a bioenergy map. Select an anchor image and at least one supplementary image, determine the number of global scaling regions and local cropping regions based on the phenological stage of the anchor image, and determine the cropping position of the local cropping region in the source image based on the corresponding bioenergy map. The anchor point image and the supplementary image are globally scaled and locally cropped, and the resulting multiple image regions are stitched together to form a hybrid training image. Based on the bioenergy integral of the bioenergy map region corresponding to each image region, the label weight of each image region is determined. The phenological stage labels of each source image are weighted according to the label weight to obtain a mixed label. The crop phenology recognition model is trained using the mixed training images as input and the mixed labels as supervision information, and the phenological stage of the crop image to be recognized is determined using the trained crop phenology recognition model.

[0007] Preferably, based on the first biological response, the second biological response, and the corresponding phenological stage adaptive weights of the crop training image, the two biological responses of the crop training image are fused to obtain a bioenergy map, including: Based on the color channel values ​​and pixel positions of each pixel in the crop training image, a first biological response and a second biological response are determined for each pixel position; wherein, the first biological response is used to characterize green leaves and green canopy, and the second biological response is used to characterize at least one of reproductive organs, mature plant tissues, senescent plant tissues and withered plant tissues; The adaptive weights for the phenological stages are determined based on the phenological stages corresponding to the crop training images. Based on the adaptive weights of the phenological stages, the first biological response and the second biological response at each pixel location are weighted and fused to obtain the bioenergy map.

[0008] Preferably, determining the adaptive weights for phenological stages based on the phenological stages corresponding to the crop training images includes: The phenological stages are assigned stage numbers according to the growth and development sequence of the crop; As the stage sequence number increases, the fusion weight of the first biological response decreases, the fusion weight of the second biological response increases, and the fusion weight between adjacent phenological stages transitions smoothly.

[0009] Preferably, determining the number of global scaling regions and local cropping regions based on the phenological stage of the anchor point image includes: According to the order of the multiple phenological stages, the multiple phenological stages are divided into an early stage group, a middle stage group, and a late stage group; When the anchor point image belongs to the early stage group, the number of local cropping regions is set to be greater than the number of global scaling regions. When the anchor point image belongs to the intermediate stage group, the number of local cropping regions is set to be equal to the number of global scaling regions; When the anchor point image belongs to the late stage group, the number of global scaling regions is set to be greater than the number of local cropping regions.

[0010] Preferably, determining the cropping location of the local cropping region in the source image based on the corresponding bioenergy map includes: Based on the bioenergy value of each pixel position in the bioenergy map corresponding to the source image, determine the sampling probability of each pixel position being selected as the cropping center; An initial cropping center is selected based on the sampling probability, and the cropping window is determined based on the target size of the local cropping area. When the cropping window exceeds the image boundary of the source image, the position of the cropping window is adjusted so that the cropping window is within the effective range of the source image.

[0011] Preferably, the label weight of each image region is determined based on the bioenergy integral of the corresponding bioenergy map region, including: The bioenergy map corresponding to the source image is subjected to the same global scaling or local cropping process as the source image to obtain a region bioenergy map that corresponds one-to-one with each image region. Integrate the bioenergy values ​​in the bioenergy maps of each region to obtain the bioenergy integral for each image region. The label weight of each image region is determined based on the proportion of the bioenergy integral of each image region to the sum of the bioenergy integrals of the mixed training images.

[0012] Preferably, the phenological stage labels of each source image are weighted according to the label weights to obtain a mixed label, including: Based on the phenological stages of each source image and the order of arrangement among multiple phenological stages, an ordinal soft label is generated for each source image, and the label value of the ordinal soft label decreases as the stage distance between the corresponding phenological stage and the phenological stage of the source image increases. The corresponding ordinal soft labels are weighted according to the label weights of each source image to obtain mixed ordinal labels, which are then used as the mixed labels of the mixed training images.

[0013] Preferably, at least one supplementary image is selected, including: Obtain the current training progress of the crop phenology recognition model; When the current training progress is in the first training interval, a supplementary image is selected from the crop training images with the same phenological stage label as the anchor image; When the current training progress is in the second training interval, as the current training progress increases, the probability of selecting supplementary images from crop training images that have adjacent phenological stage labels to the anchor image increases. When the current training progress is in the third training interval, a supplementary image is selected from crop training images that have the same or adjacent phenological stage labels as the anchor image. The first training interval, the second training interval, and the third training interval are set sequentially according to the training progress.

[0014] Preferably, training a crop phenology recognition model using the mixed training images as input and the mixed labels as supervisory information includes: The hybrid training image is input into the crop phenology recognition model to obtain the stage prediction probability of each phenological stage corresponding to the hybrid training image; The expected prediction ordinal of the hybrid training image is determined based on the stage ordinal number of each phenological stage and the corresponding stage prediction probability. The target expected ordinal number of the mixed training image is determined based on the stage ordinal number of each phenological stage and the corresponding label value in the mixed ordinal label. The ordinal constraint loss is determined based on the deviation between the predicted expected ordinal and the target expected ordinal, and the crop phenology identification model is trained based on the ordinal constraint loss.

[0015] The present invention also discloses a training system for a crop phenology recognition model, comprising: a bioenergy modeling module, used to acquire multiple crop training images with phenological stage labels, and to fuse the two biological responses of the crop training images according to the first biological response, the second biological response and the corresponding phenological stage adaptive weights to obtain a bioenergy map; The image region determination module is used to select an anchor image and at least one supplementary image, determine the number of global scaling regions and local cropping regions based on the phenological stage of the anchor image, and determine the cropping position of the local cropping region in the source image based on the corresponding bioenergy map. The hybrid image generation module is used to perform global scaling and local cropping on the anchor point image and the supplementary image, and to stitch together the resulting multiple image regions to form a hybrid training image. The hybrid label generation module is used to determine the label weight of each image region based on the bioenergy integral of the bioenergy map region corresponding to each image region, and to weight the phenological stage labels of each source image according to the label weight to obtain hybrid labels. The model training and recognition module is used to train a crop phenology recognition model with the mixed training image as input and the mixed label as supervision information, and to use the trained crop phenology recognition model to determine the phenological stage of the crop image to be recognized.

[0016] This invention adaptively adjusts the fusion weights of the first and second biological responses based on the phenological stages of crop training images, enabling the bioenergy map to change its response focus as the crop develops, taking into account green canopy, reproductive organs, and mature / senescent tissues, without relying on the model training state. It dynamically sets the number of global scaling regions and local cropping regions based on the phenological stages of the anchor image, and determines the cropping position based on the bioenergy map, thus taking into account early local details, mid-stage canopy and organ features, and late-stage overall maturity. By applying the same spatial processing to the source image and bioenergy map, and determining the label weights based on the regional bioenergy integral, it reduces the interference of low-bioinformation regions such as bare soil and shadows on the mixed labels. Utilizing ordinal soft labels and ordinal constraint loss to express the sequence and proximity of phenological stages reduces misjudgments across long distances and stages; combined with course learning and dynamic multi-view... Figure 1 Consistency constraints can reduce training conflicts and improve the model's accuracy and stability in recognizing adjacent stages and different field environments. Attached Figure Description

[0017] Figure 1 This is a flowchart of the crop phenology identification model training method provided in the embodiments of the present invention; Figure 2 This is a flowchart of the bioenergy map generation method provided in the embodiments of the present invention; Figure 3 This is a flowchart of the adaptive weight determination method for phenological stages provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of adaptive weight changes in phenological stages provided in an embodiment of the present invention; Figure 5 This is a flowchart of the method for setting the number of global scaling areas and local cropping areas provided in the embodiments of the present invention; Figure 6 This is a flowchart of the method for determining the location of a local cropping area provided in an embodiment of the present invention; Figure 7 This is a flowchart of a method for selecting supplementary images based on training progress provided in an embodiment of the present invention; Figure 8 This is a flowchart of the image region label weight determination method provided in the embodiments of the present invention; Figure 9 This is a flowchart of the hybrid ordinal tag generation method provided in the embodiments of the present invention; Figure 10 This is a flowchart of a method for training a model based on ordinal constraint loss provided in an embodiment of the present invention; Figure 11 This is a schematic diagram of the arrangement and grouping of phenological stages provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] The embodiments disclosed in this invention use maize as the crop to be identified, dividing the phenological stages of maize into nine stages: sowing, emergence, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain-filling stage, maturity stage, and harvesting stage. The corresponding phenological stage labels are sequentially set to 0, 1, 2, 3, 4, 5, 6, 7, and 8. In other embodiments, other phenological stages can be set according to the crop type and actual identification needs.

[0020] Among them, the phenological stage label also serves as the stage ordinal number of the corresponding phenological stage, which is used to determine the adaptive weight of the phenological stage, generate ordinal soft labels, and calculate the preset expected ordinal number and the target expected ordinal number.

[0021] like Figure 1 As shown, the training method for a crop phenology identification model provided in this embodiment includes steps S100 to S500.

[0022] S100: Acquire multiple crop training images with phenological stage labels. Based on the first biological response, the second biological response, and the corresponding phenological stage adaptive weights of the crop training images, fuse the two biological responses of the crop training images to obtain a bioenergy map.

[0023] Training images can be acquired by cameras set up at fixed observation locations in farmland, mobile inspection equipment, or drones. The acquired images are then manually labeled or automatically labeled after manual review, so that each crop training image has a corresponding phenological stage label.

[0024] Images from the sowing period mainly show the ground surface, sowing furrows, or soil-covered areas after sowing; images from the emergence to the tasseling period gradually show seedlings, green leaves, green canopy, and tassels; images from the grain-filling period to the harvest period show a gradual decrease in green tissues and a gradual increase in ear, mature tissues, senescent tissues, dried tissues, post-cut stubble, and exposed areas on the ground.

[0025] The crop training images are subjected to size unification and color channel normalization. In this embodiment, the values ​​of each color channel are normalized to the range of 0 to 1. A bioenergy map is calculated for each crop training image, so that each training image has a corresponding bioenergy map when it is subsequently selected as an anchor image or supplementary image.

[0026] like Figure 2 As shown, generating a bioenergy map includes steps S110 to S130.

[0027] S110, based on the color channel value and pixel position of each pixel in the crop training image, determine the first biological response and the second biological response for each pixel position; wherein, the first biological response is used to characterize green leaves and green canopy, and the second biological response is used to characterize at least one of reproductive organs, mature plant tissues, senescent plant tissues and withered plant tissues.

[0028] Specifically, the crop training image is a color image containing red, green, and blue channels. Each pixel in the crop training image is iterated through to obtain the red channel value corresponding to each pixel position (x, y). Green channel value and blue channel value The values ​​of each color channel are normalized so that they fall within the range of 0 to 1.

[0029] The first biological response is used to characterize green leaves and green canopy. Since the green channel values ​​of green plant tissues are typically higher than the red and blue channel values, the first biological response at pixel location (x, y) is determined based on the difference between the green channel value and the red and blue channel values.

[0030] in, This represents the first biological response at pixel location (x, y); max(*) is used to set calculation results less than 0 to 0. The larger the first biological response, the higher the probability that the corresponding pixel belongs to a green leaf or green canopy.

[0031] The second biological response is used to characterize at least one of reproductive organs, mature plant tissues, senescent plant tissues, and withered plant tissues. As crops progress from the vegetative growth stage to the reproductive growth, grain-filling, maturation, and senescence stages, the green channel value of the relevant plant tissues gradually decreases, while the red, yellow, or brown components gradually increase. Therefore, the second biological response at pixel location (x, y) is determined based on the difference between the red channel value and the green channel value.

[0032] During the harvest season, the second biological response table can be used to collect the dried stems, stubble, and residual mature tissues remaining after harvest. In other embodiments, auxiliary responses can be added to characterize the sowing furrow, the soil covering area, or the post-harvest surface condition, but these auxiliary responses do not change the basic processing relationship between the first and second biological responses, which are fused according to the adaptive weights of phenological stages.

[0033] in, The second biological response represents the pixel location (x, y). The larger the second biological response, the higher the probability that the corresponding pixel belongs to the male spike, mature spike, senescent leaf, withered stem, or other mature and senescent plant tissue.

[0034] The first biological response and the second biological response are calculated for all pixel locations in the crop training image, resulting in a first biological response map and a second biological response map with the same size as the crop training image. The first biological response map mainly highlights the green leaves and canopy at the seedling stage, three-leaf stage, seven-leaf stage, and jointing stage, while the second biological response map mainly highlights the reproductive organs, mature tissues, senescent tissues, dried tissues, and post-harvest stubble at the tasseling stage, grain-filling stage, maturity stage, and harvesting stage.

[0035] In other embodiments, the first and second biological responses can also be determined based on calibrated color space components. For example, crop training images can be converted from RGB color space to HSV or Lab color space, and the responses of green plant tissues and mature / senescent plant tissues can be determined based on differences in hue, saturation, or color components. Alternatively, different combination coefficients can be set for each color channel based on the organ color characteristics of the target crop, but the two biological responses should respectively characterize plant tissues with identifiable value at different growth stages.

[0036] S120, determine the adaptive weight of the phenological stage based on the phenological stage corresponding to the crop training image.

[0037] Specifically, according to the growth and development sequence of maize, the phenological stages of maize are divided into the sowing period, emergence period, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain-filling stage, maturity stage, and harvesting period, and the corresponding phenological stage labels are set as 0, 1, 2, 3, 4, 5, 6, 7, and 8, respectively. The phenological stage labels also serve as the stage numbers of the corresponding phenological stages.

[0038] Different plant tissues are involved in identification at different phenological stages. From emergence to the seven-leaf stage, identification is primarily based on green seedlings, green leaves, and the green canopy; during the jointing and tasseling stages, the green canopy, stems, and reproductive organs need to be considered; from the grain-filling stage to harvest, identification relies more on the panicle, mature tissues, senescent tissues, dried tissues, and post-harvest stubble. Therefore, the fusion ratio of the first and second biological responses should be adjusted according to the phenological stage corresponding to the crop training image.

[0039] like Figure 3 As shown, the flowchart of the adaptive weight determination method for phenological stages includes steps S1201 to S1202. Simultaneously, combined with... Figure 4 , Figure 4 This is a schematic diagram of the adaptive weight change of phenological stages provided in an embodiment of the present invention, used to represent the fusion ratio of the two types of biological responses as they change with the phenological stages.

[0040] S1201 Assign stage numbers to multiple phenological stages according to the growth and development sequence of the crop. S1202 As the stage number increases progressively, the fusion weight of the first biological response decreases, the fusion weight of the second biological response increases, and the fusion weight between adjacent phenological stages transitions smoothly.

[0041] Following the growth and development sequence of maize, nine phenological stages were assigned stage numbers from 0 to 8. As the stage number increases, the fusion weight of the first biological response gradually decreases, while the fusion weight of the second biological response gradually increases.

[0042] This embodiment uses a sigmoid function to determine the fusion weights of the second biological response:

[0043] in, Indicates the ordinal number of the phenological stage. The weights represent the rate of change, and μ represents the stage at which the weights of the two biological responses undergo a significant shift.

[0044] In this embodiment, the following settings are provided: ; The fusion weights for the first biological response are:

[0045] The fusion weights corresponding to the nine phenological stages are as follows:

[0046] As shown in the table above, the first biological response is mainly adopted from the sowing period to the three-leaf stage; the two biological responses transition smoothly from the seven-leaf stage and the jointing stage; and the contribution of the second biological response is gradually increased from the tasseling stage to the harvest stage, so that the bioenergy map can adapt to the changes of maize from green vegetative growth to reproductive growth, maturity and senescence and post-harvest state.

[0047] For images from the sowing period, the total bioenergy of the bioenergy map may be low because green plant tissue or mature senescent tissue may not yet be present in the image. When the bioenergy integral of the bioenergy map is lower than a preset minimum energy threshold, the sampling probability of each pixel location can be set to a uniform distribution, so that the local cropping area covers the sowing furrow, the soil covering area and its surrounding environment, avoiding the inability to determine the cropping location due to the overall low response of all pixels.

[0048] Taking the three-leaf stage as an example, the phenological stage label corresponding to the three-leaf stage is 2, and the fusion weight of its second biological response is:

[0049] The fusion weights for the first biological response are:

[0050] Therefore, the three-leaf stage mainly utilizes the first biological response to highlight green leaves and green canopy, while retaining a low proportion of the second biological response.

[0051] Taking the maturity stage as an example, the phenological stage label corresponding to the maturity stage is 7, and the fusion weight of its second biological response is:

[0052] The fusion weights for the first biological response are:

[0053] Therefore, during the maturity stage, the second biological response is mainly used to highlight the mature spike, senescent leaves and dried stems, while a small amount of the first biological response is retained to characterize the green tissues that have not yet fully aged.

[0054] In images taken at the sowing stage, obvious plant tissue may not yet be visible. The adaptive weights for the phenological stages corresponding to the sowing stage are still determined in the manner described above. When the bioenergy integral of the fused bioenergy map is lower than the preset minimum energy threshold, a uniform sampling method can be used when determining the local cropping location to cover the sowing furrow, the soil covering area, and the surrounding surface environment.

[0055] In other implementations, k and μ can be adjusted according to the growth cycle and number of phenological stages of different crops. Piecewise linear functions, smoothed polynomial functions, or other monotonic functions that smoothly change with the stage number can also be used to determine the adaptive weights of phenological stages, as long as the fusion weight of the first biological response gradually decreases with crop growth and development, and the fusion weight of the second biological response gradually increases.

[0056] S130, according to the adaptive weight of the phenological stage, the first biological response and the second biological response at each pixel location are weighted and fused to obtain the bioenergy map.

[0057] Specifically, for each pixel location in the crop training image, the first biological response and the second biological response are weighted and summed pixel-by-pixel according to the adaptive weights of the phenological stage corresponding to the crop training image to obtain the bioenergy value of the corresponding pixel location. After weighted fusion of all pixel locations in the crop training image, a bioenergy map with the same size as the crop training image is formed.

[0058] Let t be the phenological stage label corresponding to the crop training image, and let the pixel position be... The first biological response is The second biological response is The fusion weight of the second biological response is The fusion weight of the first biological response is Then the bioenergy value at pixel position (x, y) is:

[0059] in, This represents the bioenergy value corresponding to pixel position (x, y) when the phenological stage label is t.

[0060] The weighted fusion described above does not apply a uniform bioenergy value to the entire image. Instead, it acquires the first and second bioenergy responses at each pixel location and fuses them according to adaptive weights based on the phenological stage corresponding to the image. Therefore, the resulting bioenergy map preserves the spatial distribution of plant tissues in the original crop training image.

[0061] For example, the phenological stage label of a training image of a crop at the three-leaf stage is 2. The normalized color channel value of a green leaf pixel in this image is:

[0062] The first biological response at this pixel location is:

[0063] The second biological response of this bio-pixel is:

[0064] The weight of the first biological response corresponding to the three-leaf stage is 0.9526, and the weight of the second biological response is 0.0474. Weighted fusion is performed, therefore the biological energy value of this pixel is:

[0065] For example, the phenological stage label of a training image of a mature crop is 7. The normalized color channel value of a mature tissue pixel in this image is: 20 The first biometric response of this pixel is 0, and the second biometric response is:

[0066] The weight of the first biological response corresponding to maturity is 0.1192, and the weight of the second biological response is 0.8808. Therefore, the biological energy value of this pixel is:

[0067] In another embodiment, the response of the same pixel changes under different phenological stages.

[0068] The adaptive weighting of phenological stages allows the same combination of first and second biological responses to yield different bioenergy values ​​at different phenological stages.

[0069] For example, the first biological response of a certain pixel is 0.80, and the second biological response is 0.20.

[0070] When this pixel belongs to the trifoliate stage image, its bioenergy value is:

[0071] When this pixel belongs to a mature image, its bioenergy value is:

[0072] A bioenergy map is obtained by weighted fusion of all pixels in a crop training image of size W×H.

[0073] The bioenergy map has the same width and height as the crop training image, and each pixel in the bioenergy map stores the corresponding bioenergy value.

[0074] To reduce the scale differences in bioenergy values ​​between training images of different crops caused by variations in shooting brightness, exposure, or color range, the bioenergy map can be normalized.

[0075] in, This represents the normalized bioenergy value; and These represent the maximum and minimum bioenergy values ​​in the current bioenergy graph, respectively; ε represents a preset non-zero correction value.

[0076] After normalization, the bioenergy value of each pixel is in the range of 0 to 1.

[0077] Furthermore, Gaussian filtering can be used to spatially smooth the normalized bioenergy map:

[0078] in, Indicates the Gaussian filter kernel; The standard deviation of the Gaussian filter kernel; symbol This indicates convolution processing.

[0079] Spatial smoothing can reduce the impact of a single isolated high-response pixel on the subsequent cropping position and enable adjacent pixels belonging to the same leaf, ear, or mature tissue to form a continuous high-bioenergy region.

[0080] The resulting bioenergy map is used for: Determine the sampling probability of each pixel position in the source image being selected as the cropping center; Guide the partial pruning area to prioritize covering plant tissues related to the current phenological stage; Integrate the bioenergy map of the region corresponding to each image region in the mixed training image; The label weights are determined based on the actual bioenergy contained in each image region. By fusing the first and second biological responses pixel by pixel according to the adaptive weights of phenological stages, the bioenergy map can change the focus of the response as maize gradually develops from the sowing and seedling stages to the maturity and harvest stages, providing spatial biological information for subsequent adaptive local cropping and hybrid label generation.

[0081] S200, select an anchor point image and at least one supplementary image, determine the number of global scaling regions and local cropping regions based on the phenological stage of the anchor point image, and determine the cropping position of the local cropping region in the source image based on the corresponding bioenergy map.

[0082] Specifically, one crop training image is selected as the anchor image from multiple crop training images labeled with phenological stages, and at least one crop training image different from the anchor image is selected as a supplementary image. The anchor image and the supplementary images are collectively referred to as source images, and each source image has a corresponding phenological stage label and bioenergy map.

[0083] Anchor images are used to determine the stage group of the current fusion training image, as well as the number of global zoom regions and local cropped regions; supplementary images are used to provide information on other crop individuals, shooting environment, and local plant tissues in the same or adjacent phenological stages.

[0084] Anchor images can be randomly selected from the current training batch or selected evenly based on the number of samples for each phenological stage. When the number of training images for a certain phenological stage is small, the probability of an image from that stage being selected as an anchor image can be increased. Anchor images do not need to be the images with the largest bioenergy integral.

[0085] like Figure 5 The diagram shown is a flowchart of a method for setting the number of global scaling regions and local cropping regions provided in an embodiment of the present invention. Figure 11 The diagram shown is a schematic diagram of the arrangement and grouping of phenological stages provided in an embodiment of the present invention.

[0086] S210, according to the arrangement order of multiple phenological stages, the multiple phenological stages are divided into an early stage group, a middle stage group, and a late stage group.

[0087] The early stage group includes the sowing period, seedling emergence period, and three-leaf stage, corresponding to labels 0 to 2; the mid-stage group includes the seven-leaf stage, jointing stage, and tasseling stage, corresponding to labels 3 to 5; and the late stage group includes the grain-filling stage, maturity stage, and harvesting stage, corresponding to labels 6 to 8.

[0088] S220, when the anchor image belongs to the early stage group, the number of local cropped regions is set to be greater than the number of global zoomed regions. S230, when the anchor image belongs to the middle stage group, the number of local cropped regions is set to be equal to the number of global zoomed regions. S240, when the anchor image belongs to the late stage group, the number of global zoomed regions is set to be greater than the number of local cropped regions.

[0089] In this embodiment, the target blending canvas is divided into four image regions, and the number of regions corresponding to each stage group is as follows:

[0090] The global scaling region is obtained by scaling the complete image content of the source image to the size of the target image region, and is used to preserve the overall plant structure, canopy distribution and field environment information.

[0091] The local cropping region is obtained by extracting local image patches from the source image and scaling them to the size of the target image region. It is used to increase the proportion of local phenological features such as seedlings, green leaves, stems, tassels, mature tissues, dry tissues, or post-cut stubble in the mixed training images.

[0092] In the early stages, the target plants are relatively small at the emergence and three-leaf stages, so setting more localized cropping areas can increase the retention rate of seedlings and green leaves. When no obvious plant tissue appears at the sowing stage, uniform sampling can be used to determine the localized cropping areas. In the mid-stage, both the overall canopy structure and information on local leaves, stems, and tassels are considered, so the same number of global scaling areas and localized cropping areas are set. In the late-stage, phenological assessment relies more on overall canopy color, maturity, senescence, and post-harvest ground condition, so more global scaling areas are set.

[0093] like Figure 6 As shown, the cropping position of the local cropping region in the source image is determined according to the corresponding bioenergy map, and the specific steps include steps S250 to S270.

[0094] S250, based on the bioenergy value of each pixel position in the bioenergy map corresponding to the source image, determine the sampling probability of each pixel position being selected as the cropping center.

[0095] Specifically, the source image and the corresponding bioenergy map have the same width and height, and each pixel in the source image corresponds one-to-one with a bioenergy value in the bioenergy map. The larger the bioenergy value, the more plant tissue information related to the current phenological stage is contained at that pixel location; therefore, that pixel location is set to have a higher probability of being sampled as the cropping center.

[0096] Before determining the sampling probability, spatial smoothing can be performed on the bioenergy map to reduce the impact of a single isolated high-response pixel on the selection of the cropping center, and to form continuous high-bioenergy regions between adjacent pixels belonging to the same leaf, ear, or mature tissue. Spatial smoothing can be performed using Gaussian filtering, mean filtering, or neighborhood-weighted filtering.

[0097] Let the spatially smoothed bioenergy map be denoted as E(x,y). Then the sampling probability corresponding to the pixel position (x,y) is:

[0098] in Indicates pixel position The sampling probability of being selected as the cropping center; Indicates pixel position The bioenergy value; W and H represent the width and height of the bioenergy map, respectively; This represents a preset non-zero correction value, used to avoid the inability to calculate the sampling probability when all bioenergy values ​​in the bioenergy map are zero.

[0099] In this embodiment, Set to 10 8 When the bioenergy integral of the bioenergy map is greater than the preset minimum energy threshold, the sampling probability is mainly determined based on the bioenergy value of each pixel position. When the bioenergy integral of the image during the sowing period or the image with a large area of ​​bare land after harvest is not greater than the preset minimum energy threshold, the sampling probability region of each pixel position is evenly distributed after non-zero correction, so that the cropping center can still be selected from the source image.

[0100] For example, in a simplified bioenergy map, the bioenergy values ​​of four candidate pixel locations A, B, C, and D are 0.10, 0.20, 0.30, and 0.40, respectively, and the sum of their bioenergy values ​​is 1.00. Ignoring the influence of non-zero correction values, the sampling probabilities for the four candidate pixel locations are 0.10, 0.20, 0.30, and 0.40, respectively. Therefore, the probability of candidate pixel location D being selected as the cropping center is four times that of candidate pixel location A, allowing the cropping window to cover high-bioenergy plant tissue with a higher probability.

[0101] S260, select an initial cropping center based on the sampling probability, and determine the cropping window based on the target size of the local cropping area.

[0102] Specifically, a cumulative probability distribution is constructed based on the sampling probability of each pixel location, and random sample values ​​between 0 and 1 are generated. The corresponding pixel location is selected as the initial cropping center based on the position of the random sample value in the cumulative probability distribution. Using probability sampling instead of directly selecting the location with the highest bioenergy allows for priority coverage of high-bioenergy regions while preserving variations in the cropping location.

[0103] In other embodiments, multiple candidate trimming centers can be selected from high to low bioenergy values, and one of the multiple candidate trimming centers can be randomly selected as the initial trimming center; or distance constraints can be set between multiple local trimming regions to reduce the probability that multiple local trimming regions repeatedly cover the same plant tissue.

[0104] Let the selected initial cropping center be:

[0105] The target width of the cropped region in the source image is The target altitude is The initial cropping window formed with the initial cropping center as the center is:

[0106] The target size of the local cropping region can be determined based on the size of the corresponding target image region in the mixed training image and a preset scaling ratio. For example, if the target mixed image is 640×640 pixels and is divided into four 320×320 pixel image regions, a 400×400 pixel local cropping region can be extracted from the source image, scaled to 320×320 pixels, and then stitched to the corresponding image region.

[0107] The target size of the local cropping area can also be adjusted according to the stage group to which the anchor point image belongs. For early stages such as seedling emergence and three-leaf stage, a smaller cropping window can be used to highlight the local details of the seedling and leaves; for seven-leaf stage, jointing stage, and tasseling stage, the cropping window can be appropriately enlarged to take into account the leaves, stems, and ears; for grain filling stage, maturity stage, and harvesting stage, a larger cropping window can be used to preserve the spatial relationship between mature tissues, senescent tissues, and the post-harvest surface state.

[0108] S270, when the cropping window exceeds the image boundary of the source image, adjust the position of the cropping window so that the cropping window is within the effective range of the source image.

[0109] Specifically, let the width of the source image be W, the height be H, and the width of the cropping window be... The height is Then the coordinates of the top left corner of the adjusted cropping window will be... Determined as:

[0110]

[0111] Therefore, the adjusted cropping window is represented as follows:

[0112] By making the above adjustments, without changing the size of the cropping window, the cropping window that exceeds the boundary of the source image will be moved horizontally or vertically into the effective range of the source image.

[0113] For example, if the source image size is 1024×1024 pixels, the target size of the local cropping region is 400×400 pixels, and the initial cropping center selected based on the sampling probability is (850, 760), then the initial cropping window formed around this initial cropping center is:

[0114] Since the horizontal endpoint 1050 is greater than the horizontal boundary 1024 of the source image, the cropping window is moved 26 pixels to the left horizontally to obtain the adjusted cropping window:

[0115] The adjusted cropping window width and height remain 400 pixels, and are entirely within the valid area of ​​the source image. Subsequently, the source image content within the adjusted cropping window is cropped, and the cropped image patch is scaled to the size of the corresponding image region in the mixed training image.

[0116] Simultaneously, the bioenergy map corresponding to the source image is cropped using the same clipping window and scaled using the same scaling parameters as the source image patch, resulting in a regional bioenergy map that corresponds one-to-one with the image region. This allows for subsequent integration of the actual bioenergy retained in the image region, and determination of the label weight of that image region based on the bioenergy integral.

[0117] Through steps S250 to S270, pixels with higher bioenergy have a higher probability of being sampled at the cropping center, and the cropping size remains unchanged when the cropping window is close to the image boundary. This increases the probability that plant information related to phenological stages, such as seedlings, green canopy, spikes, mature tissues, and withered tissues, is retained in the mixed training image, while maintaining spatial consistency between the local cropping region and the corresponding region's bioenergy map.

[0118] like Figure 7 As shown, the specific steps for selecting at least one supplementary image include: S201, Obtain the current training progress of the crop phenology recognition model.

[0119] Specifically, before training the crop phenology recognition model, the total number of training rounds is preset, and the model is divided into a first training interval, a second training interval, and a third training interval according to the training progress. The training progress can be represented by the current training round, the proportion of the current training round to the total number of training rounds, or the number of times the model parameters are updated.

[0120] In this embodiment, the total number of training rounds is set to 200 rounds, rounds 1 to 80 are set as the first training interval, rounds 81 to 140 are set as the second training interval, and rounds 141 to 200 are set as the third training interval.

[0121] In each training round, a crop training image with phenological stage labels is selected from the current training batch or the complete training set as an anchor image, and the phenological stage labels of the anchor image are obtained.

[0122] The nine phenological stages of corn correspond to labels 0 to 8, with the labels for sowing, emergence, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain filling stage, maturity stage, and harvesting stage being 0, 1, 2, 3, 4, 5, 6, 7, and 8, respectively.

[0123] If the phenological stage label of the anchor image is 'c', then a crop training image with the same phenological stage label as the anchor image refers to a crop training image with the same phenological stage label as 'c'; a crop training image with an adjacent phenological stage label as the anchor image refers to a crop training image with the phenological stage label as 'c'. Crop training images of 1 or c+1.

[0124] For the sowing and harvesting periods located at the ends of the phenological sequence, only the phenological stage on one side is considered the adjacent phenological stage. Specifically, the sowing period is labeled 0, and its adjacent phenological stage is the emergence period corresponding to label 1; the harvesting period is labeled 8, and its adjacent phenological stage is the maturity period corresponding to label 7.

[0125] S202, when the current training progress is in the first training interval, select a supplementary image from the crop training images with the same phenological stage label as the anchor image.

[0126] Specifically, in the early stages of model training, the crop phenological recognition model has not yet formed a stable representation of phenological stage features. To reduce the training difficulty caused by mixing images of different phenological stages, within the first training interval, the candidate range of supplementary images is limited to crop training images with the same phenological stage label as the anchor image.

[0127] For example, when the current training round is the 50th round and the anchor image is the jointing stage image corresponding to label 4, a supplementary image is selected from other training images of crops at the jointing stage, and images at the seven-leaf stage or tasseling stage are not selected as supplementary images.

[0128] When the mixed training images include four image regions, one anchor image and three supplementary images can be selected. The three supplementary images all have the same phenological stage label as the anchor image. The supplementary images can be randomly selected from the current training batch or from an image index built according to the phenological stage labels in the complete training set.

[0129] When selecting supplementary images, it is preferable to exclude the anchor image itself and to ensure that multiple supplementary images are different from each other to improve the content diversity of the mixed training images. When the number of available images for the same phenological stage is insufficient, it is also permissible to allow the same source image to be cropped in different local areas to form different image regions.

[0130] By using image mixing of the same stage in the first training interval, the model can learn the common visual features of different shooting angles, different plots of land and different environmental conditions within the same phenological stage, thus avoiding large prediction fluctuations in the early stage of model training due to the simultaneous participation of different phenological stages in the mixing.

[0131] S203, when the current training progress is in the second training interval, as the current training progress increases, the probability of selecting supplementary images from crop training images that have adjacent phenological stage labels to the anchor image is increased.

[0132] Specifically, after the model enters the second training interval, crop training images with adjacent phenological stage labels to the anchor image are gradually added to the supplementary image candidate range, so that the model gradually learns the continuous change features between adjacent phenological stages.

[0133] For example, if the anchor image is the jointing stage image corresponding to label 4, then the adjacent phenological stages include the seven-leaf stage corresponding to label 3 and the tasseling stage corresponding to label 5. When determining to select supplementary images from adjacent phenological stages, images can be selected with equal probability from the seven-leaf stage image set and the tasseling stage image set, or the selection probability can be set according to the number of training samples for the two adjacent phenological stages.

[0134] In other implementations, the probability of selecting supplementary images for adjacent phenological stages can also be increased with the training progress using a piecewise linear function, a sigmoid function, or a cosine variation function, as long as the probability of images from adjacent phenological stages participating in the mixing increases overall with the current training progress.

[0135] S204, when the current training progress is in the third training interval, select a supplementary image from crop training images that have the same or adjacent phenological stage labels as the anchor image.

[0136] Specifically, when the model enters the third training interval, it has already learned the basic visual features of the same phenological stage and has gradually been exposed to mixed images of adjacent phenological stages in the second training interval. Therefore, in the third training interval, crop training images of the same phenological stage and adjacent phenological stages are used together as a supplementary image candidate range.

[0137] In this embodiment, within the third training interval, the selection probability of images of adjacent phenological stages is set to 0.8, and the selection probability of images of the same phenological stage is set to 0.2, so as to maintain a certain proportion of mixed training of the same stage, while focusing on training the model's ability to recognize continuous changes in adjacent phenological stages.

[0138] For example, when the current training round is 160 and the anchor image is the male extraction period image corresponding to label 5, the supplementary image can be selected from the following image set: The set of images of the ejaculation period corresponding to label 5; The set of images corresponding to the jointing stage for label 4; The set of images during the grouting period corresponding to label 6. If three supplementary images are required, one image of the tasseling stage, one image of the jointing stage, and one image of the grain-filling stage can be selected; alternatively, two images of adjacent phenological stages and one image of the same phenological stage can be selected according to a preset probability. The specific combination of stages for each supplementary image can be determined randomly.

[0139] For the sowing period anchor image corresponding to label 0, the supplementary image is selected from the sowing period and seedling emergence period images; for the harvesting period anchor image corresponding to label 8, the supplementary image is selected from the maturity period and harvesting period images to avoid selecting a previous or subsequent stage that does not exist.

[0140] In other implementations, the selection probability can be adjusted based on the number of samples in each phenological stage. When the number of training images for a certain adjacent phenological stage is small, the probability of images from that stage being selected as supplementary images is increased to reduce the impact of the imbalance in the number of samples between different phenological stages on model training.

[0141] Through steps S201 to S204, the stage relationships of the supplementary images are gradually changed according to the training progress of the crop phenological recognition model. This allows the model training process to gradually transition from feature learning within the same phenological stage to learning the continuous changes between adjacent phenological stages. Initially, using images from the same stage reduces the learning difficulty of the model. Gradually increasing the participation probability of images from adjacent stages in the middle stage avoids sudden changes in the distribution of training samples. In the later stage of training, using a joint mixture of images from the same stage and adjacent stages strengthens the model's ability to recognize phenological stage boundaries and growth continuity, thereby reducing confusion between adjacent stages such as the seven-leaf stage and the jointing stage, the jointing stage and the tasseling stage, and the grain-filling stage and the maturity stage.

[0142] S300, the anchor point image and the supplementary image are globally scaled and locally cropped, and the resulting multiple image regions are stitched together to form a hybrid training image.

[0143] Specifically, after determining the number of global scaling regions and local cropping regions based on the phenological stage group to which the anchor point image belongs, the target blending canvas is divided into multiple target regions, and the image regions corresponding to each target region are obtained from the anchor point image and the supplementary image, respectively. The global scaling region is obtained by scaling the corresponding source image as a whole, and the local cropping region is obtained by extracting local image patches from the corresponding source image and scaling them to the target region size.

[0144] The source image includes an anchor image and a supplementary image. The anchor image is used to determine the number of global scaling regions and local cropping regions used in the current hybrid training image; the supplementary image is used to add information about changes in the shooting environment, individual crops, and adjacent phenological stages in the hybrid training image. One source image can correspond to one image region, or it can correspond to multiple image regions after different cropping processes.

[0145] In this embodiment, the size of the target fusion training image is set to 640×640 pixels, and the target fusion canvas is divided into four rectangular target regions of the same size, namely the first target region, the second target region, the third target region, and the fourth target region.

[0146] The positions of the four target regions in the target blending canvas are as follows: Ω1=[0,320)×[0,320) Ω2 = [320, 640) × [0, 320) Ω3 = [0, 320) × [320, 640) Ω4 = [320, 640) × [320, 640) In this context, Ω1 represents the first target area located in the upper left, Ω2 represents the second target area located in the upper right, Ω3 represents the third target area located in the lower left, and Ω4 represents the fourth target area located in the lower right.

[0147] The four target regions are adjacent to each other and do not overlap. The four target regions together cover the entire target blending canvas, thereby avoiding unfilled areas or overlapping areas in the blended training image.

[0148] In other implementations, the target blending canvas can be divided into two, three, six, or other numbers of target regions, each of which may have the same size or different sizes. The number and size of the target regions can be determined based on the size of the training image and the model input size.

[0149] Based on the stage group to which the anchor point image belongs, determine the regions in the four target regions that use global scaling and the regions that use local cropping.

[0150] In this embodiment, the nine phenological stages of maize are divided according to their growth and development sequence as follows: The sowing period, emergence period, and three-leaf stage belong to the early stage group; The seven-leaf stage, jointing stage, and stamen emergence stage belong to the mid-stage group; The grouting stage, maturity stage, and harvesting stage belong to the late stage group. The corresponding region processing method is as follows:

[0151] Global scaling regions are used to preserve the overall canopy morphology, plant density, field environment, and spatial context of the source image. Local cropping regions are used to increase the proportion of local phenological features such as seedlings, green leaves, stems, tassels, mature tissues, withered tissues, or post-harvest stubble in the mixed training images.

[0152] The target regions corresponding to each processing method can be configured in a preset order, or they can be randomly assigned each time a mixed training image is generated. For example, in the intermediate stage group, the first and second target regions can be set as global scaling regions, and the third and fourth target regions can be set as local cropping regions; alternatively, two of the four target regions can be randomly selected as global scaling regions.

[0153] For a target region that is determined to be a global scaling region, obtain the complete image content of the corresponding source image and scale the complete source image to the size of the corresponding target region.

[0154] In this embodiment, the source image size is 1024×1024 pixels and the target area size is 320×320 pixels. Therefore, the complete source image is scaled from 1024×1024 pixels to 320×320 pixels.

[0155] Image scaling can employ bilinear interpolation, bicubic interpolation, or area interpolation. This embodiment uses bilinear interpolation. During scaling, the color channel order of the source image is preserved, and pixel normalization is applied in the same manner as the model input.

[0156] For a target region identified as a local cropping region, the cropping center is determined based on the bioenergy map of the corresponding source image, and a cropping window is formed based on the target size of the local cropping region.

[0157] While performing global scaling or local cropping on the source image, the same spatial transformation is applied to the corresponding bioenergy map of the source image.

[0158] For the global scaling region, the complete bioenergy map is scaled to the size of the corresponding target region; for the local cropping region, the same cropping window as the source image is used to crop the bioenergy map, and the cropped local bioenergy map is scaled to the size of the corresponding target region.

[0159] By performing the same cropping and scaling operations on the source image and the corresponding bioenergy map, the pixel position correspondence between each image region in the mixed training image and the corresponding region bioenergy map is maintained, so that the bioenergy actually contained in each image region can be integrated in the subsequent process.

[0160] Multiple image regions obtained through global scaling or local cropping are filled into the corresponding target regions in the target blending canvas to form a blended training image.

[0161] At the stitching boundaries of image regions, adjacent image regions can be directly connected, or a transition band with a width of 1 to 3 pixels can be used to smooth the adjacent image regions. This embodiment uses the direct stitching method, which keeps the source range of each image region clear and facilitates accurate calculation of the bioenergy integral corresponding to each image region.

[0162] S400: Determine the label weight of each image region based on the bioenergy integral of the bioenergy map region corresponding to each image region, and weight the phenological stage labels of each source image according to the label weight to obtain a mixed label.

[0163] like Figure 8 The diagram shows a flowchart of the image region label weight determination method provided in an embodiment of the present invention, including steps S410 to S430.

[0164] S410, the same global scaling or local cropping process as the source image is applied to the bioenergy map corresponding to the source image to obtain a regional bioenergy map that corresponds one-to-one with each image region.

[0165] Specifically, each source image has a bioenergy map that corresponds one-to-one with its pixel location. When generating the hybrid training images, the source image, region processing method, cropping window, scaling size, and position in the target hybrid canvas corresponding to each image region are recorded, and the corresponding bioenergy map is processed in the same way according to the recorded spatial processing parameters.

[0166] When an image region is generated using global scaling, the complete bioenergy map of the corresponding source image is scaled to the size of the image region. Let the k-th source image be I. k The corresponding bioenergy diagram is E. k The width and height of the target image region are respectively and The resulting regional bioenergy map after global scaling is as follows:

[0167] When an image region is generated using a local cropping method, the corresponding bioenergy map is cropped using the same cropping window as the source image, and the cropped local bioenergy map is scaled to the size of the target image region.

[0168] Let the cropping window used for the k-th source image be... The resulting bioenergy map after partial cropping is as follows:

[0169] For example, the source image and its corresponding bioenergy map are both 1024×1024 pixels in size. When the source image is globally scaled to generate a 320×320 pixel image region, the complete 1024×1024 pixel bioenergy map is simultaneously scaled to 320×320 pixels. When the source image is cropped using a 400×400 pixel cropping window to extract a local image patch, and the local image patch is scaled to 320×320 pixels, the same 400×400 pixel cropping window is used to extract the bioenergy map, and the extracted local bioenergy map is simultaneously scaled to 320×320 pixels.

[0170] The source image and the corresponding bioenergy map use the same cropping window, scaling size, and region location to maintain a spatial correspondence between the position of each pixel in the mixed training image and the bioenergy value in the region bioenergy map.

[0171] Multiple bioenergy maps are arranged at the same stitching positions as multiple image regions to form a hybrid bioenergy map with the same size as the mixed training image. Each region in the hybrid bioenergy map records the bioenergy actually retained in the corresponding image region.

[0172] S420, integrate the bioenergy values ​​in the bioenergy maps of each region to obtain the bioenergy integral corresponding to each image region.

[0173] Specifically, all pixel positions in the bioenergy map of each region are traversed, and the bioenergy values ​​in the regional bioenergy map are accumulated to obtain the bioenergy integral of the corresponding image region.

[0174] Let the first The bioenergy map of each region is as follows: Its pixel area is Ω k Then the first The bioenergy integral corresponding to each image region is:

[0175] Among them, S k Indicates the first Bioenergy integral corresponding to each image region; Indicates the first The bioenergy value at pixel position (x, y) in the bioenergy map of a region.

[0176] For example, the mixed training image includes four image regions, and the bioenergy integrals of the bioenergy maps of the four regions are as follows:

[0177] The bioenergy integral of the first region is 320, meaning the sum of the bioenergy values ​​of all pixels within the first region is 320. A higher bioenergy integral indicates that the corresponding image region contains more plant tissue information related to phenological stages.

[0178] For image regions of the same size, the bioenergy integrals of each region can be directly compared. For image regions of different sizes, the influence of region area can be retained in the bioenergy integral, so that regions with larger areas and containing more effective plant information receive corresponding weights; alternatively, the bioenergy per unit area can be calculated first based on the number of pixels in the region, and then the bioenergy integral can be determined by combining the area ratio of the image region in the mixed training images.

[0179] S430, determine the label weight of each image region based on the proportion of the bioenergy integral of each image region to the sum of the bioenergy integrals of the mixed training images.

[0180] Specifically, the bioenergy integrals of all regions' bioenergy maps are summed to obtain the sum of the bioenergy integrals of the mixed training images:

[0181] in, This indicates the number of image regions in the mixed training images.

[0182] No. The label weights for each image region are:

[0183] in, Indicates the first The label weights of each image region satisfy the following conditions:

[0184] Taking the four image regions mentioned above as examples, the sum of the bioenergy integrals of the mixed training images is:

[0185] The label weight for the first region is:

[0186] The label weights for the second region are:

[0187] The label weight for the third region is:

[0188] The label weights for the fourth region are:

[0189] The sum of the label weights of the four image regions is: 0.32 + 0.24 + 0.18 + 0.26 = 1 Compared to assigning label weights based on image region area, determining label weights based on bioenergy integrals allows image regions containing more green canopy, tassels, mature tissue, or withered tissue to receive higher weights, reducing the impact of bare soil, shadows, and other low-bioinformation content on mixed labels.

[0190] When the sum of the bioenergy integrals of all image regions is 0 or lower than a preset minimum energy threshold, the label weight can be determined according to the area ratio of each image region. Since the four image regions in this embodiment have the same area, the label weight of each of the four image regions can be set to 0.25.

[0191] like Figure 9 As shown, the phenological stage labels of each source image are weighted according to the label weights to obtain a mixed label. The steps include: S440, based on the phenological stages of each source image and the arrangement order among multiple phenological stages, generate ordinal soft tags corresponding to each source image, such that the tag value of the ordinal soft tag decreases as the stage distance between the corresponding phenological stage and the phenological stage of the source image increases. Specifically, the phenological stages of maize are divided into the sowing period, seedling stage, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain-filling stage, maturity stage, and harvesting stage according to the order of growth and development, and the corresponding labels are set to 0 to 8 in sequence.

[0192] For any source image, label values ​​are assigned to nine phenological stages, centered on the actual phenological stage label of that source image. The actual phenological stage has the largest label value, the phenological stages adjacent to the actual phenological stage have the second largest label values, and the label values ​​gradually decrease as the stage distance increases.

[0193] This embodiment uses a Gaussian distribution to generate ordinal soft tags:

[0194] in, The original image labeled c represents the phenological stage in the first... The label value for each phenological stage; c represents the actual phenological stage label of the source image; and The value range is from 0 to 8; This represents the distribution width of the ordinal soft tags. In this embodiment, σ is set to 0.8.

[0195] Following the sequence of sowing period, seedling emergence period, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain filling stage, maturity stage, and harvesting period, the true phenological stage label for the source image at the seven-leaf stage is 3, and its ordinal soft label is: Y (3) =[0.000441,0.021910,0.228310,0.498676,0.228310,0.021910,0.000441,0.000002,0]; The true phenological stage label of the source image at the jointing stage is 4, and its ordinal soft label is: Y. (4) =[0.000002,0.000441,0.021910,0.228310,0.498675,0.228310,0.021910,0.000441,0.000002] The true phenological stage label for the source image during the male emergence period is 5, and its ordinal soft label is: Y (5)=[0,0.000002,0.000441,0.021910,0.228310,0.498676,0.228310,0.021910,0.000441] The sum of the nine tag values ​​in each ordinal soft tag is 1. Taking the jointing stage ordinal soft tag as an example, the tag value corresponding to the jointing stage is the largest, followed by the tag values ​​corresponding to the seven-leaf stage and the tasseling stage. As the distance between the phenological stage and the jointing stage increases, the corresponding tag value gradually decreases.

[0196] Therefore, ordinal soft labels can express the order and proximity of the nine phenological stages, avoiding treating each phenological stage as a completely independent category.

[0197] S450, the corresponding ordinal soft labels are weighted according to the label weights of each source image to obtain mixed ordinal labels, and the mixed ordinal labels are used as the mixed labels of the mixed training images.

[0198] Specifically, no. Each image region corresponds to a source image with an ordinal soft label. The label weight of this image region is λ. k Then the mixed ordinal labels are:

[0199] When multiple source images with the same source image or the same phenological stage label correspond to multiple image regions, the label weights of the multiple image regions can be added together to obtain the total label weight of the corresponding source image or the corresponding phenological stage, and then the ordinal soft label can be weighted.

[0200] For example, the four image regions mentioned above correspond to: The first region represents the jointing stage, with a label weight of 0.32. The second region is the male ejaculation period, with a label weight of 0.24; The third region represents the seven-leaf stage, with a label weight of 0.18; The fourth region is the male ejaculation period, with a label weight of 0.26. Therefore, the mixed ordinal labels are:

[0201] Both the second and fourth regions correspond to the male ejaculation period, so their label weights can be combined: 0.24 + 0.26 = 0.50 therefore:

[0202] Substituting the ordinal soft labels for the seven-leaf stage, jointing stage, and stamen emergence stage, we get: Y mix =[0.000080,0.004086,0.048327,0.173776,0.314827,0.326341,0.121246,0.011097,0.000221] Among them, the label value corresponding to the jointing stage is 0.314827, and the label value corresponding to the tasseling stage is 0.326341. Both are significantly greater than the label values ​​corresponding to the sowing stage, seedling stage, maturity stage and harvesting stage, which are far apart. This is consistent with the actual situation that the mixed training images are mainly composed of effective biological information from the jointing stage and tasseling stage.

[0203] During model training, the mixed ordinal labels are used as supervision information for the mixed training images, so that the predicted probabilities of the nine stages output by the model are closer to the mixed ordinal labels.

[0204] Through steps S410 to S450, the same spatial processing is performed on the source images and bioenergy maps to ensure a one-to-one correspondence between each image region and the region bioenergy map. The label weights are determined based on the actual bioenergy contained in the region so that the label weights can reflect the amount of effective plant tissue information. Furthermore, ordinal soft labels are generated and weighted using the arrangement order of phenological stages so that the mixed labels simultaneously contain the bioinformation contribution of each source image and the proximity relationship of phenological stages, thereby improving the consistency between the mixed training images and the supervision labels.

[0205] S500, the crop phenology recognition model is trained using the mixed training image as input and the mixed label as supervision information, and the phenological stage of the crop image to be recognized is determined using the trained crop phenology recognition model. like Figure 10 The diagram shown is a flowchart of a method for training a model based on ordinal constraint loss provided in an embodiment of the present invention, including steps S510 to S540.

[0206] S510, input the mixed training image into the crop phenology recognition model to obtain the stage prediction probability of each phenological stage corresponding to the mixed training image.

[0207] Specifically, the crop phenology identification model can employ a convolutional neural network, a visual Transformer network, or a hybrid network composed of convolutional and attention structures. The output layer of the crop phenology identification model has nine output nodes, corresponding to the sowing, emergence, three-leaf, seven-leaf, jointing, tasseling, grain-filling, maturity, and harvesting stages of maize, with stage numbers ranging from 0 to 8.

[0208] The hybrid training image generated in step S300 is input into the crop phenology recognition model. The model's feature extraction layer extracts the color, texture, morphology, canopy structure, ear and mature senescent tissue features from the hybrid training image, and the output layer obtains the prediction scores corresponding to the nine phenological stages.

[0209] Let the predicted score corresponding to the i-th phenological stage be denoted as . The nine prediction scores are converted into stage prediction probabilities using a normalized exponential function:

[0210] in, This indicates that the mixed training images belong to the stage ordinal. The stage prediction probability corresponding to the phenological stage; The first output of the model represents the... Each predicted score; and The values ​​range from 0 to 8.

[0211] The stage prediction probabilities of the nine phenological stages satisfy the following:

[0212] In this embodiment, the stage prediction probability output by the model for a mixed training image is: P=[0,0.01,0.03,0.12,0.34,0.36,0.11,0.03,0] According to the order of stage numbers 0 to 8, the predicted probabilities of the above stages correspond to the sowing period, seedling emergence period, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain filling stage, maturity stage, and harvesting period, respectively.

[0213] The predicted probability for the tasseling stage is 0.36, and the predicted probability for the jointing stage is 0.34, indicating that the model believes the mixed training image mainly contains phenological features of the jointing and tasseling stages.

[0214] S520, determine the expected prediction ordinal of the mixed training image based on the stage ordinal number of each phenological stage and the corresponding stage prediction probability.

[0215] Specifically, the stage ordinal number of each phenological stage is multiplied by the corresponding stage prediction probability, and the results are summed to obtain the expected prediction ordinal number of the mixed training image:

[0216] in, Indicates the expected ordinal number of the prediction; Indicates the ordinal number of the phenological stage; Represents stage ordinal number The corresponding stage prediction probability.

[0217] The expected prediction ordinal is used to represent the phenological development position determined by the model based on the combined prediction probabilities of the nine stages. Compared with using only the highest stage prediction probability to determine the prediction result, the expected prediction ordinal can utilize the prediction probabilities of all phenological stages and reflect which two adjacent phenological stages the model's prediction result falls between.

[0218] Substituting the stage prediction probability from step S510, we get:

[0219] The predicted expected ordinal number is 4.46, indicating that the model comprehensively judges that the phenological development position of the mixed training image is between the jointing stage corresponding to label 4 and the tasseling stage corresponding to label 5, and slightly biased towards the middle position between the jointing stage and the tasseling stage.

[0220] S530, determine the target expected ordinal number of the mixed training image based on the stage ordinal number of each phenological stage and the corresponding label value in the mixed ordinal label.

[0221] Specifically, the mixed ordinal labels generated in step S450 include label values ​​corresponding to the nine phenological stages. The stage ordinal number of each phenological stage is multiplied by the corresponding label value in the mixed ordinal labels, and the results are summed to obtain the target expected ordinal number.

[0222] in, Represents the expected ordinal number of the target; Indicates the ordinal number of the phenological stage; Indicates the stage ordinal number in the mixed ordinal label. The corresponding tag value.

[0223] The target expectation ordinal is used to represent the target phenological development position determined by the phenological stages of each source image and their bioenergy contributions.

[0224] In this embodiment, the mixed ordinal label corresponding to the mixed training image is: Y mix= [0.000080,0.004086,0.048327,0.173776,0.314827,0.326341,0.121246,0.011097,0.000221] Substituting the above mixed ordinal labels, we get: t tar=0×0.000080+1×0.004086+2×0.048327+3×0.173776+4×0.314827+5×0.326341+6×0.121246+7×0.011097+8×0.000221=4.319998 The target expected ordinal number is approximately 4.32, indicating that, based on the phenological stage and bioenergy contribution of each source image in the mixed training image, the target phenological development position of the mixed training image is located between the jointing stage and the tasseling stage, and is closer to the jointing stage.

[0225] S540, determine the ordinal constraint loss based on the deviation between the predicted expected ordinal and the target expected ordinal, and train the crop phenology identification model based on the ordinal constraint loss.

[0226] Specifically, the ordinal deviation between the predicted expected ordinal and the target expected ordinal is calculated: -

[0227] Alternatively, the absolute deviation can be expressed as:

[0228] In this embodiment, the predicted expected ordinal number is 4.46, and the target expected ordinal number is 4.319998. Therefore, the absolute value of the ordinal number deviation is: 2 This embodiment uses smoothed L1 loss to determine the ordinal constraint loss based on the ordinal deviation:

[0229] in, This represents the ordinal constraint loss.

[0230] Since the absolute value of the ordinal deviation in this embodiment is 0.140002, which is less than 1, the ordinal constraint loss is:

[0231] The greater the ordinal deviation, the farther the phenological development location predicted by the model is from the target phenological development location represented by the mixed ordinal labels, and the greater the corresponding ordinal constraint loss.

[0232] For example, when the target expected ordinal number is 4.32, and the model primarily predicts the maturity stage corresponding to label 7 for the mixed training images, resulting in a predicted expected ordinal number of 6.80, the absolute value of the ordinal bias is:

[0233] The corresponding ordinal constraint loss is:

[0234] The ordinal constraint loss is significantly greater than the ordinal constraint loss when the model predicts mixed training images to near the jointing or tasseling stage. Therefore, the ordinal constraint loss can impose a larger penalty on long-distance misjudgments spanning multiple phenological stages.

[0235] During model training, the classification loss can be determined based on the stage prediction probability and mixed ordinal labels output by the model. This embodiment uses the cross-entropy form of classification loss:

[0236] in, Indicates classification loss; Indicates the first ordinal number in the mixed ordinal label The label value corresponding to each phenological stage; The first output of the model represents the... The predicted probability of each phenological stage.

[0237] We weight the classification loss and the ordinal constraint loss to obtain the training loss used to update the parameters of the crop phenology identification model:

[0238] Where L represents the training loss; β represents the weight coefficient of the ordinal constraint loss. In this embodiment, β is set to 0.5.

[0239] Backpropagation is performed on the crop phenology identification model based on the training loss to calculate the gradients corresponding to the model parameters, and the model parameters are updated using the gradient descent optimization method. After multiple training epochs, the trained crop phenology identification model is obtained.

[0240] By using classification loss, the predicted probabilities of the nine stages output by the model are made close to the mixed ordinal labels; by using ordinal constraint loss, the overall phenological development position predicted by the model is made close to the target phenological development position. The two losses work together to both constrain the predicted probabilities of each phenological stage and distinguish different degrees of prediction errors by utilizing the stage sequence from sowing to harvesting.

[0241] After training, the crop image to be identified is input into the trained crop phenology recognition model to obtain the predicted probabilities of the nine phenological stages corresponding to the crop image. The phenological stage corresponding to the stage with the highest predicted probability is determined as the phenological stage of the crop image to be identified. The multi-region mixing, mixed ordinal labeling, and ordinal constraint loss used in the training phase do not change the inference structure of the model. Therefore, no additional mixed training images need to be generated in the actual phenology recognition process.

[0242] In another implementation, during training, two training views are constructed for the anchor images participating in the current training process, denoted as the first view and the second view, respectively. The first view is obtained by perturbing the anchor images with environmental degradation, and the second view is obtained by a stage-adaptive multi-region fusion image containing the anchor images.

[0243] The first and second views are input to the same deep neural network with shared network parameters to obtain corresponding stage prediction probability distributions. A consistency loss is then determined based on these two distributions. The strength of the consistency loss is adjusted according to the current training progress and the bioenergy contribution of the anchor image in the second view.

[0244] 1. Generate the first view An environmental degradation perturbation is applied to the anchor point image participating in the current training process to obtain the first view.

[0245] Environmental disturbances include at least one of image blurring, brightness variation, contrast variation, saturation variation, color temperature variation, and noise addition, used to simulate fog, lens contamination, lighting variations, sensor noise, and image quality degradation in real field environments.

[0246] In this embodiment, the environmental degradation perturbation includes Gaussian blur and color perturbation. The convolution kernel size of the Gaussian blur is randomly selected from 5, 7, 9, 11, 13, and 15, and the standard deviation is set to a random value in the range of 0.5 to 3.0.

[0247] The luminance perturbation coefficient, contrast perturbation coefficient, and saturation perturbation coefficient are randomly selected from preset ranges. In this embodiment, the luminance perturbation coefficient is 0.8 to 1.2, the contrast perturbation coefficient is 0.8 to 1.2, and the saturation perturbation coefficient is 0.8 to 1.2.

[0248] After environmental degradation perturbation, the first view undergoes the same size adjustment and normalization process as the model training input. For example, the first view is adjusted to 640×640 pixels and normalized according to the mean and standard deviation of each color channel.

[0249] 2. Generate the second view The second view is obtained from a stage-adaptive multi-region blended image. The number of global scaling regions and local cropping regions is determined based on the phenological stage of the anchor image, and multiple image regions are obtained from the anchor image and at least one supplementary image. These multiple image regions are then stitched together to form a blended image.

[0250] In the second view, at least one image region originates from the anchor point image, and the remaining image regions may originate from supplementary images that have the same or adjacent phenological stage labels as the anchor point image.

[0251] The second view is subjected to the same sizing and input normalization as the first view, so that the two views meet the input requirements of the same deep neural network.

[0252] The first view is used to simulate the appearance changes of the same anchor point image under different field imaging conditions, while the second view is used to add structural changes formed by the combination of different source images, different image regions, and adjacent phenological stages.

[0253] 3. Obtain the predicted probability distributions for the two views. The first view and the second view are input into the same deep neural network, and the first view and the second view share the same set of network parameters. They are not set up with independent recognition networks.

[0254] The stage prediction probability distribution corresponding to the first view is: The stage prediction probability distribution corresponding to the second view is: .

[0255] The phenological stages of maize are divided into sowing period, emergence period, three-leaf stage, seven-leaf stage, jointing stage, tasseling stage, grain-filling stage, maturity stage, and harvesting stage. Therefore, the stage prediction probability distributions in both the first and second views include nine probability values:

[0256]

[0257] in, and These represent the predicted probabilities that the first view and the second view belong to the phenological stage corresponding to stage ordinal number i, respectively.

[0258] 4. Determine the consistency loss The consistency loss is determined based on the stage prediction probability distributions of the first and second views. This embodiment uses symmetric KL divergence to calculate the consistency loss.

[0259] in, The Kullback-Leibler divergence represents the stage prediction probability distribution of the first view relative to the stage prediction probability distribution of the second view.

[0260] The reverse KL divergence is:

[0261] Here, ε is a preset non-zero correction value used to avoid the inability to calculate the logarithm when the predicted probability is 0. In this embodiment, ε is set to 10. 8 .

[0262] Symmetric KL divergence considers both the distributional differences of the first view relative to the second view and the distributional differences of the second view relative to the first view. The smaller the value, the closer the stage prediction probability distributions of the two views are.

[0263] In other implementations, mean squared error, Jensen-Shannon divergence, or difference in predicted expected ordinal numbers can also be used to determine the consistency loss.

[0264] 5. Adjust consistency constraints according to the hybrid strategy The first view is derived solely from the anchor image, while the second view is formed by combining the anchor image and supplementary images, and its mixed label may include multiple phenological stages. Therefore, it is not required that the first and second views have exactly the same stage prediction probability distribution at any training stage.

[0265] This embodiment adjusts the strength of the consistency constraint based on the label weights of the anchor point images in the second view. Let λa be the sum of the label weights of one or more image regions corresponding to the anchor point image, then the mixing strategy adjustment coefficient is: γ mix =λ a， Where, 0≤λ a ≤1.

[0266] When the bioenergy contribution of the anchor image in the second view is large, the second view has a strong phenological semantic association with the first view, and the consistency constraint strength should be appropriately increased; when the bioenergy contribution of the anchor image is small, the second view contains more phenological information from the supplementary image, and the consistency constraint strength should be reduced.

[0267] For example, if the second view includes four image regions, two of which originate from the anchor image and have corresponding label weights of 0.32 and 0.18 respectively, then the label weight of the anchor image in the second view is: λ a =0.32+0.18=0.50, the corresponding adjustment coefficient of the mixed strategy is 0.50.

[0268] Furthermore, the mixing strategy adjustment coefficient can also be determined based on the similarity between the ordinal soft label of the anchor point image and the mixed ordinal label of the second view. The closer the two labels are, the stronger the consistency constraint; the greater the difference between the two labels, the weaker the consistency constraint.

[0269] 6. Dynamically adjust consistency weights To avoid excessive consistency constraints when the model has not yet formed stable phenological features in the early stages of training, this embodiment gradually increases the weight of consistency loss according to the current training progress.

[0270] Let the current training epoch be e, and the training epoch where consistency constraints are first added be e. sThe number of training epochs in which the consistency weight reaches the preset maximum value is The training progress adjustment coefficient is:

[0271] In this embodiment, the total number of training rounds is 200. Set to 80, Set it to 140.

[0272] Therefore, multi-view should not be applied before round 80. Figure 1 Consistency constraints are applied; between rounds 80 and 140, the consistency constraint strength increases linearly with the number of training rounds; after round 140, the training progress adjustment coefficient is kept at 1.

[0273] The dynamic weights for consistency loss are:

[0274] in, This represents the preset maximum weight for consistency loss. In this embodiment, Set it to 0.2.

[0275] For example, in round 110, the training progress adjustment factor is:

[0276] If the label weight of the anchor image in the second view is 0.60, then the dynamic weight of the consistency loss is:

[0277] Therefore, the consistency constraint is kept weak or not enabled in the early stage of training, and the consistency constraint is increased after the model gradually forms stable phenological features. The strength of its effect is further adjusted according to the actual phenological semantic correlation between the first view and the second view.

[0278] 7. Jointly train crop phenology recognition model The first view uses ordinal soft labels of anchor point images as supervision information, while the second view uses mixed ordinal labels generated based on the bioenergy integrals of each image region as supervision information.

[0279] Let L be the classification loss of the first view. cls,1 The classification loss for the second view is L. cls,2 If the ordinal constraint loss is Lord, then the joint training loss of the model is:

[0280] Where, β ord The weights representing the ordinal constraint loss; β cons(e)This represents the consistency loss weight that changes with training progress and hybrid strategies.

[0281] The network parameters of the deep neural network are backpropagated and updated based on the joint training loss, so that the model can learn the classification of phenological stages and the ordinal relationship of stages, while reducing its dependence on unstable visual factors such as fog, brightness changes, contrast changes, sensor noise and local mixing perturbations.

[0282] Through the above multi-view Figure 1 Consistency learning involves using a first view to provide the model with visual changes of the same anchor point image under environmental degradation conditions, and a second view to provide the model with structural changes formed by global scaling regions and local cropping regions. The consistency constraints are dynamically adjusted according to the training progress and the bioenergy contribution of the anchor point image in the second view, which can reduce training conflicts caused by the inconsistency between the two view labels and improve the model's ability to represent stable plant structures, adjacent phenological stages, and different field imaging conditions.

[0283] The present invention also provides a training system for a crop phenology identification model, comprising: a bioenergy modeling module, used to acquire multiple crop training images with phenological stage labels, and to fuse the two biological responses of the crop training images according to the first biological response, the second biological response and the corresponding phenological stage adaptive weights to obtain a bioenergy map.

[0284] The image region determination module is used to select an anchor image and at least one supplementary image, determine the number of global scaling regions and local cropping regions based on the phenological stage of the anchor image, and determine the cropping position of the local cropping region in the source image based on the corresponding bioenergy map.

[0285] The hybrid image generation module is used to perform global scaling and local cropping on the anchor point image and the supplementary image, and stitch the resulting multiple image regions together to form a hybrid training image.

[0286] The hybrid label generation module is used to determine the label weight of each image region based on the bioenergy integral of the bioenergy map region corresponding to each image region, and to weight the phenological stage labels of each source image according to the label weight to obtain hybrid labels.

[0287] The model training and recognition module is used to train a crop phenology recognition model with the mixed training image as input and the mixed label as supervision information, and to use the trained crop phenology recognition model to determine the phenological stage of the crop image to be recognized.

[0288] In one specific embodiment, the training system includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it implements the functions of a bioenergy modeling module, an image region determination module, a hybrid image generation module, a hybrid label generation module, and a model training and recognition module.

[0289] The processor may include a central processing unit, a graphics processing unit, a neural network processor, or a computing unit composed of at least two of these processors. The memory is used to store crop training images, phenological stage labels, bioenergy maps, regional location information, mixed training images, mixed labels, model parameters, and intermediate data during the training process.

[0290] Each of the above modules can be implemented by an independent software program, or by different functional units within the same software program; each module can also be deployed on different computing devices and transmit data through a communication network. For example, the construction of bioenergy maps and the generation of hybrid training images can be performed by a data processing server, model training can be performed by a training server equipped with a graphics processor, and the trained crop phenology recognition model can be deployed on field monitoring terminals, agricultural drones, mobile devices, or cloud servers.

[0291] Through the above system structure, stage-adaptive bioenergy maps can be generated based on the changes in plant tissues at different phenological stages. The bioenergy maps can be used to guide the selection of local regions and the allocation of mixed labels, so that the plant tissue information retained in the mixed training images matches the supervision labels. Combined with the stage-adaptive global scaling region and local cropping region configuration, the crop phenology recognition model can improve its ability to recognize adjacent phenological stages, key local organs, and different field imaging conditions.

[0292] This application also provides the following comparative experiments and ablation experiments.

[0293] Comparative Experiment 1: To verify the effectiveness of the training method disclosed in this invention, a comparative experiment was conducted on the continuous phenological recognition task of maize in this embodiment.

[0294] The experimental data were obtained from independent test sets at three heterogeneous field sites: Site A: Foggy and low contrast; Site B: High pixel count, low sensor quality; Site C: Small target and high-altitude obstruction.

[0295] The training set covers all nine phenological stages, and the evaluation metrics include: 1. Overall accuracy: Overall accuracy represents the proportion of all test images in which the model correctly identifies the phenological stage; 2. Macro-average F1 score: The F1 score for each of the nine phenological stages is calculated, and the F1 scores of the nine stages are then averaged using equal weights. Since each phenological stage has the same weight in the calculation process, this index can reduce the impact of imbalanced sample size on the overall evaluation results, and is mainly used to reflect the model's ability to identify different phenological stages in a balanced manner.

[0296] 3. Macro-average F1 score of difficult subset: Select phenological stages with similar visual features, blurred stage boundaries, or easy confusion from the complete test set to form a difficult subset, and then calculate the macro-average F1 score of each phenological stage in the difficult subset; this index mainly reflects the model's ability to identify transition periods and similar stages, and compared with the overall accuracy, it can better reflect whether the stage adaptive enhancement method of the present invention has improved the problem of blurred stage boundaries.

[0297] 4. Macro-average F1 score of adjacent stages: This evaluates the model's ability to identify the boundaries between adjacent phenological stages. The macro-average F1 score can be calculated for each group of adjacent stages, and then averaged across all combinations of adjacent stages. A higher F1 score indicates that the model is better able to distinguish visually similar adjacent stages, and the probability of adjacent stage confusion is lower. This score has a strong correlation with the ordinal soft labeling, local region preservation, and global-local dynamic configuration of this invention.

[0298] 5. Quadratic Weighted Kappa (QWK): This reflects the ordinal consistency between the predicted results and the actual phenological stages. Unlike ordinary accuracy, QWK applies different penalties based on the distance between the predicted and actual stages. QWK directly reflects whether the ordinal consistency monitoring of this invention reduces misjudgments across long distances and stages.

[0299] 6. Mean Absolute Error (MAE): MAE represents the average distance between the predicted stage ordinal and the actual stage ordinal. The smaller the MAE, the closer the model's prediction is to the actual stage in terms of phenological sequence. Both MAE and QWK reflect the quality of ordinal prediction, but MAE more intuitively represents how many stages the prediction has deviated from the actual sequence.

[0300] The above six evaluation indicators were selected based on different experimental objectives.

[0301] In the comparative experiment provided in this embodiment, training and testing were conducted at site A, site B, and site C using schemes A, B, and C, respectively.

[0302] Option A: The ratio of global scaling to local cropping is 1:3; Option B: The ratio of global scaling to local cropping is 2:2; Option C: The ratio of global scaling to local cropping is 3:1.

[0303] The table below compares metrics based on different schemes and different sites.

[0304]

[0305] Overall accuracy analysis: Among the three fixed schemes, fixed scheme B has the highest average accuracy across the three stations, at 0.7534. This invention achieves an accuracy of 0.7970, an improvement of 0.7970 compared to fixed scheme B. 0.7534 = 0.0436, which means an increase of 4.36 percentage points.

[0306] This shows that the advantage of the present invention is not just that a certain fixed ratio is chosen to be more effective, but that by changing the number of global scaling areas and local clipping areas according to different phenological stages, it achieves comprehensive performance that exceeds all fixed ratio schemes.

[0307] Furthermore, the accuracy of this invention at site A is 0.8042, which is an improvement compared to the three fixed schemes: This represents an increase of 6.57 percentage points compared to fixed option A; This represents an improvement of 4.40 percentage points compared to the fixed option B; This represents an improvement of 5.14 percentage points compared to the fixed option C. Fixed scheme A consistently uses a 1:3 global-to-local ratio, resulting in significant local cropping. In foggy, low-contrast environments, individual local image patches may lack the canopy extent, plant density, and field context necessary for determining phenological stages. Long-term use of high-ratio local cropping can lead to over-reliance on local color or texture in the model.

[0308] This invention adds a global scaling area in the later stages, which helps preserve the overall canopy state, plant community structure, and field context information, thus exhibiting high recognition accuracy in foggy, low-contrast scenarios. The accuracy of this invention at site B is 0.8215, the highest among the three sites. This represents an improvement of 0.8215 compared to the best-performing fixed scheme B. 0.7898 = 0.0317, which means an increase of 3.17 percentage points.

[0309] Although Site B has a high image resolution, its sensor quality is low, and it may suffer from noise, color cast, or local detail distortion. Using a fixed large local cropped area can amplify sensor noise; using a fixed large global area may shrink the early crop target.

[0310] This invention adjusts the ratio of global and local information according to phenological stages, so that the model does not continuously rely on information at a single scale, thus maintaining high recognition performance even when the sensor quality is not ideal.

[0311] The accuracy of this invention at site C is 0.7653, which is an improvement of 0.7653 compared to the fixed scheme A. 0.6891 = 0.0762, which means an increase of 7.62 percentage points; Compared to the fixed scheme C, the improvement is: 0.7653. 0.6985 = 0.0668, which means an increase of 6.68 percentage points.

[0312] Fixed scheme C consistently uses a 3:1 global-local ratio. Global scaling may further reduce the effective representation scale of small seedlings and local plant structures in the original image in the mixed image, thereby increasing the risk of losing fine-grained features. This invention uses a 1:3 ratio during the sowing, emergence, and three-leaf stages, allowing more local cropping areas to be concentrated and retain seedlings and green leaves, which is beneficial for improving the identification of phenological features of small targets.

[0313] It is worth noting that although fixed scheme A also adopts a 1:3 ratio, it maintains this ratio throughout all phenological stages, which cannot adapt to the requirements of the overall structure and context during the grouting, maturity and harvesting stages. Therefore, its overall performance is still lower than that of the present invention.

[0314] Analysis of the average F1 score of macros in difficult subsets: In the method provided by this invention, the average F1 score of the difficult subset macro is 0.6884, with the highest among the three fixed schemes being 0.6387 for fixed scheme B. This invention improves the F1 score by 0.0497, an increase of 4.97 percentage points, representing a relative improvement of approximately 7.78%.

[0315] This indicates that the present invention not only improves the accuracy of easily identifiable phenological stages, but also has a more significant improvement effect on difficult phenological stages such as the three-leaf stage to the maturity stage, where visual characteristics change continuously and stage boundaries are blurred.

[0316] Analysis of macro-average F1 values ​​in adjacent stages: The method disclosed in this invention has an adjacent-stage macro average F1 value of 0.8126, which is higher than the other three fixed-ratio schemes.

[0317] Compared to the best-performing solution B, it improved by 0.0131, or 1.31 percentage points. This improvement is less than the improvement in the macro-average F1 score of the difficult subset, indicating that the dynamic region proportion significantly improves the overall identification of the difficult stage, while its improvement on the boundaries of adjacent stages is relatively mild.

[0318] Mean Absolute Error (MAE) Analysis: In the method disclosed in this invention, the MAE is 0.2558, which is lower than all fixed-ratio schemes. Among the three fixed schemes, the lowest MAE is 0.3618 for fixed scheme B, which is 0.1060 lower than that of this invention, a reduction of approximately 29.30%.

[0319] The MAE is 0.2558, which can be interpreted as the model prediction result differing from the actual phenological stage by an average of 0.256 stages. This result indicates that even if the present invention makes an incorrect prediction, the prediction result is usually closer to the actual phenological stage, and the misjudgment across long distances and stages is significantly reduced.

[0320] However, since all schemes in this experiment include ordinal supervision, it can be demonstrated here that, under the same ordinal supervision conditions, the stage adaptive region ratio further reduces the average deviation of the prediction stage.

[0321] Cross-site stability analysis: Cross-site fluctuations can be roughly observed by the difference between the highest and lowest accuracy rates across the three sites: Fixed scheme A: 0.7712 0.6891 = 0.0821; Fixed Option B: 0.7898 0.7103 = 0.0795; Fixed scheme C: 0.7824 0.6985 = 0.0839; This invention: 0.8215 0.7653 = 0.0562. The accuracy range between stations in this invention is 0.0562, which is less than that of the three fixed ratio schemes, indicating that it has more stable recognition performance in heterogeneous scenarios such as fog, low-quality sensors, and small target occlusion.

[0322] Technical conclusions from the experimental data: Based on the available data, the following conclusions can be drawn: 1. Fixed ratios of 1:3, 2:2, or 3:1 cannot simultaneously adapt to the complete phenological cycle of maize, and there is no fixed ratio that can achieve the performance of the present invention at three heterogeneous sites.

[0323] 2. The adaptive solution of this invention has the highest accuracy at all three sites, indicating that its effectiveness is not due to chance at a single site or in a single environment.

[0324] 3. Compared with the optimal fixed scheme B, the present invention improves the average accuracy of the three stations by 4.36 percentage points, the average F1 value of the difficult subset macro by 4.97 percentage points, and the average absolute error (MAE) by approximately 29.30%, indicating that the dynamic ratio not only improves the overall recognition rate, but also improves the recognition of difficult stages and reduces misjudgments at long distances.

[0325] 4. The improvement at site C was the most significant, demonstrating that increasing the local pruning area in the early stages helps preserve small-sized crop targets and local plant structure.

[0326] 5. The improvement of site A shows that adding a global zoom area in the later stages helps to preserve the overall canopy and contextual information in low-contrast environments.

[0327] 6. The accuracy range of this invention is smaller across the three sites, which initially indicates that the stage adaptive ratio has good cross-environment stability.

[0328] Ablation Experiment 2 To verify the actual contribution of each component module of the present invention to the crop phenology identification performance, this embodiment takes the benchmark model as the starting point, adds corresponding processing modules one by one according to the technical process, and conducts ablation experiments on independent test sets of three heterogeneous field sites.

[0329] The baseline model uses a ResNet-50 network and is trained using random flipping and random pruning. The meanings of each experimental configuration are as follows: +S1 indicates the addition of a bioenergy modeling module to the baseline model; +S2 indicates the addition of a hybrid image generation module for staged adaptive multi-region hybrid enhancement; + S2-λ indicates the addition of a hybrid label generation module, used to allocate label weights according to the regional bioenergy integral; + S3 indicates the further addition of ordinal soft labels and ordinal constraint loss; + S4 indicates further integration of multi-view functionality. Figure 1 Dedicated learning; Full indicates that all relevant processing modules have been added.

[0330]

[0331] Apart from the module differences shown in the table, all experimental configurations used the same training data, test data, network structure, number of training epochs, and model parameter settings. Cross-site stability was uniformly measured using either the range or standard deviation; lower values ​​indicate higher stability. The following analysis of experimental results uses the standard deviation data as the basis for cross-site stability analysis.

[0332] The experimental results are shown in the table below.

[0333] 1. Contribution Analysis of the +S1 Bioenergy Modeling Module After adding the stage-adaptive bioenergy modeling module to the baseline, the overall accuracy of the three stations improved from 0.7841 to 0.7925, an increase of 0.84 percentage points; the macro-average improved from 0.6882 to 0.6968; the F1 score of the macro-average of the difficult subset improved from 0.6418 to 0.6612, an increase of 1.94 percentage points; and the mean absolute error decreased from 0.2847 to 0.2712. The quadratic weighted QWK improved from 0.9281 to 0.9310, an increase of 0.0029, indicating that bioenergy modeling not only improved the localization of difficult areas but also improved the ordinal consistency of phenological stage predictions.

[0334] Meanwhile, the cross-site stability decreased from 0.0616 to 0.0564, indicating that dynamically fusing the first and second biological responses according to the phenological stage not only improved the overall recognition performance but also improved the recognition stability between different sites.

[0335] The results show that, compared with conventional training methods that do not include biological priors, stage-adaptive bioenergy maps can highlight green leaves and green canopies in the early stages, and improve the response to reproductive organs, mature tissues, senescent tissues and withered tissues in the middle stages, providing spatial information that is adapted to the phenological stages for subsequent image enhancement.

[0336] 2. Contribution Analysis of the +S2 Hybrid Image Generation Module A hybrid image generation module was further added to the S1 configuration for adaptive multi-region blending enhancement. The overall accuracy across the three stations improved from 0.7925 to 0.7987, an increase of 0.62 percentage points; the macro-average F1 score for the difficult subset improved from 0.6612 to 0.6748, an increase of 1.36 percentage points; the macro-average F1 score for adjacent stages improved from 0.7954 to 0.8041; and the mean absolute error decreased from 0.2712 to 0.2625. The quadratic weighted QWK decreased from 0.9310 to 0.9295, a change of [missing information]. 0.0015. Since QWK primarily reflects the ordinal consistency between the predicted results and the actual phenological stages, this slight decrease indicates that the main effect of stage-adaptive multi-region hybrid enhancement is not to directly improve ordinal consistency, but rather to improve the model's ability to recognize difficult visual features by increasing training samples at different scales and in different spatial regions. This change did not weaken the improvement effect of S2 on overall accuracy, macro-average F1 score of difficult subsets, macro-average F1 score of adjacent stages, and MAE.

[0337] The cross-site standard deviation decreased from 0.0564 to 0.0508, indicating that stage-adaptive multi-region blending enhancement can improve the stability of the model under different shooting environments.

[0338] The reason is that setting more local pruning areas in the early stage is conducive to preserving fine-grained characteristics such as seedlings, seedlings and green leaves; setting a balanced global scaling area and local pruning areas in the middle stage is conducive to taking into account both the overall canopy structure and local organs; setting more global scaling areas in the late stage is conducive to preserving overall information such as the maturity of the plant, the aging state and the state of the ground after harvest.

[0339] 3. Contribution analysis of the addition of the regional bioenergy integral allocation label weight module to +S2--λ After replacing the method of assigning label weights based on the proportion of image region area with assigning label weights based on the regional bioenergy integral, the overall accuracy of the three stations improved from 0.7987 to 0.8002, an increase of 0.15 percentage points; the macro-average F1 score improved from 0.7003 to 0.7025; the macro-average F1 score for the difficult subset improved from 0.6748 to 0.6796, an increase of 0.48 percentage points; the macro-average F1 score for adjacent stages improved from 0.8041 to 0.8073; and the mean absolute error decreased from 0.2625 to 0.2578. The quadratic weighted QWK improved from 0.9295 to 0.9302, an increase of 0.0007. These changes are relatively small, indicating that the main role of the regional bioenergy weighted label is to improve the consistency between supervisory information and effective plant regions in the mixed image, while its improvement on overall ordinal consistency is relatively limited.

[0340] Cross-site stability further decreased from 0.0508 to 0.0487.

[0341] The above results indicate that even regions of the same area may contain different amounts of effective plant tissue information. Determining label weights based on the bioenergy integral of each region allows regions containing more green canopy, spikelets, mature tissue, or withered tissue to receive higher supervisory weights, while reducing the impact of bare soil, shade, and other low-bioinformational regions on the mixed labels. Therefore, generating mixed labels based on bioenergy contribution is more consistent with the actual phenological information distribution in the mixed training images than generating mixed labels based on region area.

[0342] 4. Contribution of Ordinal Consistency Supervision After adding ordinal soft labels and ordinal constraint loss to the “+S2-λ” configuration, the overall accuracy of the three stations improved from 0.8002 to 0.8021; the macro average F1 score improved from 0.7025 to 0.7041; the macro average F1 score of the difficult subset improved from 0.6796 to 0.6812; and the macro average F1 score of adjacent stages improved from 0.8073 to 0.8088.

[0343] The weighted quadratic QWK improved from 0.9302 to 0.9336, and the mean absolute error decreased from 0.2578 to 0.2510, both reaching their optimal values ​​in the experimental configuration. Cross-site stability decreased from 0.0487 to 0.0471.

[0344] The increase in QWK and decrease in MAE indicate that ordinal soft labeling and ordinal constraint loss can incorporate the order of arrangement from sowing to harvesting into the model training process. The model produces a small ordinal bias when misclassifying a real stage as an adjacent stage, and a large ordinal bias when misclassifying a real stage as a distant stage, thereby reducing misclassification of distant cross-stage stages.

[0345] 5.+S4 Multi-view Figure 1 Contribution analysis of homogeneous learning Add multi-view Figure 1 After consistency learning, the overall accuracy of the three stations decreased from 0.8021 to 0.7926, the macro average F1 score decreased from 0.7041 to 0.7015, the macro average F1 score of the difficult subset decreased slightly from 0.6812 to 0.6798, the double-weighted QWK decreased from 0.9336 to 0.9320, and the mean absolute error (MAE) increased from 0.2510 to 0.2550.

[0346] The changes in the above indicators indicate that multi-view Figure 1 Consistency learning did not further improve classification accuracy and ordinal accuracy under single test conditions; its main effect was on cross-environment stability. Meanwhile, the macro-average F1 score between adjacent stages increased from 0.8088 to 0.8109, and the cross-site standard deviation decreased from 0.0471 to 0.0423, indicating that this module improved the model's adaptability to different environmental disturbances while sacrificing a small amount of single-point classification performance.

[0347] 6. Contributions to course learning strategies and overall solutions After adding a course learning strategy to the +S4 configuration, the overall accuracy of the three stations improved from 0.7926 to 0.7970, the average F1 score of the difficult subset macros improved from 0.6798 to 0.6884, the average F1 score of the adjacent stage macros improved from 0.8109 to 0.8126, and the cross-site stability further decreased from 0.0423 to 0.0378.

[0348] Among them, the average F1 score of the difficult subset macro increased by 0.86 percentage points, and the standard deviation decreased by 0.0045. This indicates that using image mixing of the same stage in the early stage of training, gradually increasing the participation probability of images of adjacent stages in the middle stage of training, and jointly mixing images of the same stage and adjacent stages in the later stage of training can alleviate the learning fluctuations caused by sudden changes in the stage relationship of training samples and improve the model's ability to recognize continuous phenological changes.

[0349] Compared to the baseline, the overall accuracy of the complete solution across the three stations improved from 0.7841 to 0.7970, an increase of 1.29 percentage points; the macro average F1 score improved from 0.6882 to 0.7008, an increase of 1.26 percentage points; the macro average F1 score for the difficult subset improved from 0.6418 to 0.6884, an increase of 4.66 percentage points, a relative improvement of approximately 7.26%; the macro average F1 score for adjacent stages improved from 0.7863 to 0.8126, an increase of 2.63 percentage points; the quadratic weighted QWK improved from 0.9281 to 0.9321; the mean absolute error (MAE) decreased from 0.2847 to 0.2558, a relative decrease of approximately 10.15%; and the standard deviation decreased from 0.0616 to 0.0378, a relative decrease of approximately 38.64%.

[0350] The complete solution achieves the best results in three indicators: macro-average F1 score of difficult subset, macro-average F1 score of adjacent stages, and cross-site stability. This indicates that after the components work together, the solution can improve the identification stability of adjacent stages, difficult stages, and different field stations while maintaining the overall identification performance higher than the baseline.

[0351] It should be noted that the +S3 configuration achieved the best performance in overall accuracy across the three stations, macro-average F1 score, QWK, and MAE metrics, while the complete solution achieved the best performance in macro-average F1 score for the difficult subset and macro-average F1 score for adjacent stages. This indicates that different modules have different technical focuses: ordinal consistency supervision is more conducive to improving overall classification and stage ordinal accuracy, while multi-view... Figure 1 Consistent learning and curriculum learning strategies are more conducive to improving the ability to identify difficult stages and adjacent stages.

[0352] Comparative Experiment 3: Comparative Experiment of Dual-Response Fusion and CAM (Class Activation Map, CAM) Single-Response To verify the technical effectiveness of the adaptive dual-response fusion mechanism of this invention compared with single biological response and salient region extraction methods based on model features, this embodiment conducts a comparative experiment on the independent test set corresponding to site C.

[0353] The crop images at site C are mainly taken from high altitudes, and are characterized by small crop target size, mutual occlusion between leaves and ears, and a low proportion of crop area in the complete image. Therefore, they are more sensitive to saliency maps or bioenergy maps used to guide local cropping.

[0354] This embodiment sets up the following four saliency or bioenergy modeling schemes: 1. Single ExG Index Scheme: This scheme generates vegetation response maps based solely on the combination relationships between green, red, and blue channels. The calculation method is 3G. 2.4R B, do not change the response method according to the phenological stage; 2. Single RG index scheme: Generates response map of mature and aging tissues based solely on the difference between red channel values ​​and green channel values, without changing the response mode according to phenological stages; 3. CAM-guided scheme: This scheme employs a SnapMix-like method to generate class activation maps based on the internal features of the crop phenology recognition model, and determines the region sampling locations in the mixed image based on these class activation maps. This scheme relies on the features currently learned by the model; therefore, the quality of the class activation maps varies with the model's training state. 4. The dual-response fusion scheme of the present invention: a first biological response for characterizing green leaves and green canopy, and a second biological response for characterizing reproductive organs, mature plant tissues, senescent plant tissues and withered plant tissues are determined respectively, and the fusion weights of the two biological responses are adaptively adjusted according to the phenological stage of the crop training image to obtain a bioenergy map.

[0355] Aside from differences in saliency or bioenergy modeling methods, all four experimental schemes used the same training data, testing data, network structure, training parameters, stage-adaptive multi-region hybrid enhancement, label assignment, ordinal consistency supervision, and multi-view... Figure 1 Consistent learning and curriculum learning strategies lead to differences in experimental results primarily stemming from bioenergy modeling methods.

[0356] The experimental results are shown in the table below.

[0357]

[0358] 1. Experimental results of the single ExG exponent scheme The single ExG index scheme achieved an overall accuracy of 0.7124 at site C, with a macro-average F1 score of 0.5836 for the difficult subset, a macro-average F1 score of 0.7628 for adjacent stages, an F1 score of 0.4125 for male spikes, and an F1 score of 0.3014 for senescent stages.

[0359] The ExG index leverages the advantage of the green channel over the red and blue channels to highlight green leaves and canopy at the seedling, three-leaf, seven-leaf, and jointing stages. Therefore, this scheme provides relatively stable green plant regions for images during the vegetative growth phase.

[0360] However, as maize enters the tasseling, grain-filling, maturity, and harvesting stages, the dominance of green channels in the ear, mature tissues, senescent tissues, and withered tissues gradually weakens. The single ExG index still uses a fixed green response pattern, resulting in insufficient response of related tissues in the saliency plot. Therefore, the F1 score for tassel-type maize under this scheme is only 0.4125, and the F1 score for senescent maize is only 0.3014.

[0361] The above results indicate that a single green plant response can cover some visual features during the vegetative growth period, but it is difficult to adapt to the complete phenological cycle from sowing to harvest.

[0362] 2. Experimental results of the single RG exponent scheme The single RG index scheme achieved an overall accuracy of 0.6891 at site C, with a macro-average F1 score of 0.5502 for the difficult subset and a macro-average F1 score of 0.7356 for adjacent stages, all of which were the lowest among the four schemes. Its F1 score for male spikelets was 0.5621, and its F1 score for senescent stages was 0.4897.

[0363] The RG index showed a high response to plant tissues with enhanced red, yellow, or brown components relative to green components. Therefore, compared to the single ExG index, the single RG index increased the F1 tassel type from 0.4125 to 0.5621, and the senescent F1 type from 0.3014 to 0.4897.

[0364] However, this scheme is insufficient in responding to green seedlings and green canopies at the emergence, three-leaf, and seven-leaf stages, and cannot cover the main identification information during the vegetative growth period. Therefore, its overall accuracy, the macro-average F1 score of the difficult subset, and the macro-average F1 score of adjacent stages are all lower than the single ExG index scheme.

[0365] The above results indicate that while enhancing the response of mature and senescent tissues alone can improve the identification of some phenological stages in the middle and late stages, it weakens the ability to locate early green plant tissues and still cannot meet the requirements for continuous phenological identification.

[0366] 3. Experimental Results of the CAM Guidance Scheme The CAM-guided scheme achieved an overall accuracy of 0.7238 at site C, with a macro-average F1 score of 0.5947 for the difficult subset and 0.7721 for adjacent stages, all higher than the single ExG index scheme and the single RG index scheme. Its macro-average F1 score for male spikelets was 0.4534, and its macro-average F1 score for senescence was 0.3621.

[0367] The results show that generating class activation maps based on internal model features can locate image regions with classification functions after the model has learned certain phenological features, and its overall performance is better than the fixed single response scheme.

[0368] However, CAM is generated from the current features of the crop phenological recognition model. In the early stages of model training, the network has not yet fully learned the stable features of each phenological stage, and the class activation map may respond to the background, illumination, or high-frequency textures, making it difficult to stably locate small tassels and mature senescent tissues that appear less frequently. When imaging conditions change at different sites, the location of the model feature response may also change.

[0369] Therefore, although the CAM-guided approach can identify salient regions using model features, its region localization quality is affected by the current model training status and cross-site feature changes.

[0370] 4. Experimental results of the dual-response fusion scheme of the present invention The dual-response fusion scheme of this invention achieves an overall accuracy of 0.7653 at site C, a macro-average F1 score of 0.6234 for the difficult subset, a macro-average F1 score of 0.8058 for adjacent stages, a macro-average F1 score of 0.6187 for male spikelets, and a macro-average F1 score of 0.5543 for senescence. All evaluation indicators are higher than the other three schemes.

[0371] Compared to the CAM-guided scheme, the overall accuracy of the dual-response fusion scheme of this invention is improved by 0.7653. 0.7238 = 0.0415, which means an increase of 4.15 percentage points.

[0372] The average F1 score of the difficult subset macros improved by 0.6234. 0.5947 = 0.0287, which means an increase of 2.87 percentage points.

[0373] The macro-average F1 score increased by 0.8058 in adjacent phases. 0.7721 = 0.0337, which means an increase of 3.37 percentage points.

[0374] The increase in F1 tassel varieties was 0.6187. 0.4534 = 0.1653, which means an increase of 16.53 percentage points.

[0375] F1 increased by 0.5543 during aging. 0.3621 = 0.1922, which means an increase of 19.22 percentage points.

[0376] The above results show that the improvement of the dual-response fusion scheme of the present invention is mainly reflected in the identification of male spikes and mature senescent tissues, and also improves the comprehensive differentiation ability between difficult phenological stages and adjacent phenological stages.

[0377] The reason for this is that the present invention does not employ a fixed green response or a fixed maturity / senescence response throughout all phenological stages. Instead, it adjusts the fusion weights of the two biological responses based on the phenological stage corresponding to the crop training image. The fusion ratio of the first biological response is increased during the seedling, three-leaf, and seven-leaf stages, while the fusion ratio of the second biological response is gradually increased during the tasseling, grain-filling, maturity, and harvest stages. This allows the response focus of the bioenergy map to adapt to the major plant tissues at different phenological stages.

[0378] Furthermore, the bioenergy map of this invention is generated directly from the color channel values ​​and phenological stage labels of the input image, without waiting for the crop phenological recognition model to form stable internal features. Therefore, a bioenergy map for guiding local cropping can be generated before or in the early stages of model training, avoiding the inability to reliably determine the cropping region due to unstable features in the early stages of model training.

[0379] 5. Experimental Conclusions The above comparative experiments show that the single ExG index is mainly applicable to the green vegetative growth period, while the single RG index is mainly applicable to the reproductive, maturation, and senescence stages. Neither single response can cover the complete phenological process of maize from sowing to harvest.

[0380] The CAM-guided approach can identify salient regions using the classification features learned by the model, but its salient region localization results are affected by the model's current training state and the imaging conditions of the test site. Its stability is limited in the early stages of training and under conditions of small targets, occlusion, and cross-site conditions.

[0381] This invention adaptively integrates the first and second biological responses based on phenological stages, enabling the bioenergy map to simultaneously consider green leaves, green canopy, tassels, mature tissues, senescent tissues, and withered tissues, and adjust the response focus as phenological stages progress. This bioenergy map is independent of the model's current training state and provides a stable spatial basis for determining local pruning locations and regional bioenergy integration.

[0382] Therefore, compared with single color response and CAM guidance, the dual-response stage adaptive fusion mechanism of this invention can improve the recognition performance of difficult phenological stages, adjacent phenological stages, male spikelets and senescence in small target and occlusion scenarios, verifying the effectiveness of stage adaptive dual-response bioenergy modeling.

[0383] Comparative experiments show that, compared to a fixed region ratio, the stage-adaptive region ratio of this invention improves the overall accuracy, the macro-average F1 score of the difficult subset, and the macro-average F1 score of adjacent stages for three heterogeneous sites, while reducing MAE. The dual-response fusion scheme, compared to single ExG index, single RG index, and CAM-guided schemes, exhibits better comprehensive responses to tassels and mature / senescent tissues in small target and occlusion scenarios. The complete scheme achieves superior results in difficult subset identification, adjacent stage identification, and cross-site stability, indicating that each component module synergistically improves the performance of continuous crop phenology identification by effectively locating tissues, constructing training samples, matching labels, constraining stage ordinal numbers, and enhancing cross-environment robustness.

[0384] In summary, this invention adaptively adjusts the fusion weights of the first biological response, which characterizes green leaves and green canopy, and the second biological response, which characterizes reproductive organs, maturation, senescence, and withered tissues, based on the phenological stages of crop training images. This allows the generated bioenergy map to change its response focus as the crop grows and develops, taking into account effective visual information from different stages from sowing to harvesting. Furthermore, it does not depend on the current training state of the model and can stably provide regional sampling basis in the early stages of training.

[0385] Secondly, the number of global scaling regions and local cropping regions is dynamically set according to the phenological stage of the anchor image, and the local cropping position is determined based on the bioenergy map. This ensures that the early stage focuses on preserving local details of seedlings and leaves, the middle stage takes into account canopy structure and reproductive organs, and the late stage focuses on preserving overall maturity, senescence, and post-harvest surface information, reducing the probability of missing key plant tissues due to fixed enhancement ratios or random cropping. By applying the same spatial processing to the source image and the corresponding bioenergy map, and determining the label weight based on the actual bioenergy integral of each region, the interference of low-bioinformatic areas such as bare soil and shadows on the mixed labels can be reduced, improving the consistency between the mixed training images and the supervision information.

[0386] Ordinal soft labels are used to represent the sequential and proximity relationships among the nine phenological stages. Ordinal constraint loss is determined based on the deviation between the predicted expected ordinal and the target expected ordinal, thus reducing misclassification of adjacent stages and significantly constraining misclassification of long-distance cross-stage stages, thereby lowering the average deviation of phenological stage predictions. By gradually transitioning from image mixing within the same stage to image mixing of adjacent stages according to the training progress, the model first learns stable intra-stage features and then learns continuous changes between adjacent stages, reducing training difficulty and stage boundary confusion.

[0387] Furthermore, multi-view implementation is achieved through a hybrid view combining an environment degradation view and a stage-adaptive view. Figure 1Consistency learning, and dynamically adjusting the consistency constraint strength according to the training progress and the bioenergy contribution of anchor point images in the mixed view, can reduce the impact of unstable factors such as fog, brightness changes, sensor quality and local mixing perturbation on the model, while reducing training conflicts caused by the inconsistency of the two view labels, and improving the stability of the model under different field sites and imaging conditions.

[0388] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A training method for a crop phenology recognition model, characterized in that, include: Multiple crop training images labeled with phenological stages are acquired. Based on the first biological response, the second biological response, and the corresponding phenological stage adaptive weights of the crop training images, the two biological responses of the crop training images are fused to obtain a bioenergy map. Select an anchor image and at least one supplementary image, determine the number of global scaling regions and local cropping regions based on the phenological stage of the anchor image, and determine the cropping position of the local cropping region in the source image based on the corresponding bioenergy map. The anchor point image and the supplementary image are globally scaled and locally cropped, and the resulting multiple image regions are stitched together to form a hybrid training image. Based on the bioenergy integral of the bioenergy map region corresponding to each image region, the label weight of each image region is determined. The phenological stage labels of each source image are weighted according to the label weight to obtain a mixed label. The crop phenology recognition model is trained using the mixed training images as input and the mixed labels as supervision information, and the phenological stage of the crop image to be recognized is determined using the trained crop phenology recognition model.

2. The training method for the crop phenology recognition model according to claim 1, characterized in that, Based on the first biological response, the second biological response, and the corresponding phenological stage adaptive weights of the crop training image, the two biological responses of the crop training image are fused to obtain a bioenergy map, including: Based on the color channel values ​​and pixel positions of each pixel in the crop training image, a first biological response and a second biological response are determined for each pixel position; wherein, the first biological response is used to characterize green leaves and green canopy, and the second biological response is used to characterize at least one of reproductive organs, mature plant tissues, senescent plant tissues and withered plant tissues; The adaptive weights for the phenological stages are determined based on the phenological stages corresponding to the crop training images. Based on the adaptive weights of the phenological stages, the first biological response and the second biological response at each pixel location are weighted and fused to obtain the bioenergy map.

3. The training method for the crop phenology recognition model according to claim 2, characterized in that, Determine the adaptive weights for phenological stages based on the phenological stages corresponding to the crop training images, including: The phenological stages are assigned stage numbers according to the growth and development sequence of the crop; As the stage sequence number increases, the fusion weight of the first biological response decreases, the fusion weight of the second biological response increases, and the fusion weight between adjacent phenological stages transitions smoothly.

4. The training method for the crop phenology recognition model according to claim 1, characterized in that, The number of global scaling regions and local cropping regions is determined based on the phenological stage of the anchor point image, including: According to the order of the multiple phenological stages, the multiple phenological stages are divided into an early stage group, a middle stage group, and a late stage group; When the anchor point image belongs to the early stage group, the number of local cropping regions is set to be greater than the number of global scaling regions. When the anchor point image belongs to the intermediate stage group, the number of local cropping regions is set to be equal to the number of global scaling regions; When the anchor point image belongs to the late stage group, the number of global scaling regions is set to be greater than the number of local cropping regions.

5. The training method for the crop phenology recognition model according to claim 1, characterized in that, Determining the cropping location of the local cropping region in the source image based on the corresponding bioenergy map includes: Based on the bioenergy value of each pixel position in the bioenergy map corresponding to the source image, determine the sampling probability of each pixel position being selected as the cropping center; An initial cropping center is selected based on the sampling probability, and the cropping window is determined based on the target size of the local cropping area. When the cropping window exceeds the image boundary of the source image, the position of the cropping window is adjusted so that the cropping window is within the effective range of the source image.

6. The training method for the crop phenology recognition model according to claim 1, characterized in that, The label weights of each image region are determined based on the bioenergy integral of the corresponding bioenergy map region, including: The bioenergy map corresponding to the source image is subjected to the same global scaling or local cropping process as the source image to obtain a region bioenergy map that corresponds one-to-one with each image region. Integrate the bioenergy values ​​in the bioenergy maps of each region to obtain the bioenergy integral for each image region. The label weight of each image region is determined based on the proportion of the bioenergy integral of each image region to the sum of the bioenergy integrals of the mixed training images.

7. The training method for the crop phenology recognition model according to claim 1, characterized in that, The phenological stage labels of each source image are weighted according to the aforementioned label weights to obtain a mixed label, including: Based on the phenological stages of each source image and the order of arrangement among multiple phenological stages, an ordinal soft label is generated for each source image, and the label value of the ordinal soft label decreases as the stage distance between the corresponding phenological stage and the phenological stage of the source image increases. The corresponding ordinal soft labels are weighted according to the label weights of each source image to obtain mixed ordinal labels, which are then used as the mixed labels of the mixed training images.

8. The training method for the crop phenology recognition model according to claim 1, characterized in that, Select at least one supplementary image, including: Obtain the current training progress of the crop phenology recognition model; When the current training progress is in the first training interval, a supplementary image is selected from the crop training images with the same phenological stage label as the anchor image; When the current training progress is in the second training interval, as the current training progress increases, the probability of selecting supplementary images from crop training images that have adjacent phenological stage labels to the anchor image increases. When the current training progress is in the third training interval, a supplementary image is selected from crop training images that have the same or adjacent phenological stage labels as the anchor image. The first training interval, the second training interval, and the third training interval are set sequentially according to the training progress.

9. The training method for the crop phenology recognition model according to claim 7, characterized in that, Training a crop phenology recognition model using the hybrid training images as input and the hybrid labels as supervisory information includes: The hybrid training image is input into the crop phenology recognition model to obtain the stage prediction probability of each phenological stage corresponding to the hybrid training image; The expected prediction ordinal of the hybrid training image is determined based on the stage ordinal number of each phenological stage and the corresponding stage prediction probability. The target expected ordinal number of the mixed training image is determined based on the stage ordinal number of each phenological stage and the corresponding label value in the mixed ordinal label. The ordinal constraint loss is determined based on the deviation between the predicted expected ordinal and the target expected ordinal, and the crop phenology identification model is trained based on the ordinal constraint loss.

10. A training system for a crop phenology recognition model, characterized in that, include: The bioenergy modeling module is used to acquire multiple crop training images with phenological stage labels. Based on the first biological response, the second biological response, and the corresponding phenological stage adaptive weights of the crop training images, the two biological responses of the crop training images are fused to obtain a bioenergy map. The image region determination module is used to select an anchor image and at least one supplementary image, determine the number of global scaling regions and local cropping regions based on the phenological stage of the anchor image, and determine the cropping position of the local cropping region in the source image based on the corresponding bioenergy map. The hybrid image generation module is used to perform global scaling and local cropping on the anchor point image and the supplementary image, and to stitch together the resulting multiple image regions to form a hybrid training image. The hybrid label generation module is used to determine the label weight of each image region based on the bioenergy integral of the bioenergy map region corresponding to each image region, and to weight the phenological stage labels of each source image according to the label weight to obtain hybrid labels. The model training and recognition module is used to train a crop phenology recognition model with the mixed training image as input and the mixed label as supervision information, and to use the trained crop phenology recognition model to determine the phenological stage of the crop image to be recognized.

Images