Egret identification method and system based on rice-fish symbiotic environment

Abstract

The invention provides a rice-fish symbiotic environment-based egret identification method and system, and the method comprises the steps: taking an egret as a target object, taking rice as a shielding object, and generating a rice shielding egret sample through a UNLOCK model; the rice sheltered egret sample is input into a U2D2Net model to simulate severe weather, and a rice sheltered-severe weather egret sample is obtained; inputting the rice shielding-severe weather egret sample as a part of a training sample into a UnnaskFormer model for training to obtain an egret identification model; and acquiring a real-time environment image in the rice-fish symbiotic environment, and inputting the real-time environment image into the egret identification model to obtain an egret identification result. According to the method, the universal shielding generation capability of UNLOCK and the universal unsupervised defogging and denoising capability of U2D 2Net are cooperatively applied to a rice-fish symbiosis specific scene, so that the identification accuracy of the egret in the rice-fish symbiosis environment is improved.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of point computer vision and smart agriculture, and in particular to a method and system for egret recognition based on a rice-fish symbiotic environment. Background Technology

[0002] In rice-fish co-culture systems, egrets, as major natural predators of fish, directly impact aquaculture profitability due to their population size and activity patterns. Current egret monitoring relies on manual patrols or general visual models, which presents three core problems: Data scarcity: It is difficult to collect egret samples in severe weather and rice fields, resulting in insufficient training data coverage and poor model generalization ability. Poor model adaptability: The target detection model performs well in recognizing egrets in unobstructed situations, but it is prone to missing detection when egrets are obscured by more than 50% of rice. General semantic segmentation models such as DeepLabV3+ and Mask R-CNN cannot handle the problem of texture confusion between rice leaves and egret feathers, resulting in low detection accuracy. Limitations of generation techniques: Existing GAN generation methods suffer from the problem of generated samples being disconnected from the real rice-fish symbiotic environment.

[0003] In other words, there is currently a lack of a method that can accurately identify egrets in a rice-fish symbiotic environment. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this invention provides a method and system for egret identification based on a rice-fish symbiotic environment, thereby improving the accuracy of egret identification in such an environment.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for egret identification based on a rice-fish symbiotic environment, comprising: Step S1: Using egrets as the target object and rice as the occlusion, generate rice occlusion egret samples using the UNLOCK model; Step S2: Input the rice paddy shading egret sample into U 2 D 2 The Net model simulates severe weather to obtain egret samples under severe weather conditions with rice paddy obstruction. Step S3: Input the egret samples of rice shading and severe weather as part of the training samples into the UnmaskFormer model for training to obtain the egret recognition model; Step S4: Obtain real-time environmental images of the rice-fish co-culture environment, input the real-time environmental images into the egret recognition model, and obtain egret recognition results.

[0006] The beneficial effects of this invention are as follows: This invention utilizes UNLOCK, a non-modal driven context learning mechanism, to generate rice-shading egret samples that conform to the characteristics of a rice-fish symbiotic environment; subsequently, this sample is input into U... 2 D 2 The Net model, through its unsupervised processing capabilities, optimizes defogging and denoising under adverse weather conditions such as fog and rain, thereby combining the general occlusion generation capabilities of UNLOCK with U 2 D 2 Net's general unsupervised dehazing and denoising capabilities are applied in conjunction with specific scenarios of rice-fish co-culture, effectively solving the core pain point of scarce egret samples in scenarios with severe weather and rice shading in existing technologies. This fills the technical gap in the application of general generative models in the specific agricultural scenario of rice-fish co-culture, thereby improving the recognition accuracy of egrets in the rice-fish co-culture environment.

[0007] Optionally, in step S1, the rice paddies obscuring the egret samples are marked with key egret features, and step S3 further includes: Differential weights were assigned to different key parts of the egret, with the beak / eye area having a greater weight than the trunk area, which in turn had a greater weight than the leg area.

[0008] Optionally, the UNLOCK model is input with core parameters of the paddy field environment, including rice plant height and leaf distribution. 2 D 2 The Net model is input with fog concentration parameters and raindrop density specific to the paddy field environment.

[0009] Optionally, the training process of the UnmaskFormer model in step S3 further includes: Extract the segmentation error output by the UnmaskFormer model and feed the segmentation error back to the UNLOCK model and U at the generation end. 2 D 2 The Net model was optimized, and the rice shading-severe weather egret samples were regenerated and trained on the UnmaskFormer model.

[0010] Optionally, the training process of the UnmaskFormer model in step S3 specifically includes: Extract the segmentation result output by the UnmaskFormer model, and then convert the segmentation result Y... pred and input sample Y gt Perform pixel-by-pixel comparison of the egret's missed detection rate R miss And occlusion misjudgment error rate R false The segmentation error was obtained, and the egret false detection rate R was calculated. miss And occlusion misjudgment error rate R false for: ; ; ; ; ; ; In the formula, N miss This represents the number of missed pixels, and the subsequent equation represents the input sample Y. gt Labeled as egret but segmentation result Y pred Undivided pixel region, N total,白鹭 N represents the total number of egret pixels. false The number of misclassified pixels is given by the following equation, which represents the input sample Y. gt Labeled as rice but segmentation result Y pred Pixel regions misidentified as egrets, N total,水稻 This represents the total number of pixels representing rice. The segmentation error is fed back to the UNLOCK model and U at the generation end. 2 D 2 The Net model, the UNLOCK model, is subjected to occlusion density optimization adjustment, and the U 2 D 2 Adjustments were made to optimize the dehazing and denoising intensity of the Net model.

[0011] Optionally, step S1 includes: Step S11: Using egrets as the target object and rice paddies as occlusions, construct a target object library and an occlusion library. The target object library consists of the instance mask M of each pixel in an egret image under a rice-fish co-culture environment. 白鹭 (x,y), where the instance mask is 1 to represent egret pixels and 0 to represent background pixels, and the rice images in the occlusion library are classified according to the rice growth stage, leaf outline data are collected, and morphological parameters are extracted; Step S12: Input a panoramic image of a rice-fish co-culture environment and generate an egret category mask m. 白鹭 Using the egret category mask m 白鹭 Correcting the prediction P of the instance branch ins Only retain the egret category mask m described in the instance branch. 白鹭 Predictions of birds landing within the egret region are used to generate initial pseudo-labels. Then, an optimal threshold is determined using a category-adaptive threshold calculation function. These initial pseudo-labels are then filtered according to this optimal threshold. Finally, an uncertainty-guided loss function is used to optimize the filtered initial pseudo-labels, yielding the final pseudo-labels. The category-adaptive threshold calculation function and the uncertainty-guided loss function are as follows: ; ; In the formula, τ fix For a fixed confidence threshold, τ per For percentage thresholds, For the number of class pixels / instances with a confidence level exceeding τ, L OPLL Let N be the OPLL loss function. 确定 To determine the total number of pixels in a region, M 确定 (c,x,y) is the region mask, BCE() is the binary cross-entropy loss, and P... 初始 (c,x,y) represents the initial predicted probabilities of the model, 0.1 represents the weight of the uncertain region, and N 不确定 M represents the total number of pixels in the uncertain region. 不确定 (x,y) is the mask for the uncertain region; Step S13: Divide the rice into different shading positions according to different rice growth stages, and merge them to generate rice shading egret samples.

[0012] Optionally, step S2 includes: Step S21: Normalize and preprocess the pixel values ​​of the rice shading egret sample; Step S22: Simulate severe weather by processing the pre-processed rice-shading egret samples to obtain initial severe weather egret samples; Step S23: Calculate each local block P in the initial severe weather egret sample. i,j Fog feature similarity S i,j Based on the fog feature similarity S i,j Constructing a transmissivity estimation loss L t The transmittance estimation loss L is minimized by gradient descent. t The final transmittance map t is obtained. opt The final transmittance map t opt Substitute into the atmospheric scattering model to retrieve the dehazed sample I dehaze The function for step S23 above is as follows: ; ; ; In the formula, Let σ be the mean of the global fog features, and σ be the similarity decay coefficient. For local blocks The initial transmittance, t i,j For local blocks The current transmittance, A est For U 2 D 2Atmospheric light values ​​estimated unsupervised by the Net model =10 -6 ; Step S24: Perform wavelet transform on the dehazed sample I dehaze Decomposed into low-frequency component I L and high frequency component I H Then construct a bidirectional loss L denoise To optimize low-frequency component I L and high frequency component I H Finally, through U 2 D 2 The decoder of the Net model optimizes the low-frequency component I. L and high frequency component I H Reconstruction is performed to obtain the denoised sample I. dehaze+denoise The bidirectional loss L denoise for: ; In the formula, W T () represents wavelet transform, I denoised For the sample to be denoised, I L,denoised λ represents the low-frequency components, and λ is the weight that balances texture preservation and noise removal. Step S25, via U 2 D 2 The Net model calls the multi-scale feature fusion module to extract the denoised sample I. dehaze+denoise The feature maps at multiple scales are weighted and summed to obtain the multi-scale feature fusion map F. fusion For the multi-scale feature fusion map F fusion Upsampling is performed to obtain the final optimized sample; Step S26: Inversely normalize the pixel values ​​of the final optimized sample to the [0, 255] range and save it as a preset image format.

[0013] Optionally, the recognition process of the UnmaskFormer model includes: Step S31: Assign differentiated weights to different key parts of the egret, with the weight of the beak / eye area being greater than that of the trunk area, which is greater than that of the leg area. Step S32: Retain the cross-attention module and multi-scale feature fusion module of the UnmaskFormer model; Step S33: After input normalization, distortion adaptive feature embedding, and UA module feature enhancement, the input image is output as a multi-scale feature. Step S34: After performing cross-attention occlusion completion on the multi-scale features, multi-scale feature fusion is performed to obtain a multi-scale fused feature map. The textures of rice and egrets are distinguished based on the multi-scale fused feature map. Step S35: Perform branch fusion on the prediction results of the semantic branch, the prediction results of the visible instance branch, and the prediction results of the complete instance branch in the UnmaskFormer model to obtain the final semantic segmentation result.

[0014] Optionally, the semantic branch outputs a probability map of the categories of all pixels in the image through the multi-scale fused feature map; the visible instance branch generates egret instance candidate boxes with the help of RPN, and filters out instance masks containing only the visible area of ​​egrets by combining the results of the semantic branch; the complete instance branch fills in the invisible area of ​​egrets covered by rice on the instance mask containing only the visible area of ​​egrets through cross-attention reasoning of global and local features, and outputs an instance mask of the complete form of egrets. The branch fusion process is as follows: the instance mask output by the complete instance branch is determined to be the egret region, and all non-egret regions other than the egret region are classified according to the category corresponding to the probability map of the semantic branch.

[0015] Secondly, the present invention provides an egret identification system based on a rice-fish symbiotic environment, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the egret identification method based on the rice-fish symbiotic environment of the first aspect.

[0016] The technical effects of the egret identification system based on the rice-fish symbiotic environment provided in the second aspect are described in the relevant description of the egret identification method based on the rice-fish symbiotic environment provided in the first aspect. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the main process of the egret identification method based on the rice-fish symbiotic environment in an embodiment of the present invention; Figure 2 This is a schematic diagram of an egret identification system based on a rice-fish symbiotic environment, according to an embodiment of the present invention.

[0018] Explanation of reference numerals in the attached figures: 1. Egret recognition system based on rice-fish symbiosis environment; 2. Processor; 3. Memory. Detailed Implementation

[0019] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0020] Example 1 Please refer to Figure 1 A method for egret identification based on a rice-fish symbiotic environment includes: Step S1: Using egrets as the target object and rice as the occlusion, generate rice occlusion egret samples using the UNLOCK model.

[0021] In this embodiment, the UNLOCK model is a passive occlusion perception and segmentation framework designed specifically for panoramic visual scene understanding. Its input includes core parameters of the paddy field environment such as rice plant height and leaf distribution, to improve the adaptability of the UNLOCK model in rice-fish co-culture scenarios.

[0022] In this embodiment, step S1 includes: Step S11: Using egrets as the target object and rice paddies as occlusions, construct a target object library and an occlusion library. The target object library consists of the instance mask M of each pixel in the egret image under the rice-fish co-culture environment. 白鹭 (x,y), where an instance mask of 1 represents a egret pixel and 0 represents a background pixel. Rice images in the occlusion library are classified according to the rice growth stage. Leaf contour data are collected, and morphological parameters are extracted.

[0023] In step S11, the rice paddy occlusion of the egret sample is a 512×512 RGB image, on which key egret parts are marked, including the beak, eyes, torso, and legs, ultimately yielding the instance mask M. 白鹭 (x,y).

[0024] The morphological parameters for each growth stage of rice are shown in Table 1 below.

[0025] Table 1. Morphological parameters of rice during its main growth stages

[0026] Step S12: Input a panoramic image of a rice-fish co-culture environment and generate an egret category mask m. 白鹭 Using egret category mask m 白鹭 Correcting the prediction P of the instance branch ins Only retain the egret category mask m in the instance branch. 白鹭Predictions of birds landing within the egret region generate initial pseudo-labels. Then, an optimal threshold is determined using a class-adaptive threshold calculation function. The initial pseudo-labels are then filtered according to this optimal threshold. Finally, an uncertainty-guided loss function is used to optimize the filtered initial pseudo-labels, yielding the final pseudo-labels. The class-adaptive threshold calculation function and the uncertainty-guided loss function are as follows: ; ; In the formula, τ fix For a fixed confidence threshold, τ per For percentage thresholds, For the number of class pixels / instances with a confidence level exceeding τ, L OPLL Let N be the OPLL loss function. 确定 To determine the total number of pixels in a region, M 确定 (c,x,y) is the region mask, BCE() is the binary cross-entropy loss, and P... 初始 (c,x,y) represents the initial predicted probabilities of the model, 0.1 represents the weight of the uncertain region, and N 不确定 M represents the total number of pixels in the uncertain region. 不确定 (x,y) is the mask for the uncertain region.

[0027] Among them, when given a panoramic image of a rice-fish symbiotic environment, a semantic prediction P is first generated. 初始 (c,x,y), where c = egret / rice / water / soil. If argmaxP 初始 (c,x,y)=1, then m 白鹭 (x,y)=1 indicates that the pixel belongs to the egret region, otherwise it is 0.

[0028] Among them, the prediction P of the egret-classified instance branch is corrected using the egret category mask m. ins This is to filter out misjudgments of rice-growing areas.

[0029] The fixed confidence threshold for egrets can be set to 0.7, for rice to 0.6, and the percentage threshold to 50%. Therefore, BCE() is used to measure the difference between the predicted probability and the true label; the smaller the loss, the more accurate the prediction. Predictions in precise regions are strengthened, while noise in ambiguous regions is reduced by minimizing L... OPLL The model will continuously adjust its predictions to make the definite regions more accurate and the uncertain regions more conservative, resulting in a significant reduction in the fuzzy regions of the final output pseudo-labels.

[0030] Step S13: Divide the rice into different shading positions according to different rice growth stages, and merge them to generate rice shading egret samples.

[0031] Among them, during the tillering stage when the plant height h ≤ 60, the pixel area R where the egret is obscured by the rice is... 遮挡 for: ; That is, only the regions with a y-coordinate ≥ 350 in the egret pixels are selected.

[0032] When the plant height h > 90cm is at the heading stage, the pixel area R where the egret is obscured by the rice is... 遮挡 for: ; That is, only the region with y coordinates between 200 and 400 in the egret pixels is selected; Regarding the elongation period of 61-90cm, the region with y-coordinates of 250-450 in the egret's pixels is selected.

[0033] Then, pixel-level fusion is performed to obtain the final pixel value I(x,y): ; In the formula, I 水稻 (x,y) represents the RGB value of a rice leaf at pixel (x,y). 白鹭 (x,y) represents the RGB value of the egret at this pixel; 0.8 and 0.2 are the transparency weights, with the rice leaves providing a semi-transparent overlay on the egret.

[0034] Step S2: Input the rice paddy shading egret samples into U 2 D 2 The model simulates severe weather to obtain egret samples of rice paddy shading during severe weather.

[0035] In this embodiment, U 2 D 2 The Net (Unsupervised Unified Image Dehazing and Denoising Network) model takes into account the fog concentration parameters and raindrop density specific to the paddy field environment as inputs. The fog concentration parameter is 0.2-0.8, and the raindrop density is 50-200 droplets / m², in order to improve U 2 D 2 The adaptability of the Net model in rice-fish co-culture scenarios.

[0036] Step S2 includes: Step S21: Normalize the pixel values ​​of the egret samples obscured by rice and perform local block segmentation preprocessing.

[0037] Among them, the rice paddy obscuring egret sample is an RGB image with a size of 512×512 and a pixel value of 0-255. Mapping the pixel values ​​to the [0,1] interval yields I. norm =IUNLOCK / 255 eliminates the impact of pixel size on network gradients.

[0038] Among them, I norm The data was divided into 32×32=1024 local blocks. Each local block contains local features such as the edge of rice leaves and the texture of egret feathers, providing basic units for subsequent two-way comparison.

[0039] Step S22: Simulate severe weather by using the pre-processed rice-shading egret samples to obtain initial severe weather egret samples.

[0040] Step S23: Calculate P for each local block in the initial severe weather egret sample. i,j Fog feature similarity S i,j Based on fog feature similarity S i,j Constructing a transmissivity estimation loss L t The loss L is estimated by minimizing the transmittance through gradient descent. t The final transmittance map t is obtained. opt The final transmittance map t opt Substitute into the atmospheric scattering model to retrieve the dehazed sample I dehaze The function for step S23 above is as follows: ; ; ; In the formula, Let σ be the mean of the global fog features, and σ be the similarity decay coefficient. For local blocks The initial transmittance, t i,j For local blocks The current transmittance, A est For U 2 D 2 Atmospheric light values ​​estimated unsupervised by the Net model =10 -6 .

[0041] Where σ is 0.1, the above L t The first constraint is to make the value of t in the fog region close to the true value, and the second constraint is to make the value of t in the dark region of rice close to 0.9 in order to preserve leaf details. =10 -6 This is to avoid the denominator being 0.

[0042] Step S24: Use wavelet transform to transform the dehazed sample I dehaze Decomposed into low-frequency component I L and high frequency component I H Then construct a bidirectional loss L denoiseTo optimize low-frequency component I L and high frequency component I H Finally, through U 2 D 2 The decoder of the Net model optimizes the low-frequency component I. L and high frequency component I H Reconstruction is performed to obtain the denoised sample I. dehaze+denoise Two-way loss L denoise for: ; In the formula, W T () represents wavelet transform, I denoised For the sample to be denoised, I L,denoised λ represents the low-frequency component, and λ is the weight that balances texture preservation and noise removal.

[0043] Where λ is 10.

[0044] Step S25, via U 2 D 2 The Net model calls the multi-scale feature fusion module to extract the denoised sample I. dehaze+denoise The feature maps at multiple scales are weighted and summed to obtain the multi-scale feature fusion map F. fusion For multi-scale feature fusion map F fusion Upsampling is performed to obtain the final optimized sample.

[0045] The scale feature map includes three sizes: 1 / 2 resolution, 1 / 4 resolution, and 1 / 8 resolution. The 1 / 2 scale preserves the fine features of the egret's beak and the edges of the rice leaves; the 1 / 4 scale preserves the relationship between the egret's body and the occlusion position of the rice; and the 1 / 8 scale preserves the lighting consistency of the entire rice paddy scene. In this embodiment, the weights of the three are 0.5, 0.3, and 0.2, respectively.

[0046] Among them, the multi-scale feature fusion map F fusion Upsample to 512×512 resolution.

[0047] Step S26: Inverse normalize the pixel values ​​of the final optimized sample to the [0,255] range and save it as a preset image format.

[0048] The default image format is RGB.

[0049] Step S3: Input the egret samples of rice paddy shading and severe weather as part of the training samples into the UnmaskFormer model for training to obtain the egret recognition model.

[0050] In this embodiment, the UnmaskFormer model is a panoramic image semantic segmentation method, and the recognition process applied to this embodiment includes: Step S31: Assign differentiated weights to different key parts of the egret, with the weight of the beak / eye area being greater than that of the trunk area, which is greater than that of the leg area.

[0051] In this embodiment, the weight of the beak / eye region is 2.0, the weight of the torso region is 1.2, and the weight of the leg region and the non-egret region is 1.0.

[0052] Step S32: Retain the cross-attention module and multi-scale feature fusion module of the UnmaskFormer model.

[0053] In this embodiment, the cross-attention module is retained, preserving its cross-regional feature association capabilities. This allows for long-distance inference of egret features in rice-covered areas, resolving the feature fragmentation problem caused by local occlusion. Simultaneously, the multi-scale feature fusion module is retained, preserving the original 1 / 4, 1 / 8, and 1 / 16 feature scales. The 1 / 4 scale high resolution aims to capture the fine features of egret feather texture and rice leaf edges; the 1 / 8 scale medium resolution aims to associate the occlusion relationship between the egret's body and the surrounding rice; and the 1 / 16 scale low resolution aims to globally model the integrated spatial distribution of egrets, rice, and water in the rice paddy scene, avoiding local texture confusion.

[0054] Step S33: After input normalization, distortion adaptive feature embedding, and UA module feature enhancement, output multi-scale features from the input image.

[0055] In this embodiment, step S33 specifically includes: (1) Input normalization: Normalize the I of the input image gen Pixel values ​​are normalized to [0,1] to obtain I. norm =I gen / 255, eliminates the influence of pixel size on gradient.

[0056] (2) Distortion adaptive feature embedding: The adaptive offset Δ is calculated through the DPE module. dpe , to I norm Distortion correction and patch embedding are performed to obtain patch features X. patch ∈ N×C Where N = H × W / 16 2 =1024, C=256, the formula is as follows: X patch =DPE(I norm ,Δ dpe =Conv(I norm +Δdpe ); In the formula, Δ dpe Generated by the offset predictor of the native DPE, used to correct edge distortion in rice paddy panorama samples.

[0057] (3) Enhancement of UA module features: X patch The input UA module enhances egret features and suppresses rice interference through self-attention and occlusion-aware pooling: ① Self-attention computation to capture global feature associations X attn =Self-Attention(X patch )=Softmax((QK T ) / √C)V, where Q / K / V is X patch The query / key / value matrix obtained by linear projection; ② Global Average Pooling (GAP) is used to calculate the occlusion-aware query q: q = GAP(X) attn )∈ ; ③Sigmoid generates occlusion mask (q), for X attn Weighting is applied to highlight the unobstructed features of the egret, X ua =X attn ⊙ (q), where ⊙ represents element-wise multiplication. (·) = Sigmoid(·), with values ​​ranging from 0 to 1. The value approaches 0 in the occluded area and approaches 1 in the unoccluded egret area.

[0058] (4) Multi-scale feature output: X is processed by a four-stage Transformer encoder (Stages 1-4) ua Feature maps F mapped to scales of 1 / 4, 1 / 8, and 1 / 16 1 / 4 F 1 / 8 F 1 / 16 This provides multi-granularity feature support for subsequent occlusion inference.

[0059] Step S34: After performing cross-attention occlusion completion on the multi-scale features, perform multi-scale feature fusion to obtain a multi-scale fused feature map. Then, distinguish the textures of rice and egrets based on the multi-scale fused feature map.

[0060] In this embodiment, step S34 specifically includes: (1) Cross-attention occlusion completion: F 1 / 4 With F 1 / 16 Input the cross-attention module to infer the complete morphology of the egret under rice occlusion through global-local feature association: ; When the egret's legs are obscured by rice leaves, the module uses F... 1 / 16 The global position of the egret's torso in the image guides F 1 / 4 Complete the missing pixels in the legs; (2) Multi-scale feature fusion: For F cross (1 / 4), F 1 / 8 F 1 / 16 A weighted blending method is used to balance fine textures and global relationships, as shown in the following formula: F fusion =α1·Upsample(F cross )+α2·Upsample(F 1 / 8 )+α3·Upsample(F 1 / 16 ); In the formula, α1=0.5, α2=0.3, α3=0.2, and Upsample(·) is bilinear upsampling to 512×512 resolution.

[0061] (3) Distinguishing the textures of rice paddies and egrets: through the fused F fusion Calculate the category similarity matrix: S cls ∈ H ×W×4 Texture characteristics that distinguish rice paddies from egrets: ; In the formula, c=1 represents egret, c=2 represents rice, c=3 represents water body, and c=4 represents soil.

[0062] Step S35: Perform branch fusion on the prediction results of the semantic branch, the visible instance branch, and the complete instance branch in the UnmaskFormer model to obtain the final semantic segmentation result.

[0063] In step S35, the semantic branch outputs a probability map of the pixel categories in the entire image through multi-scale fusion feature maps. It can be seen that the instance branch uses RPN to generate egret instance candidate boxes, and combines them with the results of the semantic branch P sem Filter out instance masks M that contain only the visible area of ​​egrets ins The complete instance branch, or amodal instance branch, uses cross-attention reasoning of global and local features to complete the instance mask containing only the visible area of ​​the egret, filling in the invisible area of ​​the egret obscured by rice paddies, and outputting the instance mask M of the complete egret form. amodal .

[0064] At this point, the branch fusion process is as follows: the instance mask output by the complete instance branch is used to determine the egret region; all non-egret regions other than the egret region are categorized using the class corresponding to the probability map in the semantic branch. The effective confidence level of the complete instance branch is greater than or equal to 0.9.

[0065] In this embodiment, the training process of the UnmaskFormer model in step S3 further includes: Extract the segmentation error from the UnmaskFormer model output and feed the segmentation error back to the UNLOCK model and U at the generation end. 2 D 2 The Net model was optimized, and the rice shading-severe weather egret samples were regenerated and trained on the UnmaskFormer model.

[0066] The UnmaskFormer model itself also calculates the Amodal Panoptic Quality (APQ) and IoU of egrets to ensure smooth segmentation boundaries. APQ 白鹭 = ; In the formula, Q is the set of egret instances, TP = true positive, FP = false positive, and FN = false negative.

[0067] Specifically, the above steps are implemented as follows: Extract the segmentation results from the UnmaskFormer model output, and then convert the segmentation results Y... pred and input sample Y gt Perform pixel-by-pixel comparison of the egret's missed detection rate R miss And occlusion misjudgment error rate R false The segmentation error and the egret false negative rate R were obtained. miss And occlusion misjudgment error rate R false for: ; ; ; ; ; ; In the formula, N miss This represents the number of missed pixels, and the subsequent equation represents the input sample Y. gt Labeled as egret but segmentation result Y pred Undivided pixel region, N total,白鹭 N represents the total number of egret pixels. falseThe number of misclassified pixels is given by the following equation, which represents the input sample Y. gt Labeled as rice but segmentation result Y pred Pixel regions misidentified as egrets, N total,水稻 This represents the total number of pixels representing rice. The segmentation error is fed back to the UNLOCK model and U at the generation end. 2 D 2 The Net model, the UNLOCK model, is used to optimize and adjust the occlusion density. 2 D 2 Adjustments were made to optimize the dehazing and denoising intensity of the Net model.

[0068] Among them, the false negative rate R of egrets miss If the rate is greater than 5%, it indicates severe missed detection, and the density of the obscured area needs to be reduced. The reduced obscuration density ρ occ,new : ; In the formula, ρ occ,old The original occlusion density is represented by ceil(), which rounds up.

[0069] Among them, the occlusion misjudgment error rate R false If the rate is greater than 3%, it indicates a serious misjudgment, requiring optimization of the dehazing and noise reduction intensity. Therefore, the improved noise suppression intensity λ is... denoise,new : .

[0070] Step S4: Obtain real-time environmental images of the rice-fish co-culture environment, input the real-time environmental images into the egret recognition model, and obtain egret recognition results.

[0071] In summary, this invention utilizes UNLOCK, a non-modal driven context learning mechanism, to generate rice-shading egret samples that conform to the characteristics of a rice-fish symbiotic environment; subsequently, this sample is input into U... 2 D 2 The Net model, through its unsupervised processing capabilities, optimizes defogging and denoising under adverse weather conditions such as fog and rain, thereby combining the general occlusion generation capabilities of UNLOCK with U 2 D 2 Net's general unsupervised dehazing and denoising capabilities are applied in conjunction with specific scenarios of rice-fish co-culture, effectively solving the core pain point of scarce egret samples in scenarios with severe weather and rice shading in existing technologies. This fills the technical gap in the application of general generative models in the specific agricultural scenario of rice-fish co-culture, thereby improving the recognition accuracy of egrets in the rice-fish co-culture environment.

[0072] Example 2 Please refer to Figure 2The egret identification system 1 based on the rice-fish symbiotic environment includes a memory 3, a processor 2, and a computer program stored on the memory 3 and run on the processor 2. When the processor 2 executes the computer program, it implements the steps in the above embodiment 1.

[0073] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0074] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0076] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0077] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0078] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0079] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A method for identifying a white egret based on a rice-fish symbiotic environment, characterized in that, The method comprises the following steps: Step S1, taking Egret as a target object, taking rice as an occlusion object, and generating an Egret-occluded-by-rice sample through an UNLOCK model; Step S2, input the rice sheltered egret sample into U 2 D 2 Net model to simulate severe weather, get the rice sheltered-severe weather egret sample; Step S3, inputting the Egret-occluded-by-rice-weather sample as part of the training sample into an UnmaskFormer model for training to obtain an Egret recognition model; Step S4, obtaining a real-time environment image in a rice-fish symbiotic environment, inputting the real-time environment image into the Egret recognition model, and obtaining an Egret recognition result.

2. The method according to claim 1, wherein the rice-fish symbiotic environment is a rice field. The Egret-occluded-by-rice sample in the step S1 is labeled with an Egret key part, and the step S3 further comprises: Different Egret key parts are assigned with different weights, wherein the weight of the beak / eye region is greater than that of the trunk region, and the weight of the trunk region is greater than that of the leg region. 3.The method of claim 1, wherein the method comprises: The input of the UNLOCK model includes the core parameters of rice field environment such as rice plant height and leaf distribution, and the U 2 D 2 The input of the Net model includes the parameters specific to the rice field environment, such as fog concentration and raindrop density.

4. The method according to claim 1, wherein the white egret recognition method based on the rice-fish symbiotic environment is characterized by, The training process of the UnmaskFormer model in the step S3 further comprises: extracting segmentation errors output by the UnmaskFormer model, and feeding the segmentation errors back to the UNLOCK model and U 2 D 2 Net model to optimize the model, and regenerate the rice occlusion- bad weather egret samples to the UnmaskFormer model for training.

5. The method according to claim 4, wherein the white egret recognition method based on the rice-fish symbiotic environment is characterized by, The training process of the UnmaskFormer model in the step S3 specifically comprises: extracting a segmentation result output by the UnmaskFormer model, taking the segmentation result Y pred and an input sample Y gt performing pixel-by-pixel comparison of the white stork missing detection rate R miss and the occlusion misjudgment error rate R false , obtaining a segmentation error, the white stork missing detection rate R miss and the occlusion misjudgment error rate R false is: ； ； ； ； ； ； where N miss is the number of missed pixels, and the equation after it is the input sample Y gt is labeled as egret but the segmentation result Y pred is the pixel region that is not segmented, N total,白鹭 is the total number of egret pixels, N false is the number of misclassified pixels, and the equation after it is the input sample Y gt is labeled as rice but the segmentation result Y pred is the pixel region that is misclassified as egret, N total,水稻 is the total number of rice pixels; feedback the segmentation error to the UNLOCK model and U 2 D 2 Net model, and the UNLOCK model is adjusted in occlusion density optimization, and the U 2 D 2 Net model is adjusted in fog and noise removal intensity optimization.

6. The egret identification method based on a rice-fish symbiotic environment according to any one of claims 1 to 5, characterized in that, The step S1 comprises: Step S11, taking Egret as a target object, and taking rice as an occlusion object, constructing a target object library and an occlusion object library, the target object library being an instance mask M of each pixel on an Egret image in a rice-fish symbiotic environment, the instance mask being 1 for an Egret pixel and 0 for a background pixel, the occlusion object library having rice images classified according to rice growth periods, collecting leaf profile data, and extracting morphological parameters; 白鹭 (x,y), the instance mask being 1 for an Egret pixel and 0 for a background pixel, the occlusion object library having rice images classified according to rice growth periods, collecting leaf profile data, and extracting morphological parameters; Step S12, inputting a panoramic image in a rice-fish symbiotic environment to generate a heron class mask m 白鹭 , using the heron class mask m 白鹭 correcting a prediction P of an instance branch ins , only retaining the heron class mask m in the instance branch 白鹭 falling within a heron area, generating an initial pseudo label, then determining an optimal threshold value through a class adaptive threshold calculation function, screening the initial pseudo label according to the optimal threshold value, and finally optimizing the screened initial pseudo label through an uncertainty guided loss function to obtain a final pseudo label, wherein the class adaptive threshold calculation function and the uncertainty guided loss function are as follows: ； ； where τ fix is a fixed confidence threshold, τ per is a percentage threshold, is the number of class pixels / instances with confidence exceeding τ, L OPLL is the OPLL loss function, N 确定 is the total number of determined region pixels, M 确定 (c, x, y) is the determined region mask, BCE() is the binary cross-entropy loss, P 初始 (c, x, y) is the model initial prediction probability, 0.1 is the uncertain region weight, N 不确定 is the total number of uncertain region pixels, M 不确定 (x, y) is the uncertain region mask; Step S13, different occlusion positions are divided according to different rice growth periods, and an Egret-occluded-by-rice sample is generated by fusion.

7. The method according to any one of claims 1 to 5, wherein the white stork is identified based on a symbiotic environment of rice fish. The step S2 comprises: Step S21, performing normalization processing and local block division preprocessing on pixel values of the Egret-occluded-by-rice sample; Step S22, simulating adverse weather for the preprocessed Egret-occluded-by-rice sample to obtain an initial adverse-weather Egret sample; Step S23, calculating the fog feature similarity S of each local block P in the initial severe weather white egret sample i,j i,j i,j t t opt opt dehaze The function of the above step S23 is as follows:​​​​​​​ ； ； ； wherein, is the global fog feature mean, and σ is the similarity decay coefficient, is the initial transmittance of the local block t i,j is the current transmittance of the local block A est is the U 2 D 2 Net model unsupervised estimated atmospheric light value, =10 -6 ; Step S24, decomposing the dehazed sample I dehaze into a low-frequency component I L and a high-frequency component I H , then constructing a bidirectional loss L denoise to optimize the low-frequency component I L and the high-frequency component I H , and finally reconstructing the optimized low-frequency component I 2 and the high-frequency component I 2 by a decoder of a U L D H Net model to obtain a de-noised sample I dehaze+denoise , wherein the bidirectional loss L denoise is: ； In the formula, W T is a wavelet transform, I denoised is a sample to be denoised, I L,denoised is a low-frequency component, and λ is a weight for balancing texture reservation and noise removal. Step S25, by U 2 D 2 The Net model calls the multi-scale feature fusion module, extracts a plurality of scale feature maps of the denoised sample I dehaze+denoise , and obtains a multi-scale feature fusion map F by weighted summing the plurality of scale feature maps. fusion , and performs up-sampling on the multi-scale feature fusion map F fusion to obtain a final optimized sample. Step S26, reversing the normalization of pixel values of the final optimized sample to the [0, 255] interval and saving it as a preset image format.

8. The method according to any one of claims 1 to 5, wherein the white egret is identified based on a symbiotic environment of rice fish. The recognition process of the UnmaskFormer model comprises: Step S31, different Egret key parts are assigned with different weights, wherein the weight of the beak / eye region is greater than that of the trunk region, and the weight of the trunk region is greater than that of the leg region; Step S32, the cross-attention module and the multi-scale feature fusion module of the UnmaskFormer model are retained; Step S33, input normalization, distortion adaptive feature embedding, and UA module feature enhancement are performed on the input image, and then multi-scale features are output; Step S34, after cross-attention occlusion completion is performed on the multi-scale features, multi-scale feature fusion is performed to obtain a multi-scale fusion feature map, and texture differentiation between rice and Egret is performed according to the multi-scale fusion feature map; Step S35, branch fusion is performed on the prediction results of the semantic branch, the prediction results of the visible instance branch, and the prediction results of the complete instance branch in the UnmaskFormer model to obtain a final semantic segmentation result.

9. The method according to claim 8, wherein the white egret recognition method based on the rice-fish symbiotic environment is characterized by, The semantic branch outputs a probability map of all image pixels belonging to a class through the multi-scale fusion feature map, the visible instance branch generates an Egret instance candidate box with the help of an RPN, and combines the results of the semantic branch to filter out an instance mask containing only the visible region of the Egret; the complete instance branch completes the invisible region of the Egret occluded by the rice on the instance mask containing only the visible region of the Egret through cross-attention reasoning of global features and local features, and outputs an instance mask of the complete form of the Egret; The process of the branch fusion is that: the instance mask output by the complete instance branch is determined as a heron region, all non-heron regions except the heron region use the category corresponding to the semantic branch.

10. A system for identifying a white egret based on a rice-fish symbiotic environment, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the white heron identification method based on the rice-fish symbiotic environment in any of claims 1 to 9 when executing the computer program.

Images