An ai pest situation identification and early warning method and system based on multispectral imaging

Abstract

The application discloses an AI pest situation identification and early warning method and system based on multispectral imaging. The method uses a multispectral imaging unit to obtain multi-band images of pests, and synchronously obtains time, point, weather, and crop growth period information. Through correction, registration, and band reliability evaluation, standardized multispectral images are obtained. Based on a background spectrum dictionary and sparse reconstruction residual, the pest area is extracted and the adhered pests are separated. Combined with the prior key parts, the spectral shape part joint feature is constructed. The collaborative network with a neural controlled differential equation spectral line coding branch and a part hypergraph attention branch is used to complete pest identification. Then, the unknown pest is identified in combination with the prototype memory library, and a pest situation spatio-temporal hypergraph is constructed to realize risk early warning. The method can improve the pest identification accuracy, unknown pest identification ability, and pest situation early warning accuracy in complex scenes.

Description

Technical Field

[0001] This invention relates to the fields of smart agriculture and artificial intelligence technology, specifically to an AI-based insect pest identification and early warning method and system based on multispectral imaging. Background Technology

[0002] Agricultural pests are a significant factor affecting crop yield, quality, and agricultural ecological security. They are also susceptible to infestations by invasive species, necessitating timely analysis, extraction, and identification. Current pest monitoring primarily relies on manual field inspections, manual counting using traps, and automatic identification based on ordinary visible light images. Manual inspections suffer from high labor intensity, long monitoring cycles, strong subjectivity, and difficulty in achieving large-scale continuous monitoring, failing to meet the demands of modern precision agriculture for early detection, identification, and early warning of pests. With the development of image sensors and artificial intelligence technologies, utilizing camera devices for automatic detection and classification of insects on traps has become an important direction for intelligent pest monitoring.

[0003] Existing insect infestation identification methods based on visible light images typically combine ordinary RGB images with convolutional neural networks, object detection networks, or image segmentation networks to detect, classify, and count insects. While these methods can achieve certain results under standard sample conditions, they still face significant limitations in real agricultural scenarios. First, the surface of trapping boards often presents complex conditions such as adhesive reflections, dust contamination, plant debris obscuring the image, incomplete insect bodies, and overlapping insect bodies. The spectral differences between insect bodies and the background in ordinary RGB images are limited, making it difficult to accurately extract insect regions. Second, different insect species, especially closely related species, may have similar colors, textures, and contours under visible light, while the same species exhibits significant differences at different ages, postures, and collection conditions, leading to prominent issues of false detection, missed detection, and misclassification. Third, most existing methods only utilize single-frame images for static identification, lacking the ability to comprehensively model key parts of the insect body, cross-band reflection differences, and the spatiotemporal relationships of insect infestation propagation, making it difficult to balance identification accuracy and early warning accuracy.

[0004] To improve insect species identification capabilities, some solutions have begun to incorporate multispectral imaging technology. By collecting reflectance information across multiple bands, they enhance the distinguishability between insects and the background, as well as between different insect species. However, current multispectral insect monitoring technologies largely remain at the level of directly stitching together multi-band images and inputting them into the recognition network, without fully considering the differences in imaging quality between different bands in the actual environment. For example, some bands may be affected by overexposure, blurring, specular reflection, or low signal-to-noise ratio. If these are directly used in subsequent identification without distinguishing their reliability, the overall recognition stability can be easily reduced. Furthermore, existing methods typically lack structured modeling of key areas such as the insect's head, thorax, abdomen, wing region, and antennae, failing to effectively utilize the reflectance variations and interrelationships of these key areas across different bands, resulting in insufficient utilization of multispectral information.

[0005] On the other hand, the ultimate goal of pest monitoring is not only to identify the types of insects currently on the trapping carriers, but also to provide early warnings of future pest risks by combining temporal changes, regional spread, and environmental conditions. Current technologies for pest early warning often employ simple threshold statistics, traditional time series analysis, or ordinary graphical models, which struggle to simultaneously characterize the long-memory, gradual changes in insect population density and the many-to-many transmission relationships formed between multiple monitoring points due to field connectivity, wind direction, irrigation pathways, and similar crop growth stages. Furthermore, for new insect species not covered in the training samples, abnormal samples, or identification results with low confidence levels, existing methods often lack effective open-set discrimination mechanisms, easily misclassifying unknown insects as known species, thus affecting the reliability of subsequent statistical and early warning results.

[0006] Therefore, there is an urgent need for an AI-based insect infestation identification and early warning method and system that can integrate multispectral imaging, modeling of key parts of the insect body, open set discrimination, and spatiotemporal propagation early warning, in order to improve the accuracy and robustness of insect extraction, insect species identification, and risk warning in complex agricultural scenarios. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention discloses an AI-based insect infestation identification and early warning method and system based on multispectral imaging. This method utilizes a multispectral imaging unit to acquire multi-band images of insects, simultaneously acquiring information on time, location, weather, and crop growth stage. Standardized multispectral images are obtained through correction, registration, and band reliability assessment. Insect regions are extracted based on a background spectral dictionary and sparse reconstruction residuals, and adherent insects are separated. Combined with prior knowledge of key locations, spectral features are constructed, and insect species identification is achieved using a collaborative network containing spectral line coding branches of neurally controlled differential equations and attention branches of location hypergraphs. Finally, a prototype memory is used to identify unknown insects, and a spatiotemporal hypergraph of insect infestations is constructed to achieve risk warning. This method can improve the accuracy of insect identification in complex scenarios, the ability to identify unknown insects, and the accuracy of insect infestation early warning.

[0008] This application provides an AI-based insect infestation identification and early warning method based on multispectral imaging, including the following steps:

[0009] S1. Use a multispectral imaging unit to perform multi-band imaging of the insects on the trapping carrier to obtain a multispectral image cube at the same monitoring time; and simultaneously acquire the collection time, monitoring point information, environmental meteorological information and crop growth period information.

[0010] S2. Perform dark current correction, reflectivity correction and inter-band spatial registration on the multispectral image cube to obtain a standardized multispectral image, and determine the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflection interference and edge fidelity of each band.

[0011] S3. Based on the band reliability coefficient and background spectral dictionary, perform background sparse reconstruction on the multiband spectral vectors of each pixel or local region in the standardized multispectral image. Extract candidate insect body regions and insect body masks based on the reconstruction residual and combined with multi-scale edge response, and perform instance separation on the adhered insect bodies.

[0012] S4. Generate prior maps of key parts of the insect body within the candidate insect body region and construct a joint feature tensor of spectral parts.

[0013] S5. Input the spectral region joint feature tensor into a collaborative network containing a neural controlled differential equation spectral line encoding branch and a region hypergraph attention branch, and output the insect body category, confidence level and insect body feature vector;

[0014] S6. Match the insect body feature vector with the insect species prototype memory database, and perform open set discrimination based on the recognition confidence. When the matching distance exceeds the threshold and the classification entropy is higher than the threshold, or the recognition confidence is lower than the preset threshold, it is determined to be an unknown insect body or a suspected new insect species.

[0015] S7. Aggregate the identified insect categories by insect species and time window, and construct a spatiotemporal hypergraph of insect conditions that integrates information on field connectivity, wind direction, irrigation relationship and growth period. Output the risk level and early warning results using a fusion structure of fractional-gated temporal convolution branch and spatiotemporal hypergraph attention branch.

[0016] This application also provides an AI-based insect infestation identification and early warning system based on multispectral imaging, including:

[0017] The multispectral imaging unit is used to perform multi-band imaging of insects on the trapping carrier to obtain a multispectral image cube at the same monitoring time; the information acquisition unit is used to simultaneously acquire acquisition time, monitoring point information, environmental meteorological information and crop growth period information.

[0018] The preprocessing module is used to perform dark current correction, reflectivity correction and inter-band spatial registration on the multispectral image cube to obtain a standardized multispectral image, and to determine the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflective interference and edge fidelity of each band.

[0019] The insect body extraction module is used to perform background sparse reconstruction of the multi-band spectral vectors of each pixel or local region in the standardized multispectral image based on the band reliability coefficient and background spectral dictionary, extract candidate insect body regions and insect body masks according to the reconstruction residual and combined with multi-scale edge response, and perform instance separation of adhered insect bodies.

[0020] The feature construction module is used to generate prior maps of key parts of the insect body within the candidate insect body region and construct a joint feature tensor of spectral parts.

[0021] The collaborative identification module is used to input the joint feature tensor of the spectral region into a collaborative network containing a spectral line encoding branch of a neural controlled differential equation and a region hypergraph attention branch, and output the insect category, confidence level and insect feature vector;

[0022] The open set discrimination module matches the insect feature vector with the insect species prototype memory database and performs open set discrimination based on the recognition confidence. When the matching distance exceeds the threshold and the classification entropy is higher than the threshold, or the recognition confidence is lower than the preset threshold, it is judged as an unknown insect or a suspected new insect species.

[0023] The insect infestation early warning module aggregates the identified insect categories by insect species and time window, and constructs a spatiotemporal hypergraph of insect infestation that integrates information on field connectivity, wind direction, irrigation relationship and growth period. It outputs risk level and early warning results using a fusion structure of fractional-order gated temporal convolution branch and spatiotemporal hypergraph attention branch.

[0024] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0025] (1) This invention improves the quality of input data from the source by performing dark current correction, reflectivity correction, and inter-band spatial registration on a multispectral image cube, and further determining the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflective interference, and edge fidelity of each band. Simultaneously, by combining a background spectral dictionary and a background sparse reconstruction mechanism, candidate insect regions are extracted based on the reconstruction residual and multi-scale edge response, and instances of adhered insects are separated, thereby effectively suppressing interference from the background color of the trapping board, adhesive reflection, plant debris, dust, and non-insect impurities. Compared to methods that rely solely on visible light images or directly stitch together multi-band images, this invention can more accurately complete insect segmentation and candidate extraction in complex backgrounds, laying a stable foundation for subsequent identification.

[0026] (2) This invention generates a priori maps of key parts within the candidate insect body region, spatially aligns and fuses information from parts such as the head, thorax, abdomen, wing region, and antennae region with multispectral features to construct a joint feature tensor of spectral parts. Furthermore, through a collaborative network containing a neurally controlled differential equation spectral line encoding branch and a part hypergraph attention branch, it jointly models the continuous reflection patterns of key parts along the band and the higher-order structural relationships between different parts. A bidirectional cross-modulation fusion unit is used to achieve synergistic enhancement of spectral line information and part relationship information. Therefore, this invention not only improves the fine-grained discrimination ability between closely related insect species and insect species with similar appearances, but also combines the insect species prototype memory bank, recognition confidence, and classification entropy to achieve open set discrimination, reducing the risk of misclassifying unknown insects as known insect species.

[0027] (3) This invention does not stop at identifying insect types in a single image acquisition, but aggregates the identification results according to insect species and time windows, and integrates field connectivity, prevailing wind direction propagation, irrigation pathways, and crop growth period information to construct a spatiotemporal hypergraph of insect infestation. On this basis, fractional-order gated temporal convolution branches are used to represent the long-memory asymptotic change characteristics of insect population density, and spatiotemporal hypergraph attention branches are used to represent the many-to-many propagation associations between multiple monitoring points. Then, cross-branch fusion units are used to achieve joint modeling of temporal evolution characteristics and spatial propagation characteristics. As a result, this invention can more accurately output the insect infestation risk level and early warning results for a future preset time period, providing a more forward-looking and targeted decision-making basis for agricultural pest control. Attached Figure Description

[0028] Figure 1 This is a flowchart of an AI-based insect infestation identification and early warning method based on multispectral imaging according to the present invention;

[0029] Figure 2 This is a structural diagram of an AI insect infestation identification and early warning system based on multispectral imaging, according to the present invention. Detailed Implementation

[0030] Those skilled in the art will understand that, in order to make the above-mentioned objects, features, and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This application illustrates an AI-based insect infestation identification and early warning method based on multispectral imaging, comprising the following steps:

[0031] S1. Use a multispectral imaging unit to perform multi-band imaging of the insects on the trapping carrier to obtain a multispectral image cube at the same monitoring time; and simultaneously acquire the collection time, monitoring point information, environmental meteorological information and crop growth period information.

[0032] In one embodiment, the AI pest situation recognition and warning method based on multispectral imaging is deployed in a pest situation monitoring terminal in a farmland, orchard, tea garden, vegetable planting base or protected agriculture area. The specific content of step S1 is as follows: First, at least one pest situation monitoring point is set in the area to be monitored. Each monitoring point is provided with a trapping carrier, and the trapping carrier can be a component of a sticky pest board, an insect-catching board supporting an insect-trapping lamp, a sex attractant trapping board, a food attractant trapping device, or other carriers capable of fixing, aggregating or receiving pest bodies. Preferably, the surface of the trapping carrier is a light-colored or standard-color bottom plate to facilitate subsequent background spectral modeling and pest body area extraction.

[0033] A multispectral imaging unit is set above each pest situation monitoring point. The multispectral imaging unit includes a multi-band imaging module, a lens assembly, a light supplement component and a mounting bracket. Among them, the multi-band imaging module is used to collect image information of the same scene in multiple bands. Preferably, the multi-band imaging module covers at least several bands among the blue light, green light, red light, red edge and near-infrared bands; in another embodiment, the ultraviolet band can be further included to enhance the characterization ability of the wing surface texture and body surface reflection difference of specific pests. The multispectral imaging unit is fixedly arranged along the normal direction of the trapping carrier or at a preset angle relative to the normal direction, so that the imaging field of view completely covers the effective insect-catching area of the trapping carrier.

[0034] To ensure the comparability of images in each band under the same time condition, the multispectral imaging unit performs synchronous sampling or quasi-synchronous sampling according to a preset acquisition period. The acquisition period can be set to 5 minutes, 10 minutes, 30 minutes, 1 hour or other preset time intervals according to the crop type, pest activity pattern and regional monitoring requirements. Preferably, the sampling frequency is increased during the high pest occurrence period and decreased during the night or low activity period to balance the monitoring accuracy and equipment energy consumption. Each time of sampling, the multispectral imaging unit outputs an image set corresponding to multiple bands at the same monitoring moment, and stacks them in the order of bands to form a multispectral image cube.

[0035] To improve the sampling quality, a light supplement component is set near the multispectral imaging unit. The light supplement component can include a white light supplement source and a narrow-band light supplement source, which are used to provide stable light in low illuminance, shadow occlusion or twilight environments. Preferably, the light supplement component is briefly turned on before the acquisition trigger and turned off after the imaging ends to reduce power consumption and the influence of continuous light emission on the insect-trapping behavior. Further, a reference reflection block or calibration sheet can be set at the edge of the trapping carrier for subsequent reflectivity correction and inter-band registration.

[0036] Simultaneously, an environmental information acquisition unit is set up at each monitoring point. This unit includes at least one of a temperature sensor, humidity sensor, light sensor, wind speed and direction sensor, and rainfall sensor, used to synchronously acquire environmental meteorological information. Preferably, the environmental meteorological information includes at least temperature, humidity, light intensity, wind speed, and wind direction; in other embodiments, it may also include rainfall status, air pressure, or leaf surface humidity. The environmental information acquisition unit and the multispectral imaging unit are synchronized using a unified clock or a unified timestamp to ensure temporal consistency between image data and environmental data.

[0037] In addition, the monitoring points are equipped with positioning units for acquiring monitoring point location information. This location information may include one or more of the following: latitude and longitude coordinates, plot number, administrative division identifier, field boundary code, and monitoring device number. The location information can be obtained through a satellite positioning module, pre-entered geographic information system data, or device registration information from a farm management platform.

[0038] For crop growth period information, this embodiment can obtain it using any of the following methods: First, the agricultural management platform pre-enters information such as crop type, sowing date, transplanting date, flowering date, heading date, and fruiting date for the plot corresponding to the monitoring point, and then maps the growth period stage according to the current date; Second, the current crop stage, such as seedling stage, tillering stage, jointing stage, heading stage, flowering stage, fruiting stage, and maturity stage, is manually entered; Third, the crop growth period is automatically inferred by combining remote sensing data, ground image data, or historical agricultural database. Preferably, the crop growth period information is stored in the form of structured tags to facilitate subsequent participation in the construction of a spatiotemporal hypermap of insect infestations.

[0039] During a complete sampling process, the system starts with a sampling trigger signal and executes the following steps in sequence: triggering the multispectral imaging unit to acquire multi-band images, recording the sampling timestamp, reading the location information, synchronously reading environmental and meteorological information, and querying or receiving crop growth period information. All of the above data is then packaged into a single data record for the same monitoring time. The multi-band image set is organized into a multispectral image cube, and the sampling time, monitoring location information, environmental and meteorological information, and crop growth period information are stored together as auxiliary attribute information associated with this multispectral image cube.

[0040] Preferably, the data records are numbered, cached, and subjected to preliminary quality checks in the local edge computing unit; when a missing image in a certain band, abnormal environmental data, or inconsistent timestamps are detected, it can be marked as an abnormal sample and resampling can be triggered. After the quality check is completed, the multispectral image cube and its corresponding auxiliary attribute information are sent to the subsequent preprocessing module to perform dark current correction, reflectivity correction, inter-band spatial registration, and band reliability assessment.

[0041] In one embodiment, the monitored pests are migratory insects in rice paddies. A pest monitoring terminal is installed at the edge of each paddy field. This terminal acquires multispectral images every 30 minutes and simultaneously records temperature, humidity, wind speed, wind direction, and the current growth stage of the rice. This method allows for the acquisition of multi-band image information of the insects, as well as environmental and agricultural information closely related to pest outbreaks, at the same monitoring time. This provides a unified data foundation for subsequent insect identification, identification of unknown insects, and pest risk warning.

[0042] S2. Perform dark current correction, reflectivity correction and inter-band spatial registration on the multispectral image cube to obtain a standardized multispectral image, and determine the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflection interference and edge fidelity of each band.

[0043] In some embodiments, step S2 is executed on the edge computing unit or server side, and is used to preprocess and assess the quality of the multispectral image cube acquired in step S1, thereby outputting a standardized multispectral image and band reliability coefficients that can be used by the subsequent insect extraction module. Step S2 specifically includes the following sub-steps:

[0044] S21. Dark Current Correction. The detector output of the multispectral imaging unit at different bands typically contains bias components caused by sensor thermal noise, readout noise, and dark current. Therefore, dark current correction is first performed on the original multispectral images. Preferably, during the equipment factory calibration phase or routine maintenance phase, dark field images under shading conditions are pre-acquired to form dark field reference matrices corresponding to each band. For the currently acquired original image of the b-th band... Using the dark field reference matrix corresponding to this band Correction is performed to obtain the image after dark current correction. In one embodiment, processing can be performed using a pixel-level differential method; in another embodiment, temperature drift compensation can be applied to the dark field reference matrix based on ambient temperature to improve correction accuracy.

[0045] S22. Reflectance Correction. Due to differences in natural lighting conditions, supplementary lighting intensity, lens transmittance, and response gain across different wavelength bands, images directly corrected using dark current cannot accurately reflect the true reflectance characteristics of the insect and background. Therefore, further reflectance correction is necessary. Preferably, a white reference area and a dark reference area are set at the edge of the trapping carrier, or a standard reflectance reference sheet is pre-set in the monitoring terminal. The system synchronously records the response values ​​of the reference areas in each wavelength band during each sampling and, combined with the dark current correction results, maps the image pixel values ​​to the relative reflectance space. After reflectance correction, multi-band images from different sampling times and under different lighting conditions have better comparability, thus facilitating the subsequent construction of a unified background spectral dictionary and insect spectral characteristics.

[0046] In one example, the nominal reflectance of the white reference area is a preset constant, while the reflectance of the dark reference area is approximately zero. The system performs linear normalization on the pixel response based on the observed values ​​of the white and dark reference areas in each current band. For scenarios without a physical reference sheet, approximate reflectance correction can also be performed by calling the band gain parameters and light source compensation parameters from the device calibration database.

[0047] S23. Inter-band Spatial Registration. Since multi-band images may be acquired from different filter channels, time-division acquisition modules, or multi-camera arrays, there may be parallax, scaling differences, rotational shifts, or local distortions between the images of different bands. Therefore, inter-band spatial registration is necessary. Preferably, a two-stage approach of "coarse registration + fine registration" is adopted. In the coarse registration stage, based on the intrinsic parameters, extrinsic parameters, or channel geometric mapping relationships obtained from equipment calibration, initial geometric correction is performed on each band image to project it onto a unified coordinate system. In the fine registration stage, the initial alignment result can be further optimized using the edge calibration points of the decoy carrier, the corner points of the reference reflector block, feature points of stable texture regions, or the mutual information maximization criterion between adjacent bands to obtain the geometric transformation matrix of each band relative to the reference band. Preferably, the green or red band image is used as the reference band, and the remaining bands are resampled to the same spatial resolution and pixel coordinates as the reference band after translation, rotation, scale correction, or local deformation correction. After completing inter-band spatial registration, the corrected images of each band are re-stacked according to a preset band order to form a standardized multispectral image. The standardized multispectral image not only retains the reflectance information of each band, but also ensures that pixels at the same spatial location on different bands have a corresponding relationship, providing a basis for subsequent sparse background reconstruction based on multi-band spectral vectors of pixels or local regions.

[0048] S24. Band Quality Assessment and Band Reliability Coefficient Determination: In this embodiment, the imaging quality of different bands may vary significantly under the current sampling conditions. For example, a certain band may have a low signal-to-noise ratio due to insufficient supplemental lighting, a certain band may have local overexposure due to strong reflection, and a certain band may be blurred due to focus shift. If these quality bands are not differentiated and are equally weighted in subsequent processing, the stability of insect identification will be reduced. Therefore, after obtaining the standardized multispectral image, the system further performs quality assessment on each band and determines the corresponding band reliability coefficient.

[0049] For the b-th band, the system calculates at least the following quality metrics: Signal-to-noise ratio (SNR): The SNR of this band is estimated based on the mean difference between the target and background regions and noise fluctuations, characterizing the relationship between the effective information intensity and noise level of this band. Saturation level: The proportion of pixels in the image that reach or are close to the sensor's upper limit is statistically analyzed to measure the overexposure level of this band. Blur level: The sharpness of this band is evaluated based on the Laplacian response, gradient energy, edge sharpness, or high-frequency components in the frequency domain. Reflection interference level: The interference level of adhesive layer reflection, dew reflection, etc., is evaluated by the proportion of bright spot area, the proportion of specular reflection candidate regions, or the distribution of local extrema. Edge fidelity level: The ability to preserve the insect boundary is evaluated by comparing the consistency, contour continuity, or boundary alignment of the edge map of this band with the edge map of the reference band.

[0050] In a preferred embodiment, the system first normalizes the above-mentioned quality indicators, then performs weighted combination according to preset weights to obtain the original reliability score of the b-th band; subsequently, it normalizes the original reliability scores of all bands to obtain the final band reliability coefficient for each band. The preset weights can be obtained through historical sample statistics or determined through offline training. Preferably, when the saturation level or reflective interference level of a certain band exceeds a threshold, the reliability coefficient of the corresponding band is significantly reduced; when the signal-to-noise ratio of a certain band is high, the edge fidelity is good, and the blur level is low, the reliability coefficient of the corresponding band is increased. S25. Output standardized results. After completing the above processing, the system outputs two types of results: first, a standardized multispectral image obtained after dark current correction, reflectivity correction, and inter-band spatial registration; second, a band reliability coefficient vector corresponding to each band. The standardized multispectral image and band reliability coefficient vector are transmitted together to the subsequent insect extraction module so that high-reliability bands are given higher weights and the influence of low-reliability bands are suppressed during the background sparse reconstruction and insect region extraction process.

[0051] Taking a pest monitoring terminal deployed at the edge of a rice paddy as an example, the multispectral imaging unit acquires images in five bands: blue, green, red, red-edge, and near-infrared. The system first calls a pre-stored dark-field reference matrix to perform dark current correction, then uses the white reference area at the edge of the trapping board to complete reflectivity correction. Subsequently, using the green band image as the reference band, it performs inter-band spatial registration on the remaining band images using a combination of reference block corner registration and mutual information optimization. After registration, the system calculates the signal-to-noise ratio, saturated pixel ratio, blurriness, reflective interference, and edge fidelity for each of the five bands, forming a five-dimensional band reliability coefficient vector. For example, the near-infrared band, which is significantly affected by dew reflection, has a higher reflective interference level, resulting in a lower reliability coefficient; while the red-edge band, with its clear edges and high signal-to-noise ratio, has a relatively higher reliability coefficient. Finally, the system sends the standardized multispectral images and band reliability coefficients to step S3 for background sparse reconstruction and insect extraction.

[0052] S3. Based on the band reliability coefficient and background spectral dictionary, perform background sparse reconstruction on the multiband spectral vectors of each pixel or local region in the standardized multispectral image. Extract candidate insect body regions and insect body masks based on the reconstruction residual and combined with multi-scale edge response, and perform instance separation on the adhered insect bodies.

[0053] In some embodiments, step S3 is executed on the edge computing unit or server side, and is used to extract insect candidate regions from the standardized multispectral image output in step S2, and separate mutually adhered or overlapping insects into multiple independent instances. Step S3 specifically includes the following sub-steps: S31, Constructing a background spectral dictionary. Since the surface of the trapping carrier may contain, in addition to insects, the trapping board background color, adhesive layer, plant debris, dust, mud spots, water stains, and other non-insect impurities, a background spectral dictionary is pre-constructed in this embodiment. The background spectral dictionary can be constructed in one of the following ways: First, in the initial stage of equipment deployment, multispectral images of the trapping carriers in a state with no or few insects are collected. The background color area of ​​the trapping board, the adhesive layer area, the plant debris area, the dust area, and other impurity areas are selected from these images. Multi-band spectral vectors are extracted accordingly to form a background sample library, and the background spectral dictionary is obtained through clustering, dictionary learning, or sparse coding training. Second, during long-term operation, the system continuously updates the background sample library based on manually labeled samples or high-confidence background areas, dynamically correcting the background spectral dictionary to adapt to background changes in different regions, seasons, and aging states of different trapping carriers. Third, in a simplified embodiment, the background spectral dictionary can also be composed of multiple typical background prototype vectors, each representing the typical spectral response of the trapping board background color, the high-reflectivity area of ​​the adhesive layer, debris, dust, and non-insect residues. Preferably, each dictionary atom in the background spectral dictionary is a vector with the same number of bands as the standardized multispectral image, and corresponds one-to-one with the band reliability coefficient obtained in step S2.

[0054] S32. Construct the multi-band spectral vector to be reconstructed. For each pixel in the standardized multispectral image, or each local region composed of several adjacent pixels, extract its response value in each band to form the corresponding multi-band spectral vector. In a preferred embodiment, to improve robustness to local noise and point reflections, a local region-level spectral vector is used instead of a single-pixel spectral vector. The local region can be a 3×3, 5×5, or other preset-scale neighborhood block centered on the target pixel, and the pixels within the neighborhood block can be weighted by Gaussian weights to form a representative multi-band spectral vector for that region. Further, when constructing the spectral vector, the band reliability coefficient obtained in step S2 is introduced into the weighting process. For bands with high reliability, their spectral responses retain higher weights; for bands with low reliability, their spectral responses are weakened in subsequent reconstruction, thereby avoiding interference from low-quality bands in the extraction of candidate insects.

[0055] S33. Background Sparse Reconstruction and Reconstruction Residual Calculation. In this embodiment, it is assumed that the multi-band spectral vector of the background region can be sparsely represented by a small number of atoms in the background spectral dictionary. However, the insect body region, due to its significantly different surface material, wing texture, and multi-band reflectance characteristics compared to the background, is generally difficult to accurately reconstruct from the background spectral dictionary. Therefore, background sparse reconstruction is performed on the multi-band spectral vector of each pixel or local region to be detected.

[0056] Specifically, the sparse representation coefficients of the spectral vector to be detected are solved using a background spectral dictionary, allowing it to be approximated by a linear combination of as few background dictionary atoms as possible. In a preferred embodiment, a weighted reconstruction error is constructed by adding a band reliability coefficient, making the contribution of high-reliability bands to the reconstruction error greater, while the error contribution of low-reliability bands is relatively weakened. After sparse reconstruction is completed, the residual between the original spectral vector and the reconstructed spectral vector is calculated to obtain the reconstruction residual value of the pixel or local region. If a region belongs to background components such as the trapping board background color, adhesive layer, debris, or dust, its spectrum can usually be reconstructed well by the background spectral dictionary, resulting in a smaller reconstruction residual; if a region belongs to an insect, its spectrum differs greatly from the background dictionary, making accurate reconstruction difficult, resulting in a larger reconstruction residual. Therefore, the system can perform preliminary screening of suspected insect regions based on the reconstruction residual. In one embodiment, the system maps the reconstruction residuals of all pixels or local regions to a residual response map and obtains initial insect candidate regions through threshold segmentation. The threshold can be a global threshold or adaptively determined based on the local residual distribution.

[0057] S34. Multi-scale edge response enhancement and insect body mask generation. While relying solely on reconstruction residuals can effectively distinguish between the background and the insect body, issues such as missing edges or incomplete masks may still occur when the insect body boundaries are blurred, overlapped, or locally reflective. Therefore, this embodiment further incorporates multi-scale edge response to enhance the initial insect body candidate region. Specifically, multi-scale edge detection is performed on one or more high-discrimination band images in a standardized multispectral image to obtain edge response maps at different scales. Preferably, gradient operators, LoG operators, structural tensors, or guided filtering-based edge extraction methods can be used to capture the overall outline edge of the insect body, the detailed edges of local wing margins, and the edges of small antennae, respectively. Then, the multi-scale edge response map is fused with the reconstruction residual response map, so that regions with higher residuals and obvious edge responses obtain higher insect body confidence values. In a preferred embodiment, morphological closing operations, hole filling, and small connected component removal are first performed on the fused response map. Then, false targets that do not conform to the morphology of the worm are screened out according to rules such as connected component area, aspect ratio, boundary continuity, and region compactness, and finally, a worm mask is generated.

[0058] S35. Separation of Adhesive Insect Instances. On the trapping carrier, multiple insects may form a single connected region due to adhesive adhesion, overlapping postures, or body contact. To achieve accurate counting and subsequent fine-grained identification, this embodiment separates the adhered regions in the insect mask. First, the connected regions in the candidate insect mask are analyzed to identify suspicious adhered regions with significantly larger areas, complex shapes, multiple indentations at the boundaries, or multiple branches in the skeleton. Then, a candidate connectivity graph is constructed for each adhered region. The nodes of the candidate connectivity graph represent local insect core areas or key points of the skeleton, and the edges represent the connection relationships between these regions that may belong to the same insect or different insects. When constructing candidate connectivity graphs, at least the following information is considered: Contour indentation information: Contact points between adherent worms typically form concave boundaries on the contour; extracting these indentations can reveal potential segmentation locations. Skeleton branching point information: Refining connected components yields a skeleton graph; the presence of multiple branch endpoints or nodes indicates the potential inclusion of multiple worm instances within the region. Inter-band discontinuity boundary information: Different worms may exhibit abrupt changes in reflectivity or texture direction at certain wavelengths; analyzing multi-band boundary consistency and inter-band discontinuities helps determine segmentation boundaries. Local morphological feature information: Such as variations in region width, wing margin orientation, and segment center distribution, used to further constrain the rationality of segmentation. Subsequently, graph segmentation, minimum cut, path truncation, or energy optimization separation processes are performed on the candidate connectivity graphs to split a single connected component into multiple instance masks. Preferably, the segmentation results can be post-processed using area constraints, morphological rationality constraints, and boundary smoothing constraints to avoid over-segmentation or under-segmentation.

[0059] S36. Output Results. After completing the above processing, step S3 outputs the following results: First, candidate insect body regions, used to indicate the location range of suspected insect bodies in the standardized multispectral image; second, insect body mask, used to accurately characterize the pixel region corresponding to each insect body instance; third, instance separation results, used to extract the joint features of key parts and spectral parts of each insect body instance in subsequent steps. Taking the five-band multispectral image collected by the monitoring terminal as an example, the system first calls the background spectral dictionary to perform weighted sparse reconstruction of the five-dimensional spectral vector of each 5×5 local region. Among them, the red edge and green light bands are given larger weights due to their higher reliability, while the near-infrared band is given lower weights due to its strong reflective interference. Subsequently, a residual response map is generated based on the reconstruction residual, and multiple initial insect body candidate regions are extracted by combining the multi-scale edge responses on the green light band and the red edge band. For one of the connected regions with a significantly larger area, the system detected two prominent contour depressions and one skeleton branch point. Furthermore, it identified inter-band discontinuities at the local boundary under the red-edge band. Therefore, a candidate connectivity graph was constructed, and graph segmentation was performed to separate the connected region into two independent worm instances. The final output worm mask and instance separation results were passed to step S4 to generate a priori maps of key regions and construct a joint feature tensor for spectral regions.

[0060] S4. Generate prior images of key parts of the insect body within the candidate insect body region and construct a joint feature tensor of spectral parts. In this embodiment, step S4 is executed on the edge computing unit or server side, used to locate key parts of each insect body instance mask output in step S3, and to fuse multispectral information, morphological structure information and prior information of parts to form a joint feature tensor of spectral parts. Step S4 specifically includes the following sub-steps: S41. Candidate insect body region cropping and normalization. For each insect body instance obtained in step S3, according to its instance mask or bounding rectangle, the corresponding insect body region image block is cropped from the standardized multispectral image. Preferably, a preset edge extension pixel is added around the bounding rectangle of the insect body to retain the wing edges, antennae or local shadow information near the insect body boundary, and to avoid the loss of part information due to overly tight cropping. In a preferred embodiment, the cropped worm region image patch is scale-normalized to ensure that worms of different sizes have a uniform size when input into the subsequent network. Simultaneously, the image patch can be rotated and corrected according to the worm's principal axis direction to make the worm's longitudinal posture more consistent, reducing the impact of different orientations on localization. Furthermore, the instance mask is simultaneously cropped and normalized to form a mask map corresponding to the worm region image patch space.

[0061] S42. Generation of Prior Maps for Key Parts. In this embodiment, the key parts of the insect body include at least the head, thorax, abdomen, wing region, and antennal region. Prior maps for key parts can be generated in any of the following ways: First, a lightweight segmentation subnetwork is used to directly segment the insect body region image patch, outputting a probability map corresponding to each key part. The lightweight segmentation subnetwork can be an encoder-decoder structure, a shallow feature pyramid structure, or other small semantic segmentation structures suitable for edge deployment. Its input can be a multispectral image patch, a pseudo-color fused image patch, or a combination of multispectral and mask inputs. Each channel of the network output represents the positional confidence distribution of the head, thorax, abdomen, wing region, and antennal region, respectively. Second, in scenarios with limited training samples, a hybrid approach of "morphological rule guidance + lightweight network correction" is used to generate prior maps for key parts. Specifically, based on the principal axis direction, aspect ratio, skeletal centerline, and boundary curvature of the insect instance, the insect is longitudinally divided into multiple candidate region regions. The pointed anterior region is preferentially marked as the head candidate region, the wider central region as the thorax candidate region, the continuously extending posterior region as the abdomen candidate region, the laterally spreading regions as the wing region candidate region, and the slender, protruding region near the head as the antennae candidate region. Then, a lightweight correction network refines the coarse priors, outputting more accurate key region prior maps. Thirdly, in scenarios with high real-time requirements, a template-aligned approach can be used to directly generate coarse key region prior maps. This involves matching the insect instance mask with a pre-stored typical insect structure template, and mapping the head, thorax, abdomen, wing region, and antennae region from the template onto the current insect instance through affine or non-rigid transformations, thus obtaining the prior regions for each key region. This method can serve as a simplified alternative when the segmentation sub-network is unavailable. Preferably, this embodiment uses a combination of lightweight segmentation subnetworks and morphological rules to generate prior maps of key parts, balancing accuracy and interpretability. For each key part, the system outputs a part location confidence map with the same spatial resolution as the insect body region image block, and can perform thresholding, boundary smoothing, and hole filling processing on it to form a corresponding part mask.

[0062] S43. Prior Optimization and Consistency Constraints for Key Parts. Due to variations in insect posture, wing occlusion, local defects, and adhesion / separation errors, the initially generated prior images of key parts may have inaccurate boundaries, overlapping parts, or partial missing parts. Therefore, this embodiment further optimizes the prior images of key parts. In a preferred embodiment, the system applies the following consistency constraints to the prior images of each key part: Spatial mutual exclusion constraint: The sum of the probability of the same pixel belonging to the head, thorax, abdomen, wing area, and antennae area is limited to a preset range to reduce large-area overlap between different parts; Topological order constraint: The head, thorax, and abdomen satisfy a preset arrangement order along the main axis of the insect body, with the antennae area preferentially adjacent to the head, and the wing area preferentially adjacent to the thorax; Area rationality constraint: The area ratio of each key part falls within a reasonable range for the corresponding insect species or type to avoid abnormal expansion or shrinkage of a certain part; Boundary continuity constraint: The outline of the key part should maintain a certain continuity with the overall boundary and skeletal orientation of the insect body. Through the above constraints, the prior images of key parts can be post-processed and corrected to better conform to the actual structure of the insect body.

[0063] S44. Constructing Joint Features of Spectral Regions. In this embodiment, the "joint features of spectral regions" includes three parts: multispectral features, morphological and structural features, and prior features of key regions. Multispectral feature extraction: For each pixel or local region in the insect body image patch, its response value in each band is extracted to form basic spectral features. Preferably, the difference between adjacent bands, band ratios, local spectral line slopes, or inter-band gradients can be further calculated to enhance the characterization of differences in insect body material, pigment distribution, and wing texture. Morphological and structural feature extraction: For the insect body image patch and corresponding instance mask, morphological and structural features such as contour edges, skeletal center lines, local orientation fields, region areas, aspect ratios, boundary curvature, and texture statistics are extracted. Preferably, edge responses and orientation features at different scales are jointly encoded to simultaneously preserve the overall contour shape and local detailed structure. Prior feature extraction: Prior images or region masks of the head, thorax, abdomen, wing region, and antennae region are used as additional channel inputs, or used to weight the multispectral features of the corresponding spatial regions. Specifically, higher attention weight is given to the spectral features of a key region, while the spectral features of non-key regions are suppressed, thereby strengthening the feature expression guided by the region.

[0064] S45. Spatial Alignment and Correlation Fusion. To express multispectral features, morphological features, and site-specific prior features in a unified spatial coordinate system, this embodiment performs spatial alignment and correlation fusion on these three types of features. In a preferred embodiment, the standardized insect body region image patch coordinate system is used as a unified reference, and the feature maps of each band, edge structure map, skeleton map, and key site-specific prior maps are aligned at the pixel level. Subsequently, channel stitching, element-wise weighting, gated fusion, or local attention weighting are used to fuse various features into a unified tensor. For example, the original multi-band image patch can be used as the first set of channels, the morphological feature map composed of edges, skeleton, and orientation field can be used as the second set of channels, and the prior maps of the head, thorax, abdomen, wing region, and antennal region can be used as the third set of channels. These are then stitched together according to the channel dimension to form a high-dimensional feature stack; or, the three types of features can be mapped to the same dimension through shallow convolution, and then spatial weighting is performed on the multispectral features based on the key site-specific prior maps to form a site-sensitive multispectral response.

[0065] S46. Form a joint feature tensor for spectral regions. After spatial alignment and correlation fusion, the system outputs a joint feature tensor for spectral regions. This joint feature tensor can be represented as a multidimensional feature representation with spatial, channel, and band dimensions, including at least: normalized and aligned multi-band spectral responses; morphological features such as contours, skeletons, and orientation fields; prior maps or region masks of key regions such as the head, thorax, abdomen, wing region, and antennal region; and auxiliary spectral difference features composed of band differences and local spectral gradients. This joint feature tensor for spectral regions serves as the input to the collaborative network in step S5, used for subsequent insect category identification, confidence estimation, and insect feature vector generation.

[0066] Taking a specific insect specimen obtained through step S3 as an example, the system first crops the corresponding insect region from the five-band standardized multispectral image and scales it to a fixed size. Then, a lightweight segmentation sub-network is used to output probability maps of five key regions: head, thorax, abdomen, wing region, and antennal region. The boundaries of these regions are then corrected by combining the insect's principal axis direction and the skeletal centerline. Next, five-band reflectance features, adjacent band difference features, contour edge maps, skeletal orientation fields, and prior maps of the five key regions are extracted from the insect region. These are then stitched together and weighted in a unified coordinate system to form a joint feature tensor of the spectral regions. This joint feature tensor preserves both the reflectance differences of the insect in different bands and the structural relationships of the head, thorax, abdomen, wing region, and antennal region, and can be directly fed into the collaborative network in step S5 for subsequent identification.

[0067] S5. Input the spectral region joint feature tensor into a collaborative network containing a neural controlled differential equation spectral line encoding branch and a region hypergraph attention branch, and output the insect body category, confidence level and insect body feature vector;

[0068] In this embodiment, step S5 is executed on the edge computing unit or server side, and is used to perform insect category identification, identification confidence estimation, and insect feature vector extraction on the spectral part joint feature tensor output by step S4. Step S5 is implemented using a collaborative network, which includes: a spectral line encoding branch, a part hypergraph attention branch, a bidirectional cross-modulation fusion unit, and a hierarchical fusion decoder. The basic idea is to dynamically model the continuous reflectance changes of key parts of the insect along the band dimension on the one hand, and to perform high-order modeling of the spatial structural relationships and collaborative discrimination relationships between the head, thorax, abdomen, wing region, and antennal region on the other hand, and then achieve the collaborative enhancement of spectral line information and part structural information through bidirectional modulation and fusion decoding.

[0069] S51. Input and Precoding of the Joint Feature Tensor of Spectral Parts: For the joint feature tensor of spectral parts obtained in step S4 for each insect instance, it is first input into the precoding layer. The precoding layer may consist of one or more lightweight convolutional layers, normalization layers, and nonlinear mapping layers, used to map feature channels from different sources to a unified dimension. Preferably, the multi-band spectral features, morphological features, and key part prior features in the joint feature tensor are each processed through independent shallow convolutional mappings, and then concatenated in the channel dimension to obtain an initial fused feature map. In a preferred embodiment, the feature map output by the precoding layer maintains the spatial layout of the insect region and retains the band dimension information so that it can be called by the subsequent spectral line encoding branch and the part hypergraph attention branch.

[0070] S52. Spectral Line Encoding Branch: The spectral line encoding branch is used to dynamically model the continuous reflectance changes of key parts of the insect body across multiple wavelengths. Its core is to use neurally controlled differential equations to fit continuous trajectories and perform state evolution on discrete wavelength sequences. Partial Sub-Tensor Partitioning: Based on the prior maps of the head, thorax, abdomen, wing region, and antennal region generated in step S4, the pre-encoded initial fused feature map is divided into regions. Specifically, mask clipping or weighted pooling is performed on the region corresponding to each key part to obtain the partial sub-tensor corresponding to that key part. Each partial sub-tensor contains the local reflectance features, edge structure features, and prior information of that part in each wavelength band.

[0071] The spectral response sequence is constructed by organizing the sub-tensors of each region along the band dimension to form a spectral response sequence arranged in wavelength order. In one embodiment, global average pooling or local statistical pooling is performed on the spatial region of each key region to obtain a region-level feature vector corresponding to each band, which is then stacked in band order to form the spectral response sequence of that region. In another embodiment, local grid-level spatial information can also be retained to form a multi-channel spectral sequence composed of multiple local locations.

[0072] The neural controlled differential equation (NDE) dynamic modeling method uses the spectral response sequence of each key region as a control signal input to the NDE module. This module learns the continuous evolution trajectory of the reflectance characteristics of the key region at different wavelengths by establishing a dynamic equation for the continuous evolution of the hidden state across wavelengths. Compared to simple one-dimensional convolutional or discrete sequence recurrent networks, NDEs can more naturally express the continuous dependencies between wavelengths and are more suitable for modeling phenomena such as local peaks, absorption valleys, gradually varying slopes, and transband transitions in multispectral data. Specifically, the region's spectral response sequence is used as the control path, and the hidden state is used as the state variable to be updated. The state equation is solved by integration within the wavelength range to obtain the spectral dynamic representation vector of the corresponding key region. The spectral dynamic representations of each key region are further concatenated or weighted to form a spectral line encoding branch output.

[0073] In order to highlight the inter-band variation information with higher discriminative power, the spectral line encoding branch preferably also includes a spectral difference enhancement unit. The spectral difference enhancement unit is used to calculate the difference between adjacent bands, local spectral line gradients, band ratios, or peak-valley variation responses, and uses them as auxiliary inputs to jointly encode the hidden states of the neurally controlled differential equation, thereby enhancing the characterization ability of differences in wing texture, body surface pigment absorption, and material differences in different parts of the insect.

[0074] S53. Location Hypergraph Attention Branch: The location hypergraph attention branch is used to model the topological relationships, biological structural relationships, and cooperative discriminative relationships between key parts of the insect body, to overcome the problem that relying solely on spectral sequences is insufficient to characterize spatial structure. Hypergraph nodes are constructed using the head, thorax, abdomen, wing region, and antennal region as hypergraph nodes. The initial features corresponding to each node are composed of morphological structural features, multi-band statistical features, edge orientation field features, and corresponding prior features within the key location region. In a preferred embodiment, the initial embedding representation of each node is obtained by performing region pooling and shallow convolution mapping on the sub-tensors of each location. Unlike traditional graph structures that only model pairwise relationships, this embodiment uses a hypergraph structure to express higher-order relationships in which multiple locations jointly participate in insect species discrimination. Preferably, the hyperedge is constructed based on the following relationships: spatial adjacency relationship: for example, the head is adjacent to the thorax, and the thorax is adjacent to the abdomen; biological structural relationship: for example, the antennal region is associated with the head, and the wing region is associated with the thorax; collaborative discrimination relationship: for example, some insect species need to be judged by comprehensively considering the combined proportional relationship of the thorax-wing region-abdomen, or by comprehensively considering the morphological combination characteristics of the head-antennae region-thorax.

[0075] In one implementation, several fixed hyperedge templates can be pre-defined; in another implementation, hyperedges can be adaptively generated or filtered based on the correlation between different parts according to training data. Hypergraph attention propagation involves propagating and updating node features using a hypergraph attention mechanism after the hypergraph is constructed. Specifically, the contribution of each node to its respective hyperedge is first calculated, and then the feedback weight of each hyperedge to each node is calculated, thereby achieving information interaction between nodes, hyperedges, and nodes. Through attention weighting, the importance of different key parts in insect species discrimination is adaptively learned. For example, when the wing texture and thorax proportion of a certain insect species are more discriminative, nodes and hyperedges related to the wing and thorax will receive higher attention weights. After one or more rounds of propagation, a representation of the structural relationships between parts is obtained.

[0076] S54. Bidirectional cross-modulation fusion unit: To avoid insufficient information utilization caused by the independent and simple splicing of spectral coding branches and part hypergraph attention branches, this embodiment sets up a bidirectional cross-modulation fusion unit to realize mutual modulation and synergistic enhancement between spectral dynamic information and part structural information.

[0077] Spectral-guided structural modulation maps the dynamic spectral representations of key components output from the spectral line coding branch to modulation weights, which are used to recalibrate the node representations and hyperedge weights in the part hypergraph attention branch. For key components exhibiting stronger discriminative power in specific bands, the propagation intensity of their corresponding nodes in the hypergraph is enhanced; for components with insignificant spectral changes and low discriminative contribution, their node responses are appropriately suppressed. Structure-guided spectral modulation maps the part structural relationship representations output from the part hypergraph attention branch to spatially gating weights, which are used to inversely modulate the band responses of key components in the spectral line coding branch. For structurally stable components that are more critical in topological relationships, their corresponding band responses receive higher weights; for components that may be unreliable due to local occlusion or incompleteness, their spectral responses are downweighted. Cross-attention fusion, after completing bidirectional modulation, inputs the dynamic spectral representations and part structural relationship representations into the cross-attention module. The cross-attention module uses features from one branch as queries and features from the other branch as keys and values ​​to learn the correspondence between the two types of features, thereby outputting a unified spatiotemporal-spectral fusion representation. This fusion representation includes information on "how key components change in different bands" and "how different key components together constitute the structural pattern of a certain insect species".

[0078] S55, Layered Fusion Decoder, the fused features are input to the layered fusion decoder. The layered fusion decoder may include a global pooling layer, a location-level pooling layer, a nonlinear mapping layer and a multi-task output head. (1) Global pooling layer, global average pooling, global max pooling or attention-weighted pooling are performed on the fused features to obtain the overall discrimination representation of the insect body. This representation reflects the comprehensive features of the insect body instance at the spectral location level. (2) Location-level pooling layer, in addition to overall pooling, the local fused features corresponding to the head, thorax, abdomen, wing area and antennae area can be pooled separately, and weighted and combined according to the preset or learned location weights, thereby enhancing the expression of differences in key locations. (3) Nonlinear mapping layer, in a preferred embodiment, the layered fusion decoder uses the spectral difference enhanced Mish gated activation function to perform nonlinear mapping on the pooled features. The gated activation function introduces a dynamic spectral difference gate coefficient, which is determined at least based on the band reliability coefficient and modulated in combination with the local inter-band gradient and the key location reliability, thereby enhancing the effective response of the high-reliability band in the key region and suppressing the ineffective activation of the low-reliability band or non-key region.

[0079] (4) Multi-task output head, the hierarchical fusion decoder includes at least three output heads: category output head: outputs the probability distribution of insect category, used to determine the insect species category to which the current insect belongs; confidence output head: outputs the recognition confidence, used to reflect the reliability of the current classification result; embedding output head: outputs the insect feature vector, used for insect species prototype matching and open set discrimination in subsequent step S6.

[0080] In a preferred embodiment, the category output head is implemented using a fully connected layer plus Softmax, the confidence output head can be implemented using a Sigmoid or temperature scaling calibration layer, and the embedding output head can be implemented using a fully connected mapping layer combined with normalization processing to obtain a fixed-dimensional feature vector.

[0081] In this embodiment, the working principle of the cooperative network can be summarized as follows: First, the "spectral line variation pattern" and "partial structural relationship" are extracted from the joint feature tensor of the spectral shape parts. Then, the two are mutually enhanced through bidirectional cross-modulation. Finally, the classification and representation output are completed through a hierarchical fusion decoder. Specifically, the spectral line encoding branch focuses on the continuous reflectance variation trajectory of key parts of the same insect body under different bands. For example, the wing regions of some insect species have more obvious reflectance differences in the red-edge and near-infrared bands, and the head and abdomen of some insect species have distinguishable features in the spectral line gradients of adjacent bands. The part hypergraph attention branch focuses on how the proportional relationships, adjacency relationships, and combination relationships between the head, thorax, abdomen, wing regions, and antennae regions together constitute the morphological pattern of a certain insect species. For example, although closely related insect species have similar overall colors, their structural coupling patterns between the wing regions and thorax, and the spatial relationship between the head and antennae regions may have stable differences.

[0082] Through bidirectional cross-modulation fusion units, spectral variation information is no longer used in isolation but is used to guide the modeling of part structures; part structure information is no longer limited to geometric relationships but, in turn, strengthens the high-discrimination band response corresponding to key parts. Therefore, the final fused features possess both strong spectral resolution and morphological resolution, which is beneficial for improving the recognition accuracy of closely related insect species, insect species with similar appearances, and insects with complex postures, and provides a more stable insect feature vector for subsequent open set discrimination.

[0083] Taking a joint feature tensor of spectral regions constructed in step S4 as an example, the system first divides the joint feature tensor into five region sub-tensors based on prior maps of the head, thorax, abdomen, wing region, and antennae region, forming five sets of multi-band spectral response sequences for each region. Subsequently, the neural controlled differential equation module continuously and dynamically models the spectral response sequences of each region, obtaining the spectral dynamic representations of the five regions. Simultaneously, the system uses the five key regions as hypergraph nodes and constructs hyperedges based on structural relationships such as head-antennae region, thorax-wing region, and thorax-abdomen, obtaining the region structural relationship representations through hypergraph attention propagation. Then, the bidirectional cross-modulation fusion unit enhances the propagation weights of highly discriminative regions in the hypergraph using the spectral dynamic representations and performs gated modulation on the responses of each band using the region structural relationship representations, obtaining unified fused features. Finally, the hierarchical fusion decoder outputs the insect species category, recognition confidence, and fixed-dimensional insect feature vector for the insect instance, and passes the insect feature vector to step S6 for prototype matching and unknown insect discrimination.

[0084] S6. Match the insect body feature vector with the insect species prototype memory database, and perform open set discrimination based on the recognition confidence. When the matching distance exceeds the threshold and the classification entropy is higher than the threshold, or the recognition confidence is lower than the preset threshold, it is determined to be an unknown insect body or a suspected new insect species.

[0085] In some embodiments, step S6 is executed on the edge computing unit or server side, and is used to further discriminate the insect feature vector, insect category result and identification confidence score output in step S5, so as to identify unknown insects, abnormal insects or suspected new insect species not covered in the training samples. Step S6 adopts the method of "insect species prototype memory matching + multi-index joint discrimination" to realize open set discrimination, and specifically includes the following sub-steps.

[0086] S61. Construction of Insect Species Prototype Memory Database: In this embodiment, an insect species prototype memory database is pre-constructed. The insect species prototype memory database is used to store representative prototype vectors of each known insect species in the feature space. Preferably, the insect species prototype memory database can be constructed using labeled samples during the offline training phase. Specifically, multiple insect samples belonging to the same insect species in the training set are input into the collaborative network in step S5, corresponding insect feature vectors are extracted, and then similar feature vectors are aggregated to obtain the category prototype vector of the insect species. In a preferred embodiment, a global prototype vector is set for each insect species to represent the central position of the insect species in the overall feature space; in another embodiment, multiple sub-prototype vectors can be further constructed for the same insect species to correspond to intra-class distribution changes under different insect age stages, male-female differences, typical postures, or different regional collection conditions, thereby enhancing the adaptability of the prototype memory database to intra-class differences. Preferably, the insect species prototype memory database adopts an updatable storage structure, which, in addition to storing the prototype vectors of each insect species, also stores the category identifier, sample quantity, prototype update time, and prototype stability parameters corresponding to the prototype. For samples that are subsequently confirmed as new insect species or subtypes through manual verification, they can be incrementally written into the insect species prototype memory bank when the update conditions are met, so as to achieve continuous expansion of the prototype bank.

[0087] S62. Insect Feature Vector Matching: For the current insect instance feature vector output in step S5, the system matches it with the prototype vectors of each insect species in the insect species prototype memory. During matching, Euclidean distance, cosine distance, Mahalanobis distance, or other metrics representing vector similarity can be used. Preferably, the distance is calculated between the normalized feature vector and the prototype vector to reduce the impact of feature amplitude fluctuations. Specifically, the system calculates the matching distance between the current insect feature vector and the prototype vectors of each insect species, and selects the insect species corresponding to the prototype with the smallest distance as the most similar insect species category. Simultaneously, the minimum distance, the second smallest distance, and the distance distribution of the top few candidate categories can be obtained for subsequent discrimination. In one embodiment, if a multi-sub-prototype structure is used, the current insect feature vector needs to first calculate the distance with multiple sub-prototypes under the same insect species, and then take the minimum distance as the matching distance for that insect species. This can reduce the impact of intra-class offsets caused by differences in posture, insect age, or collection conditions on the matching results.

[0088] S63. Classification Entropy Calculation: Since it is sometimes difficult to distinguish between "unknown insects that are dissimilar to all known insect species" and "fuzzy samples near the boundaries of two similar categories" based solely on the minimum matching distance, this embodiment further introduces classification entropy as a discrimination index. Specifically, the classification entropy of the insect category probability distribution generated by the category output head in step S5 is calculated. If the category probability distribution of a certain insect instance is too scattered, the classification entropy is high, indicating that the model is uncertain about the category of the sample; if the category probability is highly concentrated in a certain category, the classification entropy is low, indicating that the model's discrimination is relatively clear.

[0089] In a preferred embodiment, the system considers the classification entropy and the prototype matching distance of the insect feature vector together: when the distance between the sample and the nearest prototype is large and the category probability distribution is relatively dispersed, it can be determined with a high probability that the sample does not belong to a known insect species.

[0090] S64. Recognition Confidence Acquisition: In this embodiment, the recognition confidence is generated by the confidence output header from step S5, reflecting the reliability of the current insect instance recognition result. The recognition confidence can be represented as a continuous value between 0 and 1. A higher value indicates that the current sample better matches the known category pattern and the network's discrimination is more stable; a lower value indicates that the current sample may have partial occlusion, abnormal posture, incomplete insect body, insufficient band quality, or no insect species observed. Preferably, the recognition confidence is calibrated offline to reduce the inconsistency between the classification output probability and the true reliability. Calibration methods can include temperature scaling, ordinal-preserving regression, or a segmented mapping method based on the validation set, making the confidence more accurately reflect the model's judgment reliability of the sample.

[0091] S65. Joint Discrimination of Open Sets: In this embodiment, matching distance, classification entropy, and recognition confidence are used to jointly determine whether the current insect instance is an unknown insect or a suspected new insect species. Specifically, the system sets a matching distance threshold, a classification entropy threshold, and a recognition confidence threshold. For the current insect instance, it is determined to be an unknown insect or a suspected new insect species when one of the following conditions is met: First, the matching distance between the current insect feature vector and the nearest insect species prototype exceeds the matching distance threshold, and the corresponding classification entropy is higher than the classification entropy threshold; this indicates that the sample deviates from the known insect species prototype and lacks a clear category classification, and should be determined as an unknown insect or a suspected new insect species. Second, the recognition confidence of the current insect instance is lower than the preset threshold; even if its nearest prototype distance does not significantly exceed the threshold, the sample can still be considered to have a large degree of uncertainty and can be further marked as a low-confidence sample, and in the preferred embodiment, it is included in the set of unknown insects or suspected new insect species. In a preferred embodiment, samples that simultaneously meet the criteria of "matching distance significantly exceeding the threshold", "classification entropy significantly higher", and "identification confidence significantly lower" are given a higher priority for unknown determination; samples that only partially meet the criteria can be marked as samples to be reviewed first, and whether they belong to suspected new insect species can be further confirmed based on the repeated occurrence of similar samples in subsequent time windows.

[0092] S66. Unknown Insect Processing and Sample Pool Management: For samples identified as unknown insects or suspected new insect species through joint discrimination, this embodiment does not directly incorporate them into the statistical results of a known insect species. Instead, it outputs them as "unknown insect" or "suspected new insect species" labels and writes them into the sample pool to be reviewed. The sample pool to be reviewed can store the insect image patch, multispectral image cube index, insect feature vector, matching distance, classification entropy, identification confidence, collection time, monitoring point, and related environmental information of the sample. In one embodiment, the system can cluster unknown samples in the sample pool to be reviewed according to sample similarity. If clusters of unknown samples with similar characteristics appear consecutively in multiple time windows or multiple monitoring points, the unknown sample cluster can be marked as a high-priority suspected new insect species event and submitted for manual review or expert confirmation. After manual confirmation, if it is determined to be an abnormal individual belonging to an existing insect species, it can be added to the training set as a difficult sample; if it is determined to be a new insect species, a new category label can be assigned to it, and the insect species prototype memory bank can be updated accordingly.

[0093] S67. Output Results: After completing the open set discrimination, step S6 outputs the following results: First, for known insect species samples, output the insect species category, matching distance, recognition confidence, and insect body feature vector confirmed by prototype matching, and pass them to step S7 for aggregation statistics by insect species category and time window; Second, for unknown insect species or suspected new insect species samples, output unknown labels, relevant discrimination indicators, and original sample indexes, and write them into the sample pool to be reviewed for subsequent manual confirmation, incremental learning, or abnormal event tracking.

[0094] The open set discrimination principle of this embodiment is as follows: After being mapped by a collaborative network, known insect species samples are usually distributed around the corresponding insect species prototype in the feature space; while unknown insects, abnormal samples, or new insect species samples often cannot stably fall near any known prototype, or even if they are near a prototype, their category probability distribution still shows high uncertainty, and the recognition confidence is usually low. By simultaneously examining "how far away from the known prototype," "whether the classification is sufficiently concentrated," and "whether the model has sufficient confidence in its own output," it is possible to distinguish known insect species from unknown insects more accurately than single threshold discrimination.

[0095] Specifically, prototype matching distance reflects the degree of deviation of a sample from the center of known insect species in the feature space, classification entropy reflects the degree of uncertainty in class assignment, and recognition confidence reflects the model's reliability assessment of the final output. Using these three together can reduce the misclassification of unknown insects as known species and avoid excessive false rejections of known but morphologically unique samples, thereby improving the overall stability and practicality of the system.

[0096] Taking a specific insect instance output from step S5 as an example, the collaborative network predicts the insect species as "rice leaf roller" with a confidence level of 0.42, and outputs a fixed-dimensional insect feature vector. The system matches this feature vector with the prototypes of various insect species in the insect prototype memory bank. It finds that the minimum matching distance with the "rice leaf roller" prototype is still higher than the preset distance threshold, and its category probability distribution is relatively dispersed among "rice leaf roller," "rice stem borer," and "unknown category," with the corresponding classification entropy higher than the preset entropy threshold. Since this sample simultaneously meets the criteria of "matching distance exceeding the threshold and classification entropy higher than the threshold," and its confidence level is lower than the preset confidence threshold, the system classifies it as an unknown insect or a suspected new insect species, and does not directly include it in the "rice leaf roller" insect population statistics. Subsequently, this sample is written into the sample pool to be reviewed, along with its collection time, monitoring location, and environmental information. If, within multiple subsequent time windows, the system continuously detects unknown samples with similar characteristics at adjacent monitoring points, it will further report them as suspected new insect species events for manual verification and subsequent updates to the prototype database.

[0097] S7. Aggregate the identified insect categories by insect species and time window, and construct a spatiotemporal hypergraph of insect conditions that integrates information on field connectivity, wind direction, irrigation relationship and growth period. Output the risk level and early warning results using a fusion structure of fractional-gated temporal convolution branch and spatiotemporal hypergraph attention branch.

[0098] In this embodiment, step S7 is executed in a collaborative environment between the edge computing unit and the cloud server. It is used to predict the pest risk within a preset future time period and output an early warning result based on the known insect species identification results output in step S6, combined with temporal evolution information, spatial propagation relationships, and agricultural environmental information. Step S7 specifically includes the following sub-steps.

[0099] S71. The identification results are aggregated according to insect species and time window. For the known insect species identification results output in step S6, the system first aggregates and statistically analyzes them according to insect species, monitoring points, and preset time windows. The preset time window can be set to 10 minutes, 30 minutes, 1 hour, 6 hours, 12 hours, 1 day, or other fixed time lengths according to actual application needs. Preferably, a shorter time window is used for migratory pests with a fast occurrence rhythm; a longer time window can be used for resident pests with slow changes. Within each time window at each monitoring point, the insect identification results of the same insect species are summarized to obtain the insect population, insect population density per unit area, increase or decrease compared to the previous time window, growth rate, number of consecutive increases, and proportion of high-confidence identification samples within that time window. Preferably, the identification confidence level output in step S5 can also be used as a weighting factor in the statistics, so that high-confidence samples contribute more to the insect population statistics, while the statistical contribution of low-confidence samples is relatively reduced. In one implementation, the system establishes independent time series for different insect species; in another implementation, the population sizes of multiple insect species can be organized together into a multivariate time series to reflect possible symbiotic or substitution relationships between different insect species.

[0100] S72. Construct a state feature vector for each monitoring point. After completing time window aggregation, the system constructs a state feature vector for each monitoring point under each time window. The state feature vector includes at least: the number, density, growth rate, and continuous growth characteristics of the target insect species within the current time window; environmental meteorological information within the current and adjacent time windows, including temperature, humidity, light intensity, rainfall status, wind speed, and wind direction; crop type, growth stage, and growth stage coding characteristics of the plot corresponding to the current monitoring point; and static attribute information of the monitoring point, such as plot number, trapping device type, and monitoring point location coordinates. Preferably, the insect population sequence and environmental sequence under continuous time windows are normalized and missing value imputation is performed. For occasional missing environmental data, interpolation of nearby times, imputation using the mean of adjacent monitoring points, or backfilling using historical means can be used; for samples with abnormal jumps in insect population statistics, anomaly correction can be performed by combining identification confidence and equipment status logs.

[0101] S73. Constructing a spatiotemporal hypergraph of insect infestation: In this embodiment, to depict the many-to-many propagation relationship among multiple monitoring points influenced by common propagation factors, a spatiotemporal hypergraph is used to model the insect infestation propagation structure. Spatiotemporal node construction: The combination of "monitoring point - time window" is used as a spatiotemporal node. That is, the i-th monitoring point corresponds to a spatiotemporal node under the t-th time window. The node features of each spatiotemporal node are the state feature vector constructed in step S72. Temporal evolution hyperedge construction: A temporal evolution hyperedge is constructed for the spatiotemporal nodes of the same monitoring point under multiple consecutive time windows to characterize the continuous change pattern of insect infestation at a single monitoring point in the time dimension. Preferably, nodes of the same monitoring point under 3, 5, or 7 consecutive time windows can be connected together to form a temporal evolution hyperedge to simultaneously reflect short-term fluctuations and medium-term accumulation trends. Spatial propagation hyperedge construction involves constructing spatial propagation hyperedges for multiple monitoring points within the same time window that satisfy preset relationships. The preset relationships include at least one or more of the following: field connectivity: multiple monitoring points correspond to adjacent, shared, contiguous, or located within the same field cluster; wind direction propagation: multiple monitoring points are located on the propagation path of the prevailing wind direction or high-frequency wind direction, and the upwind and downwind relationships satisfy the preset directional consistency condition; irrigation connectivity: multiple monitoring points are located in the same irrigation branch, the same ditch network, or have an upstream-downstream irrigation connection; growth period correlation: multiple monitoring points are located in fields with the same crop type and the same or similar growth period, reaching a preset similarity threshold.

[0102] In a preferred embodiment, the same set of monitoring points can simultaneously belong to multiple spatial propagation hyperedges. For example, if multiple monitoring points are located within the same contiguous field and along the same prevailing wind path, then "field connectivity hyperedges" and "wind propagation hyperedges" can be constructed separately to fully represent different propagation mechanisms. Hyperedge weights are determined by further defining the weights of each hyperedge to differentiate the contribution of different propagation relationships to pest risk. For temporal evolution hyperedges, the weights can be determined based on the magnitude of pest density changes, growth continuity, length of continuous high-growth periods, and environmental stability. For spatial propagation hyperedges, the weights can be determined comprehensively based on field connectivity strength, wind direction consistency, wind speed, irrigation connectivity strength, and similarity of growth periods. Preferably, the above influencing factors are first normalized, and then the final hyperedge weights are obtained using weighted combination, rule mapping, or learnable weight mapping. Candidate hyperedges with excessively low weights can be directly eliminated to reduce graph structure noise.

[0103] S74. Input for constructing the fusion prediction structure: After the spatiotemporal hypergraph of insect populations is constructed, the system further organizes the input into two types of data: First, the temporal sequence of insect population density and environmental sequence at each monitoring point under continuous time windows, used as input to the fractional-order gated temporal convolution branch; Second, the node features, hyperedge connections, and hyperedge weights in the spatiotemporal hypergraph, used as input to the attention branch of the spatiotemporal hypergraph. Preferably, the sequence inputs of different time lengths are uniformly pruned or padded; For multi-species joint prediction scenarios, inputs can be established separately for each insect species, or the population densities of different insect species can be combined into multi-channel sequence inputs.

[0104] S75. Fractional-order gated temporal convolution branch processing: The fractional-order gated temporal convolution branch is used to characterize the long-memory gradual features of insect population density, which slowly accumulates, continuously increases, and suddenly changes over time. First, the insect population density sequence, environmental meteorological sequence, and reproductive period sequence of each monitoring point within multiple consecutive time windows are input into the temporal embedding unit to form temporal features with temporal location information. Subsequently, the fractional-order convolution unit performs fractional-order convolution operations on the temporal features to enhance the ability to remember information from longer historical time windows. Unlike ordinary convolution, which only focuses on local fixed windows, fractional-order convolution can comprehensively utilize historical change information over a longer period of time in a decaying memory manner, making it more suitable for depicting the evolutionary process of insect infestation from "slow accumulation to critical outbreak." Furthermore, the gated update unit generates update gates and retention gates based on the current input, historical hidden state, and environmental factors to filter and update the temporal features, thereby suppressing non-continuous fluctuations caused by misidentification, accidental intrusion, or single sampling noise, and highlighting the truly risk-significant continuous growth trend. After multi-layer temporal convolution and gating update, the insect population evolution characteristics of each monitoring point in the time dimension are obtained.

[0105] S76. Spatiotemporal Hypergraph Attention Branching: The spatiotemporal hypergraph attention branching is used to characterize the high-order correlations between different monitoring points at the spatial propagation level. First, a spatiotemporal hypergraph is constructed using the state feature vectors of each spatiotemporal node as node inputs and the temporal evolution hyperedges and spatial propagation hyperedges as connection structure inputs. Then, the hypergraph attention propagation unit first calculates the contribution of each node to its respective hyperedge, and then calculates the feedback weights of each hyperedge to each node, realizing information propagation from node to hyperedge to node. Through attention allocation, the importance of different monitoring points and different propagation channels in risk diffusion can be adaptively learned. For example, when a monitoring point has a low current insect population but is located in the upwind main propagation channel and connected to multiple high-risk monitoring points, its propagation risk weight in the hypergraph can be increased; conversely, for monitoring points with a local insect population increase but relatively isolated spatially and with weak propagation channels, their spatial propagation risk weight can be relatively decreased. After one or more rounds of propagation, the spatial propagation correlation representation of each spatiotemporal node is obtained.

[0106] S77. Cross-branch fusion and risk decoding. To avoid the isolation of temporal evolution features and spatial propagation features from each other, a cross-branch fusion unit is set up in this embodiment to achieve two-way modulation and cross-attention fusion. Temporal guidance for spatial modulation. The temporal evolution features output by the fractional-order gated temporal convolutional branch are used to dynamically adjust the node weights and hyper-edge weights in the spatio-temporal hypergraph. For monitoring points with continuous recent growth, high growth rate or in a critical accumulation state, their propagation influence weights in the hypergraph are correspondingly increased. Spatial guidance for temporal modulation. The spatial propagation correlation features output by the spatio-temporal hypergraph attention branch are used to recalibrate the temporal response in the temporal branch. For monitoring points in high-risk propagation channels, associated with multiple high-risk nodes or located on the key irrigation propagation chain, the growth trend response in their temporal features is enhanced. Cross-attention fusion. The outputs of the two branches are used as the query, key, and value to input into the cross-attention module, to learn the correspondence between the temporal evolution features and the spatial propagation features, and output a unified spatio-temporal fusion representation. The fusion representation not only reflects the historical pest situation change trajectory of a single monitoring point, but also reflects the pest situation diffusion and propagation coupling relationship between regions. Subsequently, the risk decoding output unit decodes the spatio-temporal fusion representation and outputs the pest situation risk level and early warning result within a preset future time period. The preset time period can be 1 hour, 6 hours, 12 hours, 24 hours, 3 days or 7 days in the future. The risk level can be divided into low risk, medium risk, high risk and outbreak risk; in another implementation, a continuous risk score or pest population density prediction value can also be output.

[0107] S78. Early warning result generation and output. In this embodiment, the system generates an early warning result according to the risk decoding result. The early warning result at least includes: the target pest species; the monitoring point, field or regional scope corresponding to the early warning; the risk level within a preset future time period; the risk change trend, such as rising, remaining flat or falling; an optional pest population quantity prediction value or outbreak probability; an optional disposal suggestion, such as strengthening patrol, local trapping, key spraying or joint prevention and control. Preferably, when the risk level reaches high risk or outbreak risk, the system pushes an alarm message to the farm management platform, mobile terminal or regional early warning center; when multiple adjacent monitoring points simultaneously show continuous high risks on the same dominant wind direction path, regional linkage early warning can be triggered.

[0108] In this embodiment, the working principle of step S7 is as follows: first, an ordered time series is formed from the single-point monitoring results; then, a spatiotemporal hypergraph is used to describe the high-order propagation relationship between multiple monitoring points due to field connectivity, wind direction propagation, irrigation pathways, and similar growth periods; finally, the temporal evolution features and spatial propagation features are synergistically fused to achieve an upgrade from static identification to dynamic early warning. Specifically, the fractional-order gated temporal convolution branch is good at capturing the long-memory accumulation features of insect population density in the time dimension, and can better characterize the process of insect infestation from low-density accumulation to rapid growth; the spatiotemporal hypergraph attention branch is good at depicting the many-to-many propagation relationship between multiple monitoring points, and can express common propagation channels that are difficult to represent by traditional pairwise graph structures. Through cross-branch fusion, the system not only considers "whether there are many insects at a certain point recently", but also "whether these insects may spread from elsewhere, whether they will spread to other places, and whether the current crop is in a vulnerable stage", so that the early warning results are more consistent with the actual insect pest occurrence patterns.

[0109] Taking a specific pest in a paddy field as an example, the system aggregates the identification results for each hour within an adjacent 24-hour period by insect species, obtaining 24 time-window insect population density sequences for each monitoring point. Simultaneously, it reads temperature, humidity, wind speed and direction, and rice growth stage information for each time window. Subsequently, the "monitoring point-time window" combination is used as a spatiotemporal node, constructing a temporal evolution hyperedge for four consecutive time-window nodes at the same monitoring point. For multiple monitoring points located in the same field cluster, along the dominant southeasterly wind propagation path, and connected to the same irrigation branch, a spatial propagation hyperedge is constructed within the same time window. A fractional-gated temporal convolution branch extracts slow accumulation and continuous increase features from the 24-hour insect population sequence, while a spatiotemporal hypergraph attention branch identifies key propagation links from the spatial propagation structure. After cross-branch fusion, the system determines that the target pest risk level for a certain field cluster is high within the next 24 hours and generates an early warning result stating, "The target pest has a continuous spread trend in the next 24 hours; it is recommended to strengthen field inspections and implement local control measures," which is then sent to the management platform and mobile terminals.

[0110] Preferably, the band reliability coefficient in S2 is determined based on the signal-to-noise ratio, saturated pixel ratio, local blur, specular reflection ratio and edge fidelity of each band, and the features of each band are weighted or gated accordingly.

[0111] Preferably, the background spectral dictionary in S3 includes the background color of the trapping board, the adhesive layer, plant debris, dust, and non-insect impurities; the instance separation is achieved by constructing a candidate connectivity graph by combining contour depression points, skeleton branch points, and inter-band discontinuity boundaries and performing graph segmentation.

[0112] Preferably, the prior map of the key parts in S4 is generated by a lightweight segmentation subnetwork. The key parts include the head, thorax, abdomen, wing region and antenna region. The multispectral features of the corresponding regions are spatially aligned and correlated and fused by the part mask to form a spectral part joint feature tensor.

[0113] Preferably, the collaborative network in S5 includes: a spectral coding branch, a part hypergraph attention branch, a bidirectional cross-modulation fusion unit, and a hierarchical fusion decoder; wherein, the spectral coding branch is used to divide the joint feature tensor of spectral parts into multiple part sub-tensors based on the prior of key parts, and construct a spectral response sequence of each part sub-tensor along the band dimension, and dynamically model the continuous change of reflectance of each key part through neural controlled differential equations to output the dynamic spectral representation of the part; the part hypergraph attention branch is used to use the head, thorax, abdomen, wing region, and antennal region as hypergraph nodes, and construct hyperedges based on spatial adjacency, biological structural relationship, and cooperative discrimination relationship, and obtain the part structural relationship representation through hypergraph attention propagation; the bidirectional cross-modulation fusion unit is used to recalibrate the part structural relationship representation using the dynamic spectral representation of the part, and to perform gated modulation on each band response using the part structural relationship representation to obtain fused features; the hierarchical fusion decoder is used to output the insect category, confidence level, and insect feature vector according to the fused features.

[0114] In some embodiments, the hierarchical fusion decoder employs a spectral difference-enhanced Mish-gated activation function to enhance the response of the discriminative band in key regions. The spectral difference-enhanced Mish-gated activation function... :

[0115]

[0116] in, The dynamic spectral difference gating coefficient is obtained by weighted mapping of the band reliability coefficient, local inter-band gradient, and key location confidence, and is used to characterize the degree of activation enhancement of the current band in the current key location region. Let ln be the hyperbolic tangent function; ln represents the natural logarithm. This is the output of the convolutional layer.

[0117] The layered fusion decoder employs a spectral difference-enhanced Mish-gated activation function. By introducing a dynamic spectral difference gating coefficient β into the softplus term of the standard Mish activation function, the output of the convolutional layer is modulated based on the reliability and discrimination value of the current band before entering the nonlinear mapping. β is determined at least by the band reliability coefficient and jointly corrected by the local inter-band gradient and the location confidence of key regions. For feature responses with high reliability, significant inter-band variations, and located in key regions such as the head, thorax, abdomen, wing area, or antennae, β is set relatively large to enhance effective spectral band activation. For feature responses significantly affected by reflection, blurring, or noise, or located in non-key regions, β is set relatively small to suppress the propagation of invalid responses.

[0118] Enhance the response of discriminative bands in key regions. By introducing the band reliability coefficient into the Mish activation function, this invention can adaptively enhance the effective response of high-reliability bands in key regions such as the head, thorax, abdomen, wing region, and antennal region during the nonlinear mapping stage, so that spectral features more valuable for insect species differentiation are preferentially preserved, thereby improving the fine-grained identification ability of closely related insect species and insect species with similar appearance.

[0119] Since some bands may be affected by specular reflection, local blurring, noise, or background contamination, directly using conventional activation functions can easily amplify invalid responses. This invention modulates the Mish activation function using a dynamic spectral difference gating coefficient β, which can reduce the response intensity in low-reliability bands and non-critical regions, reduce false activations caused by background impurities, trap plate reflection, and insect adhesion, and improve the robustness of feature extraction.

[0120] This invention does not uniformly process all bands and all spatial locations. Instead, it incorporates band reliability, local inter-band differences, and key component information into the activation stage, making the feature fusion process more scenario-specific. Compared to standard Mish or other general activation functions, this improved approach better aligns with the actual pattern in multispectral insect infestation identification where "different bands contribute differently, and different components have different importance," thus improving the overall recognition accuracy and stability of the collaborative network.

[0121] Preferably, the collaborative network is obtained through training with a joint loss function, which includes at least focus classification loss, part consistency loss, angle interval prototype loss and open set energy loss, so as to simultaneously constrain class discrimination, part representation consistency, same-class aggregation and different-class separation, and the ability to reject unknown insects.

[0122] Preferably, the fusion structure in S7 includes an input construction unit, a fractional-gated temporal convolution branch, a spatiotemporal hypergraph attention branch, a cross-branch fusion unit, and a risk decoding output unit. The input construction unit constructs a temporal input based on insect population density, environmental factors, and crop growth period information at each monitoring point within a continuous time window, and constructs a spatiotemporal hypergraph based on field connectivity, wind direction propagation, irrigation pathway relationships, and growth period correlations. The fractional-gated temporal convolution branch performs fractional-gated convolution and gating updates on the insect population density sequence at each monitoring point, extracting long-memory progressive change features. The spatiotemporal hypergraph attention branch performs attention propagation with each monitoring point as a node and the common propagation relationship of multiple monitoring points as a hyperedge, extracting spatial propagation correlation features. The cross-branch fusion unit modulates the node and hyperedge weights using temporal evolution features and recalibrates the temporal feature response using spatial propagation correlation features, obtaining a spatiotemporal fusion representation through cross-attention. The risk decoding output unit outputs the insect infestation risk level and early warning results for a future preset time period based on the spatiotemporal fusion representation.

[0123] Preferably, the construction of the insect spatiotemporal hypermap in S7 includes: aggregating the insect population number, insect population density, growth rate, environmental factors, and crop growth period information of each monitoring point in each time window to form a state feature vector corresponding to each monitoring point in each time window, and using the combination of monitoring point and time window as spatiotemporal node; constructing temporal evolution hyperedges for spatiotemporal nodes of the same monitoring point in multiple consecutive time windows, and constructing spatial propagation hyperedges for spatiotemporal nodes corresponding to multiple monitoring points in the same time window that satisfy field connectivity, prevailing wind direction propagation, irrigation connectivity, or crop growth period similarity; determining the weight of each hyperedge based on the insect population density change amplitude, growth continuity, field connectivity strength, wind direction consistency, irrigation connectivity strength, and growth period similarity, thereby forming the insect spatiotemporal hypermap.

[0124] This application also provides an AI-based insect infestation identification and early warning system based on multispectral imaging, such as... Figure 2 The device includes at least the following components: a trapping carrier 1, a multispectral imaging unit 2, a supplementary lighting unit 3, an environmental information acquisition unit 4, a positioning and timing unit 5, an edge computing unit 6, a storage unit 7, a communication unit 8, a power supply unit 9, a cloud server 10, and a user terminal 11.

[0125] The trapping carrier is used to carry, gather, or fix insects, and can be a receiving plate for sticky insect boards, insect-attracting lamps, pheromone trapping boards, a receiving plate or insect-collecting tray in food-attracting devices. The trapping carrier is placed at the center of the sampling area of ​​the monitoring point and is located within the field of view of the multispectral imaging unit.

[0126] The multispectral imaging unit is used to acquire image information of insects in multiple spectral bands, and includes at least a multispectral camera module, a lens assembly, and a mounting bracket. In one embodiment, the multispectral camera module can adopt: a time-division switching filter structure; a multi-channel beam splitting structure; or a multi-camera array structure. Preferably, the multispectral imaging unit acquires at least four spectral bands from blue light, green light, red light, red edge, and near-infrared. The multispectral imaging unit is mounted above the trapping carrier and fixedly connected to the chassis or pole via a bracket, so that the imaging optical axis faces the surface of the trapping carrier.

[0127] The supplementary lighting unit is used to provide stable illumination under low light, backlight, shadow, or nighttime conditions, and includes at least a white light supplementary light source and / or a narrowband supplementary light source. The supplementary lighting unit is positioned around the multispectral imaging unit and forms a fixed relative position with it. Preferably, the supplementary lighting unit is controlled to turn on and off by an edge computing unit and is triggered synchronously with the multispectral imaging unit.

[0128] The environmental information acquisition unit is used to collect environmental meteorological information and includes at least one or more of the following: temperature sensor, humidity sensor, light sensor, wind speed sensor, wind direction sensor, and rainfall sensor. The environmental information acquisition unit is located near the monitoring point and is connected to the edge computing unit via wired or wireless means to synchronously upload data such as temperature, humidity, light intensity, wind speed, wind direction, and rainfall status.

[0129] The positioning and timing unit is used to acquire monitoring point location information and unified time information, and may include a GNSS positioning module, a Beidou positioning module, a GPS module, an RTC clock module, or a network timing module. The positioning and timing unit is connected to the edge computing unit and is used to provide the edge computing unit with latitude and longitude coordinates, device number, timestamp, and sampling time information.

[0130] The edge computing unit is the core processing hardware of this system, used to perform image preprocessing, insect extraction, feature construction, collaborative recognition, open set discrimination, and early warning calculation. The edge computing unit may include a processor, memory, an interface controller, and a local operating system, wherein the processor may be a CPU, GPU, NPU, FPGA, or a combination thereof.

[0131] Preferably, the edge computing unit integrates: an image acquisition and control module; a preprocessing module; an insect extraction module; a feature construction module; a collaborative recognition module; an open set discrimination module; and an insect infestation early warning module. The storage unit stores raw multispectral images, standardized multispectral images, a background spectral dictionary, an insect species prototype memory bank, model parameters, recognition results, and early warning results. The storage unit may include eMMC, SD card, solid-state drive, flash memory chip, or mechanical hard drive. The storage unit is directly connected to the edge computing unit for local caching and historical data tracing. The communication unit enables data communication between the monitoring terminal and a cloud server or user terminal, and may include one or more of the following: 4G module, 5G module, NB-IoT module, LoRa module, Wi-Fi module, and Ethernet interface. The communication unit is connected to the edge computing unit, uploading image data, recognition results, risk levels, and early warning results to the cloud server, and receiving model update instructions, threshold configuration parameters, and device management instructions from the cloud. The power supply unit provides power to the multispectral imaging unit, supplementary lighting unit, environmental information acquisition unit, positioning and timing unit, edge computing unit, and communication unit. The power supply unit may include a solar panel, a battery, a power management module, and a mains power adapter module. Preferably, the power supply unit provides a stable operating voltage to each hardware module through the power management module and supports low-power wake-up and timed sampling. The cloud server is used to perform historical data storage, cross-monitoring point data aggregation, centralized model training, threshold updates, early warning information aggregation, and regional cascade analysis. The cloud server connects to multiple monitoring terminals via a network and communicates with user terminals. The user terminal is used to display insect infestation identification results and early warning information, and may be a mobile phone, tablet, PC terminal, or agricultural management platform. The user terminal communicates with the cloud server and / or monitoring terminals to receive insect species, insect population density, risk level, early warning area, and treatment recommendations.

[0132] In a preferred embodiment, the connection relationships between the hardware components are as follows: (1) Acquisition link connection: The multispectral imaging unit is electrically connected and data signal connected to the edge computing unit, used to send multi-band image data to the edge computing unit. The connection method can be one of MIPI, USB3.0, GigE, Camera Link, LVDS or CSI interface. The supplementary lighting unit is controlled by the edge computing unit, and the edge computing unit controls the supplementary lighting unit to work synchronously through GPIO, PWM or relay drive interface. (2) Environmental and positioning data link connection: The environmental information acquisition unit is communicatively connected to the edge computing unit, preferably using RS485, RS232, UART, I2C, SPI, CAN bus or wireless short-range communication. The positioning and timing unit is communicatively connected to the edge computing unit, preferably using UART or USB interface to transmit timestamps and location information. (3) Local computing and storage connection: The multispectral imaging unit, environmental information acquisition unit and positioning and timing unit are all connected to the edge computing unit, and the edge computing unit performs joint processing after receiving the above inputs. The storage unit is connected to the edge computing unit at high speed, preferably using SATA, PCIe, eMMC or SDIO interfaces, to store image data, feature data and model parameters. (4) Remote communication connection: The edge computing unit communicates with the cloud server through the communication unit to upload multispectral images, recognition results, time window statistical results and early warning results, and receive model parameter update and configuration instructions. The cloud server further communicates with the user terminal to push risk levels, early warning information and prevention and control suggestions to the user terminal.

[0133] (5) Power supply connection: The power supply unit is electrically connected to the multispectral imaging unit, the supplementary lighting unit, the environmental information acquisition unit, the positioning and timing unit, the edge computing unit, the storage unit, and the communication unit, respectively. Preferably, the supplementary lighting unit and the communication unit are independently powered by the power management module to reduce the impact of voltage fluctuations on the stability of imaging and communication.

[0134] In one specific embodiment, the system is deployed at the edge of a rice paddy and includes: a yellow sticky insect trap as a trapping carrier; a five-band multispectral camera as a multispectral imaging unit; a ring-shaped LED supplementary light as a supplementary lighting unit; a set of temperature, humidity, wind speed, wind direction, and light sensors as an environmental information acquisition unit; a BeiDou / GPS positioning module as a positioning and timing unit; an embedded AI edge computing box as an edge computing unit; an industrial-grade solid-state drive as a storage unit; a 4G / 5G communication module as a communication unit; and a solar panel and battery as a power supply unit. The multispectral camera is connected to the edge computing box via USB 3.0; the environmental sensors are connected to the edge computing box via an RS485 bus; the BeiDou positioning module is connected to the edge computing box via UART; the supplementary light is connected to the edge computing box via a GPIO control interface; the solid-state drive is connected to the edge computing box via SATA; and the 4G / 5G communication module is connected to the edge computing box via USB or Mini PCIe. The edge computing box communicates with the cloud server through the communication module, and the cloud server then sends the pest warning results to the agricultural management platform and mobile terminals.

[0135] The AI ​​insect infestation identification and early warning system based on multispectral imaging provided in this application integrates a multispectral imaging unit, an environmental information acquisition unit, a positioning and timing unit, an edge computing unit, a storage unit, a communication unit, and a power supply unit into one unit. These units are connected through an image acquisition link, a sensor data link, a local computing and storage link, and a remote communication link. This achieves a closed-loop hardware support for multi-band insect image acquisition, synchronous perception of environmental and agricultural conditions, insect identification, open set discrimination, and insect infestation risk early warning.

[0136] The multispectral imaging unit is used to perform multi-band imaging of insects on the trapping carrier to obtain a multispectral image cube at the same monitoring time; the information acquisition unit is used to simultaneously acquire acquisition time, monitoring point information, environmental meteorological information and crop growth period information.

[0137] The preprocessing module is used to perform dark current correction, reflectivity correction and inter-band spatial registration on the multispectral image cube to obtain a standardized multispectral image, and to determine the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflective interference and edge fidelity of each band.

[0138] The insect body extraction module is used to perform background sparse reconstruction of the multi-band spectral vectors of each pixel or local region in the standardized multispectral image based on the band reliability coefficient and background spectral dictionary, extract candidate insect body regions and insect body masks according to the reconstruction residual and combined with multi-scale edge response, and perform instance separation of adhered insect bodies.

[0139] The feature construction module is used to generate prior maps of key parts of the insect body within the candidate insect body region and construct a joint feature tensor of spectral parts.

[0140] The collaborative identification module is used to input the joint feature tensor of the spectral region into a collaborative network containing a spectral line encoding branch of a neural controlled differential equation and a region hypergraph attention branch, and output the insect category, confidence level and insect feature vector;

[0141] The open set discrimination module matches the insect feature vector with the insect species prototype memory database and performs open set discrimination based on the recognition confidence. When the matching distance exceeds the threshold and the classification entropy is higher than the threshold, or the recognition confidence is lower than the preset threshold, it is judged as an unknown insect or a suspected new insect species.

[0142] The insect infestation early warning module aggregates the identified insect categories by insect species and time window, and constructs a spatiotemporal hypergraph of insect infestation that integrates information on field connectivity, wind direction, irrigation relationship and growth period. It outputs risk level and early warning results using a fusion structure of fractional-order gated temporal convolution branch and spatiotemporal hypergraph attention branch.

[0143] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products, and therefore this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0144] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An AI-based insect infestation identification and early warning method based on multispectral imaging, characterized in that, Including the following steps: S1. Use a multispectral imaging unit to perform multi-band imaging of the insects on the trapping carrier to obtain a multispectral image cube at the same monitoring time; and simultaneously acquire the collection time, monitoring point information, environmental meteorological information and crop growth period information. S2. Perform dark current correction, reflectivity correction and inter-band spatial registration on the multispectral image cube to obtain a standardized multispectral image, and determine the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflection interference and edge fidelity of each band. S3. Based on the band reliability coefficient and background spectral dictionary, perform background sparse reconstruction on the multiband spectral vectors of each pixel or local region in the standardized multispectral image. Extract candidate insect body regions and insect body masks based on the reconstruction residual and combined with multi-scale edge response, and perform instance separation on the adhered insect bodies. S4. Generate prior maps of key parts of the insect body within the candidate insect body region and construct a joint feature tensor of spectral parts. S5. Input the spectral region joint feature tensor into a collaborative network containing a neural controlled differential equation spectral line encoding branch and a region hypergraph attention branch, and output the insect body category, confidence level and insect body feature vector; S6. Match the insect body feature vector with the insect species prototype memory database, and perform open set discrimination based on the recognition confidence. When the matching distance exceeds the threshold and the classification entropy is higher than the threshold, or the recognition confidence is lower than the preset threshold, it is determined to be an unknown insect body or a suspected new insect species. S7. Aggregate the identified insect categories by insect species and time window, and construct a spatiotemporal hypergraph of insect conditions that integrates information on field connectivity, wind direction, irrigation relationship and growth period. Output the risk level and early warning results using a fusion structure of fractional-gated temporal convolution branch and spatiotemporal hypergraph attention branch.

2. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The reliability coefficient of each band in S2 is determined based on the signal-to-noise ratio, saturated pixel ratio, local blur, specular reflection ratio and edge fidelity of each band, and the features of each band are weighted or gated accordingly.

3. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The background spectral dictionary in S3 includes the background color of the trapping board, the adhesive layer, plant debris, dust, and non-insect impurities; the instance separation is achieved by combining contour depression points, skeleton branch points, and inter-band discontinuity boundaries to construct a candidate connectivity graph and performing graph segmentation.

4. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The prior map of key parts in S4 is generated by a lightweight segmentation subnetwork. The key parts include the head, thorax, abdomen, wing region and antenna region. The multispectral features of the corresponding regions are spatially aligned and correlated and fused by the part mask to form a spectral part joint feature tensor.

5. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The collaborative network in S5 includes: a spectral coding branch, a site hypergraph attention branch, a bidirectional cross-modulation fusion unit, and a hierarchical fusion decoder. The spectral coding branch divides the joint feature tensor of spectral sites into multiple site sub-tensors based on key site priors, constructs a spectral response sequence along the band dimension for each site sub-tensor, and dynamically models the continuous reflectance changes of each key site using neural controlled differential equations to output a dynamic spectral representation of the site. The site hypergraph attention branch uses the head, thorax, abdomen, wing region, and antennal region as hypergraph nodes, constructs hyperedges based on spatial adjacency, biological structural relationships, and cooperative discrimination relationships, and obtains a site structural relationship representation through hypergraph attention propagation. The bidirectional cross-modulation fusion unit recalibrates the site structural relationship representation using the dynamic spectral representation of the site, and uses the site structural relationship representation to perform gated modulation on the band responses to obtain fused features. The hierarchical fusion decoder outputs the insect category, confidence level, and insect feature vector based on the fused features.

6. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 5, characterized in that, The hierarchical fusion decoder employs a spectral difference-enhanced Mish-gated activation function. The gating weights of the activation function are jointly determined by the band reliability coefficient, the local inter-band gradient, and the key location confidence, in order to enhance the response of the discriminative band in the key region.

7. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The collaborative network is obtained through training with a joint loss function, which includes at least the focus classification loss, the part consistency loss, the angle interval prototype loss, and the open set energy loss, so as to simultaneously constrain the class discrimination, part representation consistency, class aggregation and heterogeneous separation, and the ability to reject unknown insects.

8. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The fusion structure in S7 includes an input construction unit, a fractional-gated temporal convolution branch, a spatiotemporal hypergraph attention branch, a cross-branch fusion unit, and a risk decoding output unit. The input construction unit constructs a temporal input based on insect population density, environmental factors, and crop growth period information at each monitoring point within a continuous time window, and constructs a spatiotemporal hypergraph based on field connectivity, wind direction propagation, irrigation pathways, and growth period correlations. The fractional-gated temporal convolution branch performs fractional-gated convolution and gating updates on the insect population density sequence at each monitoring point, extracting long-memory progressive change features. The spatiotemporal hypergraph attention branch performs attention propagation with each monitoring point as a node and the common propagation relationship of multiple monitoring points as a hyperedge, extracting spatial propagation correlation features. The cross-branch fusion unit modulates the node and hyperedge weights using temporal evolution features and recalibrates the temporal feature response using spatial propagation correlation features, obtaining a spatiotemporal fusion representation through cross-attention. The risk decoding output unit outputs the insect infestation risk level and early warning results for a future preset time period based on the spatiotemporal fusion representation.

9. The AI-based insect infestation identification and early warning method based on multispectral imaging according to claim 1, characterized in that, The construction of the insect spatiotemporal hypermap in S7 includes: aggregating the insect population number, population density, growth rate, environmental factors, and crop growth period information of each monitoring point in each time window to form a state feature vector corresponding to each monitoring point in each time window, and using the combination of monitoring point and time window as spatiotemporal node; constructing temporal evolution hyperedges for spatiotemporal nodes of the same monitoring point in multiple consecutive time windows, and constructing spatial propagation hyperedges for spatiotemporal nodes corresponding to multiple monitoring points in the same time window that satisfy field connectivity, prevailing wind direction propagation, irrigation connectivity, or crop growth period similarity; determining the weight of each hyperedge based on the insect population density change amplitude, growth continuity, field connectivity strength, wind direction consistency, irrigation connectivity strength, and growth period similarity, thereby forming the insect spatiotemporal hypermap.

10. An AI-based insect infestation identification and early warning system based on multispectral imaging, characterized in that, include: The multispectral imaging unit is used to perform multi-band imaging of insects on the trapping carrier and obtain a multispectral image cube at the same monitoring time. The information acquisition unit is used to simultaneously acquire acquisition time, monitoring point information, environmental meteorological information, and crop growth period information; The preprocessing module is used to perform dark current correction, reflectivity correction and inter-band spatial registration on the multispectral image cube to obtain a standardized multispectral image, and to determine the band reliability coefficient based on the signal-to-noise ratio, saturation, blurring, reflective interference and edge fidelity of each band. The insect body extraction module is used to perform background sparse reconstruction of the multi-band spectral vectors of each pixel or local region in the standardized multispectral image based on the band reliability coefficient and background spectral dictionary, extract candidate insect body regions and insect body masks according to the reconstruction residual and combined with multi-scale edge response, and perform instance separation of adhered insect bodies. The feature construction module is used to generate prior maps of key parts of the insect body within the candidate insect body region and construct a joint feature tensor of spectral parts. The collaborative identification module is used to input the joint feature tensor of the spectral region into a collaborative network containing a spectral line encoding branch of a neural controlled differential equation and a region hypergraph attention branch, and output the insect category, confidence level and insect feature vector; The open set discrimination module matches the insect feature vector with the insect species prototype memory database and performs open set discrimination based on the recognition confidence. When the matching distance exceeds the threshold and the classification entropy is higher than the threshold, or the recognition confidence is lower than the preset threshold, it is judged as an unknown insect or a suspected new insect species. The insect infestation early warning module aggregates the identified insect categories by insect species and time window, and constructs a spatiotemporal hypergraph of insect infestation that integrates information on field connectivity, wind direction, irrigation relationship and growth period. It outputs risk level and early warning results using a fusion structure of fractional-order gated temporal convolution branch and spatiotemporal hypergraph attention branch.

Images