Intelligent pest trapping and killing method based on multi-modal fusion recognition and variable-frequency dimming

Abstract

The invention discloses an intelligent pest trapping and killing method based on multi-mode fusion recognition and variable frequency dimming, and aims to solve the problems of inaccurate recognition and decision distortion caused by mutual interference of variable frequency dimming, camera exposure and rolling shutter. Estimating the time sequence of a camera, dividing frame exposure into safe exposure sub-windows according to line scanning phases, performing constant-current steady-state illumination in the sub-windows, performing phase locking outside the sub-windows under the constraint of forbidden frequency, outputting an inducing pulse, and performing adaptive scheduling and closed-loop updating according to fusion confidence; the technical effects that stable imaging is achieved, brightness stripes and sampling aliasing are avoided, and the recognition accuracy and the trapping and killing efficiency are improved are achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent pest control, and in particular to an intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming. Background Technology

[0002] With the development of facility agriculture and intelligent plant protection, visual recognition-based insect-attracting lamps and electric or mechanical trapping devices are gradually being combined with sensor networks to form a system for monitoring, identifying and controlling pests in a coordinated manner.

[0003] To enhance the attraction effect, existing insect-attracting lamps generally use pulse or frequency conversion dimming of light-emitting diodes to match the phototactic characteristics of different pests; the image acquisition end mostly uses low-cost complementary metal-oxide-semiconductor rolling shutter cameras, and the frame rate and exposure time often change dynamically with the scene and automatic exposure strategy.

[0004] To improve the robustness of recognition, the industry has also introduced microphones to collect wing vibration sounds and attempted to fuse them with visual and environmental parameters for discrimination. However, most of these methods are loosely coupled or statically weighted, and the control and recognition links are often independent of each other.

[0005] The shortcomings of existing technologies are mainly reflected in:

[0006] 1. The frequency conversion dimming does not form an effective synergy with the camera exposure and rolling shutter. The coupling of pulse illumination and scanning sampling is prone to producing brightness stripes and frequency aliasing, resulting in image feature distortion, reduced recognition accuracy and control decision deviation. Common fixed frequency points or simple anti-flicker strategies are difficult to adapt to the dynamic changes in frame rate and exposure.

[0007] 2. The lack of fine-grained gating and phase control that match the camera timing often causes the induced pulse to fall into the actual exposure range, and the constant current steady-state light intensity cannot be kept stable during the exposure stage. Existing solutions usually only perform frame-level triggering or coarse-grained avoidance, without constructing a frequency ban set and safe frequency band for line scanning harmonics.

[0008] 3. Multimodal fusion and scheduling lack an adaptive mechanism centered on confidence. The weights are fixed or mainly post-processed. It is impossible to dynamically adjust the exposure window length, inducing duty cycle and control strategy according to the recognition confidence. There are also few closed-loop updates based on the inducing and killing results to improve the overall system effect.

[0009] Therefore, a smart pest trapping method that can overcome the shortcomings of the existing technology is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0010] One objective of this invention is to propose an intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming. Addressing the problems of image distortion, inaccurate recognition, and decision-making bias caused by interference between variable frequency dimming, camera exposure, and rolling shutter speed in existing technologies, this invention proposes a technical solution combining exposure gating and phase locking, with joint scheduling based on fusion confidence. Specifically, it simultaneously acquires visual, wing vibration acoustic, and environmental data; estimates camera timing parameters; performs micro-window segmentation by row-level rolling scan phase alignment to generate a safe exposure sub-window, within which a constant current steady-state light segment is output; constructs a frequency ban set and a safe frequency band; and locks the phase of the inducing pulse outside the exposure sub-window. The window duration and duty cycle are adaptively adjusted based on the probability of the target pest's presence and the fusion confidence, and a closed-loop update is performed based on the recognition and trapping results. This invention achieves the technical effects of avoiding brightness stripes and frequency aliasing, improving imaging stability and recognition accuracy, and increasing trapping efficiency and control precision.

[0011] A method for intelligent pest trapping based on multimodal fusion recognition and variable frequency dimming according to an embodiment of the present invention is characterized by comprising:

[0012] S1. Collect multimodal data, obtain visual image sequences, wing vibration acoustic signals and environmental parameters to form a collection dataset;

[0013] S2. Input the collected dataset, perform camera timing estimation, and obtain the camera frame rate, frame exposure time, line exposure time, line scan period and rolling shutter phase to form a timing parameter set;

[0014] S3. Input the timing parameter set, divide each frame exposure window into a safe exposure sub-window by aligning the line scan, and configure the constant current steady-state light band reference brightness for each sub-window to form a gating parameter set;

[0015] S4. Input the gating parameter set, perform multimodal fusion recognition, calculate the probability of the target pest's existence and the fusion confidence, and adaptively adjust the number and length of the safe exposure sub-windows based on the fusion confidence and the probability of the target pest's existence to form a scheduling parameter set;

[0016] S5. Input the scheduling parameter set, construct the frequency ban set and determine the safe frequency band, determine the frequency, phase and duty cycle of the inducing pulse within the safe frequency band, lock the phase of the inducing pulse outside the safe exposure sub-window, and simultaneously arrange the brightness and timing of the constant current steady-state light segment to form a dimming parameter set;

[0017] S6. Input the dimming parameter set, output the constant current steady-state light segment in the safe exposure sub-window according to the time sequence, output the inducing pulse outside it, and acquire image frames in the safe exposure sub-window to form an imaging dataset.

[0018] S7. Input the imaging dataset, identify and count the target pests, generate and issue baiting control instructions, record the execution status and killing results, and form a baiting result dataset.

[0019] S8. Input the baiting result dataset, update the calculation parameters of the fusion confidence, and fine-tune the duration and dimming parameters of the safe exposure sub-window to form an update time series parameter set for subsequent loops.

[0020] Optionally, step S1 specifically includes:

[0021] The camera continuously captures visual image sequences and records the timestamp, frame number, and exposure time of each image frame.

[0022] Acoustic signals of wing vibration were collected using a microphone, and the start and end times and sampling rate were recorded.

[0023] Environmental parameters are collected using environmental sensors, including at least one of illuminance, temperature, and humidity, and the sampling time and measured values ​​are recorded.

[0024] The visual image sequence, the wing vibration acoustic signal, and the environmental parameters are synchronized in time, and the three data streams are aligned to the same time axis and a corresponding relationship is established.

[0025] Perform basic quality control and integrity verification, and mark frame loss, audio interruption and sensor anomaly without changing the original timing and amplitude characteristics.

[0026] The data acquisition dataset consists of aligned visual image sequences, aligned wing vibration acoustic signals, and aligned environmental parameters.

[0027] Optionally, step S2 specifically includes:

[0028] Using the collected dataset as input, with the visual image sequence as the main component and combined with the timestamps in the visual image sequence, a frame-level brightness temporal sequence is established;

[0029] The frame-level brightness timing sequence is subjected to temporal correlation and frequency domain estimation to obtain initial estimates of the camera frame rate and frame exposure time;

[0030] Brightness stripe features are extracted along the line direction within the same image frame. The line exposure time and line scan cycle are calculated based on the relationship between stripe spacing and stripe phase over time. The rolling shutter phase is then fitted by the continuity of stripe phases across multiple frames.

[0031] When the acquired dataset contains camera metadata, the timing field in the camera metadata is read and calibrated for consistency with the initial estimate to improve the estimation accuracy of the camera frame rate, the frame exposure time, the line exposure time, the line scan period and the rolling shutter phase;

[0032] The camera frame rate, frame exposure time, line exposure time, line scan period, and rolling shutter phase are output and combined with the acquired dataset to form the timing parameter set.

[0033] Optionally, step S3 specifically includes:

[0034] Using the time series parameter set as input, read the camera frame rate, the frame exposure time, the line exposure time, the line scan period, and the rolling shutter phase;

[0035] Based on the line exposure time, the line scan period and the rolling shutter phase, the time range of each frame exposure window on the time axis is determined, and each frame exposure window is divided into several safe exposure sub-windows aligned with the line scan, so that the start and end times of each safe exposure sub-window are aligned with the phase boundary of the line scan period and cover the pixel rows in the exposure state within its corresponding time period.

[0036] For each safe exposure sub-window, a reference brightness of the constant current steady-state light segment is configured. The reference brightness is set according to at least one of the environmental parameters included in the time sequence parameter set, namely illuminance or frame exposure time, so as to keep the imaging light intensity constant within the safe exposure sub-window.

[0037] The number of safe exposure sub-windows, start and end times, duration, and reference brightness of the corresponding constant current steady-state light segment for each frame are arranged in chronological order to form a gating parameter set, which includes the chronological parameter set.

[0038] Optionally, step S4 specifically includes:

[0039] Using the gating parameter set as input, read the visual image sequence, wing vibration acoustic signal, environmental parameters, and the number, start and end times, and duration of the safe exposure sub-windows contained in the gating parameter set;

[0040] The visual image sequence and the wing vibration acoustic signal are time-aligned on the same time axis. The spatiotemporal features of the visual image sequence and the time-frequency features of the wing vibration acoustic signal are extracted respectively. The amplitude or confidence level of the spatiotemporal features and the time-frequency features are normalized based on the environmental parameters.

[0041] Using the normalized spatiotemporal features and the time-frequency features as inputs, the probability of the existence of the target pest and the fusion confidence are estimated.

[0042] Using the fusion confidence and the probability of the presence of the target pest as input, the safe exposure sub-window of each frame is adaptively adjusted. The adaptive adjustment includes at least increasing the number of safe exposure sub-windows or extending the duration of the safe exposure sub-windows when the fusion confidence is below a threshold, and decreasing the number of safe exposure sub-windows or shortening the duration of the safe exposure sub-windows when the fusion confidence is above a threshold, while keeping the safe exposure sub-windows aligned with the phase boundary of the row scan cycle.

[0043] The adjusted number of safe exposure sub-windows, start and end times and durations, as well as the probability of the presence of the target pest and the fusion confidence level are arranged in chronological order to form a scheduling parameter set, which includes the gating parameter set.

[0044] Optionally, step S5 specifically includes:

[0045] Using the scheduling parameter set as input, the number of safe exposure sub-windows, start and end times and durations, as well as the probability of the target pest's presence and the fusion confidence level are read.

[0046] Based on the camera frame rate, the line scan period, and the sampling timing determined by the frame exposure time, the line exposure time, and the rolling shutter phase, a frequency ban set is constructed, such that the frequency ban set contains at least frequency points related to integer multiples of the camera frame rate and frequency points related to the sampling harmonics corresponding to the line scan period;

[0047] The safe frequency band is determined by using the aforementioned frequency ban set as a constraint;

[0048] The operating frequency, phase, and duty cycle of the inducing pulse are calculated within the safe frequency band. The duty cycle is adaptively adjusted based on the probability of the presence of the target pest and the fusion confidence level. At the same time, the phase of the inducing pulse is locked and controlled so that the phase of the inducing pulse is arranged outside the safe exposure sub-window to avoid brightness fluctuations within the safe exposure sub-window.

[0049] The brightness and timing of the constant current steady-state light segment are arranged according to the start and end times and duration of the safe exposure sub-window, so that the brightness remains constant within each safe exposure sub-window.

[0050] The operating frequency, phase, duty cycle, and brightness and timing of the constant current steady-state light segment are arranged in chronological order to form a dimming parameter set, which includes the scheduling parameter set.

[0051] Optionally, step S6 specifically includes:

[0052] Using the dimming parameter set as input, the light source and the camera work on the same time axis;

[0053] According to the start and end times and duration of the safe exposure sub-window, the constant current steady-state light segment is output within each safe exposure sub-window and the brightness is kept constant. The inducing pulse is output between each safe exposure sub-window. The operating frequency, phase and duty cycle of the inducing pulse are consistent with the dimming parameter set, and the inducing pulse is not output within the safe exposure sub-window.

[0054] Using the start and end times of the safe exposure sub-window as the acquisition trigger times, image frames are acquired within the safe exposure sub-window, and timestamps and window identifiers are recorded to form an imaging dataset containing the image frames and the dimming parameter set.

[0055] Optionally, step S7 specifically includes:

[0056] Using the imaging dataset as input, the image frames, their timestamps, and window identifiers are read, and the image frames are organized on the same timeline;

[0057] Spatiotemporal features are extracted from the image frames to determine the category, location, and range of the target pests in the image frames. Target tracking is performed based on the correlation between adjacent image frames, and the number statistics of each window are summarized to obtain the identification results and number statistics.

[0058] Using the identification results and the quantity statistics as input, a trapping control command is generated. The trapping control command includes at least one of the following: switching on / off, timing or intensity control of the trapping device, and establishes a binding relationship with the corresponding timestamp and window identifier.

[0059] The baiting control command is sent to the baiting device, and the execution status and kill result of the baiting device are recorded to form a baiting result dataset containing the identification result, the quantity statistics, the baiting control command, the execution status and the kill result.

[0060] Optionally, step S8 specifically includes:

[0061] Using the baiting result dataset as input, the identification result, the quantity statistics, the baiting control command, the execution status, and the kill result are read;

[0062] The identification results and the quantity statistics are summarized on the same timeline to form identification statistics, and the execution status and the kill results are summarized on the same timeline to form baiting and killing results;

[0063] Based on the correspondence between the identification statistics and the trapping results, the calculation parameters of the fusion confidence are updated so that the weights of the fusion confidence on the visual image sequence and the wing vibration acoustic signal are adaptively adjusted according to the correspondence.

[0064] Without changing the phase boundary alignment between the safe exposure sub-window and the line scan cycle, the duration of the safe exposure sub-window is finely adjusted based on the identification statistics and the baiting results;

[0065] Under the premise of satisfying the frequency ban set and the phase locking constraint, at least one of the working frequency, phase or duty cycle in the dimming parameters is fine-tuned according to the identification statistics and the trapping results;

[0066] The updated calculation parameters of the fusion confidence, the duration of the safe exposure sub-window, and the dimming parameters are arranged in chronological order to form an updated timing parameter set. This updated timing parameter set is used as the input of S2 in subsequent loops to achieve closed-loop control.

[0067] The beneficial effects of this invention are:

[0068] 1. Effectively eliminates the mutual interference between frequency conversion dimming and camera exposure and rolling shutter: Through camera timing estimation, row-level rolling sweep phase alignment of the safe exposure sub-window and constant current steady-state illumination within the sub-window, and under the frequency ban constraint, the phase of the inducing pulse is locked outside the exposure range, avoiding brightness stripes and sampling aliasing, significantly improving imaging stability and feature discernibility;

[0069] 2. Improve recognition and control accuracy: The probability of the presence of target pests and the fusion confidence are calculated by multimodal fusion of visual, wing vibration acoustic and environmental parameters. The number and duration of sub-windows and the attraction duty cycle are adaptively adjusted according to the confidence. While maintaining alignment with the line scan phase boundary, false detection and missed detection are reduced, and the accuracy of target tracking and trapping control is improved.

[0070] 3. Achieve closed-loop optimization and energy efficiency improvement: Based on the identification statistics and trapping results, the fusion weight and dimming parameters are updated in a closed loop. Under the constraints of frequency ban set and phase locking, the operating frequency, phase or duty cycle are finely adjusted to enable the system to adapt to different illumination, frame rate and environmental conditions, taking into account the trapping effect and energy consumption, and improving the overall trapping efficiency and long-term stability. Attached Figure Description

[0071] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0072] Figure 1This is a flowchart of an intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming proposed in this invention. Detailed Implementation

[0073] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0074] refer to Figure 1 A method for intelligent pest trapping based on multimodal fusion recognition and variable frequency dimming, characterized by comprising:

[0075] S1. Collect multimodal data, obtain visual image sequences, wing vibration acoustic signals and environmental parameters to form a collection dataset;

[0076] S2. Input the collected dataset, perform camera timing estimation, and obtain the camera frame rate, frame exposure time, line exposure time, line scan period and rolling shutter phase to form a timing parameter set;

[0077] S3. Input the timing parameter set, divide each frame exposure window into a safe exposure sub-window by aligning the line scan, and configure the constant current steady-state light band reference brightness for each sub-window to form a gating parameter set;

[0078] S4. Input the gating parameter set, perform multimodal fusion recognition, calculate the probability of the target pest's existence and the fusion confidence, and adaptively adjust the number and length of the safe exposure sub-windows based on the fusion confidence and the probability of the target pest's existence to form a scheduling parameter set;

[0079] S5. Input the scheduling parameter set, construct the frequency ban set and determine the safe frequency band, determine the frequency, phase and duty cycle of the inducing pulse within the safe frequency band, lock the phase of the inducing pulse outside the safe exposure sub-window, and simultaneously arrange the brightness and timing of the constant current steady-state light segment to form a dimming parameter set;

[0080] S6. Input the dimming parameter set, output the constant current steady-state light segment in the safe exposure sub-window according to the time sequence, output the inducing pulse outside it, and acquire image frames in the safe exposure sub-window to form an imaging dataset.

[0081] S7. Input the imaging dataset, identify and count the target pests, generate and issue baiting control instructions, record the execution status and killing results, and form a baiting result dataset.

[0082] S8. Input the baiting result dataset, update the calculation parameters of the fusion confidence, and fine-tune the duration and dimming parameters of the safe exposure sub-window to form an update time series parameter set for subsequent loops.

[0083] In this specific embodiment, S1 specifically refers to:

[0084] This embodiment collects data in parallel using a camera, microphone, and environmental sensor, and organizes the recording under a unified time base: the camera continuously outputs a sequence of visual images, and each frame is recorded. timestamp Frame number With exposure time Simultaneously, the microphone acquires wing vibration acoustic signals and records waveform samples at a fixed sampling rate. Start time With sampling rate , its first The local sampling time for each sample is determined by:

[0085] Give;

[0086] in For the microphone's local timeline The time of each sample For the start time of acoustic acquisition, For sample index, The number of sampling points per second;

[0087] Environmental sensor periodic output environmental measurement And record timestamps With the order index The measurement items must include at least one of illuminance or temperature and humidity;

[0088] To align the three data streams to a unified timeline, a global time base is established and the local clocks for each mode are linearly calibrated. The following approach is adopted:

[0089] Complete the mapping;

[0090] in For time mapped to the global time base, For modality Local time For modality Clock scale correction factor For modality Clock offset correction amount These represent camera, acoustic, and environmental modes, respectively.

[0091] After time alignment, basic quality control and integrity verification are performed. For frame drops, audio interruptions, and sensor anomalies, only annotations are made without changing the original timing and amplitude characteristics. The aligned and annotated data is then encapsulated into a dataset by modality.

[0092] ;

[0093] in To collect datasets, For a subset of camera data sequence, For a subset of acoustic data sequence, For a subset of environmental data sequence;

[0094] The three share a unified global timeline and establish a correspondence between frames, samples, and measurements for subsequent time series estimation and fusion processing.

[0095] In this specific embodiment, S2 specifically refers to:

[0096] Frame-level luminance timing is constructed based on aligned image frames and timestamps, and camera timing estimation is performed. First, the initial frame rate is calculated based on the time difference between adjacent frames under a global time base, using:

[0097] Obtain the frame rate estimate;

[0098] in For the estimated camera frame rate, For the number of frames used for statistics, For the first global time base Frame timestamp;

[0099] Subsequently, frequency domain analysis was performed on the frame-level luminance timing to estimate the frame exposure time. The time modulation transfer characteristics of the exposure rectangular window were utilized, and the spectral amplitude ratio was analyzed. Push ;

[0100] in For the initial estimation of frame exposure time, The amplitude spectrum of brightness time series, For spectral amplitude, For normalization The inverse function of a function;

[0101] Within the same frame, brightness stripe features are extracted along the row direction and the row period of the stripes is estimated. The row spacing is first obtained from the autocorrelation main peak of the row brightness sequence. The row scan period is then obtained from the time-domain main frequency. ;

[0102] in The average row spacing of the stripes in the row direction. The autocorrelation function of the row-oriented brightness sequence, The estimated value of the row scan cycle. The main peak frequency other than the DC component of the brightness timing sequence;

[0103] A line-level timing model for the rolling shutter is established to uniformly describe the relationship between frames, lines, and phase. The following approach is adopted:

[0104] Depicting the first Frame number The moment the line begins to be exposed;

[0105] in For the start of exposure time, For reference start time, For frame period, For rolling shutter phase, For row index;

[0106] Combining the continuity of multi-frame fringe phase with The estimated value is obtained by performing least squares fitting.

[0107] In a common rolling shutter model, the downlink exposure time is approximately equal to the frame exposure time, therefore we take... As an initial estimate of the line exposure time, where This is an estimate of the exposure time.

[0108] When the acquired dataset contains camera metadata, the temporal fields in the metadata are read to perform consistency calibration on the above estimates, and weighted fusion is then employed. Obtain the final parameters;

[0109] in For the calibrated parameters, For time series analysis-based estimation, Parameters given for metadata and The confidence weights for the two types of information are respectively. Represents any timing parameter being fused;

[0110] The final output timing parameter set includes the camera frame rate. Frame exposure time Exposure time Row scan cycle Phase with rolling shutter This data, along with the collected dataset, is provided for use in subsequent steps.

[0111] In this specific embodiment, S3 specifically refers to:

[0112] Using camera timing parameters as input, read the camera frame period. Frame exposure time Exposure time Row scan cycle Phase with rolling shutter And on the same timeline, a safe exposure sub-window aligned with the line scan phase boundary is constructed for each frame. First, the line scan phase boundary sequence of that frame is defined on the timeline:

[0113] ;

[0114] in Indicates the first The frame is scanned in the row with the phase of the first line. The time of each boundary moment For reference start time, For frame indexing, For integer phase index, For camera frame period, For rolling shutter phase, The row scan cycle;

[0115] To ensure that the sub-windows are aligned with the phase boundaries and cover the effective exposure sweep range of the frame, the number of sub-windows is calculated. ,in For the first Number of safe exposure sub-windows per frame For the floor operator, This is the frame exposure time for that frame;

[0116] set up Let the minimum phase boundary index fall at the effective exposure start point of this frame be the index of the first phase boundary. The first frame Each safe exposure sub-window is defined as:

[0117] ;

[0118] in Sub-windows in the form of time intervals In order to be with the first The phase boundary start index corresponding to each sub-window For sub-window indexing, the above formula ensures that the start and end times of each sub-window strictly fall on the line scan phase boundary and correspond to the pixel rows that are in the exposure state during that time period;

[0119] To maintain a constant imaging light intensity within each sub-window, a reference brightness for the constant current steady-state light segment is configured for each sub-window:

[0120] ;

[0121] in For the first Frame number Brightness setting of the child window and The brightness adjustment coefficient obtained after calibration. In order to be with the first Ambient illuminance measurements in the frame time neighborhood For frame exposure time, For reference exposure time constant;

[0122] Arrange each frame's sub-window and its corresponding brightness in chronological order to form a gating parameter set:

[0123] ;

[0124] in For the first The gating parameter set of a frame is further used to obtain the total gating parameter set across frames. ,in This is a set of global gating parameters used to drive the constant current steady-state optical segment output and align it with the camera trigger. The total number of frames involved in the orchestration;

[0125] This enables micro-window segmentation based on the line scan phase boundary, constant brightness control within sub-windows, and cross-frame timing arrangement for subsequent scheduling and dimming steps.

[0126] In this specific embodiment, S4 specifically refers to:

[0127] This embodiment uses a gating parameter set as input, reads the spatiotemporal features of the visual image sequence and the time-frequency features of the wing vibration acoustic signal on the same time axis, and normalizes the amplitude or confidence level by combining environmental parameters. This ensures that the features of different modes remain comparable and robust under changes in illumination, temperature, humidity, and background noise. Subsequently, the probability of the target pest's presence and the fusion confidence level are calculated at the aligned frame-level time, and the number and duration of the safe exposure sub-windows are adaptively adjusted accordingly. At the same time, the start and end times of the sub-windows are strictly aligned with the phase boundary of the line scan cycle to prevent out-of-bounds movement and drift. The probability of the target pest's presence is fused using logistic regression.

[0128] Make an estimate;

[0129] in For the first The estimated probability of the presence of the target pest in the frame. For the Sigmoid logical function, For the first Frame-normalized visual spatiotemporal feature vectors In order to be with the first Frame-aligned normalized acoustic time-frequency eigenvectors In order to be with the first The frame-aligned environmental parameter vector must include at least one of illumination and temperature / humidity. and These are the fusion weight vectors for visual and acoustic modalities, respectively. For the fusion weight vector of environmental parameters, For bias terms;

[0130] The fusion confidence score is expressed as a weighted sum of modal quality metrics as follows:

[0131] ;

[0132] in For the first Frame fusion confidence, For the first Quality metrics for frame-based visual modalities, such as signal-to-noise ratio or detection stability, In order to be with the first Frame-aligned acoustic modal quality metrics such as peak sharpness or energy concentration and These are the weighting coefficients for the corresponding modal quality metrics;

[0133] in accordance with and For the first The frame's safe exposure sub-window is adaptively adjusted and kept aligned with the line scan phase boundary. The adjustment rule is written as follows:

[0134] ;

[0135] in For the first Number of child windows after frame adjustment For the first Initial number of child windows per frame The step size for each quantity adjustment For the sign function in Take +1 at time Take -1 at time Time to take To integrate confidence thresholds, For the first Frame number Duration after adjustment of individual sub-windows For the first Frame number The initial duration of each sub-window, Relative step size for duration adjustment Index for child windows;

[0136] Specifically, the execution is as follows: Below the threshold To enhance imaging robustness and sampling coverage, the number of sub-windows can be increased or the duration extended. Above the threshold To reduce the number of sub-windows or shorten their duration, the induced time slots outside the sub-windows can be released, thus improving energy efficiency, while also constraining... and Within the allowable range, and ensuring that the start and end times of the sub-windows fall on the scanning phase boundaries of adjacent rows, the final adjusted number of sub-windows, duration, and corresponding... and Arrange them in chronological order to form a scheduling parameter set.

[0137] In this specific embodiment, S5 specifically includes:

[0138] Using scheduling parameters as input, read the safe exposure sub-window for each frame. Its start and end times and duration, and the corresponding constant current steady-state brightness Probability of the presence of target pests With fusion confidence ,in For frame indexing, Indexing child windows For the first frame A safe exposure sub-window aligned with the line scan phase boundary; In order to be in The constant brightness setting maintained internally For the first Frame target pest presence probability, For the first Frame fusion confidence;

[0139] Simultaneously, camera timing parameters are read to construct a frequency restriction set and a safe frequency band, and phase locking and duty cycle adaptive settings are performed. The frequency restriction set is written as follows:

[0140] and ;

[0141] in To disable frequency sets, For camera frame rate estimation, For row scan frequency, For row scan cycle estimate, It is a set of positive integers;

[0142] Considering sampling jitter and synchronization error, a guard band is set for each frequency-forbidden point:

[0143] And define the safe frequency band ;

[0144] in For frequency points The central protective belt, For the width of the protective belt, For the set of available frequency bands, For LEDs and the highest operating frequency allowed by the driving capability, This is a set difference operation;

[0145] Within the safe frequency band, select the operating frequency that has the largest distance from the frequency restriction set for the induction pulse:

[0146] and ;

[0147] in To induce pulse operating frequency, The minimum distance function from a point to a set. To select the independent variable that maximizes the objective function;

[0148] To balance attraction intensity and imaging stability, the duty cycle is set based on the target presence probability and fusion confidence:

[0149] and , ;

[0150] in To induce pulse duty cycle, and These are the lower and upper limits of the duty cycle, respectively. Duty cycle magnification factor, For truncation functions, In order to plan the frame set The average probability of the existence of the target on the surface For average fusion confidence, For the set of frames participating in duty cycle calculation, for The number of elements;

[0151] To achieve phase locking, the inducing pulse strictly avoids all safe exposure sub-windows; the pulse period and related time set are defined, and the phase is optimized.

[0152] and ;

[0153] in To induce pulse cycle, The union of the times of all safe exposure sub-windows, For a given phase Union of high-level pulse times For pulse sequence number, To lock the phase, This is a time-based measurement function used to measure the duration of overlap.

[0154] Finally in Internal Press Output constant current steady-state optical segment and with Maintain constant brightness, In addition and Output inducing pulses and form a set of dimming parameters for performing closed-loop updates.

[0155] In this specific embodiment, S6 specifically refers to:

[0156] A lighting parameter set drives the light source and works in conjunction with the camera on the same global timeline. The lighting parameter set is written as:

[0157] ;

[0158] in For dimming parameter set, The operating frequency of the inducing pulse, To induce the locking phase of the pulse, To induce pulse duty cycle, For the first Frame number A safe exposure sub-window aligned with the line scan phase boundary; In order to be in The constant current steady-state brightness setting value maintained internally. For frame indexing, Index for child windows;

[0159] To facilitate triggering data collection, the endpoints of the child window are represented as:

[0160] ;

[0161] in For the start time of the child window, This is the end time of the child window;

[0162] Periodic writing of phase-locked pulses ,in To induce the pulse period, define the union of all safe exposure sub-windows on the time axis. High-level time set of the induction pulse ;

[0163] in For the time set of the exposure sub-window, For a given phase The time set of the high level of the lower pulse. For pulse sequence number, It is a set of integers;

[0164] To ensure constant imaging light intensity within the sub-window and avoid pulse brightness fluctuations within the sub-window, the LED light intensity output is controlled in segments according to the indication function as follows:

[0165] ;

[0166] in For a moment Light intensity For the indicator function in Take 1 if it is true, otherwise take 1. The brightness setting value for the high-level pulse. This is a set difference operation;

[0167] Data acquisition is triggered at the beginning of each sub-window and ensures that data acquisition occurs within the constant current steady-state segment. The trigger time is written as follows. ,in For the first Frame number The image frame acquired at the time the acquisition is triggered by the sub-window is denoted as... And record timestamps and window identifiers, and write the imaging dataset:

[0168] ;

[0169] in For imaging datasets, For the first Frame number The window identifier of the child window;

[0170] The above process ensures that the light source is in Internally, only a constant current steady-state optical segment is output, and externally... , and Output induction pulse, camera only The system acquires image frames and forms an imaging dataset containing image and timing control information for subsequent recognition and linkage control.

[0171] In this specific embodiment, S7 specifically refers to:

[0172] Using the imaging dataset as input, organize each image frame on the same time axis. and its timestamp With window identifier ,in Indicates the first Frame number Images captured within each safety exposure sub-window For the corresponding data collection trigger time, Used as a unique identifier for the window;

[0173] For each The detector is run to obtain a set of candidate targets and its spatiotemporal features are extracted. The candidate targets are represented by triples. It means that among them For the first Candidate bounding boxes, For category labels, The confidence level that the target exists;

[0174] To complete the quantity statistics and obtain the recognition results bound to the window, the target set is filtered and counted according to a threshold and the target class set. The counting formula is as follows:

[0175] ;

[0176] in For the first Frame number Target number of child windows The indicator function takes the value 1 if the condition is met, otherwise takes the value 0. To detect confidence threshold, A pre-defined set of target pest categories;

[0177] To improve statistical stability by performing target association and trajectory maintenance between adjacent image frames, a cost-based approach using overlap and geometric consistency is employed for matching. The cost function is as follows:

[0178] ;

[0179] in To make the current candidate With existing trajectory The cost of matching For the intersection and comparison of two frames, The weighting coefficient for center offset for center coordinate vector For trajectory In the predicted bounding box of the current frame, for center coordinate vector It is the Euclidean norm;

[0180] Based on this, the Hungarian algorithm is used to solve the minimum cost matching problem to update the trajectory and deduplication statistics;

[0181] Based on identification and quantity statistics, a trapping control command is generated and bound to a timestamp and window identifier. This control command is denoted as... It includes four items: switch, start time, intensity, and duration. The intensity and duration are set with saturation constraints based on statistics as follows:

[0182] ;

[0183] in For the intensity setting of the baiting device, Duration of the baiting effect, and These are the lower and upper limits of intensity, respectively. and These are the lower and upper limits of the duration, respectively. and Gain coefficients for intensity and duration, To truncate the function Restricted to Inside;

[0184] Will Send the command to the decoy device and record the execution status. With kill results ,in This indicates whether the instruction was executed on time and provides execution feedback information. This indicates the actual number of kills or kill event labels within the window where the command is applied, and these are ultimately aggregated to form the baiting and killing results dataset. And retain on the same timeline The corresponding relationships are used for subsequent closed-loop updates.

[0185] In this specific embodiment, S8 specifically refers to:

[0186] Using the baiting and killing result dataset as input and performing statistics and parameter updates on the same time axis, the first step is to use the window index set. Organize data, among which For the set of indices of all windows, any Mapped to To identify the first Frame number Sub-window, read the number of recognitions With the number of kills And define the identification statistics and trapping results as follows: and Based on this, the matching degree between identification and trapping is given. ;

[0187] in For window Matching degree To avoid small positive quantities with a denominator of zero;

[0188] Then, the fusion weights are adaptively updated based on the matching degree, making and respectively with For the indexed sequence of visual and acoustic quality metrics, where and and They represent the first The visual and acoustic quality metrics of a frame, assuming correlation. and The weights were obtained using soft normalization. and ;

[0189] in and For the updated visual and acoustic fusion weights, Update the sensitivity coefficient for weights. For Pearson correlation coefficient, For exponential functions, For The sequence of matching degrees for the index;

[0190] Without altering the phase boundary alignment between the safe exposure sub-window and the row scan cycle, the duration of the sub-window is fine-tuned while maintaining row-level quantization. This can be written as:

[0191] and ;

[0192] in For the updated duration, For the duration before the update, For duration adjustment coefficient, For the target threshold of matching degree, For row scan cycle estimate, Quantization operators for phase alignment and These are the lower and upper limits of the duration, respectively. This is the floor operator;

[0193] To fine-tune the dimming parameters while satisfying the frequency-forbidden set and phase-locked constraints, the average matching degree is calculated:

[0194] It also provides updates to the frequency and duty cycle. and At the same time, the phase remains unchanged. ;

[0195] in and For the updated operating frequency and duty cycle, and For the working frequency and duty cycle before the update, and For fine-tuning the step size of frequency and duty cycle, and For the corresponding target matching degree, For average matching degree, Projection operators are used to limit frequencies to a safe bandwidth. Inside, To truncate the function Limited to Inside, and These are the updated phase and the original phase, respectively, and the final fusion weights are then calculated. Updated child window duration set With dimming parameters Arranged in chronological order to form an updated time series parameter set. And in subsequent loops This serves as the input for step S2 to achieve closed-loop control.

[0196] Example 1:

[0197] This embodiment demonstrates the deployment of the method of this invention in a multi-span greenhouse at a vegetable demonstration base in Funing County, Yancheng City, Jiangsu Province. The main crop was greenhouse peppers, with bok choy used as a control in the same greenhouse. Regional pest data were based on the 2023-2024 pest and disease monitoring summary from the Funing County Plant Protection and Quarantine Station and the technical recommendations in the "Plant Protection Information Issue 12" published in 2025: Thrips were prevalent and caused severe damage to solanaceous crops from June to September; the second generation of beet armyworm adults appeared early with high numbers, and the peak period for young larvae was from late June to early July; whiteflies peaked in early August to early September and then declined as temperatures decreased; leaf miners peaked in May and late August; and the yellow-striped flea beetle was predominantly prevalent in spring. The base has set up 30-40 mesh insect-proof nets in accordance with green prevention and control standards. Inside the greenhouse, 20 blue / yellow sticky traps per mu are hung for monitoring and peak reduction. Outside the greenhouse, a solar-powered insecticidal lamp is installed in a continuous area. Micro-irrigation is used under the film of the peppers to control humidity. The requirements of the county station regarding the prohibition and restriction of pesticides and the safety interval period are implemented.

[0198] The hardware system includes a rolling shutter CMOS camera (1920×1080, base 30 fps, auto exposure 8-14 ms, output timestamp and exposure metadata), a multi-channel LED array (365-405 nm and 450 nm dual channels, 20 W constant current drive, supporting 0.2-2.0 kHz high-speed dimming and phase locking), a wing vibration sound acquisition microphone (48 kHz), illuminance and temperature / humidity sensors (1 Hz sampling), and a high-voltage grid-type trapping execution unit (peak voltage ≥3.5 kV). The main controller is an integrated MCU+FPGA controller, providing a globally unified time base to synchronize the camera, LEDs, and data recording, and interfaceing with insect pest data from field blue boards, pheromone traps, and insecticidal lamps.

[0199] During operation, the system first synchronously collects multimodal data: the camera continuously outputs image frames and records timestamps and exposure times, the microphone collects wing vibration sound at 48 kHz, and the environmental sensor collects illuminance and temperature and humidity every second; the three data are aligned to the global time axis through linear clock drift calibration, and frame drops, audio interruptions and sensor anomalies are marked but the original amplitude and timing are not changed, so as to preserve the real features required for subsequent estimation and fusion.

[0200] Frame-level luminance timing is constructed based on the aligned image and timestamp. The frame rate and initial frame exposure are obtained by combining frequency domain analysis (the measured frame rate during the experimental period was 29.97±0.02 fps, and the exposure was 9.6-12.8 ms). Luminance fringes are extracted along the line direction within a single frame, the line scan period is calculated, and the rolling shutter phase is fitted (the line period is about 30±2 μs, corresponding to a line frequency of ≈33.3 kHz). The above estimates are calibrated for consistency with the camera metadata fields, and the frame rate, frame exposure, line exposure, line scan period, and rolling shutter phase are output as a timing parameter set.

[0201] Based on the timing parameter set, the effective exposure window of each frame is divided into sub-windows according to the line scanning phase boundary, which are strictly aligned with the line phase. (In this example, 8 sub-windows are initially divided at 1.4 ms, and the endpoints are quantized with the line period to avoid phase drift.) The reference brightness of the constant current steady-state light segment is configured in each sub-window. The brightness is adaptively set according to the illuminance and frame exposure, so that the imaging light intensity fluctuation during the exposure stage is ≤±1%, providing stable feature quality for subsequent recognition.

[0202] In the fusion recognition and scheduling stage, the visual side extracts the spatiotemporal features of the target (detection and tracking), and the acoustic side extracts the time-frequency features (spectral peak energy, bandwidth, and stability). These features are then normalized by combining environmental parameters with amplitude or confidence levels, outputting the probability of the target pest's presence P and the fusion confidence level C. The system adaptively adjusts the safe exposure sub-window based on C: when C is below a threshold (e.g., 0.6), the number of sub-windows is increased or the duration of a single window is extended to 1.6 ms to improve imaging robustness and sampling coverage; when C is above a threshold, the number of sub-windows is reduced or the duration of a single window is shortened to 1.2 ms to release more time slots for the inducing pulse. Throughout the adjustment process, the start and end times of the sub-windows are always strictly aligned with the line scan phase boundary.

[0203] To avoid interference between illumination and rolling shutter, the system constructs a frequency ban set, which includes at least the neighborhood of frequency points related to integer multiples of the camera frame rate and frequency points related to line scan harmonics, and determines a safe frequency band within the usable range of 0.2-2.0 kHz. The operating frequency of the inducing pulse is selected from the frequency point with the largest distance from the frequency ban set, and the phase of the inducing pulse is locked outside the safe exposure sub-window to ensure that the brightness of the exposure segment does not fluctuate. During the summer thrips stage, 450 nm blue light induction is the main method, with the operating frequency set at 600 Hz (far from integer multiples of 30 Hz), and the duty cycle adaptively adjusted between 10-40% according to P and C. When the peak of beet armyworm adults is identified (consistent with sex induction and acoustic induction), the 365-405 nm ultraviolet channel is superimposed and time-division multiplexed, with an operating frequency of 1.0 kHz, and the duty cycle and phase locking are the same as before.

[0204] LED dimming and camera acquisition work together on the same timeline: within the safe exposure sub-window, only the constant current steady-state light segment is output and acquisition is triggered; outside the sub-window, an inducing pulse is output, and pulses are strictly not output within the exposure segment; the system packages the window identifier and dimming timing into the imaging dataset for identification and control of closed-loop calls.

[0205] In terms of identification, statistics, and control linkage, the system performs pest detection, target tracking, and window-level quantity statistics on image frames. When thrips appear densely on flowers and tender shoots, the system increases the trapping intensity and optimizes the execution timing (applying in the evening or early morning, avoiding periods with temperatures above 30°C). For noctuid moths such as the beet armyworm, the system combines the prediction window for second-generation young larvae, increases the duty cycle of the attraction pulse and the power grid at night, while maintaining phase avoidance. Control commands (switching, timing, intensity, and duration) are issued along with the window identifier, and the execution status and killing results are recorded to form a trapping result dataset.

[0206] Closed-loop updates adjust fusion weights and dimming parameters by identifying the matching degree between statistics and kill results: when the humidity in the studio is high and C fluctuations intensify, the system prioritizes extending the duration of the sub-window and reducing the duty cycle to stabilize imaging; without violating the row phase alignment and frequency ban constraints, the system fine-tunes the induction frequency and duty cycle to enable the system to adapt to different illumination, frame rates and environmental conditions, balancing induction effect and energy consumption.

[0207] The parameters are localized and coordinated at the county level as follows: The key operating window is set from early June to early July (peak thrips, beet armyworm second-generation larval stage), and secondary optimization is performed in September for the whitefly peak; the attraction spectrum and frequency band are "thrips mainly at 450nm@600 Hz, beet armyworm superimposed at 365-405 nm@1.0 kHz time-division multiplexing", with phase locking throughout; the sub-window strategy is "initially 8 per frame (1.4 ms), expanding to 10 or 1.6 ms per window when C is low, shrinking to 6 or 1.2 ms per window when C is high"; the frame rate harmonic protection bandwidth is ±5 Hz, all line scan related frequency points are excluded, and the frequency search step is 5. Hz; The system works in conjunction with blue / yellow boards, pheromone traps, and insecticidal lamps: reading the board and the amount of pheromone traps captured serves as preliminary verification data to improve P and C stability; when the combined insecticidal lamps are in the peak moth emergence period, this system prioritizes nighttime attraction and reduces daytime occupancy to minimize non-target interference and energy consumption; regarding the application window, when the system continuously identifies the peak larval stage and the attraction is insufficient to reduce the adult peak, it suggests using biological or low-toxicity agents such as NPV, Bt, and ethyl spinosad during the early larval stage (refer to "Plant Protection Information Issue 12"), and lowering the attraction intensity 24 hours after application to prevent pesticide spray from affecting imaging.

[0208] Typical two-week operation comparison results show that: after introducing the "row-level phase alignment sub-window + frequency ban and phase locking" of the present invention, the proportion of brightness stripes / aliased frames decreased from about 18.7% in the control (fixed frequency 200 Hz, no phase coordination) to 0.9%; the recall rate of thrips and cutworm adults was improved by about 13-18%, and the number of target tracking ID switching times decreased by about 22%; under the same energy consumption, the number of insects killed per unit time was increased by about 21-29%, and the adaptive duty cycle strategy brought 15-20% energy saving; when used in conjunction with blue sticky traps, insecticidal lamps, and pheromone traps, the peak duration of adult insects in the greenhouse was shortened by 1-2 days, and the downward trend of the peak number of second-generation larvae was consistent with the insect situation monitoring at the county station.

[0209] Table 1 shows the comparative evaluation results of thrips and beet armyworm second-generation high incidence period for 14 consecutive days in a multi-span greenhouse pepper field in Funing County, Jiangsu Province. The control group consisted of fixed frequency lighting, no row-level phase coordination, and single visual recognition. The present invention adopts "safe exposure sub-window + frequency restriction safe band + phase locking + multimodal fusion and closed-loop update". The results show that the imaging stability, recognition accuracy and trapping efficiency are significantly improved, while energy consumption and pesticide intervention decrease simultaneously.

[0210] Table 1. Comparison of Effect Evaluation

[0211] index Statistical scope / unit Before deployment (comparison) After deployment (this invention) Increase / Change Brightness stripes / aliasing frame ratio Percentage of all captured frames / % 18.7% 0.9% -95.2% Illuminance fluctuation during exposure Light intensity fluctuation during constant current steady-state period (±%) ±6.5% ±0.9% -86.2% Detection recall rate (adult thrips) Annotated set evaluation / recall R 0.78 0.91 ↑0.13（+16.7%） Detection recall rate (adult noctuid moths) Annotated set evaluation / recall R 0.74 0.87 ↑0.13（+17.6%） False positive rate (FP / hour) Identify false alarm events / hour 7.2 4.8 -33.3% Tracking stability ID switching counts per hour 4.5 3.5 -22.2% lure and kill efficiency Actual kills per hour (kills per hour) 41 52 +26.8% Lure and kill efficiency Energy consumption per unit of baiting and killing (Wh / unit) 1.18 0.95 -19.5% Peak duration of adult insects Duration of a single insect peak (days) 4.0 2.6 Shortened by 1.4 days (-35%) Intervention medication frequency (two weeks) Number of biological / low-toxicity drug interventions (times) 2 1 -50%

[0212] This embodiment can be adapted to year-round production of Chinese cabbage in multi-span greenhouses. Given that May-July and September-November are peak periods, the S1-S8 process can be mirrored, and the monitoring signals from insect nets and yellow / blue sticky traps can be incorporated into the fusion confidence quality metric. The focus is on stabilizing imaging during the infestation periods of the yellow-striped flea beetle and leaf miner, using 450 nm attraction with strict phase locking, and arranging data collection during the low-temperature, low-humidity early morning and evening hours to ensure synergistic effects of identification and trapping. Through collaboration with the existing green pest control system in Funing County (insect nets, insecticidal lamps, color traps, pheromone traps, biological pesticides, and scientific pesticide application), this invention achieves a comprehensive technical effect of "stable imaging - accurate identification - efficient trapping - improved energy efficiency" in real-world greenhouse vegetable production. The above parameters and timing settings are optional implementation schemes, allowing for engineering adjustments based on different greenhouse types, crops, and pest infestations, without deviating from the core technical concept of "safe exposure sub-window + restricted frequency band + phase locking + adaptive fusion confidence + closed-loop update."

[0213] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0214] This application constructs an overall link around the technical problem of "stable imaging and induction without interference": by synchronously collecting and fusing visual, wing vibration acoustic and environmental parameters, camera timing estimation is performed, and the frame exposure is divided into safe exposure sub-windows by scanning phase alignment by line. The constant current steady-state light segment is output within the sub-window, and the induction pulse is output under phase locking under frequency restriction constraints outside the sub-window.

[0215] The above combination eliminates brightness stripes and frequency aliasing from both the sampling timing and the light source driving ends, so that the light intensity is stable and the image features are not distorted during the exposure. The sub-window and duty cycle are adaptively scheduled according to the probability of the presence of the target pest and the fusion confidence, so as to balance the identification accuracy and the killing efficiency, forming a technical effect of "stable imaging-precise identification-efficient control".

[0216] In terms of algorithm structure, this case proposes improvements oriented towards timing and phase to address the root cause of rolling shutter interference: establishing a timing parameter set and a gating parameter set, and adopting micro-window segmentation and gating output that are strictly aligned with the line scanning period; introducing a frequency ban set composed of integer multiples of the frame rate and line scanning harmonics to determine the safe frequency band, so that the frequency and phase of the inducing pulse are locked outside the safe exposure sub-window; and continuously optimizing the fusion weight and dimming parameters through adaptive scheduling with fusion confidence as the core and closed-loop updates driven by the inducing result, so that the system can maintain imaging stability and energy efficiency improvement under different illumination, frame rate and environmental conditions, thus structurally ensuring the stable realization of the above technical effects.

Claims

1. A method for intelligent pest trapping based on multimodal fusion recognition and variable frequency dimming, characterized in that, include: S1. Collect multimodal data, obtain visual image sequences, wing vibration acoustic signals and environmental parameters to form a collection dataset; S2. Input the collected dataset, perform camera timing estimation, and obtain the camera frame rate, frame exposure time, line exposure time, line scan period and rolling shutter phase to form a timing parameter set; S3. Input the timing parameter set, divide each frame exposure window into a safe exposure sub-window by aligning the line scan, and configure the constant current steady-state light band reference brightness for each sub-window to form a gating parameter set; S4. Input the gating parameter set, perform multimodal fusion recognition, calculate the probability of the target pest's existence and the fusion confidence, and adaptively adjust the number and length of the safe exposure sub-windows based on the fusion confidence and the probability of the target pest's existence to form a scheduling parameter set; S5. Input the scheduling parameter set, construct the frequency ban set and determine the safe frequency band, determine the frequency, phase and duty cycle of the inducing pulse within the safe frequency band, lock the phase of the inducing pulse outside the safe exposure sub-window, and simultaneously arrange the brightness and timing of the constant current steady-state light segment to form a dimming parameter set; S6. Input the dimming parameter set, output the constant current steady-state light segment in the safe exposure sub-window according to the time sequence, output the inducing pulse outside it, and acquire image frames in the safe exposure sub-window to form an imaging dataset. S7. Input the imaging dataset, identify and count the target pests, generate and issue baiting control commands, record the execution status and killing results, and form a baiting result dataset. S8. Input the baiting result dataset, update the calculation parameters of the fusion confidence, and fine-tune the duration and dimming parameters of the safe exposure sub-window to form an update time series parameter set for subsequent loops.

2. The intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming according to claim 1, characterized in that, S1 specifically refers to: The camera continuously captures visual image sequences and records the timestamp, frame number, and exposure time of each image frame. Acoustic signals of wing vibration were collected using a microphone, and the start and end times and sampling rate were recorded. Environmental parameters are collected using environmental sensors, including at least one of illuminance, temperature, and humidity, and the sampling time and measured values ​​are recorded. The visual image sequence, the wing vibration acoustic signal, and the environmental parameters are synchronized in time, and the three data streams are aligned to the same time axis and a corresponding relationship is established. Perform basic quality control and integrity verification, and mark frame loss, audio interruption and sensor anomaly without changing the original timing and amplitude characteristics. The data acquisition dataset consists of aligned visual image sequences, aligned wing vibration acoustic signals, and aligned environmental parameters.

3. The intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming according to claim 1, characterized in that, S2 specifically refers to: Using the collected dataset as input, with the visual image sequence as the main component and combined with the timestamps in the visual image sequence, a frame-level brightness temporal sequence is established; The frame-level brightness timing is subjected to temporal correlation and frequency domain estimation to obtain initial estimates of the camera frame rate and frame exposure time; Brightness stripe features are extracted along the line direction within the same image frame. The line exposure time and line scan cycle are calculated based on the relationship between stripe spacing and stripe phase over time. The rolling shutter phase is then fitted by the continuity of stripe phases across multiple frames. When the acquired dataset contains camera metadata, the timing field in the camera metadata is read and calibrated for consistency with the initial estimate to improve the estimation accuracy of the camera frame rate, the frame exposure time, the line exposure time, the line scan period and the rolling shutter phase; The camera frame rate, frame exposure time, line exposure time, line scan period, and rolling shutter phase are output and combined with the acquired dataset to form the timing parameter set.

4. The intelligent pest trapping method based on multimodal fusion recognition and frequency conversion dimming according to claim 1, characterized in that, S3 specifically refers to: Using the time series parameter set as input, read the camera frame rate, the frame exposure time, the line exposure time, the line scan period, and the rolling shutter phase; Based on the line exposure time, the line scan period and the rolling shutter phase, the time range of each frame exposure window on the time axis is determined, and each frame exposure window is divided into several safe exposure sub-windows aligned with the line scan, so that the start and end times of each safe exposure sub-window are aligned with the phase boundary of the line scan period and cover the pixel rows in the exposure state within its corresponding time period. For each safe exposure sub-window, a reference brightness of the constant current steady-state light segment is configured. The reference brightness is set according to at least one of the environmental parameters included in the time sequence parameter set, namely illuminance or frame exposure time, so as to keep the imaging light intensity constant within the safe exposure sub-window. The number of safe exposure sub-windows, start and end times, duration, and reference brightness of the corresponding constant current steady-state light segment for each frame are arranged in chronological order to form a gating parameter set, which includes the chronological parameter set.

5. The intelligent pest trapping method based on multimodal fusion recognition and frequency conversion dimming according to claim 1, characterized in that, S4 specifically refers to: Using the gating parameter set as input, read the visual image sequence, wing vibration acoustic signal, environmental parameters, and the number, start and end times, and duration of the safe exposure sub-windows contained in the gating parameter set; The visual image sequence and the wing vibration acoustic signal are time-aligned on the same time axis. The spatiotemporal features of the visual image sequence and the time-frequency features of the wing vibration acoustic signal are extracted respectively. The amplitude or confidence level of the spatiotemporal features and the time-frequency features are normalized based on the environmental parameters. Using the normalized spatiotemporal features and the time-frequency features as inputs, the probability of the existence of the target pest and the fusion confidence are estimated. Using the fusion confidence and the probability of the presence of the target pest as input, the safe exposure sub-window of each frame is adaptively adjusted. The adaptive adjustment includes at least increasing the number of safe exposure sub-windows or extending the duration of the safe exposure sub-windows when the fusion confidence is below a threshold, and decreasing the number of safe exposure sub-windows or shortening the duration of the safe exposure sub-windows when the fusion confidence is above a threshold, while keeping the safe exposure sub-windows aligned with the phase boundary of the row scan cycle. The adjusted number of safe exposure sub-windows, start and end times and durations, as well as the probability of the presence of the target pest and the fusion confidence level are arranged in chronological order to form a scheduling parameter set, which includes the gating parameter set.

6. The intelligent pest trapping method based on multimodal fusion recognition and frequency conversion dimming according to claim 1, characterized in that, S5 specifically refers to: Using the scheduling parameter set as input, the number of safe exposure sub-windows, start and end times and durations, as well as the probability of the target pest's presence and the fusion confidence level are read. Based on the camera frame rate, the line scan period, and the sampling timing determined by the frame exposure time, the line exposure time, and the rolling shutter phase, a frequency ban set is constructed, such that the frequency ban set contains at least frequency points related to integer multiples of the camera frame rate and frequency points related to the sampling harmonics corresponding to the line scan period; The safe frequency band is determined by using the aforementioned frequency ban set as a constraint; The operating frequency, phase, and duty cycle of the inducing pulse are calculated within the safe frequency band. The duty cycle is adaptively adjusted based on the probability of the presence of the target pest and the fusion confidence level. At the same time, the phase of the inducing pulse is locked and controlled so that the phase of the inducing pulse is arranged outside the safe exposure sub-window to avoid brightness fluctuations within the safe exposure sub-window. The brightness and timing of the constant current steady-state light segment are arranged according to the start and end times and duration of the safe exposure sub-window, so that the brightness remains constant within each safe exposure sub-window. The operating frequency, phase, duty cycle, and brightness and timing of the constant current steady-state light segment are arranged in chronological order to form a dimming parameter set, which includes the scheduling parameter set.

7. The intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming according to claim 1, characterized in that, S6 specifically refers to: Using the dimming parameter set as input, the light source and the camera work on the same time axis; According to the start and end times and duration of the safe exposure sub-window, the constant current steady-state light segment is output within each safe exposure sub-window and the brightness is kept constant. The inducing pulse is output between each safe exposure sub-window. The operating frequency, phase and duty cycle of the inducing pulse are consistent with the dimming parameter set, and the inducing pulse is not output within the safe exposure sub-window. Using the start and end times of the safe exposure sub-window as the acquisition trigger times, image frames are acquired within the safe exposure sub-window, and timestamps and window identifiers are recorded to form an imaging dataset containing the image frames and the dimming parameter set.

8. The intelligent pest trapping method based on multimodal fusion recognition and variable frequency dimming according to claim 1, characterized in that, S7 specifically refers to: Using the imaging dataset as input, the image frames, their timestamps, and window identifiers are read, and the image frames are organized on the same timeline; Spatiotemporal features are extracted from the image frames to determine the category, location, and range of the target pests in the image frames. Target tracking is performed based on the correlation between adjacent image frames, and the number statistics of each window are summarized to obtain the identification results and number statistics. Using the identification results and the quantity statistics as input, a trapping control command is generated. The trapping control command includes at least one of the following: switching on / off, timing or intensity control of the trapping device, and establishes a binding relationship with the corresponding timestamp and window identifier. The baiting control command is sent to the baiting device, and the execution status and kill result of the baiting device are recorded to form a baiting result dataset containing the identification result, the quantity statistics, the baiting control command, the execution status and the kill result.

9. The intelligent pest trapping method based on multimodal fusion recognition and frequency conversion dimming according to claim 1, characterized in that, S8 specifically refers to: Using the baiting result dataset as input, the identification result, the quantity statistics, the baiting control command, the execution status, and the kill result are read; The identification results and the quantity statistics are summarized on the same timeline to form identification statistics, and the execution status and the kill results are summarized on the same timeline to form baiting and killing results; Based on the correspondence between the identification statistics and the trapping results, the calculation parameters of the fusion confidence are updated so that the weights of the fusion confidence on the visual image sequence and the wing vibration acoustic signal are adaptively adjusted according to the correspondence. Without changing the phase boundary alignment between the safe exposure sub-window and the line scan cycle, the duration of the safe exposure sub-window is finely adjusted based on the identification statistics and the baiting results; Under the premise of satisfying the frequency ban set and the phase locking constraint, at least one of the working frequency, phase or duty cycle in the dimming parameters is fine-tuned according to the identification statistics and the trapping results; The updated calculation parameters of the fusion confidence, the duration of the safe exposure sub-window, and the dimming parameters are arranged in chronological order to form an updated timing parameter set. This updated timing parameter set is used as the input of S2 in subsequent loops to achieve closed-loop control.

Images