A centimeter-scale biological sound and light combined counting method and system under arctic ice

Abstract

The application provides a centimeter-level biological sound and light combined counting method and system under the Arctic ice, and relates to the field of in-situ detection technology of polar marine environment. The method comprises the following steps: establishing a multi-coordinate system synchronous time reference; defining a multi-layer observation area containing a virtual registration surface; wideband acoustic discovery of candidate targets, background filtering and effectiveness preliminary screening, fish / plankton double-object diversion; local high-frequency acoustic fine tracking and prediction of the probability of entering the optical registration volume, triggering low-disturbance optical registration; optical image preprocessing and segmentation evaluation, fusion of sound and light feature recognition; judging effective crossing with the virtual registration surface as the boundary, generating a cross-modal identity token for deduplication; and outputting the classification and counting results. The system corresponds to the implementation of the above method. The application improves the adaptability of the Arctic ice environment, eliminates duplicate counting, and realizes stable discovery, identification and unique counting of centimeter-level fish and plankton.

Description

Technical Field

[0001] This invention relates to the field of in-situ detection technology of polar marine environment, specifically to an acoustic and optical joint detection and counting method and system applicable to the Arctic subglacial marine environment, and particularly to an acoustic and optical joint detection and counting method and system for target detection, target identification and quantity counting of fish and plankton at the centimeter level and above. Background Technology

[0002] The Arctic subglacial environment is covered by sea ice, resulting in low temperatures, low light levels, and severe multipath reverberation, which limits communication and maintenance conditions. Fish larger than centimeters coexist with plankton (such as copepods and krill) in the subglacial water layer. Their small size and uneven distribution, coupled with the overlapping acoustic scattering characteristics of some fish and larger plankton, pose significant challenges to stable detection, accurate identification, and reliable counting.

[0003] Among existing underwater detection methods, single acoustic methods have a long detection range, but the echoes are easily affected by reflections from the ice-covered interface, reverberation, and non-biological scattering, making it difficult to accurately classify and count individuals. Single optical methods can acquire morphological information, but the line of sight under the ice is limited, continuous shooting consumes a lot of energy and generates a lot of invalid data, and low-frequency shooting is prone to missing detections. Existing acoustic-optical combined solutions, such as Chinese utility model patent CN212586555U (acoustic-optical integrated detection and assessment device for fish resources in marine ranches), Chinese invention patent application CN113475429A (fish farming monitoring system based on multiple acoustic-optical sensors and satellite communication), and Chinese invention patent CN113747064B (automatic underwater biological detection system triggered by sensors), are mainly aimed at aquaculture or ranching scenarios. When directly applied to the Arctic ice, they suffer from several problems, including insufficient handling of special conditions such as strong reflection under ice and low illumination during polar nights, failure to consider centimeter-sized plankton as effective detection targets, lack of close collaboration between acoustic and optical devices driven by long-distance detection and short-distance triggering, low hit rate due to optical triggering not utilizing acoustic prediction results, lack of a unified counting interface and cross-modal identity deduplication mechanism leading to duplicate counting and undercounting, and the lack of a complete technical process for each key link. These issues prevent the simultaneous and stable detection, reliable identification, and unique counting of fish and plankton in mixed scenarios.

[0004] Therefore, there is a need to provide a combined acoustic and optical detection and counting method and system for the Arctic subglacial marine environment to overcome the above-mentioned shortcomings. Summary of the Invention

[0005] This invention aims to provide a combined acoustic and optical detection and counting method and system for the Arctic subglacial marine environment, overcoming the problems of poor environmental adaptability, incomplete target coverage, weak acoustic-optical coordination, unreliable optical triggering, serious duplicate counting, and incomplete technical processes in existing technologies. Specifically, it needs to achieve stable detection, accurate identification, and reliable counting of fish and plankton at the centimeter level and above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A centimeter-scale acoustic-optical combined counting method for organisms under Arctic ice includes the following steps: Observation nodes were set up under the Arctic ice to establish a system reference coordinate system, acoustic coordinate system and optical coordinate system and to calibrate the external parameter transformation relationship. Acoustic transmission and reception, optical exposure and supplementary light pulses were synchronized to a unified time reference. Define the broadband acoustic detection region, the local acoustic tracking region, the optical registration volume, and the virtual registration surface as the counting interface; Broadband acoustic signals are periodically scanned in the broadband acoustic detection area. The echoes are then subjected to matched filtering and pulse compression to extract initial candidate targets that meet the preset signal-to-noise ratio and equivalent acoustic scale thresholds. Background filtering and validity screening are performed on the initial candidate targets, and non-static scattering targets that appear continuously for multiple periods and whose displacements meet the preset velocity constraints are retained as valid biological candidate targets. Based on acoustic scattering and motion characteristics, effective biological candidates are categorized into fish candidates, planktonic candidates, or unidentified biological targets. A high refresh rate rescan is performed on the local sector where each candidate target is located, which is higher than the broadband operating frequency. The position and velocity are estimated based on continuous position measurements using a Kalman filter, and the probability of entering the optical registration volume or the crossing time is predicted. When the probability reaches a preset threshold or the predicted trajectory intersects with the optical registration volume, a differentiated optical triggering plan is generated according to the candidate target category, triggering the camera to expose and opening short pulse supplementary light within the exposure window to obtain an optical image; The optical images are preprocessed and morphological features are extracted. The segmentation quality of the planktonic cluster candidates is evaluated. When it is determined that the individual organisms in the cluster cannot be reliably segmented, the adaptive degradation estimation mode is switched. The target category is output by fusing acoustic features, optical features, and spatial-temporal features, and the acoustic target intensity is corrected by attitude normalization using optical attitude information. Based on the three-dimensional trajectory of the target within the optical registration volume, determine whether it has effectively crossed the virtual registration surface, and include targets that meet the crossing conditions in the candidate set for deduplication crossing. For targets in the candidate traversal set, generate cross-modal identity tokens containing traversal time, spatial location, velocity, target scale, acoustic summary features, and optical summary features. Match the current token with historical identity tokens of the same category in the active identity cache table that are within a preset historical time window. If the matching similarity exceeds the threshold and meets the conditions for turning back or retracing, it is determined to be a duplicate occurrence; otherwise, it is determined to be a new target and a counting event is generated. The system summarizes valid count events within a preset statistical time window, classifies them into fish, individual plankton, plankton clusters, and unidentified biological targets, and outputs the classification count results. For plankton clusters in adaptive degradation mode, the system outputs the number of cluster events and local quantity density estimates, and adds a degradation mode flag.

[0007] On the other hand, the present invention also provides a centimeter-level acoustic-optical joint counting system for subglacial Arctic organisms, used to perform the above-mentioned centimeter-level acoustic-optical joint counting method for subglacial Arctic organisms, including: The broadband acoustic detection unit is used to transmit broadband acoustic signals and perform periodic scanning, perform matched filtering and pulse compression processing on the echoes, and extract initial candidate targets. The biological target screening unit is used to filter out background and screen the effectiveness of initial candidate targets to obtain valid biological candidate targets. The fish / plankton dual-object diversion unit is used to divert valid biological candidate targets into two objects, marking them as fish candidates, plankton candidates, or unconfirmed biological targets. A local high-resolution acoustic tracking unit is used to perform high-frequency, high-refresh-rate rescanning and Kalman filtering tracking on the split candidate target and to predict the probability of entering the optical registration volume or the crossing time. The low-disturbance optical registration unit is used to generate a differentiated optical triggering plan based on the target category when the probability reaches a threshold or the predicted trajectory intersects with the optical registration volume, and to trigger camera exposure and short-pulse illumination to obtain an optical image. The acoustic-optical cross-modal recognition unit is used to fuse acoustic features, optical features and spatial-temporal features to output the target category and perform attitude normalization correction of acoustic target intensity; The virtual registration counting unit is used to determine whether a target has effectively crossed the virtual registration surface and to include targets that meet the crossing conditions in the candidate set for deduplication. The active identity cache deduplication unit is used to generate cross-modal identity tokens for targets in the traversal candidate set, which include traversal time, spatial location, speed, target scale, acoustic summary features, and optical summary features. The current identity token is matched with historical identity tokens of the same category in the active identity cache table within a preset historical time window. If the matching similarity exceeds the threshold and meets the conditions for turning back or reflowing, it is determined to be a duplicate occurrence; otherwise, it is determined to be a new target and a counting event is generated. The quantity statistics output unit is used to summarize valid count events, distinguish fish, individual plankton, plankton clusters and unidentified biological targets, and output the classification count results; it performs time-series analysis and feature analysis statistics on cross-modal identity tokens, and outputs time window statistics, acoustic feature statistics and optical feature statistics. A unified clock and cross-modal space calibration unit are used to provide a unified time reference and perform real-time coordinate transformation based on extrinsic parameter transformation relationships; The polar environment support unit is used to ensure the long-term operation of the system under Arctic ice, including a power management module, a low-power monitoring and scheduling module, a data storage module, and a narrowband satellite communication module; The unified clock and cross-modal spatial calibration unit is electrically connected to the broadband acoustic detection unit, the local high-resolution acoustic tracking unit, and the low-disturbance optical registration unit, respectively, providing the above units with a unified timestamp signal and real-time coordinate transformation parameters. The power management module in the polar environment support unit supplies power to all units, and the low-power monitoring and scheduling module is connected to the broadband acoustic detection unit and the local high-resolution acoustic tracking unit to control their working and sleep state switching. The narrowband satellite communication module is connected to the quantity statistics result output unit for transmitting compressed counting data products. The broadband acoustic detection unit, biological target screening unit, fish / plankton dual-object diversion unit, local high-resolution acoustic tracking unit, low-disturbance optical registration unit, acoustic-optical cross-modal recognition unit, virtual registration counting unit, active identity cache deduplication unit, and quantity statistics result output unit are connected in sequence.

[0008] Compared with the prior art, the beneficial effects of the present invention are: (1) Strong environmental adaptability. In response to the special conditions such as strong reflection of sea ice under Arctic ice, low illumination during polar night and multipath reverberation, the system effectively reduced the interference of the sub-ice environment by defining multi-layer observation areas and adopting a strategy that combines acoustic monitoring with low-disturbance triggering, thus ensuring the long-term stable operation of the system.

[0009] (2) Wide target spectrum coverage. The innovative introduction of a dual-object diversion mechanism for fish and plankton is used to adopt differentiated tracking and triggering strategies for fish and plankton with different scattering characteristics, and to use optical morphology information for confirmation. This avoids the problem of filtering out plankton as clutter or mixing it with fish in the statistics, making the attributes of the counting objects clearer.

[0010] (3) Close coordination between acoustics and optics. The tasks of wide-range detection and local fine tracking performed by acoustics are connected with the tasks of close-range morphology confirmation performed by optics through prediction triggering logic. Supplemental lighting and shooting are only initiated when the acoustically predicted target is about to enter the optical field of view, which greatly improves the effective image hit rate and solves the problems of acoustic-optic mismatch and missed shots.

[0011] (4) Reliable triggering and low energy consumption. Optical triggering is based on real-time prediction of target position and velocity from acoustics. Compared with simply relying on image difference or timed shooting, the triggering timing is more accurate. It can capture both fast-moving fish and slow-drifting plankton clusters. Combined with short-pulse supplemental lighting and on-demand shooting, it significantly reduces energy consumption and data redundancy in long-term under-ice observation.

[0012] (5) Effective elimination of duplicate counting. By establishing a virtual registration surface as the only counting interface and introducing a deduplication mechanism based on multi-dimensional feature cross-modal identity tokens and activity cache tables, it is possible to accurately identify behaviors such as the return of the same target and the drift back of clusters, which fundamentally solves the problem of duplicate counting that is common in existing technologies and makes the counting results closer to the real number of individuals.

[0013] (6) Complete technical process. This invention integrates acoustic detection, background filtering, dual-object diversion, local fine tracking, low-disturbance optical registration, acoustic-optical fusion recognition, virtual registration surface counting and identity deduplication into a complete technical chain, filling the technical gap in centimeter-level bioacoustic-optical joint detection and unique counting under Arctic ice.

[0014] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0015] Figure 1 This is a functional structure block diagram of the observation system of the present invention; Figure 2 This is a schematic diagram illustrating the relationship between observation nodes and the observation space. Figure 3 Timing diagram for cross-modal registration and optical triggering; Figure 4 This diagram illustrates biomass assessment and data product generation. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings, so as to more clearly understand the purpose, features and advantages of this invention. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of this invention, but are only for illustrating the essential spirit of the technical solutions of this invention. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0017] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0018] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0019] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.

[0020] In the following description, in order to clearly demonstrate the structure and working method of the present invention, a number of directional terms will be used. However, terms such as "front", "back", "left", "right", "outside", "inside", "outward", "inward", "up", and "down" should be understood as convenient terms and not as limiting terms.

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments described herein are merely illustrative of the invention and are not intended to limit its scope of protection.

[0022] Example 1 This embodiment provides a centimeter-level acoustic-optical combined counting system for organisms under Arctic ice. For example... Figure 1 As shown, the system includes a broadband acoustic detection unit, a biological target initial screening unit, a fish / plankton dual-object diversion unit, a local high-resolution acoustic tracking unit, a low-disturbance optical registration unit, an acoustic-optical cross-modal recognition unit, a virtual registration counting unit, an activity identity caching and deduplication unit, a quantity statistics output unit, a unified clock and cross-modal spatial calibration unit, and a polar environment support unit. The functional connections between the units are as follows: Figure 1 As indicated by the middle arrow, this constitutes a complete processing link from broadband acoustic candidate discovery to the output of quantitative statistical results.

[0023] The specific functions of each of the above units are as follows: The broadband acoustic detection unit is used to transmit broadband acoustic signals, periodically and actively scan the broadband acoustic detection area, and perform matched filtering, pulse compression processing, background subtraction and detection on the received echoes. It extracts echoes that meet the preset signal-to-noise ratio threshold and equivalent acoustic scale threshold as initial candidate targets, and records the distance, azimuth, echo intensity, echo envelope, multi-frequency amplitude distribution and timestamp of each candidate target.

[0024] The biological target screening unit is used to perform background filtering and effectiveness screening on the initial candidate targets, eliminating interference echoes caused by fixed reflections from the sea ice interface, reflections from equipment structures, and random noise, and retaining only non-static scattering targets that appear continuously for multiple periods and meet physical motion constraints as valid biological candidate targets.

[0025] The fish / plankton dual-object diversion unit is used to divert effective biological candidate targets into two categories. Based on the concentration of scattering centers, consistency of trajectory direction, continuity of motion velocity, equivalent aspect ratio, number of multiple scattering centers, boundary dispersion, and low-speed drift characteristics, candidate targets are marked as fish candidates, plankton candidates, or unconfirmed biological targets, so as to select different strategy paths for subsequent processing.

[0026] The local high-resolution acoustic tracking unit is used to perform high refresh rate rescans on the local sector where each candidate target is located after splitting, using high-frequency acoustic signals higher than the operating frequency of the broadband acoustic detection unit, to obtain a continuous position measurement sequence of the candidate targets. The Kalman filter is then used to recursively estimate the target position and velocity and extrapolate the short-time trajectory to predict the probability of the candidate target entering the optical registration volume or the crossing time.

[0027] The low-disturbance optical registration unit is used to generate differentiated optical triggering plans based on the candidate target category when the entry probability reaches a preset threshold or the predicted trajectory intersects with the optical registration volume, and triggers camera exposure and short pulse illumination according to the plan to acquire optical images containing candidate targets. The triggering plan adopts a short-frame fast triggering mode for fish candidates, a short-window multi-frame triggering mode for individual planktonic candidates, and a continuous short-time sequence triggering mode for planktonic cluster candidates.

[0028] The acoustic-optical cross-modal recognition unit is used to perform weighted fusion of acoustic features, optical features and spatial-temporal features of candidate targets and output target category results. Among them, the optical attitude information is used to perform attitude normalization correction on the acoustic target intensity, and the corrected target intensity replaces the original value to participate in acoustic confidence assessment and identity token generation.

[0029] The virtual registration counting unit is used to determine whether a target has effectively traversed the virtual registration surface based on its three-dimensional trajectory within the optical registration volume, and to include targets that meet the traversal conditions in the candidate set for deduplication.

[0030] The active identity cache deduplication unit is used to generate cross-modal identity tokens for targets in the traversal candidate set. It performs similarity matching between the current token and historical tokens in the active identity cache table that belong to the same category and are within a preset historical time window. If the similarity exceeds the threshold and meets the conditions for backtracking or reflowing, it is determined to be a duplicate occurrence of the already counted target, and no new count event is added; otherwise, it is determined to be a new target, a count event is generated, and the current token is written to the active identity cache table.

[0031] The quantity statistics output unit is used to summarize the valid count events within a preset time window, distinguish between fish, individual plankton, plankton clusters, and unidentified biological targets, and output the classification count results; it performs time-series analysis and feature analysis statistics on cross-modal identity tokens, and outputs time window statistics, acoustic feature statistics, and optical feature statistics.

[0032] The unified clock and cross-modal space calibration unit provides a unified high-precision time reference for acoustic emission time, acoustic reception time, optical exposure time, and supplementary light pulse time. Based on the external parameter transformation relationship, it transforms the acoustically measured candidate target position to the system reference coordinate system and optical coordinate system in real time, providing a consistent spatiotemporal reference for acousto-optic spatial registration, entry probability prediction, and virtual registration surface crossing judgment.

[0033] The polar environment support unit is used to ensure the stable operation of observation nodes under the conditions of low temperature, low light, limited communication, and long-term unattended operation under Arctic ice. It includes a power management module, a low-power monitoring and scheduling module, a data storage module, and a narrowband satellite communication module. It enables the hibernation or shutdown control of non-essential components during idle periods and compresses and intermittently transmits counting data products.

[0034] The above units work together to form a complete processing flow, which includes broadband acoustic candidate discovery, background filtering and initial effectiveness screening, fish / plankton dual-object diversion, local acoustic fine tracking, low-disturbance optical registration, acoustic-optical fusion recognition, virtual registration surface counting, identity token deduplication, and output of quantity statistics results.

[0035] Example 2 This embodiment uses the system of Embodiment 1 as the execution subject to detect, identify, and count fish and plankton at the centimeter level or larger in water. The specific steps of the acoustic-optical combined counting method for centimeter-level organisms under Arctic ice provided in this embodiment are as follows: Step S1: Deploy under-ice observation nodes and establish an observation coordinate system; Fixed, tethered, or quasi-fixed observation nodes are deployed beneath the Arctic ice to maintain a relatively stable attitude during observation. Internally, it integrates an embedded Linux main control board, an STM32 low-power microcontroller, a broadband acoustic transducer array (center frequency 300kHz), a local high-resolution acoustic transducer (center frequency 800kHz), a binocular optical camera (global shutter, resolution 2448×2048 pixels), a short-pulse LED illumination component, a data storage module, a power management module, and a narrowband satellite communication module.

[0036] The embedded Linux main control board serves as the core of edge computing and task scheduling, responsible for acoustic data parsing, coordinate transformation, target trajectory prediction, image processing, acoustic-optical fusion recognition, virtual registration surface counting, identity token deduplication, and data product encapsulation. The STM32 low-power microcontroller handles device power-on and sleep scheduling, camera hardware triggering, supplementary lighting pulse driving, sensor status acquisition, and watchdog reset. The broadband acoustic detection unit and the local high-resolution acoustic tracking unit are connected to the embedded Linux main control board via Ethernet. The dual-optical camera is connected to the embedded Linux main control board and controls camera triggering and supplementary lighting driving via GPIO interfaces.

[0037] Before the system officially starts operating, multi-coordinate system calibration is performed. A system reference coordinate system is established and fixed to the main node. Acoustic coordinate system fixed at the center of acoustic transducer array And the optical coordinate system fixed at the optical center of the left eye of the binocular optical camera. By using the mechanical installation dimensions and the underwater calibration target, the transformation matrix from the acoustic coordinate system to the system reference coordinate system is obtained. And the transformation matrix from the system reference coordinate system to the optical coordinate system. Therefore, any target position measured in the acoustic coordinate system... All coordinates can be transformed to the system reference coordinate system and optical coordinate system using the following coordinate transformation relationships: ; ; in, This indicates the target position in the system's reference coordinate system. This indicates the target position in the optical coordinate system.

[0038] All acoustic transmission times, reception times, camera exposure center times, and fill light pulse times are synchronized by the same high-precision clock source, ensuring that the time synchronization error is no more than 5 milliseconds.

[0039] After the observation nodes are deployed, the STM32 low-power microcontroller first starts up and performs a self-test. The self-test includes power status, cabin temperature and humidity, pressure sensor status, acoustic component status, camera power supply status, and supplementary lighting drive status. If the self-test is successful, the STM32 low-power microcontroller powers on the embedded Linux mainboard. After startup, the embedded Linux mainboard loads the acoustic data receiving program, camera control program, image processing program, counting program, data storage program, and communication scheduling program, and reads external parameter calibration parameters, optical registration volume parameters, virtual registration surface parameters, and target screening thresholds. Subsequently, the system enters a low-power acoustic monitoring state.

[0040] Step S2: Define the acoustic detection area, the local acoustic tracking area, the optical registration volume, and the virtual registration surface; like Figure 2 As shown, based on the installation direction of the observation nodes and the coverage of acoustic and optical devices, the following four types of observation areas are defined: 1. Define the broadband acoustic detection area. The broadband acoustic detection area is used for searching for candidate targets at medium to long ranges. For larger fish targets, the broadband acoustic detection distance can be set from 1 m to 30 m; for small candidate targets of 1 cm to 5 cm, the effective acoustic candidate detection distance is preferably set from 0.5 m to 8 m.

[0041] 2. Define the local acoustic tracking region. The local acoustic tracking region is a local sector or local spatial volume around the candidate target, used for high refresh rate rescanning and trajectory prediction after the candidate target is detected. The local acoustic tracking distance is preferably set to 0.5 m to 8 m.

[0042] 3. Define the optical registration volume. The optical registration volume is jointly defined by the overlapping field of view of the optical camera, the effective supplementary lighting range, and the resolvable imaging distance for centimeter-level targets. The optical registration volume is preferably set within the range of 0.3 m to 2.5 m from the observation node. Targets only enter the counting process after entering this optical registration volume and completing acousto-optic spatial registration.

[0043] 4. Define the virtual registration surface. The virtual registration surface is a hemispherical surface set within the optical registration volume, serving as the unique spatial location defining the occurrence of counting events. The function of this surface is to ensure that only when a target crosses the surface in a specific direction is it recognized as a valid crossing and enters the counting process, avoiding the direct equation of generalized acoustic detection events or image detection events with counting events.

[0044] In the system reference coordinate system In the middle, the hemispherical virtual registration surface The mathematical definition of is: in, Indicates the spatial position in the system's reference coordinate system. Indicates the center of the hemisphere. Indicates the radius of the hemisphere. The unit vector representing the orientation of the hemisphere. Represents the optical registration volume in the system's reference coordinate system. Indicates radius as Tolerance-expanding body, This represents Minkowski and... (condition) Limited to With the center of the ball, For a sphere with radius , the condition is... Preserve the spherical surface along One side of the direction it points to, thus forming a hemisphere; condition This confines the hemisphere within the optical registration volume and its tolerance boundary region, ensuring that the target can be effectively imaged when a crossing event occurs.

[0045] The orientation unit vector of the hemisphere The direction is set to be from the center of the hemisphere. Pointing towards the main optical axis of the optical camera, or towards the main direction of the acoustic candidate target, ensures that the direction of motion of the target when crossing the surface matches the counting intention.

[0046] The center of the hemisphere With radius The method for determining this is as follows: a preset counting reference point is selected on the optical principal axis. The hemispherical radius is determined based on the optical overlap field of view, effective imaging distance, acousto-optic spatial registration error, and target positioning error. Hemisphere center Represented as: This setting allows the reference point to be set. It lies precisely on the hemispherical virtual registration surface. Radius The value of should ensure that the effective part of the virtual registration surface is located within the optical registration volume or its tolerance expansion region, thereby ensuring that the target is within the effective observation range of the optical imaging unit when it crosses the surface.

[0047] Step S3: Use broadband acoustics to detect centimeter-level potential biological targets; like Figure 2As shown, in low-power acoustic standby mode, the binocular optical camera captures images intermittently, the supplementary lighting component remains off, and the local high-resolution acoustic tracking unit is in standby mode. The broadband acoustic detection unit actively scans at a set period, using a linear frequency modulated signal with a center frequency of 200kHz to 400kHz (preferably 300kHz) and a bandwidth of 50kHz to 150kHz (preferably 100kHz), scanning the broadband acoustic detection area once per second. The receiving echo link sequentially performs bandpass filtering, matched filtering, and pulse compression processing on the signal, and performs background echo subtraction, range gate detection, and azimuth estimation. For each range-azimuth unit, the echo signal-to-noise ratio, echo energy concentration, echo envelope width, and multi-frequency echo amplitude distribution are calculated. When the echo signal-to-noise ratio of a certain range-azimuth cell is not less than 10dB, and the equivalent acoustic scale derived from its echo envelope is not less than 2 cm, the system marks the echo as an initial candidate target and records summary information such as its range, azimuth, echo intensity, echo energy concentration, echo envelope, multi-frequency amplitude distribution, timestamp, and detection consistency over consecutive multi-scan cycles.

[0048] Step S4: Perform background filtering and initial effectiveness screening on the initial acoustic candidate targets; The main control board performs background filtering and validity screening on the initial candidate target set obtained in step S3 to eliminate interference caused by stable reflections from the sea ice interface, fixed reflections from the node device structure, scattering from suspended particles in the water, and random noise. An initial candidate target must simultaneously meet all of the following conditions to be retained as a valid biological candidate target: (a) The echo signal-to-noise ratio is not less than 10 dB; (b) It is detected at least M times in the most recent consecutive N acoustic scans, where N is an integer between 3 and 10, and M is an integer not less than N / 2. Preferably, it is detected at least M=3 times in the most recent consecutive N=5 acoustic scans. (c) The equivalent acoustic scale is not less than 2 cm; (d) The velocity corresponding to the estimated displacement between two adjacent scans satisfies the physical reachability constraint, wherein the upper limit of the velocity of fish candidate targets is set to 1.0 m / s to 2.0 m / s, preferably 1.5 m / s, and the upper limit of the velocity of planktonic candidate targets is set to 0.1 m / s to 0.5 m / s, preferably 0.3 m / s; (e) Its spatial location is not within the pre-defined stable reflection zone of the sub-ice interface, the reflection zone of the nodal structure, or other known static background area; (f) On the distance-azimuth plane, spatially traceable clusters can be formed, rather than isolated scattered points.

[0049] Targets that meet all of the above conditions are retained. This step eliminates most false alarms caused by sub-ice multipath reverberation, strong sea ice reflections, and random noise, effectively improving the stability of candidate detection.

[0050] Step S5: Perform dual-object diversion of fish / plankton for valid biological candidate targets; For valid biological candidates that pass the initial screening, the system further performs dual-object diversion based on their acoustic echo characteristics. Specifically, the system calculates fish tendency scores based on the candidate targets' characteristics such as the concentration of scattering centers, consistency of trajectory direction, smoothness of motion speed, and equivalent aspect ratio, using preset weighting rules; at the same time, it calculates planktonic tendency scores based on characteristics such as the number of multiple scattering centers, boundary dispersion, low-speed drift characteristics, local acoustic scattering area occupancy, and whether they are clustered.

[0051] The triage rules are as follows: If the fish tendency score is significantly higher than the plankton tendency score and exceeds the first threshold, the candidate target is marked as a fish candidate; if the plankton tendency score is significantly higher than the fish tendency score and exceeds the second threshold, it is marked as a plankton candidate; if neither score is significant and cannot be clearly classified, it is marked as an unconfirmed biological target and will continue to be tracked.

[0052] The purpose of this triage step is to select different processing paths for subsequent local acoustic tracking and optical triggering strategies, not to complete the final classification.

[0053] In this embodiment, the specific calculation method for the fish tendency score and the plankton tendency score is as follows: after normalizing the scattering center concentration, trajectory direction consistency, motion velocity smoothness, and equivalent aspect ratio, the fish tendency score is obtained by weighting and summing the results using weighting coefficients [0.3, 0.25, 0.2, 0.25]. The normalized values ​​of the number of multiple scattering centers, boundary dispersion, low-velocity drift characteristics, and local acoustic scattering region occupancy were weighted and summed using weighting coefficients [0.25, 0.25, 0.3, 0.2] to obtain the planktonic tendency score. The first and second thresholds are set based on historical data statistical analysis, and are preferably 0.7. When and When, it is marked as a fish candidate; when and When the target is identified, it is marked as a planktonic candidate; otherwise, it is marked as an unidentified biological target.

[0054] Step S6: Perform local high-resolution acoustic fine tracking and predict the probability of the target entering the optical registration volume; Once a candidate target passes the initial screening and is diverted, the main control board immediately activates the local high-resolution acoustic tracking unit. This unit uses a high-frequency signal with a center frequency of 800kHz and a bandwidth of 200kHz, and only performs high-refresh-rate rescans on a local sector centered on the current position of the candidate target. Its measurement update cycle is 20 to 50 milliseconds. The tracking unit only undertakes local continuous tracking tasks and does not perform large-scale searches.

[0055] The system employs a Kalman filter based on a constant velocity state model to recursively process the continuous acoustic measurement results. A state vector is defined. ,in The three-dimensional position of the target in the system's reference coordinate system. Three-dimensional velocity; measurement vector The distance and orientation are measured by the acoustic system. The standard Kalman prediction and update process is then used: Prediction steps: , , Update steps: , ， ； in, This is the optimal state estimate obtained after the update step at time k-1; The prior state estimate at time k (the superscript "-" represents prior / prediction) is obtained by acquiring the acoustic measurement vector at time k. Previously, the target's state at time k was predicted purely based on the optimal estimate at time k-1 and the motion model. The posterior state estimate at time k is obtained after acquiring the acoustic measurements at time k. Then, the prior state is estimated. The optimal estimate obtained after correction is the final output of the filter; F is the state transition matrix; H is the observation matrix; Let be the posterior estimation error covariance matrix at time k-1; Let be the prior estimation error covariance matrix at time k; The process noise covariance matrix; To observe the noise covariance matrix; The Kalman gain matrix; The actual measurement vector obtained by the local high-resolution acoustic system at time k; It is the identity matrix; This is the corresponding state estimation error covariance matrix.

[0056] The process noise covariance matrix of the Kalman filter Adaptive settings are applied based on the target category. When a candidate target is categorized as a fish candidate, the standard deviation of the location process noise is... Set to 0.15 meters, the standard deviation of the velocity process noise Set to 0.3 m / s; when a candidate target is diverted to a planktonic candidate, Set to 0.03 meters. Set to 0.05 m / s. Matrix The standard deviation is constructed using the discretization formula of the constant velocity model. When the target category is unconfirmed, the default median value is used.

[0057] The updated state vector is represented as follows: in, The updated candidate target locations are obtained after Kalman filtering. Let be the candidate target's velocity updated by Kalman filtering. Therefore, the candidate target's position and velocity are determined by the updated state vector. The error covariance matrix is ​​obtained directly. This is used to characterize the uncertainty in the position and velocity estimates. The system utilizes the updated candidate target positions. and speed of movement Perform short-time trajectory extrapolation. The predicted position of the candidate target at future time t is represented as: in, Let be the time corresponding to the k-th acoustic measurement. The optical registration volume is denoted as . The probability of a candidate target entering the optical registration volume is calculated based on the shortest distance between the predicted location and the optical registration volume, and can be expressed as: in, Indicates spatial location With optical registration volume The shortest Euclidean distance between them; this distance is 0 when the predicted position is already within the optical registration volume, therefore Take 1. This represents the positional uncertainty at the corresponding prediction time, expressed as the error covariance matrix obtained from Kalman filtering. The acoustic positioning error and short-term motion prediction error are determined. The system searches within a preset time range in the future. The maximum value and its corresponding time .when: Alternatively, when the predicted trajectory of the candidate target intersects with the optical registration volume, the system enters a low-disturbance optical registration preparation state; otherwise, it continues local acoustic fine-tracking. The preset entry probability threshold is preferably 0.6 to 0.9.

[0058] Step S7: Trigger low-disturbance optical registration based on acoustic prediction results; Based on the predicted target entry into the optical registration volume obtained in step S6, an optical triggering plan is generated. The optical triggering plan includes the camera trigger time, the fill light trigger time, the exposure time, the fill light pulse width, the number of trigger frames, and the frame interval.

[0059] Preferably, the triggering plan in step S7 is used to predict the arrival time. Centered on the generator. The system arranges the main exposure frame based on camera trigger delay and fill light delay. Nearby, and a few auxiliary frames are set before and after it.

[0060] The above-mentioned optical triggering process can be further combined Figure 3 To understand. Figure 3 The upper part illustrates the cross-modal spatial registration relationship of this invention: acoustic coordinate system. The target position measured below Transformed matrix via extrinsic parameters Transform to system reference coordinate system The lower position Then by Transform to optical coordinate system The lower position Ultimately, spatial alignment of the target within the optical registration volume is achieved. Figure 3 The lower section illustrates the timing logic for generating the optical triggering plan based on the prediction results: The system generates a short-frame rapid triggering mode (pre-trigger frame, main trigger frame, and subsequent trigger frame) for fish candidates based on the predicted crossing time; a short-window multi-frame triggering mode (acquiring multiple frames at equal intervals within a short time window) for individual planktonic candidates; and a continuous short-time-sequence triggering mode for planktonic cluster candidates. The supplemental lighting component is only briefly activated within the camera's exposure window, and the camera uses a global shutter mode.

[0061] For fish candidates, a short-frame, fast triggering mode is preferred, including a pre-trigger frame, a main trigger frame, and a post-trigger frame. The pre-trigger frame captures the target's state near the optical registration volume, the main trigger frame captures images of the target crossing near the virtual registration surface, and the post-trigger frame helps confirm the target's direction of passage and whether it recurs. For single planktonic candidates, a short-window, multi-frame triggering mode is preferred, acquiring multiple images within a time window of 100 ms to 1000 ms to improve contour extraction, scale estimation, and target segmentation stability. For planktonic cluster candidates, a continuous short-sequence triggering mode is preferred to determine cluster boundaries, local density, and the number of segmentable individuals. For unconfirmed biological targets, the same short-window, multi-frame triggering mode as for single planktonic candidates is used to acquire as clear an optical image as possible for subsequent definitive identification.

[0062] A preferred triggering plan parameter is as follows: For fish candidates, the short-frame fast triggering mode includes acquiring a pre-trigger frame 30 milliseconds before the predicted main trigger time, acquiring the main trigger frame at the main trigger time, and acquiring a post-trigger frame 30 milliseconds after the main trigger time. This setting can effectively capture fast-swimming fish and determine their direction of passage. For single planktonic candidates, the short-window multi-frame triggering mode acquires 5 images at equal intervals within a 200-millisecond time window near the prediction time to adapt to their slow drift characteristics and ensure that at least one image shows a clear and complete target. For planktonic cluster candidates, the continuous short-time-sequence triggering mode acquires 10 images continuously at 50-millisecond intervals near the prediction time to obtain temporal change information of the internal structure of the cluster, which assists in subsequent cluster segmentation and degradation judgment.

[0063] The fill light component is only activated briefly within the camera's exposure window, remaining off or in low-power standby mode at other times. The fill light pulse width is preferably between 50 μs and 5 ms. The camera preferably uses a global shutter mode, and the resolution is preferably no less than 2448 × 2048 pixels.

[0064] Step S8: Preprocess the optical image and extract target morphological features; After acquiring the optical image, the image is processed by dark field background subtraction, non-uniform illumination correction, suspended particle noise filtering, target region segmentation, contour extraction and scale estimation.

[0065] For centimeter-level targets, the minimum projected length of the target in the image is preferably no less than 10 pixels to ensure the stability of contour and scale extraction. For fish candidates, the body length is extracted. Features such as body height, aspect ratio, main body outline, outline closure, principal axis direction, and attitude angle are extracted. For planktonic candidates, features such as body length, body width, transparency boundary strength, segmental structure response, appendage abduction outline response, cluster boundary, cluster area occupancy, and number of individual segments are extracted. For unconfirmed biological targets, at least the equivalent scale, target outline, image location, and spatial registration location are preserved.

[0066] Preferably, step S8 further includes target three-dimensional pose estimation. The system uses calibrated binocular optical imaging units to obtain synchronous images of the same target, and calculates the target's head position through triangulation. and tail position Thus, the target volume axis unit vector is determined: in, Indicates the spatial orientation of the target subject.

[0067] Determine the unit vector of the acoustic incident direction at the target based on the acoustic transducer location and the target space registration location. The equivalent incident attitude angle between the target body axis and the acoustic incident direction is expressed as: in, It is used to characterize the change in target attitude relative to the acoustic incident direction and is used for acoustic target intensity normalization correction in step S9. When the target's head-tail direction or three-dimensional attitude cannot be reliably determined, the image principal axis direction or attitude estimation results from multiple consecutive frames can be used as weak attitude constraints.

[0068] For plankton cluster candidates, step S8 also includes image segmentation quality evaluation and adaptive degradation decision. The system first determines the target region Ω within the image region where the plankton cluster is located, and obtains candidate individual unit masks M within the cluster through threshold segmentation, edge extraction, connected component analysis, or instance segmentation. i , where i represents the candidate monomer number.

[0069] The overlap ratio (Rov) between candidate pixels represents the proportion of pixels simultaneously covered by masks of two or more candidate pixels out of the total number of candidate target pixels, and can be expressed as: Where x represents a pixel within the target region Ω, and Mi(x) is the value of the mask for the i-th candidate unit at pixel x; when the pixel belongs to this candidate unit... ,otherwise . The larger the value, the more severe the adhesion or occlusion between candidate monomers; when there is no overlap... Take 0.

[0070] Cluster contour closure This is used to evaluate whether the extracted cluster edges form a complete closed contour. Endpoint detection is performed on the cluster edges, and the shortest connection length that needs to be padded between adjacent broken endpoints is denoted as Lgap. The total length of the actual extracted cluster contour is denoted as... Then the cluster profile closure is expressed as: in, This represents the actual extracted cluster contour length from the image. The total length of the contour gaps required to fill in order for the contour to form a closed boundary. When the cluster contour is completely closed, =0, Take 1; the more severe the contour loss or breakage, the better. The smaller the size, the better. The outline notch can be determined based on the distance between the edge endpoints, directional continuity, and a preset maximum connection distance.

[0071] Cluster area occupancy The proportion of foreground pixels in a cluster to the total number of pixels in the target region Ω can be expressed as: Where M represents the union of all candidate monomer masks, i.e.: |M| represents the number of foreground pixels in the cluster, and |Ω| represents the total number of pixels in the target region Ω. The larger the value, the denser the cluster is within the target area, and the higher the likelihood of adhesion, overlap, or occlusion between candidate monomers.

[0072] The overlap pixel ratio Rov and cluster contour closure and cluster area occupancy The values ​​range from 0 to 1, and the segmentation quality score is calculated accordingly. : in, , and Let be the weight coefficient of the corresponding indicator, and satisfy: The smaller The larger and A smaller value indicates lower overlap between candidate units, more complete cluster outlines, and less image crowding, thus resulting in a higher segmentation quality score. The higher.

[0073] When the segmentation quality score satisfies: Or, if at least one of the following conditions—overlapping pixel ratio, contour closure, or area occupancy—is met, preferably, when the following conditions are met... , , or If any of the following conditions are met, the current planktonic cluster is determined to be difficult to reliably segment into individual organisms, and the adaptive degradation estimation mode is entered; otherwise, the normal individual organism identification and counting mode is maintained.

[0074] When the system enters the adaptive degradation estimation mode, it obtains the local number density estimate of planktonic clusters based on acoustic volume scattering information and optical area occupancy information, respectively. The volume backscattering coefficient is calculated based on the background subtraction of the acoustic detection area where the cluster is located. and the average monomer backscattering cross section of the corresponding target group Acoustic number density estimate obtained: in, The local number density obtained from acoustic estimation, The volume backscattering coefficient is expressed as a linear quantity. This represents the average backscattering cross section of a single target, obtained through pre-calibration or historical samples of similar targets. When acoustic measurements are expressed in decibels, they are first converted to linear quantities before number density calculations are performed. Simultaneously, the number of targets within the optical observation area is estimated based on the cluster area occupancy rate, average single-target projected area, and effective observation volume corresponding to the optical registration volume in the optical image. And obtain an optical number density estimate: in, The local number density is obtained from optical estimation. To estimate the number of targets obtained from the cluster foreground area, average single-cell projected area, and occlusion correction results. This refers to the effective observation volume of the corresponding image region within the optical registration volume.

[0075] To reduce fusion bias caused by fixed empirical weights, the system adaptively weights the results based on the uncertainties of the acoustic and optical estimates. The standard uncertainties of the acoustic and optical number density estimates are denoted as follows: and And determine their fusion weights respectively: in, The calibration error is determined based on the acoustic signal-to-noise ratio, background residual scattering, the degree of fluctuation in multiple consecutive acoustic measurements, and the average cell backscattering cross-section. In this embodiment, The following empirical formula is used for quantitative calculation: ,in The linear signal-to-noise ratio (SNR) is given, and CV is the coefficient of variation of five consecutive acoustic measurements. The coefficient represents the relative calibration error of the average monomer backscattering cross section. =0.15、 =0.50、 =0.35. The following empirical formula is used for quantitative calculation: ,in To segment the quality score, This represents the relative estimation error of the average projected area of ​​a single building. To effectively observe the relative calibration error of the volume, the coefficient =0.60、 =0.25、 =0.15. The smaller the estimation uncertainty, the larger the corresponding fusion weight.

[0076] The consistency index between acoustic and optical estimations is expressed as: in, Used to characterize the difference between acoustic number density estimates and optical number density estimates relative to the magnitude of their combined uncertainty. When Not greater than the preset consistency threshold At that time, it was assumed that the acoustic estimation and optical estimation had acceptable consistency, and the final local number density was obtained by using an uncertainty-weighted method: The uncertainty of the number density after fusion can be expressed as: when Greater than the preset consistency threshold When the system determines that there is a large difference between the acoustic estimation and the optical estimation, it reduces the fusion weight of the side with greater uncertainty based on the current acoustic signal quality, optical segmentation quality and on-site calibration results, or only outputs the number density estimate of the side with less uncertainty. At the same time, it adds an inconsistency flag for acoustic and optical estimation and a low confidence flag, and does not directly output the high confidence fusion result.

[0077] By using the adaptive fusion method based on uncertainty and acoustic-optical consistency, the final local quantity density can be dynamically adjusted according to the acoustic signal quality, optical image quality and on-site observation conditions, avoiding the simple averaging of acoustic and optical estimates using fixed empirical weights.

[0078] In the normal monomer counting mode, the monomers obtained by stable segmentation are subjected to subsequent acousto-optic fusion identification and cross-modal identity tokens are generated. After entering the adaptive degradation estimation mode, the system generates cluster-level candidate events and continues to execute the cluster category confirmation, cluster crossing judgment and cluster-level identity deduplication in steps S9 to S11. After deduplication is completed, the local number density or volume abundance estimation results are output in step S12, but the accurate monomer counting results are not output.

[0079] Step S9: Perform acousto-optic cross-modal recognition and output the target category result; The acoustic features, optical features, and spatial-temporal features are fused to obtain the target category result.

[0080] Acoustic features include echo intensity, echo envelope, multi-frequency amplitude distribution, continuous multi-ping stability, velocity estimation, and local trajectory. Optical features include target scale, geometric proportions, contour morphology, edge features, transparent boundaries, segmental structure, appendage abduction contours, and cluster segmentation results. Spatial-temporal features include timestamps, registration locations, direction of motion, and velocity.

[0081] Step S9 also includes acoustic target intensity normalization correction based on optical attitude information. For acousto-optic spatially registered targets, the measured target intensity can be obtained at the acoustic frequency f. And based on the actual incident attitude angle θ of the target obtained in step S8, it is corrected to the preset reference attitude angle. The normalized target strength is expressed as: in, The target strength obtained from actual observation. The normalized target strength, The actual acoustic incident angle of the target. To preset the reference attitude angle, For the target frequency and attitude angle Attitude scattering response function under given conditions.

[0082] The attitude scattering response function Using a pre-calibrated empirical function: in and For reference attitude angle, and These are calibration coefficients.

[0083] The corrected target strength TSnorm replaces the original target strength in the acoustic-optical fusion recognition and cross-modal identity token matching, so that the system does not simply splice acoustic data and optical data, but uses optical attitude to physically compensate for acoustic scattering differences.

[0084] The target recognition confidence score is calculated using a weighted fusion method. in, For acoustic candidate confidence levels For optical morphology confidence level. Calculate the confidence level for space-time registration. , and is the weighting coefficient, and the sum of the weighting coefficients is 1.

[0085] For targets marked as fish candidates during the diversion phase, if their acoustic trajectory continuity, optical aspect ratio, main body contour, and scale characteristics meet the fish target conditions, and If a preset fish detection threshold is reached, the fish target is output. For targets marked as plankton candidates during the diversion phase, if their optical images show segmental structures, transparent boundaries, appendage abduction contours, or cluster boundaries, and their acoustic local scattering characteristics are consistent with their spatial location, and... If the preset plankton detection threshold is reached, the plankton target will be output.

[0086] For targets marked as unidentified biological targets during the diversion phase, and targets that were diverted as fish or plankton candidates but The acoustic candidate confidence level of a target that has not reached the corresponding confirmation threshold. and optical morphology confidence The weights were all set to 0.4, and the confidence level for spatial-temporal registration was... The weight is set to 0.2 to maintain decision balance in the event of insufficient information; if the fusion is performed... If the target biometric is still below the confirmation threshold for each category, then an unconfirmed biometric is output. Unconfirmed biometrics can proceed to the counting process, but their category is marked as unconfirmed.

[0087] Step S10: Determine whether the target has successfully traversed the virtual registration surface. Within the optical registration volume, a sequence of three-dimensional motion trajectory points of the target is obtained through continuous multi-frame binocular visual localization. The system then determines frame by frame whether the target has completed its alignment with the virtual registration surface. Effective crossing. Assume two consecutive frames at times k and k+1 are in the system's reference coordinate system. The target positions below are respectively and The system reference coordinate system of the target at time k+1. The velocity vector of the motion below is (The velocity estimate output by the Kalman filter in step S6) (Obtained after time alignment). Based on the virtual registration surface defined in step S2. hemisphere center hemisphere radius and orientation unit vector ( (From the center of the hemisphere to the main direction of the acoustic candidate target or the center of the optical field of view), determine whether the crossing is valid.

[0088] like and Located on the hemisphere Both sides, that is and The signs are opposite, and their directions of motion satisfy the condition of normal crossing the counting region. (That is, if the projection of the velocity vector onto the hemisphere in the direction of the unit vector is positive), then the target is determined to have completed a valid traverse from the outside to the inside of the virtual registration surface.

[0089] For a crossing to be included in the deduplication candidate set, its corresponding target must simultaneously meet the following conditions: (1) Acoustic and optical spatial registration has been completed; (2) Category label has been obtained (fish target, planktonic target, or unidentified biological target); (3) Acousto-optical fusion confidence level. (Minimum counting threshold). Targets that fail to enter the optical registration volume or fail to complete a valid traverse will not generate a counting event.

[0090] Step S11: Generate a cross-modal identity token and perform deduplication checks; For counted candidate targets that have completed acoustic-optical spatial registration, target category confirmation, and valid virtual registration surface traversal, a cross-modal identity token is generated. For the j-th counted candidate target, its identity token is represented as: ; in, For the moment of crossing the virtual registration surface; Register the spatial location of the crossing point; The velocity vector during the crossing; Direction of crossing; The target scale is (body length for fish, and body length or equivalent diameter of clusters for plankton). The normalized acoustic summary feature vector includes one or more of the following: acoustic target intensity or echo intensity, signal-to-noise ratio, multi-frequency echo features, continuous multi-ping stability, and local trajectory features. The normalized optical summarization feature vector includes one or more of the following: target contour, aspect ratio, area, principal axis direction, transparent boundary, segmental structure, appendage abduction contour, or low-dimensional image features. For category labels.

[0091] To avoid matching newly emerging targets one by one with all historical counted targets, this method establishes an active identity cache table and an archived count record table. The active identity cache table only stores target identity tokens that are likely to re-enter the optical registration volume in the near future; identity tokens that have exceeded the corresponding time window are transferred to the archived count record table and no longer participate in real-time deduplication matching.

[0092] For a newly emerging current target j, the system first filters candidate historical targets from the active identity cache table based on the target category and historical time window. Candidate historical targets should have the same or compatible target category as the current target, and the time interval between them should not exceed the preset historical time window of the corresponding target category.

[0093] For the current target j and the candidate historical target i, the matching distance between their cross-modal identity tokens is calculated using normalized weighted Euclidean distance. : in, and These represent the time it takes for historical target i and current target j to traverse the virtual registration surface, respectively. and These are the spatial registration locations for both; and These are the speeds of the two objects; and These are the target scales for the two, respectively; and These are the normalized acoustic summary feature vectors; and These are the normalized optical summary feature vectors.

[0094] T, R, V, and S are the normalized scales for time difference, spatial position difference, velocity difference, and target scale difference, respectively. They can be determined based on the historical time window of the corresponding target category, the maximum allowable spatial recurrence distance, the maximum allowable velocity difference, and the target scale variation range. In this embodiment, the normalized scale values ​​are: T = 300 seconds, R = 0.5 meters, V = 1.0 meters / second, and S = 0.2 meters. , , , , and Let be the weight coefficients of the corresponding features, and satisfy: In this embodiment, the weighting coefficient is taken as: , , , , , .

[0095] Furthermore, the matching distance is converted into identity matching similarity between the current target j and the candidate historical target i: in, The value range is from 0 to 1. The closer it is to 1, the more similar the current target is to the candidate historical target in terms of time travel, spatial location, speed of movement, target scale, and acoustic and optical characteristics; The closer it is to 0, the lower the probability that the two are the same target.

[0096] The system calculates the identity matching similarity between the current target j and each historical target in the candidate set, and takes the maximum value among them: ; in, This represents the set of candidate historical targets corresponding to the current target j.

[0097] when Not less than the preset similarity threshold ,Should The preferred similarity is 0.85. If the current target and its corresponding historical target's crossing direction and spatial position satisfy the conditions of the same target turning back, flowing back, or re-entering the optical registration volume, the current target is determined to be a duplicate of a previously counted target, and no new counting event is added. When the identity matching similarity between the current target and all historical targets in the candidate set is less than 0.85... When the target is determined to be a new valid target, a new counting event is generated, and the cross-modal identity token of the current target is written to the active identity cache table.

[0098] For fish targets, when two crossings occur in opposite directions but exhibit high similarity in spatial location, speed, target size, acoustic summarization features, and optical summarization features, they can be identified as the same fish returning and crossing the virtual registration surface again. For planktonic cluster targets, the weights corresponding to spatial location, cluster size, acoustic scattering features, and optical cluster contour features can be appropriately increased to reduce the risk of duplicate counting caused by the drift and backflow of the same cluster.

[0099] Step S12: Output the quantity statistics results.

[0100] After completing the virtual registration surface crossing judgment and cross-modal identity token deduplication, the system summarizes the generated valid count events according to the preset observation time window and outputs the acoustic-optical joint detection count results; it performs time-series analysis and feature analysis statistics on the cross-modal identity tokens and outputs the time window statistical results, acoustic feature statistics, and optical feature statistics. The preset time window can be set to 1 minute, 10 minutes, 1 hour, etc., according to the observation task.

[0101] like Figure 4 As shown, biomass assessment is further performed by combining information such as target category and identity token. Based on the input of count events, cross-modal identity tokens, and degradation mode flags, the system generates statistics on monomer quantity, cluster density, scale distribution, total biomass, time window, bio-optical characteristics, and bio-acoustic characteristics through biomass statistical models and identity token statistical models, and further generates data product output.

[0102] In this embodiment, the biomass statistical model is used to aggregate valid count events within a preset statistical time window and output statistical indicators reflecting the biomass level of the observed area. Let the statistical time window be... T, there are K valid count events in the window, each event contains a category label O. k (Values ​​can be fish, individual planktonic organisms, planktonic clusters, or unidentified biological targets), Target scale S k (For fish, body length; for plankton, body length or equivalent diameter of clusters) and local quantity density estimates. k (Applicable only to clusters with adaptive degradation patterns). The model is computed as follows: (1) Statistics on the number of valid monomers in each category: in, This represents the number of valid individuals of different categories within the statistical time window; i represents the category, with values ​​for fish, individual planktonic organisms, or unidentified biological targets. This is an indicator function that takes the value 1 when the condition is true, and 0 otherwise.

[0103] (2) Statistical analysis of planktonic clusters: The total number of cluster-level events is For clusters in adaptive degradation mode, the total local number estimate is obtained by summing the local number estimates of each cluster (obtained by multiplying the local number density estimate by the effective estimated volume); individual clusters in normal counting mode are already included. , without duplicate statistics.

[0104] (3) Total biomass estimation: Estimating total biomass by combining the target scale with category-specific length-biomass conversion factors. : in and This is a preset category-related length-biomass conversion factor; i represents the category, with values ​​for fish, solitary plankton, or plankton clusters. Unidentified biological targets are not included in the biomass estimation.

[0105] In this embodiment, the identity token statistical model is used to perform time-series and feature statistical analysis on cross-modal identity tokens, outputting acoustic and optical feature statistics, and providing data support for empirical function optimization. Assume that K identity tokens are formed within the statistical window, and each token for a counting event contains a category label. Acoustic summary feature vector (Including acoustic target intensity, etc.), optical summary feature vector (Including target outline, aspect ratio, etc.) and crossing time The model is calculated as follows: (1) Time window statistics: Total observation time window T is divided into L consecutive sub-windows, each sub-window having a length of... Count the number of valid fish targets for each category in each sub-window. Let the number of fish targets in the l-th sub-window be denoted as . The number of individual planktonic organisms is The number of planktonic clusters is The number of unconfirmed biological targets is The calculation method is uniformly as follows: Where i represents the target category; This represents the time interval of the l-th sub-window; Let k be the time when the k-th counting event crosses the virtual registration surface.

[0106] (2) Statistical analysis of acoustic and optical characteristics: The system statistically analyzes the acoustic and optical summary features of each count event by category, and calculates the mean and standard deviation of acoustic and optical features within categories such as fish, solitary plankton, and plankton clusters, forming a baseline for the distribution of acoustic and optical features of various targets under the current observation conditions.

[0107] The system's data products employ a differentiated output strategy to address different objectives: Fish target: Output the number of valid single entities after identity deduplication. The data includes the length of each individual, the time of traversal, and its spatial location.

[0108] Individual planktonic organism target: Output the number of effective individual organisms. and its scale distribution.

[0109] Planktonic Cluster Objective: For clusters maintaining a normal monomer counting pattern, output the total number of monomers confirmed within the cluster. For clusters entering an adaptive degradation estimation mode, instead of outputting the exact number of monomers, output the number of cluster-level events. Local quantity density estimates Effective volume estimation, local quantity estimation, and degradation pattern indicators are provided. This result is not considered an accurate monomer count, nor is it combined with normal monomer counts at the same confidence level.

[0110] Unidentified biological target: Number of individual outputs It is not merged with the explicitly defined categories mentioned above.

[0111] Acoustic summary features, optical summary features, target identity token, and temporal and spatial location of crossing the virtual registration surface for each valid count event.

[0112] The final data products include: Biomass statistics include individual quantity statistics, cluster density statistics, scale distribution statistics, and total biomass statistics.

[0113] The identity token statistics table includes a time window statistics table, a bio-optical feature statistics table, and a bio-acoustic feature statistics table.

[0114] Given the limited bandwidth of communication under Arctic ice, the system does not transmit complete raw acoustic data and images; instead, it transmits compressed, structured data products. When the communication link becomes unavailable, the system temporarily stores the data in local storage and continues transmission in the next communication window.

[0115] The steps S1 to S12 described above constitute a complete acoustic-optical joint detection and counting process for fish and plankton targets larger than centimeters. After processing a target event, the system enters a low-power standby state, specifically as follows: The embedded Linux main control board sends a task completion signal to the STM32 low-power microcontroller. The STM32 low-power microcontroller then shuts down the power supply to the supplementary lighting component, putting the binocular optical camera into standby mode and restoring the local high-resolution acoustic tracking unit to standby mode. If no new candidate targets appear within a set time, the embedded Linux main control board enters a frequency-reduced standby state, while the STM32 low-power microcontroller continues to maintain acoustic standby scheduling, environmental status monitoring, and wake-up control. The system returns to the broadband acoustic low-power standby state, waiting for the next echo event of a suspected fish or plankton target larger than centimeters.

[0116] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention. Such modifications or substitutions should all fall within the scope of the invention, or any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. A centimeter-level acoustic-optical combined counting method for organisms under Arctic ice, characterized in that, Includes the following steps: Observation nodes were set up under the Arctic ice to establish a system reference coordinate system, acoustic coordinate system and optical coordinate system and to calibrate the external parameter transformation relationship. Acoustic transmission and reception, optical exposure and supplementary light pulses were synchronized to a unified time reference. Define the broadband acoustic detection region, the local acoustic tracking region, the optical registration volume, and the virtual registration surface as the counting interface; Broadband acoustic signals are periodically scanned in the broadband acoustic detection area. The echoes are then subjected to matched filtering and pulse compression to extract initial candidate targets that meet the preset signal-to-noise ratio and equivalent acoustic scale thresholds. Background filtering and validity screening are performed on the initial candidate targets, and non-static scattering targets that appear continuously for multiple periods and whose displacements meet the preset velocity constraints are retained as valid biological candidate targets. Based on acoustic scattering and motion characteristics, effective biological candidates are categorized into fish candidates, planktonic candidates, or unidentified biological targets. A high refresh rate rescan is performed on the local sector where each candidate target is located, which is higher than the broadband operating frequency. The position and velocity are estimated based on continuous position measurements using a Kalman filter, and the probability of entering the optical registration volume or the crossing time is predicted. When the probability reaches a preset threshold or the predicted trajectory intersects with the optical registration volume, a differentiated optical triggering plan is generated according to the candidate target category, triggering the camera to expose and opening short pulse supplementary light within the exposure window to obtain an optical image; The optical images are preprocessed and morphological features are extracted. The segmentation quality of the planktonic cluster candidates is evaluated. When it is determined that the individual organisms in the cluster cannot be reliably segmented, the adaptive degradation estimation mode is switched. The target category is output by fusing acoustic features, optical features, and spatial-temporal features, and the acoustic target intensity is corrected by attitude normalization using optical attitude information. Based on the three-dimensional trajectory of the target within the optical registration volume, determine whether it has effectively crossed the virtual registration surface, and include targets that meet the crossing conditions in the candidate set for deduplication crossing. For targets in the candidate traversal set, generate cross-modal identity tokens containing traversal time, spatial location, velocity, target scale, acoustic summary features, and optical summary features. Then, perform token similarity matching between the current token and historical identity tokens of the same category in the active identity cache table that are within a preset historical time window. If the similarity exceeds the threshold and the conditions for backtracking or reflow are met, it is determined to be a duplicate occurrence; otherwise, it is determined to be a new target and a counting event is generated. The system summarizes valid count events within a preset statistical time window, classifies them into fish, individual plankton, plankton clusters, and unidentified biological targets, and outputs the classification count results. For plankton clusters in adaptive degradation mode, the system outputs the number of cluster events and local quantity density estimates, and adds a degradation mode flag.

2. The method according to claim 1, characterized in that, The observation node integrates an embedded Linux main control board, an STM32 low-power microcontroller, a broadband acoustic transducer array, a local high-resolution acoustic transducer, a binocular optical camera, a short-pulse LED supplementary lighting component, a data storage module, a power management module, and a narrowband satellite communication module. The STM32 low-power microcontroller is used for device power-on and sleep scheduling, camera hardware triggering, and supplementary lighting pulse driving. The embedded Linux main control board is used for acoustic data parsing, coordinate transformation, trajectory prediction, image processing, acoustic-optical fusion recognition, counting, and identity token deduplication.

3. The method according to claim 1, characterized in that, The virtual registration surface A hemispherical surface is set within the optical registration volume, in the system reference coordinate system. The definition in Chinese is: ； in, Indicates the spatial position in the system's reference coordinate system. Indicates the center of the hemisphere. Indicates the radius of the hemisphere. The unit vector representing the orientation of the hemisphere. Represents the optical registration volume in the system's reference coordinate system. Indicates radius as Tolerance-expanding body, Minkowski and; condition Limited to With the center of the ball, For a sphere with radius , the condition is... Preserve the spherical surface along One side of the direction it points to, thus forming a hemisphere; condition This confines the hemisphere within the optical registration volume and its tolerance boundary region.

4. The method according to claim 1, characterized in that, The broadband acoustic signal is a linear frequency modulated signal with a center frequency of 200kHz to 400kHz and a bandwidth of 50kHz to 150kHz; the preset signal-to-noise ratio threshold is 10dB, and the equivalent acoustic scale threshold is 2cm.

5. The method according to claim 1, characterized in that, The specific conditions for the validity screening are that the following conditions must be met simultaneously: (a) The echo signal-to-noise ratio is not less than 10 dB; (b) It has been detected at least M times in the most recent N consecutive acoustic scans, where N is an integer between 3 and 10, and M is an integer not less than N / 2; (c) The equivalent acoustic scale is not less than 2 cm; (d) The velocity corresponding to the estimated displacement between two adjacent scans does not exceed the preset maximum velocity of the corresponding target category, wherein the maximum velocity of fish candidate targets is 1.0 m / s to 2.0 m / s, and the maximum velocity of plankton candidate targets is 0.1 m / s to 0.5 m / s; (e) The spatial location does not belong to the pre-marked static background reflection area; (f) Form traceable clusters on the distance-azimuth plane.

6. The method according to claim 1, characterized in that, The predicted probability of entering the optical registration volume or the crossing time specifically includes: the updated candidate target position using the Kalman filter. and speed of movement Short-time trajectory extrapolation is performed to predict the position of the candidate target at a future time t. Represented as: ； in, The time corresponding to the k-th acoustic measurement; the candidate target enters the optical registration volume at a future time t. probability Calculated based on the shortest distance between the predicted location and the optical registration volume: ； in, Indicates spatial location With optical registration volume The shortest Euclidean distance between them This indicates the uncertainty of the position at the corresponding prediction time; when the probability Maximum value within a future preset time range When the preset entry probability threshold is reached, the corresponding time is used. The optical triggering scheme is triggered as a predicted crossing time.

7. The method according to claim 1, characterized in that, The differentiated optical triggering scheme is specifically as follows: For fish candidates, a short-frame fast triggering mode is adopted. One frame is collected at the preset time before the main triggering time, the main triggering time, and the preset time after the main triggering time, which are respectively used as the pre-trigger frame, the main triggering frame, and the post-triggering frame. For individual planktonic candidates, a short-window multi-frame triggering mode is adopted, and a preset number of frames of images are collected at equal intervals within a finite time window near the prediction time. For planktonic cluster candidates, a continuous short-time triggering mode is adopted, and a preset number of frames of images are continuously collected at a preset time interval near the prediction time. For unconfirmed biological targets, a short-window, multi-frame triggering mode is used.

8. The method according to claim 1, characterized in that, The segmentation quality assessment includes calculating the overlap pixel ratio. Cluster contour closure and cluster area occupancy This generates a segmentation quality score. ,in These are the weighting coefficients; When the segmentation quality score Below the preset segmentation quality threshold, or the overlapping pixel ratio Cluster contour closure and cluster area occupancy When at least one of the preset segmentation failure conditions is met, it is determined that the individual cells within the cluster cannot be reliably segmented, and the system switches to adaptive degradation estimation mode; in adaptive degradation estimation mode, the local number density estimated by the acoustic volume backscattering coefficient is... and local number density estimated from optical images The final local number density estimate is obtained by weighted fusion of uncertainties. .

9. The method according to claim 1, characterized in that, The similarity matching is achieved by calculating the identity matching similarity between the current target j and the candidate historical target i. accomplish: ； in, Normalized weighted Euclidean distance: ； In the formula, and For the moment of crossing the virtual registration surface, and Register a location for the space. and For the speed of movement, and For the target scale, and This is the normalized acoustic summary feature vector. and is the normalized optical summarization feature vector; T, R, V, and S are the normalized scales for time difference, spatial position difference, velocity difference, and target scale difference, respectively. , , , , and These are the weight coefficients for the corresponding features, and their sum is 1; similarity threshold. When the maximum similarity And if the conditions for turning back or returning are met, it is determined to be a duplicate occurrence.

10. A centimeter-level acoustic-optical combined counting system for subglacial Arctic organisms, used to execute the centimeter-level acoustic-optical combined counting method for subglacial Arctic organisms as described in any one of claims 1 to 9, characterized in that, include: The broadband acoustic detection unit is used to transmit broadband acoustic signals and perform periodic scanning, perform matched filtering and pulse compression processing on the echoes, and extract initial candidate targets. The biological target screening unit is used to filter out background and screen the effectiveness of initial candidate targets to obtain valid biological candidate targets. The fish / plankton dual-object diversion unit is used to divert valid biological candidate targets into two objects, marking them as fish candidates, plankton candidates, or unconfirmed biological targets. A local high-resolution acoustic tracking unit is used to perform high-frequency, high-refresh-rate rescanning and Kalman filtering tracking on the split candidate target and to predict the probability of entering the optical registration volume or the crossing time. The low-disturbance optical registration unit is used to generate a differentiated optical triggering plan based on the target category when the probability reaches a threshold or the predicted trajectory intersects with the optical registration volume, and to trigger camera exposure and short-pulse illumination to obtain an optical image. The acoustic-optical cross-modal recognition unit is used to fuse acoustic features, optical features and spatial-temporal features to output the target category and perform attitude normalization correction of acoustic target intensity; The virtual registration counting unit is used to determine whether a target has effectively crossed the virtual registration surface and to include targets that meet the crossing conditions in the candidate set for deduplication. The active identity cache deduplication unit is used to generate cross-modal identity tokens for targets in the traversal candidate set, which include traversal time, spatial location, speed, target scale, acoustic summary features, and optical summary features. The current identity token is matched with historical identity tokens of the same category in the active identity cache table within a preset historical time window. If the matching similarity exceeds the threshold and meets the conditions for turning back or reflowing, it is determined to be a duplicate occurrence; otherwise, it is determined to be a new target and a counting event is generated. The quantity statistics output unit is used to summarize valid count events, distinguish between fish, individual plankton, plankton clusters, and unidentified biological targets, and output the classification count results; Perform time-series and feature-based statistical analysis on cross-modal identity tokens, and output time window statistical results, acoustic feature statistics, and optical feature statistics. A unified clock and cross-modal space calibration unit are used to provide a unified time reference and perform real-time coordinate transformation based on extrinsic parameter transformation relationships; The polar environment support unit is used to ensure the long-term operation of the system under Arctic ice, including a power management module, a low-power monitoring and scheduling module, a data storage module, and a narrowband satellite communication module; The unified clock and cross-modal spatial calibration unit is electrically connected to the broadband acoustic detection unit, the local high-resolution acoustic tracking unit, and the low-disturbance optical registration unit, respectively, providing the above units with a unified timestamp signal and real-time coordinate transformation parameters. The power management module in the polar environment support unit supplies power to all units, and the low-power monitoring and scheduling module is connected to the broadband acoustic detection unit and the local high-resolution acoustic tracking unit to control their working and sleep state switching. The narrowband satellite communication module is connected to the quantity statistics result output unit for transmitting compressed counting data products. The broadband acoustic detection unit, biological target screening unit, fish / plankton dual-object diversion unit, local high-resolution acoustic tracking unit, low-disturbance optical registration unit, acoustic-optical cross-modal recognition unit, virtual registration counting unit, active identity cache deduplication unit, and quantity statistics result output unit are connected in sequence.

Images