A ship black smoke automatic snapshot system

Abstract

The present application provides a kind of ship black smoke automatic snapshot system, it is related to ship environmental protection monitoring technical field, including long focus camera module, intelligent identification and trigger module, multi-source sensing control module, data fusion processing module, decision logic module and evidence video generation module.System is through long focus camera and collects water area video stream, after suspected black smoke is identified by intelligent algorithm, triggers multi-spectral and acoustic sensor synchronous data acquisition, fusion generates characteristic traceability vector, by decision logic module threshold control short focus camera record evidence video and embed characteristic data, it has solved the existing single camera monitoring by environment interference, evidence is not complete, bandwidth is occupied high and the problem that ship identity is associated difficult, realized the accurate identification of ship black smoke, multi-source evidence and low bandwidth dependence integrated automatic supervision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship environmental protection monitoring, and particularly relates to a ship black smoke automatic snapshot system. BACKGROUND

[0002] With the increasingly stringent environmental protection supervision requirements, the problem of ship black smoke emission has become a key and difficult point of maritime law enforcement and air pollution control. At present, some maritime agencies attempt to use existing video monitoring systems (CCTV) for black smoke identification, but significant defects are exposed in actual deployment and application: the existing cameras mainly serve the navigation safety monitoring, and their installation angle, focal length and field of view range cannot meet the technical requirements of black smoke identification; the cameras are usually erected high, and the shooting background is mostly water surface or buildings, which interferes with the accurate extraction of the algorithm for black smoke; a single camera cannot take into account long-time recording and picture clarity, resulting in insufficient time length of the evidence video and weak evidence effectiveness; in addition, the system generally lacks the automatic association capability of ship identity information, and cannot bind the black smoke event with the specific ship, which restricts the law enforcement efficiency. The existing scheme mostly deploys the identification algorithm on the central server, needs to continuously return the video stream, occupies a large amount of network bandwidth, causes system delay and cost rise.

[0003] Therefore, it is urgent to build a ship black smoke automatic snapshot system which can realize accurate identification, multi-source evidence collection, low bandwidth dependence and has the automatic association capability of ship identity. SUMMARY

[0004] In view of the above problems existing in the prior art, the present application provides a ship black smoke automatic snapshot system, which comprises, in sequence, a long-focus camera module, an intelligent identification and triggering module, a multi-source sensing control module, a data fusion processing module, a decision logic module and an evidence video generation module, wherein,

[0005] The long-focus camera module is used to perform step 1: based on the monitoring instruction of the target water area, the optical information is collected through the image sensor of the long-focus camera to obtain an initial ship video stream;

[0006] The intelligent identification and triggering module is used to perform step 2: based on the initial ship video stream, the intelligent analysis algorithm running on the black smoke identification computing unit is used for frame-by-frame processing, and a triggering instruction is generated when a suspected black smoke area is identified;

[0007] The multi-source sensing control module is used to perform step 3: based on the triggering instruction, the multi-spectral sensing module and the acoustic sensing module are synchronously started through the sensor control unit to respectively collect the spectral feature data and the acoustic vibration data of the ship chimney area;

[0008] The data fusion processing module is configured to perform step 4: based on the spectral feature data and the sound wave vibration data, comprehensive calculation is performed by a feature fusion module of the embedded data processing unit to generate a multi-dimensional feature trace vector;

[0009] The decision logic module is configured to perform step 5: based on the modulus of the feature trace vector, comparison is performed by a threshold comparison circuit with a preset threshold parameter to generate a camera start control signal;

[0010] The forensic video generation module is configured to perform step 6: based on the camera start control signal, the high-definition recording and encoding functions of the short-focus camera are started through the control interface of the short-focus camera to generate a forensic video embedded with the feature trace vector.

[0011] In some implementations, the spectral feature data obtained in step 3 includes:

[0012] Step 3-1: based on the target ship position information contained in the trigger instruction, the pitch angle and azimuth angle of the multi-spectral sensing module are adjusted by the pan-tilt control system to align the multi-spectral sensing module to the target chimney;

[0013] Step 3-2: based on the adjusted posture of the multi-spectral sensing module, photon collection and conversion are performed by the ultraviolet band detector of the multi-spectral sensing module to obtain an initial ultraviolet spectrum sequence;

[0014] Step 3-3: based on the initial ultraviolet spectrum sequence, noise reduction and standardization processing are performed by the signal conditioning circuit to obtain the spectral feature data.

[0015] In some implementations, the sound wave vibration data obtained in step 3 includes:

[0016] Step 3-4: based on the engine characteristic frequency provided by the ship database, the bandpass filter parameters are configured by the digital signal processor of the sound wave sensing module;

[0017] Step 3-5: based on the configured bandpass filter parameters, the original sound pressure signal is collected by the microphone array of the sound wave sensing module to obtain mixed sound wave data containing environmental noise;

[0018] Step 3-6: based on the mixed sound wave data, the digital signal processor performs a digital filtering algorithm to filter out out-of-band noise to obtain the sound wave vibration data.

[0019] In some implementations, step 3-6 includes:

[0020] Step 3-6a: based on the mixed sound wave data, time-domain to frequency-domain conversion is performed by the Fourier transform module to obtain frequency spectrum distribution data;

[0021] Step 3-6b: performing a frequency domain filtering operation based on the spectrum distribution data and a preset engine characteristic frequency range through a spectrum mask algorithm to obtain filtered spectrum data;

[0022] Step 3-6c: performing a time domain conversion operation from the frequency domain to the time domain based on the filtered spectrum data through an inverse Fourier transform module to obtain sound wave vibration data.

[0023] In some implementations, step 4 includes:

[0024] Step 4-1: based on the spectral feature data, calculating the time sequence change gradient of the spectral feature data through a first sub-module of the data processing unit to obtain a spectral change feature;

[0025] Step 4-2: based on the sound wave vibration data, performing a frequency domain conversion and calculating a spectral entropy of the sound wave vibration data through a second sub-module of the data processing unit to obtain a voiceprint feature;

[0026] Step 4-3: based on the confidence output by the intelligent analysis algorithm, performing a normalization process through a third sub-module of the data processing unit to obtain a visual recognition feature;

[0027] Step 4-4: based on the spectral change feature, the voiceprint feature, and the visual recognition feature, performing splicing and weighted calculation through a feature fusion module of the data processing unit to generate a multi-dimensional feature trace vector.

[0028] In some implementations, step 4-2 includes:

[0029] Step 4-2a: based on the sound wave vibration data, performing a windowing and framing processing operation through the second sub-module of the data processing unit to obtain a plurality of analysis frames;

[0030] Step 4-2b: based on the analysis frames, performing a discrete Fourier transform operation through a frequency domain conversion unit of the second sub-module to obtain a corresponding power spectral density distribution;

[0031] Step 4-2c: based on the power spectral density distribution, performing an entropy value calculation operation through a spectral entropy calculation unit of the second sub-module to obtain a spectral entropy value;

[0032] Step 4-2d: based on the spectral entropy value, performing a formatted output operation through a feature output unit of the second sub-module to generate a voiceprint feature.

[0033] In some implementations, step 4-4 includes:

[0034] Step 4-4a: based on the wind speed and visibility data monitored in real time by the environmental sensor, dynamically obtaining a set of environmental adaptive weight coefficients by querying a preset weight mapping table;

[0035] Step 4-4b: Based on the obtained weight coefficients, the spectral variation feature, the voiceprint feature, and the visual recognition feature are respectively weighted by the multiplier;

[0036] Step 4-4c: Based on the weighted feature values, a three-dimensional vector is generated by a vector synthesizer in a predetermined order, generating a multi-dimensional feature trace vector.

[0037] In some implementations, step 6 includes:

[0038] Step 6-1: Based on the camera start control signal, the power management circuit supplies power to the short-focus camera and initializes its image processing chip;

[0039] Step 6-2: Based on the initialized short-focus camera, the high-definition image sensor of the short-focus camera collects light signals and performs analog-to-digital conversion to obtain a high-definition video stream;

[0040] Step 6-3: Based on the high-definition video stream and the feature trace vector, the feature trace vector is written into the metadata area of the high-definition video stream by the data embedding function of the video encoder, generating a forensic video.

[0041] In some implementations, it further includes a hardware encryption module, a digital signature generation module, and a network transmission module, wherein,

[0042] The hardware encryption module is used to perform step 7: based on the forensic video, the encryption chip uses an asymmetric encryption algorithm to perform encryption operation, generating an encrypted video data packet;

[0043] The digital signature generation module is used to perform step 8: the hash algorithm is used to calculate the digest information of the encrypted video data packet, and the private key signature unit is used to sign the digest information, generating a signature file;

[0044] The network transmission module is used to perform step 9: the encrypted video data packet and the signature file are assembled according to the communication protocol, and sent to the designated cloud server.

[0045] In some implementations, step 9 includes:

[0046] Step 9-1: Based on the encrypted video data packet and the signature file, the protocol encapsulation unit adds the transport layer and network layer header information to assemble a network transmission packet;

[0047] Step 9-2: Based on the IP address of the designated cloud server, the optimal network transmission path is determined by the routing algorithm;

[0048] Step 9-3: Based on the determined optimal network transmission path, the network transmission packet is modulated and transmitted by the wireless communication module, completing the uploading operation.

[0049] Compared with the prior art, the present application has the beneficial effects that:

[0050] The ship black smoke automatic snapshot system effectively overcomes the shortcomings of the prior art through the cooperation of multiple modules. The long-focus camera module collects optical information based on the monitoring instruction of the target water area, generates an initial ship video stream, and provides stable video input for subsequent identification. The intelligent identification and triggering module runs in the black smoke identification calculation unit, processes the video stream frame by frame, can accurately identify the suspected black smoke area in the complex background and generate a triggering instruction, significantly improves the identification accuracy and response speed. The multi-source sensing control module synchronously starts the multi-spectral sensing module and the acoustic sensing module according to the triggering instruction, respectively collects the spectral feature data and acoustic vibration data of the ship chimney area, realizes the cooperative monitoring of multiple physical quantities, and enhances the robustness of the system under adverse weather or shielding conditions. The data fusion processing module performs feature fusion on the multi-source data through the embedded data processing unit, generates a multi-dimensional feature trace vector, and provides comprehensive data support for determining black smoke emission. The decision logic module compares the feature trace vector modulus value with the preset threshold value through the threshold comparison circuit, generates a camera start control signal, ensures that the evidence is only started in the case of high confidence, and avoids false triggering and resource waste. The evidence video generation module starts the short-focus camera for high-definition recording according to the control signal, and embeds the feature trace vector into the video metadata to generate an evidence video containing visual evidence and feature data, which meets the evidence integrity requirement of law enforcement.

[0051] The system realizes the full-process automation from identification, multi-source sensing, decision-making to evidence collection through modular division and process collaboration, significantly improves the black smoke ship identification efficiency, evidence credibility and system applicability. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0053] Figure 1 The structure schematic diagram of a ship black smoke automatic snapshot system provided by an embodiment of the present application is shown.

[0054] Figure 2 The streaming media processing and data management architecture schematic diagram involved in an embodiment of the present system is shown. EMBODIMENT

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0056] The specific embodiments of the present application are described below.

[0057] As shown in Figure 1 The present application proposes an automatic black smoke capturing system for ships, which comprises long-focus camera module, intelligent recognition and triggering module, multi-source sensing control module, data fusion processing module, decision logic module and evidence video generation module connected in sequence, wherein,

[0058] The long-focus camera module is used to perform step 1: based on the monitoring instruction of the target water area, the optical information is collected through the image sensor of the long-focus camera to obtain the initial ship video stream;

[0059] The intelligent recognition and triggering module is used to perform step 2: based on the initial ship video stream, the intelligent analysis algorithm running on the black smoke recognition calculation unit is used for frame-by-frame processing, and the triggering instruction is generated when the suspected black smoke area is identified;

[0060] The multi-source sensing control module is used to perform step 3: based on the triggering instruction, the multi-spectral sensing module and the acoustic sensing module are synchronously started through the sensor control unit to respectively collect the spectral feature data and the acoustic vibration data of the ship chimney area;

[0061] The data fusion processing module is used to perform step 4: based on the spectral feature data and the acoustic vibration data, the feature fusion module of the embedded data processing unit is used for comprehensive calculation to generate a multi-dimensional feature trace vector;

[0062] The decision logic module is used to perform step 5: based on the modulus of the feature trace vector, the threshold comparison circuit is compared with the preset threshold parameter to generate a camera start control signal;

[0063] The evidence video generation module is used to perform step 6: based on the camera start control signal, the high-definition recording and encoding functions of the short-focus camera are started through the control interface of the short-focus camera to generate the evidence video embedded with the feature trace vector.

[0064] Specifically, the implementation of an automatic ship black smoke capturing system, the core of which is to upgrade the process of black smoke identification from single visual judgment to intelligent decision-making of multi-source data fusion through the precise cooperation of a series of modules. The long-focus camera module, as the "eyes" of the system, starts work first. This module usually selects an industrial-grade camera with high optical zoom capability. Its image sensor (such as CMOS or CCD) is responsible for continuous monitoring of the preset target water area, capturing the dynamics of ships in the vast water area, and converting the optical signal into an electrical signal to form an initial ship video stream. This step provides the raw data basis for all subsequent analysis, ensuring the comprehensiveness of the monitoring range and the continuity of the data.

[0065] The initial ship video stream is transmitted in real time to the intelligent recognition and triggering module for processing. The core of this module is an intelligent analysis algorithm deployed on a black smoke identification computing unit (such as an embedded GPU or a dedicated AI acceleration chip). This algorithm is trained and iteratively optimized through a large number of video big data sets containing ship black smoke scenes. The algorithm performs pixel-level analysis on the input video stream frame by frame, extracts visual features such as smoke texture, motion trajectory, color features, and diffusion patterns, and compares them with the built-in black smoke model in the algorithm. When the algorithm's confidence exceeds a preset threshold, it is determined that there is a suspected black smoke area in the current picture, and a digital trigger instruction is generated. This process realizes the ability to quickly locate key suspicious events from massive video information, greatly reducing invalid data processing.

[0066] The trigger instruction is immediately sent to the multi-source sensor control module as soon as it is generated. This module serves as the "nerve center" of the system, responsible for coordinating subsequent special data collection. After receiving the instruction, the sensor control unit sends a start signal to the multi-spectral sensing module and the acoustic sensing module almost simultaneously. The multi-spectral sensing module (such as an ultraviolet band detector) is activated to capture the characteristic spectrum of combustion byproducts invisible to the human eye or difficult to distinguish by conventional cameras; at the same time, the acoustic sensing module (such as a directional microphone array) also starts working to collect specific acoustic vibration signals generated by ship engines and exhaust systems. This synchronous triggering mechanism ensures that the collection of spectral feature data and acoustic vibration data for the same suspected event is highly consistent in time, laying a solid foundation for subsequent data correlation and fusion.

[0067] The collected multi-source raw data is then fed into the data fusion processing module. This module runs on an embedded data processing unit (such as a high-performance DSP or FPGA), and its internal feature fusion module is responsible for comprehensive calculation on heterogeneous data. Instead of simply stacking data, the module first performs time series analysis on spectral feature data to capture its dynamic change rule, performs frequency domain transformation on sound wave vibration data to extract its characteristic frequency related to diesel engine combustion, and normalizes these features with the visual confidence given by the intelligent recognition algorithm. Finally, a multi-dimensional feature trace vector that can fully represent this suspected emission event is generated through weighted splicing and other methods. This vector contains richer and more objective physical information than a single visual judgment, significantly improving the accuracy of event determination.

[0068] The generated feature trace vector is passed to the decision logic module for final adjudication. The core of this module is a threshold comparison circuit or corresponding software logic. It compares the modulus (or some comprehensive score) of the input feature trace vector with a threshold parameter that is pre-calibrated through a large number of experiments and field data. This threshold parameter sets the minimum evidence strength required for a "black smoke event" determination. Only when the modulus of the vector exceeds the threshold, the decision logic module will generate a high-level camera start control signal. This step introduces a strict secondary verification mechanism, effectively filtering out visual false positives caused by water reflection, flying birds, cloud shadows, etc., ensuring that the system will only start the resource-consuming evidence collection process when sufficient evidence is obtained, thereby improving the overall reliability of the system.

[0069] The final camera start control signal activates the evidence video generation module. This module controls the short-focus camera (usually a high-definition visible light camera) to start its high-definition recording and encoding functions. The short-focus camera provides a wider field of view and higher resolution, ensuring that the recorded video can clearly show the entire ship and the process of its chimney emitting black smoke, meeting the picture integrity requirements needed as evidence for law enforcement. During the encoding process to generate video files, the data embedding function of the video encoder writes the previously generated feature trace vector as metadata (such as writing to the information area or custom data block) into the video stream, thereby generating an evidence video embedded with the feature trace vector. This video evidence not only contains intuitive visual images, but also embeds objective multi-dimensional sensor data, making the evidence chain more complete and credible, and difficult to be easily denied.

[0070] The system realizes wide area monitoring through a long-focus camera module, preliminary screening through an intelligent recognition and triggering module, deep perception through a multi-source sensing control module, information extraction through a data fusion processing module, accurate judgment through a decision logic module, and finally evidence fixation through an evidence video generation module. The modules are connected to each other, forming a complete automated evidence chain. The technical effect lies in greatly improving the automation level and accuracy of ship black smoke identification, freeing law enforcement personnel from the heavy video monitoring work; through multi-source sensing and data fusion technology, the weakness of single visual recognition being easily disturbed by the environment is effectively overcome, and the robustness of the system in complex scenes is enhanced; by completing the main identification and fusion decision on the terminal side, only the final trigger record and encrypted evidence video are uploaded, greatly reducing the continuous occupation of network bandwidth and solving the bandwidth bottleneck problem caused by the center deployment mode; the final evidence video embedded with multi-dimensional feature data provides strong and difficult-to-refute scientific evidence for maritime law enforcement, effectively solving the problems of difficult evidence collection and identification.

[0071] In some implementations, the step 3 of acquiring the spectral feature data comprises:

[0072] Step 3-1: Based on the target ship position information contained in the trigger instruction, adjust the pitch angle and azimuth angle of the multi-spectral sensing module through the pan-tilt control system to make the multi-spectral sensing module aim at the target chimney;

[0073] Step 3-2: Based on the adjusted multi-spectral sensing module posture, perform photon collection and conversion through the ultraviolet band detector of the multi-spectral sensing module to obtain an initial ultraviolet spectrum sequence;

[0074] Step 3-3: Based on the initial ultraviolet spectrum sequence, perform noise reduction and standardization processing through the signal conditioning circuit to obtain the spectral feature data.

[0075] Specifically, the process of obtaining spectral feature data in step 3 can be further refined to achieve more accurate measurement. Specifically, based on the position information of the target ship in the image contained in the trigger instruction (usually in the form of pixel coordinates), the multi-source sensor control module will drive a high-precision pan-tilt control system. This system is responsible for adjusting the pitch and azimuth of the multi-spectral sensing module. Its control algorithm will convert image coordinates into actual rotation angles of the pan-tilt based on the relative calibration parameters of the long-focus camera and the multi-spectral sensor, so that the optical axis of the multi-spectral sensing module is accurately aligned with the chimney area of the target ship, ensuring the relevance of the measurement and avoiding the collection of irrelevant background spectral data. After adjusting the posture of the multi-spectral sensing module and locking the target, the internal ultraviolet band detector begins to work. This detector is particularly sensitive to the characteristic emission spectrum of unburned hydrocarbons in the ship's exhaust in the ultraviolet band, and can convert the incident photon energy into an electrical signal. After preliminary amplification, an initial ultraviolet spectrum sequence is obtained, which contains the original light intensity variation with time or wavelength information. However, this initial sequence usually contains circuit noise and environmental stray light interference. Therefore, the subsequent signal conditioning circuit will process it, and digital filtering algorithms (such as moving average filtering or wavelet denoising) can be used to suppress random noise, and standardization processing (such as normalization to a certain reference value) can be used to eliminate the influence of absolute light intensity changes caused by distance, atmospheric attenuation, etc. Finally, stable and pure spectral feature data is output. This refined data acquisition process ensures that the collected spectral data directly and effectively reflects the characteristics of the target chimney's emissions, providing high-quality data input for subsequent high-precision fusion analysis.

[0076] Alternatively, the pan-tilt control can also use the preset strategy. For fixed monitoring points on major shipping lanes, several commonly used angles are set in advance. Once the target enters a certain area, the corresponding preset position is called to improve the response speed. Spectral detection is not limited to the ultraviolet band. In some embodiments, an infrared band detector can be added to detect thermal radiation information as a supplement or verification of ultraviolet detection.

[0077] In some implementations, obtaining sound wave vibration data in step 3 includes:

[0078] Step 3-4: Based on the engine characteristic frequency provided by the ship database, configure the bandpass filter parameters through the digital signal processor of the sound wave sensing module;

[0079] Step 3-5: Based on the configured bandpass filter parameters, collect the original sound pressure signal through the microphone array of the sound wave sensing module to obtain mixed sound wave data containing environmental noise;

[0080] Step 3-6: Based on the mixed acoustic wave data, a digital filtering algorithm is executed by the digital signal processor to filter out the out-of-band noise, and the acoustic vibration data is obtained.

[0081] Specifically, the process of obtaining the acoustic vibration data in Step 3 focuses on extracting the characteristic acoustic signals related to the target ship engine from the noisy environment. The ship database pre-stores the models of common ship engines and their corresponding characteristic frequency ranges (such as the fundamental frequency and its harmonics at the rated speed). Based on this information, the digital signal processor (DSP) of the acoustic sensing module will dynamically configure the bandpass filter parameters of its internal digital filter when starting up, setting the passband to the characteristic frequency range of the vibration noise that the target engine may produce, thereby focusing on the useful frequency band at the hardware level in advance. After the configuration is completed, the microphone array of the acoustic sensing module starts to collect the original sound pressure signals in the environment. The microphone array usually has a certain directivity, which can enhance the sound waves from the target direction and suppress the interference from other directions, but even so, the mixed acoustic wave data collected inevitably contains environmental noise such as water flow sound, wind sound, and noise from other ships. In order to separate the target signal from this acoustic reverberation, the digital signal processor executes its built-in digital filtering algorithm. This algorithm uses the previously configured bandpass filter parameters to process the mixed acoustic wave data, strongly attenuating the noise components outside the passband frequency range, thereby preliminarily purifying the acoustic vibration data that may be related to the vibration of the target ship engine. This process significantly improves the signal-to-noise ratio of the acoustic signal, laying a foundation for subsequent voiceprint analysis.

[0082] Alternatively, the setting of the filter parameters can not rely on the pre-set database, but through real-time analysis of the frequency spectrum of the mixed acoustic wave data, automatically find the frequency band with prominent energy and set the filter center frequency and bandwidth accordingly, with certain adaptive ability.

[0083] In some implementations, Step 3-6 includes:

[0084] Step 3-6a: Based on the mixed acoustic wave data, a time-to-frequency domain conversion operation is performed by the Fourier transform module to obtain the frequency spectrum distribution data;

[0085] Step 3-6b: Based on the frequency spectrum distribution data and the pre-set engine characteristic frequency range, a frequency domain filtering operation is performed by the frequency spectrum masking algorithm to obtain the filtered frequency spectrum data;

[0086] Step 3-6c: Based on the filtered frequency spectrum data, a frequency-to-time domain conversion operation is performed by the inverse Fourier transform module to obtain the acoustic vibration data.

[0087] Specifically, steps 3-6 can be further optimized by more advanced digital signal processing techniques to achieve better filtering results. Specifically, the digital signal processor first performs a windowed Fourier transform (such as STFT) on the mixed sound wave data, converting the time-domain sound wave signal to the frequency domain to obtain its spectral distribution data, which clearly shows the energy intensity of the signal at different frequency components. Subsequently, based on the pre-set engine characteristic frequency range (for example, a frequency window centered on the fundamental frequency and covering its main harmonics), the processor performs an operation called spectral mask algorithm. This algorithm generates a binary mask (Mask), with values set to 1 (allowed to pass) within the characteristic frequency range and values set to 0 (completely suppressed) outside the range, or using a soft mask generated according to the actual spectral shape. Multiplying this mask with the original spectral distribution data point by point, the frequency domain filtering operation is realized, and the energy outside the characteristic frequency range is completely removed, obtaining the filtered spectral data. Finally, by performing inverse Fourier transform on this filtered frequency domain data, it is converted back to the time domain, and finally the highly pure sound wave vibration data is obtained.

[0088] This method directly operates in the frequency domain and can accurately and completely filter out out-of-band noise, which is usually better than traditional digital filters designed directly in the time domain, and is particularly suitable for extracting periodic signals with specific frequency characteristics from strong background noise.

[0089] In some implementations, step 4 includes:

[0090] Step 4-1: Based on the spectral feature data, the time sequence change gradient of the spectral feature data is calculated by the first sub-module of the data processing unit to obtain the spectral change feature;

[0091] Step 4-2: Based on the sound wave vibration data, the spectral entropy of the sound wave vibration data is calculated by the second sub-module of the data processing unit through frequency domain transformation to obtain the voiceprint feature;

[0092] Step 4-3: Based on the confidence output by the intelligent analysis algorithm, the visual recognition feature is obtained by the normalization processing of the third sub-module of the data processing unit;

[0093] Step 4-4: Based on the spectral change feature, the voiceprint feature and the visual recognition feature, the multi-dimensional feature trace vector is generated by the feature fusion module of the data processing unit through splicing and weighted calculation.

[0094] In particular, the process of step 4 executed by the data fusion processing module is the key to generating the high-reliability feature provenance vector. The essence is to standardize and weight the fusion of features from different sources and with different physical meanings. First, the first submodule of the data processing unit (which can be a microcontroller core or a dedicated logic circuit) analyzes the spectral feature data. It calculates the rate of change or gradient of the data at consecutive time points to obtain the spectral change feature. This feature can quantify the dynamic evolution process of the spectral characteristics during the black smoke emission process, such as the sudden increase and slow decay of ultraviolet intensity, and the degree of change is an important indicator for judging whether a sudden emission has occurred. At the same time, the second submodule of the data processing unit (usually handled by a DSP core with FFT calculation capability) processes the sound wave vibration data. This module not only performs frequency domain transformation (such as FFT), but further calculates the spectral entropy of the transformed spectrum. The spectral entropy value reflects the uniformity of the frequency spectrum energy distribution: when the engine is running normally and stably, the spectral energy is concentrated in a few characteristic peaks, and the spectral entropy is low; when black smoke is produced due to insufficient combustion, the vibration may become irregular, the spectral energy is dispersed, and the spectral entropy value increases.

[0095] Therefore, the calculated spectral entropy value is output as a voiceprint feature, which describes the orderliness of the sound signal from an information theory perspective. In parallel, the third submodule of the data processing unit normalizes the confidence level output by the intelligent analysis algorithm, mapping it to a fixed numerical interval (such as between 0 and 1) to generate a visual recognition feature, which represents the strength of the pure visual evidence. Finally, the feature fusion module (which can be a coprocessor performing matrix operations) splices the above three features: the spectral change feature representing dynamic changes, the voiceprint feature representing orderliness, and the visual recognition feature representing visual credibility. When performing weighted calculation, the system can assign different weight coefficients to different features (for example, when visibility is extremely low, appropriately reduce the weight of the visual recognition feature and increase the weight of the voiceprint feature), and finally generate a multi-dimensional feature provenance vector. This vector integrates information from three modalities: light, sound, and image. Its dimensionality and comprehensiveness make the final decision based on its modulus more reliable and accurate than relying on a single information source.

[0096] In some implementations, step 4-2 includes:

[0097] Step 4-2a: based on the sound wave vibration data, performing a windowed framing operation by the second submodule of the data processing unit to obtain a plurality of analysis frames;

[0098] Step 4-2b: based on the analysis frames, performing a discrete Fourier transform operation by the frequency domain transformation unit of the second submodule to obtain the corresponding power spectral density distribution;

[0099] Step 4-2c: Based on the power spectral density distribution, an entropy value calculation operation is performed by the spectral entropy calculation unit of the second submodule to obtain a spectral entropy value;

[0100] Step 4-2d: Based on the spectral entropy value, a formatted output operation is performed by the feature output unit of the second submodule to generate a voiceprint feature.

[0101] Specifically, the voiceprint feature extraction process involved in step 4-2 can further enhance its representation ability through more detailed time-frequency analysis. In specific implementation, the second submodule of the data processing unit first performs windowing and framing processing operation on the sound wave vibration data. This is because the sound wave signal has time-varying characteristics, and windowing and framing can divide the long non-stationary signal into multiple short-time, approximately stationary signal segments (i.e. analysis frames), thereby meeting the requirement of signal stationarity for subsequent frequency domain analysis. Commonly used window functions include Hamming window or Hanning window, which can reduce the frequency spectrum leakage caused by signal truncation. After obtaining a series of consecutive analysis frames, the frequency domain transformation unit of the second submodule performs discrete Fourier transform operation on each frame of data. This transformation maps the time domain signal of each frame to the frequency domain, obtaining the power spectral density distribution reflecting the energy distribution of the signal at different frequencies within the time period. Based on this power spectral density distribution, the spectral entropy calculation unit begins to work. Spectral entropy is the application of information entropy concept in frequency spectrum analysis, which quantifies the uniformity of spectral energy through a specific entropy value calculation operation: a spectrum containing a few sharp spectral peaks (such as normal engine noise) has a lower spectral entropy value, indicating energy concentration and high order; while a signal with very uniform spectral energy distribution or containing a large number of chaotic spectral peaks (such as abnormal combustion noise or strong environmental interference) has a higher spectral entropy value, indicating energy dispersion and high disorder. The calculated spectral entropy value itself is a very representative scalar feature. Finally, the feature output unit is responsible for formatting the spectral entropy value for output operation, such as binding it with the corresponding timestamp or frame number, or combining it into a small time series, thereby generating the final voiceprint feature for fusion.

[0102] This spectral entropy-based voiceprint feature extraction method can effectively capture the internal statistical characteristics of noise signals, has good robustness to common stationary or non-correlated noise in the background environment, avoids misjudgment caused by relying only on specific frequency amplitudes, and makes the acoustic evidence more reliable and discriminative.

[0103] In some implementations, step 4-4 includes:

[0104] Step 4-4a: Based on the wind speed and visibility data monitored by the environmental sensor in real time, a set of environment-adaptive weight coefficients is dynamically obtained by querying a pre-set weight mapping table;

[0105] Step 4-4b: Based on the obtained weight coefficients, the spectral change feature, the voiceprint feature, and the visual recognition feature are respectively weighted by the multipliers;

[0106] Step 4-4c: Based on the weighted feature values, a three-dimensional vector is combined by the vector synthesizer in a predetermined order to generate a multi-dimensional feature trace vector.

[0107] Specifically, the feature fusion process of step 4-4 can be designed to be more intelligent, by introducing environmental perception capability to dynamically adjust the fusion strategy, thereby improving the adaptability of the system under different working conditions. Specifically, the system will be equipped with additional environmental sensors (such as an anemometer and a visibility meter) to monitor the wind speed and visibility data in real time. These external parameters are sent to the data fusion processing module in real time. Based on these real-time data, the feature fusion module dynamically obtains a set of environment adaptive weight coefficients by querying a pre-set weight mapping table. The weight mapping table is calibrated in advance through a large number of experiments and data analysis, which defines the allocation strategy of the confidence (i.e. weight) of the spectral change feature, the voiceprint feature, and the visual recognition feature under different wind speed and visibility conditions. For example, when the wind speed is high, the black smoke may be blown away quickly, causing the confidence of the visual recognition feature to decrease and the spectral feature to change dramatically, at which time the system may appropriately reduce the weight of the visual recognition feature. When the visibility is extremely low (such as heavy fog), the visual and optical sensors are limited, and the relative importance of the voiceprint feature is increased. Conversely, under ideal conditions of calm wind and clear weather, the weights of visual and optical evidence will dominate. After obtaining this set of dynamic weight coefficients, the multipliers in the fusion module use them to respectively weight the normalized spectral change feature, voiceprint feature, and visual recognition feature, amplify the contribution of reliable evidence, and suppress the influence of unreliable evidence. Finally, the vector synthesizer combines the three weighted feature values into a three-dimensional feature trace vector in a predetermined order (for example, [weighted spectral change feature, weighted voiceprint feature, weighted visual recognition feature]).

[0108] This environment-adaptive weighted fusion mechanism enables the feature trace vector output by the system to sensitively reflect the optimal judgment basis under the current environment, greatly enhancing the accuracy and reliability of the entire system when facing complex and variable external weather conditions. It is an important evolution from "static fusion" to "dynamic intelligent fusion".

[0109] In some implementations, step 6 includes:

[0110] Step 6-1: Based on the camera start control signal, the short-focus camera is powered and its image processing chip is initialized by the power management circuit;

[0111] Step 6-2: Based on the initialized short-focus camera, the high-definition image sensor of the short-focus camera collects light signals and performs analog-to-digital conversion to obtain a high-definition video stream;

[0112] Step 6-3: Based on the high-definition video stream and the feature trace vector, the feature trace vector is written into the metadata area of the high-definition video stream through the data embedding function of the video encoder, generating a forensic video.

[0113] Specifically, the forensic video generation process of step 6 can be further refined to ensure the integrity and traceability of the forensic video. The camera start control signal is not only a logic signal, but also used to control the power supply sequence. Based on this signal, the power management circuit is activated, which provides a stable working voltage for the short-focus camera and performs power-on timing control, while initializing its core image processing chip, loading various parameters (such as white balance, exposure value, encoding format), so that it quickly enters standby state. After initialization, the high-definition image sensor of the short-focus camera starts working, and the optical signals captured by its lens are photoelectrically converted and analog-to-digital converted to generate an uncompressed high-definition video stream. At the same time, the feature trace vector generated by the data fusion processing module is also sent to the video encoder. In addition to performing regular video compression and encoding functions, the video encoder also calls its data embedding function. This function allows non-video data (i.e. feature trace vector) to be written in a specific format to the metadata area of the final generated video file. The metadata area is a part of the video stream used to store additional information without affecting the decoding of the main video picture, such as SEI (Supplemental Enhancement Information) units or custom user data areas. In this way, the feature trace vector is seamlessly and invisibly written into the high-definition video stream, ultimately generating a forensic video that contains both visually observable emission process pictures and objectively quantified sensor data.

[0114] This video evidence realizes the close combination of visual information and multi-dimensional feature data, so that when reviewing afterwards, law enforcement personnel can not only watch the video, but also directly read the embedded machine analysis data such as spectrum and voiceprint, providing double and mutually verified iron evidence for the identification of illegal emissions, greatly enhancing the scientificity and authority of the evidence.

[0115] In some implementations, it also includes a hardware encryption module, a digital signature generation module, and a network transmission module, wherein,

[0116] The hardware encryption module is used to perform step 7: based on the forensic video, the encryption chip uses an asymmetric encryption algorithm to perform encryption operations to generate an encrypted video data packet;

[0117] The digital signature generation module is configured to perform step 8: calculating the digest information of the encrypted video data packet by a hash algorithm, and digitally signing the digest information by a private key signing unit to generate a signature file;

[0118] The network transmission module is configured to perform step 9: assembling the encrypted video data packet and the signature file according to a communication protocol, and sending to a designated cloud server.

[0119] Specifically, in order to further improve the security and legal effectiveness of the system output evidence, and ensure that the data is not tampered with during transmission and storage, the system can also integrate a complete data security reinforcement scheme. After the forensic video generation module outputs the forensic video, the hardware encryption module immediately intervenes. The module is usually composed of a dedicated encryption chip (such as a chip that meets the national standard), which uses an asymmetric encryption algorithm to encrypt the complete forensic video data. The encryption process uses a pre-set public key, and the generated encrypted video data packet cannot be decrypted and viewed without the corresponding private key, effectively preventing video evidence from being leaked or snooped during transmission. Then, the digital signature generation module starts working. It first calculates the digest information of the encrypted video data packet by a hash algorithm, which is the unique "fingerprint" of the data packet, and any minor changes to the data will cause the digest information to change greatly. Then, a signature unit (which can be another functional unit in the encryption chip) that securely stores the private key is used to digitally sign the digest information, generating an independent signature file. This signature file corresponds to the encrypted video data packet and is used to prove that the data packet was indeed generated by the system and has not been tampered with since its generation. Finally, the network transmission module is responsible for combining and packaging the encrypted video data packet and the signature file. It assembles the data according to the established communication protocol (such as the TCP / IP protocol family), adds the necessary packet header information, and then sends it to the designated cloud server through the wireless network (such as 4G / 5G) for centralized storage and subsequent processing.

[0120] This security mechanism increases the confidentiality, integrity and non-repudiation of the data, making the final uploaded evidence chain have certain evidence effectiveness, meeting the stringent requirements of maritime law enforcement for evidence security and credibility.

[0121] In some implementations, step 9 includes:

[0122] Step 9-1: based on the encrypted video data packet and the signature file, adding transport layer and network layer header information through a protocol packaging unit to assemble into a network transmission packet;

[0123] Step 9-2: based on the IP address of the designated cloud server, determine the optimal network transmission path through a routing algorithm;

[0124] Step 9-3: Based on the determined optimal network transmission path, the network transmission package is modulated and transmitted by the wireless communication module, completing the uploading operation.

[0125] Specifically, step 9 can further improve the efficiency and reliability of data transmission by optimizing the network transmission strategy. After receiving the encrypted video data package and the signature file, the protocol encapsulation unit inside the network transmission module does not send it directly, but first adds transport layer header information (such as source / destination port number, sequence number) and network layer header information (such as source / destination IP address) to the data according to the adopted transmission protocol (for example, based on TCP), and assembles these independent data units into one or more network transmission packages that conform to the network transmission specification. Based on the cloud server IP address, the built-in routing algorithm in the module starts working. This algorithm takes into account the current network conditions (such as signal strength, bandwidth, delay, and packet loss rate of each available network, which can be obtained through network detection), dynamically calculates and determines an optimal network transmission path at the current time from multiple available network paths (such as the primary 4G link and the backup wired network), to achieve the fastest speed or the highest reliability of transmission. After the optimal path is determined, the wireless communication module modulates the assembled network transmission package to a specific radio frequency band and transmits it out, and through the base station and other infrastructure, the uploading operation to the cloud server is finally completed.

[0126] This intelligent network transmission management can effectively adapt to the complex and unstable network environment on site, and in the case of limited bandwidth, it can prioritize the reliable uploading of key evidence data, avoiding data loss caused by network jitter or interruption, and ensuring the complete delivery of law enforcement evidence.

[0127] In an alternative embodiment, the system can further include a streaming media processing and data management module, the architecture diagram of which is shown in Figure 2 The module includes a streaming media processing sub-module, a database sub-module, an analyzer unit, and a permission management unit. The streaming media processing sub-module is responsible for pulling video streams from the long-focus camera module and the short-focus camera module in real time, decoding, encoding, and streaming, and achieving efficient scheduling and transmission of video data. The database sub-module is used to store various data generated during system operation, including ship feature data, black smoke event records, forensic video and its metadata, etc. The analyzer unit interfaces with the intelligent identification and triggering module, analyzes the video stream in real time, identifies black smoke events, and triggers subsequent multi-source data acquisition and fusion processes. The permission management unit is used for identity verification and permission allocation of system operators, to ensure data security and operation compliance. This system architecture supports unified management of camera IP addresses and can synchronize data with the cloud server and interact remotely, further improving the deployment flexibility and management efficiency of the system in complex water environments.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. An automatic ship black smoke snapshot system, characterized in that, The system comprises a long-focus camera module, an intelligent recognition and triggering module, a multi-source sensing control module, a data fusion processing module, a decision logic module and a forensic video generation module connected in sequence, wherein, The long-focus camera module is used to perform step 1: based on the monitoring instruction of the target water area, the optical information is collected through the image sensor of the long-focus camera to obtain an initial ship video stream; The intelligent recognition and triggering module is used to perform step 2: based on the initial ship video stream, the intelligent analysis algorithm running on the black smoke recognition calculation unit is used for frame-by-frame processing, and a triggering instruction is generated when a suspected black smoke area is identified; The multi-source sensing control module is used to perform step 3: based on the triggering instruction, the multi-spectral sensing module and the acoustic sensing module are synchronously started through the sensor control unit to respectively collect the spectral feature data and the acoustic vibration data of the ship chimney area; wherein, the acoustic vibration data obtained in step 3 includes: Step 3-4: based on the engine characteristic frequency provided by the ship database, the bandpass filter parameters are configured through the digital signal processor of the acoustic sensing module; Step 3-5: based on the configured bandpass filter parameters, the original sound pressure signal is collected through the microphone array of the acoustic sensing module to obtain mixed acoustic data containing environmental noise; Step 3-6: based on the mixed acoustic data, the digital signal processor is used to perform a digital filtering algorithm to filter out out-of-band noise to obtain the acoustic vibration data; Step 3-6 includes: Step 3-6a: based on the mixed acoustic data, the time domain to frequency domain conversion operation is performed through the Fourier transform module to obtain the frequency spectrum distribution data; Step 3-6b: based on the frequency spectrum distribution data and the preset engine characteristic frequency range, the frequency domain filtering operation is performed through the frequency spectrum mask algorithm to obtain the filtered frequency spectrum data; Step 3-6c: based on the filtered frequency spectrum data, the frequency domain to time domain conversion operation is performed through the inverse Fourier transform module to obtain the acoustic vibration data; The data fusion processing module is used to perform step 4: based on the spectral feature data and the acoustic vibration data, the feature fusion module of the embedded data processing unit is used for comprehensive calculation to generate a multi-dimensional feature trace vector; The decision logic module is used to perform step 5: based on the modulus of the feature trace vector, the threshold value comparison circuit is compared with the preset threshold value parameter to generate a camera start control signal; The forensic video generation module is used to perform step 6: based on the camera start control signal, the high-definition recording and encoding functions of the short-focus camera are started through the control interface of the short-focus camera to generate a forensic video embedded with the feature trace vector.

2. An automatic black smoke snapshot system for a marine vessel according to claim 1, characterized in that, The spectral feature data obtained in step 3 includes: Step 3-1: based on the target ship position information contained in the triggering instruction, the pitch and azimuth of the multi-spectral sensing module are adjusted through the pan-tilt control system to align the multi-spectral sensing module to the target chimney; Step 3-2: based on the adjusted posture of the multi-spectral sensing module, the photonic collection and conversion are performed through the ultraviolet band detector of the multi-spectral sensing module to obtain an initial ultraviolet spectrum sequence; Step 3-3: Based on the initial ultraviolet spectrum sequence, noise reduction and standardization processing are performed by the signal conditioning circuit to obtain the spectrum feature data.

3. An automatic black smoke snapshot system for a marine vessel as claimed in claim 1, wherein, Step 4 includes: Step 4-1: Based on the spectrum feature data, the time sequence change gradient of the spectrum feature data is calculated by the first sub-module of the data processing unit to obtain the spectrum change feature; Step 4-2: Based on the acoustic wave vibration data, the frequency domain transformation is performed by the second sub-module of the data processing unit, and the spectral entropy of the acoustic wave vibration data is calculated to obtain the voiceprint feature; Step 4-3: Based on the confidence output by the intelligent analysis algorithm, normalization processing is performed by the third sub-module of the data processing unit to obtain the visual recognition feature; Step 4-4: Based on the spectrum change feature, the voiceprint feature and the visual recognition feature, splicing and weighted calculation are performed by the feature fusion module of the data processing unit to generate a multi-dimensional feature trace vector.

4. An automatic black smoke snapshot system for a marine vessel according to claim 3, characterized in that, Step 4-2 includes: Step 4-2a: Based on the acoustic wave vibration data, windowing and framing processing operations are performed by the second sub-module of the data processing unit to obtain a plurality of analysis frames; Step 4-2b: Based on the analysis frame, the discrete Fourier transform operation is performed by the frequency domain transformation unit of the second sub-module to obtain the corresponding power spectrum density distribution; Step 4-2c: Based on the power spectrum density distribution, the entropy value calculation operation is performed by the spectral entropy calculation unit of the second sub-module to obtain the spectral entropy value; Step 4-2d: Based on the spectral entropy value, the format output operation is performed by the feature output unit of the second sub-module to generate the voiceprint feature.

5. An automatic black smoke snapshot system for a marine vessel according to claim 3, characterized in that, Step 4-4 includes: Step 4-4a: Based on the wind speed and visibility data monitored by the environmental sensor in real time, a set of environmental adaptive weight coefficients are dynamically obtained by querying the pre-set weight mapping table; Step 4-4b: Based on the obtained weight coefficients, the spectrum change feature, the voiceprint feature and the visual recognition feature are respectively weighted by the multiplier; Step 4-4c: Based on the weighted feature values, a three-dimensional vector is combined in a predetermined order by the vector synthesizer to generate a multi-dimensional feature trace vector.

6. An automatic black smoke snapshot system for a marine vessel as claimed in claim 1, wherein, Step 6 includes: Step 6-1: Based on the camera start control signal, the power supply management circuit supplies power to the short-focus camera and initializes its image processing chip; Step 6-2: Based on the initialized short-focus camera, the light signal is collected by the high-definition image sensor of the short-focus camera and subjected to analog-to-digital conversion to obtain a high-definition video stream; Step 6-3: Based on the high-definition video stream and the feature trace vector, the feature trace vector is written into the metadata area of the high-definition video stream by the data embedding function of the video encoder to generate the forensic video.

7. An automatic black smoke snapshot system for a marine vessel according to claim 1, characterized in that, It also includes a hardware encryption module, a digital signature generation module and a network transmission module, wherein The hardware encryption module is used to perform step 7: based on the forensic video, the encryption chip uses an asymmetric encryption algorithm to perform encryption operation to generate an encrypted video data packet; The digital signature generation module is used to perform step 8: calculate the digest information of the encrypted video data packet using a hash algorithm, and digitally sign the digest information using a private key signing unit to generate a signature file; The network transmission module is used to perform step 9: assembling the encrypted video data packet and the signature file according to the communication protocol, and sending them to the designated cloud server.

8. An automatic black smoke snapshot system for a marine vessel according to claim 7, characterized in that, Step 9 includes: Step 9-1: Based on the encrypted video data packet and signature file, add transport layer and network layer header information through the protocol encapsulation unit to assemble it into a network transmission packet; Step 9-2: Based on the specified cloud server IP address, determine the optimal network transmission path using a routing algorithm; Step 9-3: Based on the determined optimal network transmission path, the network transmission packet is modulated and transmitted through the wireless communication module to complete the upload operation.

Images