Real-time offshore safety intelligent video monitoring system and method thereof

Abstract

The invention relates to the technical field of marine safety monitoring, in particular to a real-time marine safety intelligent video monitoring system and method, and the system comprises a global monitoring acquisition module, a local threat analysis module, a dynamic response control module, a target trajectory prediction module and a remote recognition tracking module. According to the invention, by collecting and fusing optical brightness and thermal imaging temperature data, the innovative scheme significantly improves the accuracy of target detection in a complex marine environment, optimizes the monitoring effect under a low visual condition, enhances the rapid response capability of the system to dynamic changes, tracks the target position and motion update in real time, and improves the accuracy of target detection in a complex marine environment. The real-time performance and continuity of the monitoring system are improved, the response to emergencies is quicker, prediction of future target movement and advanced recognition of potential threats are achieved, and support is provided for reasonable allocation of emergency response resources, so that the efficiency and effect of the whole monitoring system are improved while the safety of offshore operation is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of maritime safety monitoring, and in particular to a real-time maritime safety intelligent video monitoring system and method thereof. Background Art

[0002] The field of maritime safety monitoring technology includes technical content related to the safety monitoring and management of personnel, ships and equipment involved in maritime activities. The core content of this technical field mainly includes the use of monitoring equipment to perform real-time video acquisition, information transmission, image analysis and remote monitoring, so as to achieve all-round monitoring of the maritime environment, ship operating areas and surrounding conditions. The overall technical field covers maritime radar systems, video monitoring systems, environmental data acquisition systems and artificial intelligence-based image recognition and processing technologies. Through the coordinated operation of these technologies, comprehensive management of maritime operating areas can be achieved. At the same time, this field also integrates VHF very high frequency receiving technology and AIS recognition technology. By receiving VHF signals and AIS (automatic identification system) information broadcast by ships, the identity, position, heading and speed of ships in the sea area are monitored in real time, providing more data support for maritime safety.

[0003] Among them, the real-time intelligent video surveillance system for maritime safety refers to the use of video acquisition equipment to conduct video surveillance of key areas in the maritime environment, and to realize automatic identification, tracking and data extraction of specific targets through image processing technology. The technical matters covered by this patent subject include video acquisition of real-time scenes based on optical imaging equipment, multi-frame analysis of the acquired video stream with an embedded image processing unit, and screening of abnormal targets within the monitoring range through rule-based target discrimination technology or image classification technology, while transmitting the processed monitoring information to the remote monitoring center through a wireless communication module. The system further integrates VHF very high frequency information receiving function to capture maritime communication data, and AIS identification technology to extract the navigation status information of the ship. Through the combination of these technologies, the system can achieve more accurate target identification and global situational awareness. The system completes the monitoring and analysis of dynamic scenes at sea through a unified hardware architecture and algorithm process.

[0004] Existing maritime surveillance technologies primarily rely on real-time video acquisition and image analysis, but their effectiveness is limited in low-light conditions or in adverse weather. Their effectiveness decreases significantly at night or in foggy conditions, making it difficult to accurately identify and track targets. Image processing methods that rely on fixed rules are less adaptable to non-standard target behavior, prone to misjudgments or delayed responses. Furthermore, the lack of effective data fusion strategies makes it difficult to quickly distinguish targets in emergency situations, hindering the speed and accuracy of decision-making. These shortcomings limit the application of existing technologies in complex maritime environments and hinder the comprehensiveness and practicality of surveillance systems. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a real-time maritime safety intelligent video monitoring system and method.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical solution: A real-time maritime safety intelligent video monitoring system comprises:

[0007] The global monitoring acquisition module collects optical images, VHF reception information, thermal imaging data, radar signals, and AIS identification information. It extracts the brightness distribution characteristics of optical images, thermal imaging temperature parameters, radar amplitude changes, and direction data, marks dynamic areas, and creates a distribution map of potential threat areas.

[0008] The local threat analysis module extracts optical brightness data and thermal imaging temperature parameters based on the potential threat area distribution map, analyzes the thermal imaging temperature change amplitude and distribution characteristics, extracts target speed and direction data in the dynamic area, classifies and filters target data with abnormal characteristics, and generates a threat target distribution map;

[0009] The dynamic response control module extracts the target position and dynamic motion parameters based on the threat target distribution map, analyzes the interactive relationship between the target position and the mission range of the UAV and patrol boat, records the equipment mission completion time and the captured target position, updates the captured target time series and position distribution, and generates a real-time capture list of threat targets;

[0010] The target trajectory prediction module extracts time series and movement direction data based on the real-time capture list of threat targets, records the target displacement range and path trend, analyzes the time interval and position range of the target's future activity area, and generates a map of the future threat target activity area;

[0011] The remote identification and tracking module extracts the target activity range image frame sequence based on the future threat target activity area map, extracts the brightness signal and thermal imaging signal, eliminates the background pixel data, records the target pixel trajectory, and generates a remote tracking target path data set.

[0012] As a further solution of the present invention, the potential threat area distribution map includes dynamic area marking, geographic information matching, and radar data analysis; the threat target distribution map includes speed classification results, direction analysis results, and abnormal target identification; the threat target real-time capture list includes task assignment details, equipment execution status, and target position updates; the future threat target activity area map includes path prediction correction, activity time analysis, and boundary data annotation; the remote tracking target path dataset includes image frame analysis, brightness and thermal imaging data integration, and background data elimination.

[0013] As a further solution of the present invention, the global monitoring acquisition module includes an optical feature extraction submodule, a thermal imaging parameter analysis submodule, and a radar signal dynamic marking submodule;

[0014] The optical feature extraction submodule analyzes the brightness distribution characteristics of the image based on the optical image data, calculates the brightness values ​​of the pixels, compares the spatial distribution changes of the brightness values, determines the difference between the brightness mutation area and the optical feature distribution, marks the abnormal brightness distribution area in the image, and generates the brightness distribution features;

[0015] The thermal imaging parameter analysis submodule is based on the brightness distribution characteristics and thermal imaging data. It extracts the temperature information of thermal imaging pixels, analyzes the temperature value distribution trend, screens the temperature fluctuation area, calculates the temperature change rate, determines the correlation between the abnormal temperature change area and the brightness distribution area, and generates the thermal imaging temperature characteristics.

[0016] The radar signal dynamic marking submodule analyzes the radar amplitude change trend, calculates the direction change angle, analyzes the amplitude change range, determines the dynamic signal changes in the area overlapping with the temperature characteristics, marks the potential threat position of the dynamic characteristics, and establishes a potential threat area distribution map based on the thermal imaging temperature characteristics and radar signal data.

[0017] As a further solution of the present invention, the temperature change rate calculation formula is specifically:

[0018]

[0019] Among them, R t1 represents the temperature change rate, T 1i Represents the pixel temperature value at the i-th time point in the thermal imaging data, T 1(i-1) represents the pixel temperature value at the i-1th time point in the thermal imaging data, n represents the total number of thermal imaging data in the time series, Represents the average temperature value of all thermal imaging pixels in the time series, ∑ is the summation symbol, Represents the absolute difference between the temperature value of the i-th thermal imaging pixel and the average value.

[0020] As a further solution of the present invention, the local threat analysis module includes a threat area parameter extraction submodule, a dynamic target characteristic analysis submodule, and a threat target classification and screening submodule;

[0021] The threat area parameter extraction submodule extracts optical brightness data and thermal imaging temperature parameters based on the potential threat area distribution map, analyzes the spatial variation and gradient distribution of the optical image brightness, calculates the fluctuation trend of the thermal imaging temperature data on the time axis, and analyzes the spatial gradient distribution characteristics to generate threat area characteristic data;

[0022] The dynamic target characteristic analysis submodule extracts the moving speed and direction information of the dynamic area target based on the threat area characteristic data, calculates the instantaneous change value of the speed, identifies the direction offset angle, compares the time series fluctuation characteristics of the speed data and the direction data, and generates dynamic target characteristic data;

[0023] The threat target classification and screening submodule analyzes the distribution characteristics of speed and direction data based on the dynamic target characteristic data, counts the spatial area density of dynamic targets, marks target characteristic data with differences, regroups and classifies them according to dynamic characteristics and spatial characteristics, and establishes a threat target distribution map.

[0024] As a further solution of the present invention, the calculation formula of the instantaneous change value of speed is specifically:

[0025]

[0026] Among them, V3 represents the instantaneous change value of the velocity, x3 represents the position change of the target in the x direction, y3 represents the position change of the target in the y direction, t3 represents the time interval, θ3 represents the direction change angle, and cosθ3 is the cosine value of the direction angle.

[0027] As a further solution of the present invention, the dynamic response control module includes a target interaction analysis submodule, a capture time recording submodule, and a target distribution update submodule;

[0028] The target interaction analysis submodule extracts the target position and dynamic motion parameters based on the threat target distribution map, calculates the spatial distribution weight of the target position within the mission range, analyzes the interaction frequency of the target position within the mission range of the UAV and patrol boat, and generates mission interaction target data;

[0029] The capture time recording submodule records the start and completion time of the device task based on the task interaction target data, extracts the spatial position of the capture target, analyzes and captures the matching relationship between the target position and time, generates the time series distribution characteristics of the target, and generates the target capture time series;

[0030] The target distribution update submodule updates the spatial position distribution data of the captured targets based on the target capture time series, analyzes the spatial dynamic distribution of target capture in combination with the time series data, marks the time characteristics of the captured target positions, and establishes a real-time capture list of threat targets.

[0031] As a further solution of the present invention, the target trajectory prediction module includes a target path characteristic extraction submodule, an activity area trend analysis submodule, and a future activity area prediction submodule;

[0032] The target path characteristic extraction submodule extracts the target time series and movement direction data based on the real-time capture list of threat targets, records the displacement range and path change trend of the target between consecutive time points, calculates the change amplitude of the path in the time series, integrates time and space data to analyze the path characteristics, and generates target path characteristic data;

[0033] The activity area trend analysis submodule analyzes the temporal trend of the movement direction based on the target path characteristic data, records the offset range of the movement direction, calculates the temporal continuity characteristics of the path change, analyzes the activity area dynamics in combination with the time series fluctuation characteristics of the target path, and generates activity area trend data;

[0034] The future activity area prediction submodule predicts the time interval and spatial boundary of the target activity area based on the activity area trend data and the movement law of the target time series, calculates the spatial dynamic distribution characteristics of the targets in the activity area, integrates the time and space characteristic data, and establishes a future threat target activity area map.

[0035] As a further solution of the present invention, the remote identification and tracking module includes a target image frame extraction submodule, a background data removal submodule, and a target trajectory recording submodule;

[0036] The target image frame extraction submodule extracts the image frame sequence within the activity range based on the future threat target activity area map, records the image frame time sequence and corresponding spatial coordinates, analyzes the distribution range of dynamic targets in the image frame, calculates the temporal and spatial variation characteristics of the dynamic area between image frames, integrates the image frames to form a temporal and spatial matching relationship, and generates the target image frame sequence;

[0037] The background data removal submodule extracts the brightness signal and thermal imaging signal data from the target image frame sequence, analyzes the pixel distribution and regional gradient characteristics of the signal, filters the background features of the static area in the pixel signal, removes the background data that does not conform to the dynamic target characteristics, records the effective signal within the dynamic pixel range, and generates the target effective pixel signal;

[0038] The target trajectory recording submodule records the position changes of dynamic pixels in continuous image frames based on the target effective pixel signals, calculates the time series and spatial trajectory of pixel displacement, analyzes the dynamic characteristics and continuity trend of the trajectory, integrates the spatial path and time relationship of the trajectory, and establishes a long-range tracking target path dataset.

[0039] A real-time maritime safety intelligent video monitoring method comprises the following steps:

[0040] S1: By collecting optical images, VHF reception information, thermal imaging data, radar signals, and AIS identification information, the brightness distribution characteristics of optical images, thermal imaging temperature parameters, radar amplitude changes, and direction data are extracted to establish a distribution map of potential threat areas.

[0041] S2: Based on the potential threat area distribution map, extract optical brightness data and thermal imaging temperature parameters, extract target speed and direction data in the dynamic area, classify and filter abnormal characteristic target data, and generate a threat target distribution map;

[0042] S3: extracting target positions and dynamic motion parameters based on the threat target distribution map, analyzing the interactive relationship between target positions and the mission ranges of drones and patrol boats, updating the captured target time series and position distribution, and generating a real-time capture list of threat targets;

[0043] S4: Based on the real-time capture list of threat targets, extract time series and movement direction data, analyze the time interval and location range of the target's future activity area, and generate a future threat target activity area map;

[0044] S5: Based on the future threat target activity area map, extract the target activity range image frame sequence, extract the brightness signal and the thermal imaging signal, record the target pixel trajectory, and generate a long-range tracking target path dataset.

[0045] Compared with the prior art, the advantages and positive effects of the present invention are:

[0046] In the present invention, by collecting and fusing optical brightness and thermal imaging temperature data, the innovative solution significantly improves the accuracy of target detection in complex marine environments, optimizes the monitoring effect under low visibility conditions, and enhances the system's ability to respond quickly to dynamic changes, tracking target position and motion updates in real time, improving the real-time and continuity of the monitoring system, making the response to emergencies more rapid, and predicting future target movements and identifying potential threats in advance, providing support for the rational allocation of emergency response resources, thereby ensuring the safety of marine operations while also improving the efficiency and effectiveness of the overall monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 is a system flow chart of the present invention;

[0049] Figure 2 It is a submodule flow chart of the present invention;

[0050] Figure 3 This is a flow chart of the global monitoring and acquisition module of the present invention;

[0051] Figure 4 This is a flow chart of the local threat analysis module of the present invention;

[0052] Figure 5 This is a flow chart of the dynamic response control module of the present invention;

[0053] Figure 6 This is a flow chart of the target trajectory prediction module of the present invention;

[0054] Figure 7 This is a flow chart of the remote identification and tracking module of the present invention;

[0055] Figure 8 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0057] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0058] In the embodiments of the present invention, the terms "image" and "picture" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same. The terms "of," "corresponding," and "corresponding" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same.

[0059] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0060] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0061] See also Figure 1 and Figure 2 , a real-time maritime safety intelligent video monitoring system includes:

[0062] The global monitoring acquisition module collects optical images, VHF reception information, thermal imaging data, radar signals, and AIS identification information. It extracts the brightness distribution characteristics of optical images, thermal imaging temperature parameters, radar amplitude changes, and direction data, marks dynamic areas, and creates a distribution map of potential threat areas.

[0063] The local threat analysis module extracts optical brightness data and thermal imaging temperature parameters based on the potential threat area distribution map, analyzes the thermal imaging temperature change amplitude and distribution characteristics, extracts target speed and direction data in dynamic areas, classifies and filters target data with abnormal characteristics, and generates a threat target distribution map;

[0064] The dynamic response control module combines the threat target distribution map to extract the target position and dynamic motion parameters, analyzes the interaction between the target position and the mission range of the UAV and patrol boat, records the equipment mission completion time and the captured target position, updates the captured target time series and location distribution, and generates a real-time capture list of threat targets;

[0065] The target trajectory prediction module extracts time series and movement direction data based on the real-time capture list of threat targets, records the target displacement range and path trend, analyzes the time interval and location range of the target's future activity area, and generates a map of the future threat target activity area;

[0066] The remote identification and tracking module extracts the target activity range image frame sequence based on the future threat target activity area map, extracts the brightness signal and thermal imaging signal, eliminates the background pixel data, records the target pixel trajectory, and generates a remote tracking target path dataset.

[0067] The potential threat area distribution map includes dynamic area marking, geographic information matching, and radar data analysis. The threat target distribution map includes speed classification results, direction analysis results, and abnormal target identification. The real-time threat target capture list includes task assignment details, equipment execution status, and target position updates. The future threat target activity area map includes path prediction correction, activity time analysis, and boundary data annotation. The long-range tracking target path data set includes image frame analysis, brightness and thermal imaging data integration, and background data elimination.

[0068] See also Figure 3 and Figure 2 ,The full domain monitoring and acquisition module includes an optical feature extraction ,submodule, a thermal imaging parameter analysis submodule, and a radar signal dynamic ,marking submodule;

[0069] The optical feature extraction submodule analyzes the brightness distribution characteristics of the image based on the optical image data, calculates the brightness values ​​of the pixels, compares the spatial distribution changes of the brightness values, determines the difference between the brightness mutation area and the optical feature distribution, marks the abnormal brightness distribution area in the image, and generates the brightness distribution features;

[0070] First, the brightness value of each pixel in the image is obtained and a brightness value matrix is ​​constructed. The pixel brightness values ​​are sorted according to the spatial coordinates to form a brightness distribution curve. The difference in the brightness distribution change rate is calculated. The second derivative is used to calculate the brightness mutation area and mark the brightness abnormal points. By comparing the spatial distribution position of the brightness abnormal points with the difference analysis of the optical characteristics, a calculation method based on the distribution model is used to verify the distribution probability of the optical feature abnormal area within the brightness mutation area. Finally, the optical feature brightness distribution of all abnormal areas is marked and the brightness distribution characteristics are generated.

[0071] The thermal imaging parameter analysis submodule extracts temperature information of thermal imaging pixels based on brightness distribution characteristics and thermal imaging data, analyzes temperature value distribution trends, screens temperature fluctuation areas, calculates temperature change rates, determines the correlation between abnormal temperature change areas and brightness distribution areas, and generates thermal imaging temperature characteristics;

[0072] The temperature change rate calculation formula is as follows:

[0073]

[0074] Among them, R t1 represents the temperature change rate, T 1i Represents the pixel temperature value at the i-th time point in the thermal imaging data, T 1(i-1) represents the pixel temperature value at the i-1th time point in the thermal imaging data, n represents the total number of thermal imaging data in the time series, Represents the average temperature value of all thermal imaging pixels in the time series, ∑ is the summation symbol, Represents the absolute difference between the temperature value of the i-th thermal imaging pixel and the average value.

[0075] Calculate the temperature change rate R t1 The process is divided into two parts: the numerator calculates the sum of squares of the changes in the temperature of the thermal imaging pixels between adjacent time points, and the denominator calculates the sum of the absolute values ​​of the thermal imaging pixel temperatures that deviate from the average value.

[0076] Extract pixel temperature value T in thermal imaging data time series 1i and T 1(i-1) Assume that the thermal imaging data records the temperature values ​​at 10 time points, which are:

[0077] T 11 =300, T 12 =305, T 13=310, T 14 =307, T 15 =312, T 16 =315, T 17 =320, T 18 =318, T 19 =325, T 110 =330.

[0078] Calculate the average temperature of all pixels in a time series

[0079]

[0080] Compute the sum of the squares of the numerators:

[0081] Extract the temperature change values ​​at adjacent time points and calculate their squares:

[0082] (T 12 -T 11 ) 2 =(305-300) 2 =25;

[0083] (T 13 -T 12 ) 2 =(310-305) 2 =25;

[0084] (T 14 -T 13 ) 2 =(307-310) 2 =9;

[0085] (T 15 -T 14 ) 2 =(312-307) 2 =25;

[0086] (T 16 -T 15 ) 2 =(315-312) 2 =9;

[0087] (T 17 -T 16 ) 2 =(320-315) 2 =25;

[0088] (T 18 -T 17 ) 2 =(318-320) 2 =4;

[0089] (T 19 -T 18 ) 2 =(325-318) 2 =49;

[0090] (T 110 -T 19 ) 2 =(330-325) 2 =25;

[0091] Sum:

[0092]

[0093] The calculated results of the molecular part are Compute the sum of the absolute values ​​of the denominators:

[0094] Extract the absolute value of the difference between the temperature value at each time point and the average value:

[0095]

[0096]

[0097] Sum:

[0098]

[0099] Substituting the numerator and denominator into the formula:

[0100]

[0101] This result shows that the temperature change rate R t1 The value is 0.1887, which can be used to determine the correlation between the abnormal temperature change area and the brightness distribution area in the thermal imaging temperature data, and plays an important role in the subsequent generation of thermal imaging temperature features.

[0102] The radar signal dynamic marking submodule analyzes the radar amplitude change trend, calculates the direction change angle, analyzes the amplitude change range, determines the dynamic signal changes in the area overlapping with the temperature characteristics, marks the potential threat location of the dynamic characteristics, and creates a potential threat area distribution map based on the thermal imaging temperature characteristics and radar signal data.

[0103] By dynamically analyzing the radar amplitude change trend, selecting the time series of the signal amplitude and calculating its directional change angle in space, analyzing the amplitude change interval of the dynamic signal, and combining the coordinate information of the overlapping area of ​​the temperature feature, the coordinate projection method is used to convert the dynamic mark of the radar signal to the temperature feature distribution area. The overlapping range of the dynamic signal feature and the potential threat area is further extracted. The spatial distribution is located and annotated by constructing the threat area distribution matrix. Finally, the potential threat position of the dynamic feature is annotated and a potential threat area distribution map is generated.

[0104] See also Figure 4 and Figure 2 ,The local threat analysis module includes the threat area parameter extraction submodule, the ,dynamic target characteristic analysis submodule, and the threat target classification and ,screening submodule;

[0105] The threat area parameter extraction submodule extracts optical brightness data and thermal imaging temperature parameters based on the potential threat area distribution map. It analyzes the spatial variation and gradient distribution of the optical image brightness, calculates the fluctuation trend of the thermal imaging temperature data on the time axis, and analyzes the spatial gradient distribution characteristics to generate threat area characteristic data.

[0106] By analyzing optical brightness data and thermal imaging temperature parameters, the brightness values ​​in the image are extracted and a brightness matrix is ​​constructed. The variation amplitude and gradient distribution of the brightness values ​​in the spatial distribution are analyzed, and the local change rate of the gradient distribution matrix is ​​calculated. The brightness distribution results are matched and analyzed with the time series of the thermal imaging data. The variation trend of the thermal imaging data on the time axis is extracted, and its temporal fluctuation rate is calculated. The temperature gradient distribution characteristics are further analyzed. Combined with the spatial distribution results, the characteristic data of the threat area is generated.

[0107] The dynamic target characteristic analysis submodule extracts the moving speed and direction information of the dynamic area target based on the threat area characteristic data, calculates the instantaneous change value of the speed, identifies the direction offset angle, compares the time series fluctuation characteristics of the speed data and the direction data, and generates dynamic target characteristic data;

[0108] The calculation formula of instantaneous speed change is as follows:

[0109]

[0110] Among them, V3 represents the instantaneous change value of the velocity, x3 represents the position change of the target in the x direction, y3 represents the position change of the target in the y direction, t3 represents the time interval, θ3 represents the direction change angle, and cosθ3 is the cosine value of the direction angle.

[0111] The process of calculating the instantaneous speed change value V3 is divided into the following steps:

[0112] Parameter acquisition and quantification:

[0113] The displacement changes x3 and y3 of the target in the x-direction and y-direction are determined by real-time monitoring of the position of the dynamic target in the image frame. The time interval t3 is the recording time difference between the two frames of image. The direction change angle θ3 is calculated by the trajectory analysis tool to obtain the angular change of the target movement in the image frame and the cosine value.

[0114] The actual monitoring data are as follows:

[0115] x3=6m, y3=8m, t3=2s, θ3=45°;

[0116] The cosine of the angle of change in direction is:

[0117] cosθ3=cos45°=0.707;

[0118] Molecular computing:

[0119] Calculate the sum of the squares of the target's displacement changes in the x and y directions, and multiply it by the square of the time interval:

[0120]

[0121] Square root of the numerator:

[0122]

[0123] Denominator calculation:

[0124] Calculate the absolute value of the time interval multiplied by the cosine of the angle of change in direction:

[0125] |t3·cosθ3|=|2·0.707|=1.414;

[0126] Overall calculation formula:

[0127] Substitute the numerator and denominator into the formula to calculate the instantaneous speed change parameter:

[0128]

[0129] V3≈14.14m / s;

[0130] The results show that the instantaneous change in the target's speed within the time interval recorded by the image frame is 14.14 meters per second, reflecting the motion characteristics of the dynamic target and the speed change trend in the trajectory. Combined with the direction offset angle, it can be used to further analyze the time series fluctuation characteristics and provide important support for generating dynamic target characteristic data.

[0131] The threat target classification and screening submodule analyzes the distribution characteristics of speed and direction data based on dynamic target characteristic data, calculates the spatial density of dynamic targets, annotates target characteristic data with differences, regroups and classifies them according to dynamic and spatial characteristics, and establishes a threat target distribution map;

[0132] By analyzing the distribution characteristics of speed and direction data, selecting the time series of speed and direction values, calculating the fluctuation amplitude of the speed values, extracting the local maximum and minimum values ​​in the time series, analyzing the trend of the offset angle change of the direction data in the time series, and further calculating the regional density of dynamic targets in space, the spatial distribution of the targets is re-clustered in combination with the fluctuation characteristics of speed and direction, and the different targets in the clustering results are marked. Targets with similar characteristics are grouped and classified by merging them, and finally a threat target distribution map is generated.

[0133] See also Figure 5 and Figure 2 ,The dynamic response control module includes the target interaction analysis submodule, the ,capture time recording submodule, and the target distribution updating submodule;

[0134] The target interaction analysis submodule extracts the target position and dynamic motion parameters based on the threat target distribution map, calculates the spatial distribution weight of the target position within the mission range, analyzes the interaction frequency of the target position within the mission range of the UAV and patrol boat, and generates mission interaction target data;

[0135] By extracting the spatial position and dynamic motion parameters of the target, the target position is first mapped to the mission range coordinate system, and the frequency and interaction points of the target in the mission coverage area of ​​the UAV and patrol boat are analyzed. The spatial distribution weight of the target position in the mission range is calculated, and the mission interaction points and occurrence density of the target are marked using the weight matrix. The interaction frequency data is associated with the time series of the target dynamic motion parameters. Finally, the mission interaction targets are screened out through frequency and weight calculation to generate mission interaction target data.

[0136] The capture time recording submodule records the start and completion time of the device task based on the task interaction target data, extracts the spatial position of the capture target, analyzes and captures the matching relationship between the target position and time, generates the time series distribution characteristics of the target, and generates the target capture time series;

[0137] Based on the task interaction target data, according to the formula

[0138]

[0139] Calculate the spatial distribution weight of the target position within the task range.

[0140] Where W represents the spatial distribution weight, P iRepresents the weight coefficient of the target position, F i Represents the interaction frequency between the target and the device, ΔT i Represents the time the target stays at this location.

[0141] Extract the interaction data of a target. Assume that the weight coefficients of the target at three positions are P1=0.5, P2=0.3, and P3=0.2, and the interaction frequencies are F1=4, F2=3, and F3=2. The corresponding dwell times are ΔT1=5 seconds, ΔT2=4 seconds, and ΔT3=6 seconds. Substitute the data into the formula:

[0142] W=(0.5·4·5)+(0.3·3·4)+(0.2·2·6);

[0143] W = 10 + 3.6 + 2.4 = 16;

[0144] The results show that the spatial distribution weight of the target within the mission scope is 16, which reflects the importance of the target in the execution of the mission in the relevant area and provides basic data support for subsequent dynamic analysis.

[0145] The target distribution update submodule updates the spatial position distribution data of captured targets based on the target capture time series. It combines the time series data to analyze the spatial dynamic distribution of target capture, annotates the time characteristics of the captured target position, and establishes a real-time capture list of threat targets.

[0146] Combine the spatial position distribution data of the captured target with the time series data, extract the capture interval and capture frequency of the target in the time series, analyze the dynamic distribution characteristics of the captured target in space, construct the time and spatial distribution mapping matrix of the target capture, mark and record the capture position of the target in each time period, update the spatial position distribution data of the target according to the distribution law of the target in the time series, generate a real-time capture list by matching the capture time and position characteristics, and dynamically update the targets in the list to adapt to the real-time scenario requirements, and finally establish a real-time capture list of threat targets.

[0147] See also Figure 6 and Figure 2 ,The target trajectory prediction module includes the target path feature extraction ,submodule, the activity area trend analysis submodule, and the future ,activity area prediction submodule;

[0148] The target path characteristic extraction submodule extracts target time series and movement direction data based on the real-time capture list of threat targets, records the target's displacement range and path change trend between consecutive time points, calculates the path change amplitude in the time series, integrates time and space data to analyze path characteristics, and generates target path characteristic data;

[0149] Based on the real-time capture list of threat targets, according to the formula

[0150]

[0151] Calculate the displacement range of adjacent time points in the target time series.

[0152] Where ΔL represents the displacement range of the target, and Δx and Δy represent the position changes of the target in the x and y directions, respectively.

[0153] Select the target's position information at consecutive time points. Assume that the starting position is (5, 8) and the position at the next time point is (9, 12). Calculate the position changes in the x and y directions:

[0154] Δx = 9 - 5 = 4;

[0155] Δy=12-8=4;

[0156] Substitute the data into the formula:

[0157]

[0158] The results show that the displacement range of the target in this section of the time series is 5.66 unit lengths, which reflects the characteristics of the target's motion path and provides basic data for further analysis of the target path change trend.

[0159] The activity area trend analysis submodule analyzes the temporal trend of the movement direction based on the target path characteristic data, records the offset range of the movement direction, calculates the temporal continuity characteristics of the path change, and analyzes the activity area dynamics in combination with the time series fluctuation characteristics of the target path to generate activity area trend data;

[0160] By analyzing the temporal trend of the movement direction, the offset angle information of the movement direction in the time series is extracted, and the path change amplitude of the target at different time points is calculated. By analyzing the continuity of the path change, the path characteristics in the time series are annotated, and the path characteristics are combined with spatial dynamics to construct a dynamic trend model of the activity area, and finally generate activity area trend data.

[0161] The future activity area prediction submodule is based on activity area trend data and combines the movement patterns of target time series to predict the time interval and spatial boundaries of the target activity area, calculate the spatial dynamic distribution characteristics of the target within the activity area, integrate the time and space characteristic data, and establish a future threat target activity area map:

[0162] Extract the movement patterns of the target in the time series, analyze the movement direction and speed data in the time series, combine the boundary range of the target's spatial distribution, use the regional dynamic change model to calculate the time interval and spatial boundary of the future activity area, generate the dynamic distribution matrix of the target in the area, and complete the establishment of the threat target activity area map by integrating time and space data. The boundary parameters of the activity area are updated and corrected in real time to finally generate the future threat target activity area map.

[0163] See also Figure 7 and Figure 2 ,The remote recognition and tracking module includes the target image frame extraction submodule, the background data ,removal submodule, and the target trajectory recording submodule;

[0164] The target image frame extraction submodule extracts the image frame sequence within the activity range based on the future threat target activity area map, records the image frame time sequence and corresponding spatial coordinates, analyzes the distribution range of dynamic targets in the image frame, calculates the temporal and spatial variation characteristics of the dynamic area between image frames, integrates the image frames to form a temporal and spatial matching relationship, and generates the target image frame sequence;

[0165] Extract the image frame sequence within the target's activity range, record the time series data of the image frames frame by frame, use the target's spatial position coordinates to spatially sort the image frame data, analyze the dynamic distribution range of the target in the image frame, and establish the time-space correspondence between the image frame data by calculating the time change rate and spatial movement range of the dynamic area between consecutive image frames. Finally, integrate the time and space change characteristics of the dynamic target into the image frame sequence output to generate the target image frame sequence.

[0166] The background data removal submodule extracts the brightness signal and thermal imaging signal data from the target image frame sequence, analyzes the pixel distribution and regional gradient characteristics of the signal, filters the background features of the static area in the pixel signal, removes the background data that does not conform to the dynamic target characteristics, records the effective signal within the dynamic pixel range, and generates the target effective pixel signal;

[0167] Based on the target image frame sequence, according to the formula

[0168]

[0169] Calculate the background data removal weight value in the dynamic pixel signal.

[0170] In the formula, G is the background removal weight value, L i is the brightness signal value of the image frame pixel, S i is the brightness reference value of the static background, Δt i is the time change interval.

[0171] Extract the dynamic signal data of a pixel point in the image frame sequence. Suppose the brightness signal values ​​at three time points in a certain image frame sequence are L1=120, L2=125, L3=130, the brightness reference values ​​corresponding to the static background are S1=100, S2=102, S3=105, and the time change intervals are Δt1=2, Δt2=3, Δt3=4.

[0172] Enter the formula to calculate:

[0173] G=(120-100) 2 2+ (125-102) 2 3+ (130-105) 2 4;

[0174] G=20 2 2+23 2 3+25 2 4;

[0175] G = 400·2+529·3+625·4;

[0176] G = 800 + 1587 + 2500 = 4887;

[0177] The results show that the background rejection weight value of the dynamic pixel signal is 4887, which represents the changing characteristics of the dynamic target pixel signal relative to the static background, and provides basic data support for the subsequent extraction of effective signals within the dynamic pixel range.

[0178] The target trajectory recording submodule records the position changes of dynamic pixels in continuous image frames based on the target's effective pixel signals, calculates the time series and spatial trajectory of pixel displacement, analyzes the dynamic characteristics and continuity trend of the trajectory, integrates the spatial path and time relationship of the trajectory, and establishes a long-range tracking target path dataset;

[0179] The positions of dynamic pixels in consecutive image frames are recorded as trajectory data. The pixel position information of each frame is analyzed, and the time series and spatial variation characteristics of the dynamic pixel displacement between consecutive frames are calculated. The displacement amplitude in the time series is analyzed section by section and the temporal continuity trend of the trajectory is recorded. By integrating the path and time matching relationship of the pixel trajectory, the spatial trajectory of the dynamic target is finally generated, and a path dataset of the remote target is established.

[0180] See also Figure 8 , a real-time maritime safety intelligent video monitoring method, comprising the following steps:

[0181] S1: By collecting optical images, VHF reception information, thermal imaging data, radar signals, and AIS identification information, the brightness distribution characteristics of optical images, thermal imaging temperature parameters, radar amplitude changes, and direction data are extracted to establish a distribution map of potential threat areas.

[0182] S2: Based on the potential threat area distribution map, extract optical brightness data and thermal imaging temperature parameters, extract target speed and direction data in the dynamic area, classify and filter abnormal target data, and generate a threat target distribution map;

[0183] S3: Combined with the threat target distribution map, the target location and dynamic motion parameters are extracted, the interactive relationship between the target location and the mission range of the UAV and patrol boat is analyzed, the capture target time series and location distribution are updated, and a real-time capture list of threat targets is generated;

[0184] S4: Based on the real-time capture list of threat targets, extract the time series and movement direction data, analyze the time interval and location range of the target's future activity area, and generate a map of the future threat target activity area;

[0185] S5: Based on the future threat target activity area map, extract the target activity range image frame sequence, extract the brightness signal and thermal imaging signal, record the target pixel trajectory, and generate a long-range tracking target path dataset.

[0186] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A real-time intelligent video surveillance system for maritime safety, characterized by: The system comprises: The global monitoring acquisition module collects optical images, VHF reception information, thermal imaging data, radar signals, and AIS identification information. It extracts the brightness distribution characteristics of optical images, thermal imaging temperature parameters, radar amplitude changes, and direction data, marks dynamic areas, and creates a distribution map of potential threat areas. The local threat analysis module extracts optical brightness data and thermal imaging temperature parameters based on the potential threat area distribution map, analyzes the thermal imaging temperature change amplitude and distribution characteristics, extracts target speed and direction data in the dynamic area, classifies and filters target data with abnormal characteristics, and generates a threat target distribution map; The dynamic response control module extracts the target position and dynamic motion parameters based on the threat target distribution map, analyzes the interactive relationship between the target position and the mission range of the UAV and patrol boat, records the equipment mission completion time and the captured target position, updates the captured target time series and position distribution, and generates a real-time capture list of threat targets; The target trajectory prediction module extracts time series and movement direction data based on the real-time capture list of threat targets, records the target displacement range and path trend, analyzes the time interval and position range of the target's future activity area, and generates a map of the future threat target activity area; The remote identification and tracking module extracts the target activity range image frame sequence based on the future threat target activity area map, extracts the brightness signal and thermal imaging signal, eliminates the background pixel data, records the target pixel trajectory, and generates a remote tracking target path data set.

2. The real-time maritime safety intelligent video monitoring system according to claim 1 is characterized by: The potential threat area distribution map includes dynamic area marking, geographic information matching, and radar data analysis. The threat target distribution map includes speed classification results, direction analysis results, and abnormal target identification. The threat target real-time capture list includes task assignment details, equipment execution status, and target location updates. The future threat target activity area map includes path prediction correction, activity time analysis, and boundary data annotation. The long-range tracking target path dataset includes image frame analysis, brightness and thermal imaging data integration, and background data elimination.

3. The real-time maritime safety intelligent video monitoring system according to claim 1 is characterized by: The global monitoring acquisition module includes an optical feature extraction submodule, a thermal imaging parameter analysis submodule, and a radar signal dynamic marking submodule; The optical feature extraction submodule analyzes the brightness distribution characteristics of the image based on the optical image data, calculates the brightness values ​​of the pixels, compares the spatial distribution changes of the brightness values, determines the difference between the brightness mutation area and the optical feature distribution, marks the abnormal brightness distribution area in the image, and generates the brightness distribution features; The thermal imaging parameter analysis submodule is based on the brightness distribution characteristics and thermal imaging data. It extracts the temperature information of thermal imaging pixels, analyzes the temperature value distribution trend, screens the temperature fluctuation area, calculates the temperature change rate, determines the correlation between the abnormal temperature change area and the brightness distribution area, and generates the thermal imaging temperature characteristics. The radar signal dynamic marking submodule analyzes the radar amplitude change trend, calculates the direction change angle, analyzes the amplitude change range, determines the dynamic signal changes in the area overlapping with the temperature characteristics, marks the potential threat position of the dynamic characteristics, and establishes a potential threat area distribution map based on the thermal imaging temperature characteristics and radar signal data.

4. The real-time maritime safety intelligent video monitoring system according to claim 3 is characterized by: The temperature change rate calculation formula is specifically: Among them, R t1 represents the temperature change rate, T 1i Represents the pixel temperature value at the i-th time point in the thermal imaging data, T 1(i-1) represents the pixel temperature value at the i-1th time point in the thermal imaging data, n represents the total number of thermal imaging data in the time series, Represents the average temperature value of all thermal imaging pixels in the time series, ∑ is the summation symbol, Represents the absolute difference between the temperature value of the i-th thermal imaging pixel and the average value.

5. The real-time maritime safety intelligent video monitoring system according to claim 1 is characterized by: The local threat analysis module includes a threat area parameter extraction submodule, a dynamic target characteristic analysis submodule, and a threat target classification and screening submodule; The threat area parameter extraction submodule extracts optical brightness data and thermal imaging temperature parameters based on the potential threat area distribution map, analyzes the spatial variation and gradient distribution of the optical image brightness, calculates the fluctuation trend of the thermal imaging temperature data on the time axis, and analyzes the spatial gradient distribution characteristics to generate threat area characteristic data; The dynamic target characteristic analysis submodule extracts the moving speed and direction information of the dynamic area target based on the threat area characteristic data, calculates the instantaneous change value of the speed, identifies the direction offset angle, compares the time series fluctuation characteristics of the speed data and the direction data, and generates dynamic target characteristic data; The threat target classification and screening submodule analyzes the distribution characteristics of speed and direction data based on the dynamic target characteristic data, counts the spatial area density of dynamic targets, marks target characteristic data with differences, regroups and classifies them according to dynamic characteristics and spatial characteristics, and establishes a threat target distribution map.

6. The real-time maritime safety intelligent video monitoring system according to claim 5, characterized in that: The calculation formula of the instantaneous speed change value is specifically: Among them, V3 represents the instantaneous change value of the velocity, x3 represents the position change of the target in the x direction, y3 represents the position change of the target in the y direction, t3 represents the time interval, θ3 represents the direction change angle, and cosθ3 is the cosine value of the direction angle.

7. The real-time maritime safety intelligent video monitoring system according to claim 1, characterized in that: The dynamic response control module includes a target interaction analysis submodule, a capture time recording submodule, and a target distribution update submodule; The target interaction analysis submodule extracts the target position and dynamic motion parameters based on the threat target distribution map, calculates the spatial distribution weight of the target position within the mission range, analyzes the interaction frequency of the target position within the mission range of the UAV and patrol boat, and generates mission interaction target data; The capture time recording submodule records the start and completion time of the device task based on the task interaction target data, extracts the spatial position of the capture target, analyzes and captures the matching relationship between the target position and time, generates the time series distribution characteristics of the target, and generates the target capture time series; The target distribution update submodule updates the spatial position distribution data of the captured targets based on the target capture time series, analyzes the spatial dynamic distribution of target capture in combination with the time series data, marks the time characteristics of the captured target positions, and establishes a real-time capture list of threat targets.

8. The real-time maritime safety intelligent video monitoring system according to claim 1, characterized in that: The target trajectory prediction module includes a target path feature extraction submodule, an activity area trend analysis submodule, and a future activity area prediction submodule; The target path characteristic extraction submodule extracts the target time series and movement direction data based on the real-time capture list of threat targets, records the displacement range and path change trend of the target between consecutive time points, calculates the change amplitude of the path in the time series, integrates time and space data to analyze the path characteristics, and generates target path characteristic data; The activity area trend analysis submodule analyzes the temporal trend of the movement direction based on the target path characteristic data, records the offset range of the movement direction, calculates the temporal continuity characteristics of the path change, analyzes the activity area dynamics in combination with the time series fluctuation characteristics of the target path, and generates activity area trend data; The future activity area prediction submodule predicts the time interval and spatial boundary of the target activity area based on the activity area trend data and the movement law of the target time series, calculates the spatial dynamic distribution characteristics of the targets in the activity area, integrates the time and space characteristic data, and establishes a future threat target activity area map.

9. The real-time maritime safety intelligent video monitoring system according to claim 1, characterized in that: The remote identification and tracking module includes a target image frame extraction submodule, a background data removal submodule, and a target trajectory recording submodule; The target image frame extraction submodule extracts the image frame sequence within the activity range based on the future threat target activity area map, records the image frame time sequence and corresponding spatial coordinates, analyzes the distribution range of dynamic targets in the image frame, calculates the temporal and spatial variation characteristics of the dynamic area between image frames, integrates the image frames to form a temporal and spatial matching relationship, and generates the target image frame sequence; The background data removal submodule extracts the brightness signal and thermal imaging signal data from the target image frame sequence, analyzes the pixel distribution and regional gradient characteristics of the signal, filters the background features of the static area in the pixel signal, removes the background data that does not conform to the dynamic target characteristics, records the effective signal within the dynamic pixel range, and generates the target effective pixel signal; The target trajectory recording submodule records the position changes of dynamic pixels in continuous image frames based on the target effective pixel signals, calculates the time series and spatial trajectory of pixel displacement, analyzes the dynamic characteristics and continuity trend of the trajectory, integrates the spatial path and time relationship of the trajectory, and establishes a long-range tracking target path dataset.

10. A real-time intelligent video surveillance method for maritime safety, characterized in that: According to any one of claims 1 to 9, a real-time maritime safety intelligent video surveillance system is implemented, comprising the following steps: S1: By collecting optical images, VHF reception information, thermal imaging data, radar signals, and AIS identification information, the brightness distribution characteristics of optical images, thermal imaging temperature parameters, radar amplitude changes, and direction data are extracted to establish a distribution map of potential threat areas. S2: Based on the potential threat area distribution map, extract optical brightness data and thermal imaging temperature parameters, extract target speed and direction data in the dynamic area, classify and filter abnormal characteristic target data, and generate a threat target distribution map; S3: extracting target positions and dynamic motion parameters based on the threat target distribution map, analyzing the interactive relationship between target positions and the mission ranges of drones and patrol boats, updating the captured target time series and position distribution, and generating a real-time capture list of threat targets; S4: Based on the real-time capture list of threat targets, extract time series and movement direction data, analyze the time interval and location range of the target's future activity area, and generate a future threat target activity area map; S5: Based on the future threat target activity area map, extract the target activity range image frame sequence, extract the brightness signal and the thermal imaging signal, record the target pixel trajectory, and generate a long-range tracking target path dataset.

Images