An inland waterway safety supervision method, system, device and storage medium

Abstract

The application discloses an inland waterway safety supervision method, system, device and storage medium, the method comprises the following steps: collecting the dual-spectrum video, river data and environment data of the target inland waterway, monitoring the dual-spectrum video transmission channel bandwidth data; the real-time collected data is used as an encoding compression strategy decision factor, the encoding compression scheme and the multi-code stream setting scheme matched with the real-time collected data are obtained; the video transmission matching strategy matched with the transmission bandwidth data is obtained; the compression, transmission and storage of the dual-spectrum video are completed according to the matching result; the transmission video is received and decoded; the real-time collected data is used as a risk assessment factor, the comprehensive risk level is determined, and the multi-scale registration strategy matched with the comprehensive risk level is obtained; feature extraction, registration and fusion are carried out according to the matching result, whether there is a prohibited entry ship is identified; and warning instructions are generated according to the identification result and linkage warning is carried out. The application can improve the accuracy and timeliness of the inland waterway safety monitoring and identification.

Description

Technical Field

[0001] This application relates to the technical field of waterway safety supervision, specifically to a method, system, equipment, and storage medium for inland waterway safety supervision. Background Technology

[0002] Inland waterways, as vital transportation channels, require strict safety management to ensure smooth water transport and the safety of people and property. With increasingly busy inland waterway transport, waterway safety has received growing attention. Effective waterway safety supervision can reduce accidents, improve transport efficiency, and promote the sustainable development of inland waterway transport. The complex geographical environment of inland waterways, coupled with variable factors such as water flow and weather, places extremely high demands on waterway safety supervision. Simultaneously, the increasing number of barges makes their standardized operation and mooring a key factor affecting waterway safety.

[0003] In the past, inland waterway safety management mainly relied on manual monitoring and traditional video surveillance systems. Manual monitoring required staff to be on duty for extended periods to observe and assess waterway conditions. Traditional video surveillance systems primarily used visible light cameras for waterway monitoring, acquiring real-time images of the waterway by installing a certain number of cameras along the waterway. While these methods could monitor waterway conditions to some extent, they had significant limitations.

[0004] Existing inland waterway safety management methods have significant shortcomings. Manual monitoring and traditional video surveillance systems struggle to achieve accurate 24 / 7 monitoring under complex weather conditions and cannot provide real-time warnings of barge violations. Traditional monitoring systems lack all-weather monitoring capabilities and event-linked alarm functions, resulting in low accuracy in identifying barges in the waterway at night and in adverse weather conditions, leading to inefficient waterway safety monitoring. Furthermore, when barges enter restricted areas, real-time warnings are not issued, affecting the timeliness of waterway safety management and lowering its overall safety management level. Summary of the Invention

[0005] In order to achieve 24 / 7 monitoring of inland waterway safety, improve the accuracy of monitoring and identification, and ensure the timeliness of monitoring and identification, this application provides an inland waterway safety supervision method, system, equipment, and storage medium.

[0006] Firstly, this application provides a method for safety supervision of inland waterways, including:

[0007] Acquire bispectral video of the target inland waterway; simultaneously acquire river channel data and environmental data of the target inland waterway while acquiring bispectral video; monitor the bandwidth data of the bispectral video transmission channel in real time;

[0008] Based on real-time collected river data, environmental data, and predicted data of prohibited vessels in the current bispectral video as decision factors for encoding and compression strategies, a preset encoding and compression scheme and a preset multi-stream setting scheme are obtained to match them; combined with real-time collected transmission bandwidth data, a preset video transmission matching strategy is obtained to match it; the predicted data of prohibited vessels in the current bispectral video is obtained using a prohibited vessel data prediction model built based on deep learning algorithms.

[0009] The current dual-spectrum video is compressed, transmitted, and stored according to the matching preset encoding compression scheme, preset multi-stream settings, and preset video transmission matching strategy.

[0010] Receive and decode the transmitted video; use real-time collected river data, environmental data, and predicted data on prohibited vessels in the current bispectral video as risk assessment factors, determine the comprehensive risk level according to preset risk assessment rules, and obtain a matching multi-scale registration strategy; perform feature extraction, registration, and fusion on the decoded bispectral video according to the matching multi-scale registration strategy to identify and obtain whether there are prohibited vessels.

[0011] Based on the identification results of the prohibited vessels, a warning command is generated and the alarm light is activated to issue a warning.

[0012] By adopting the above scheme, dual-spectral video, river data, and environmental data are collected, and transmission bandwidth data is monitored. Adaptive encoding and compression schemes, multi-stream settings, and video transmission matching strategies are acquired to complete video compression, transmission, and storage, improving video transmission efficiency and storage utilization. Deep learning algorithms are used to predict data on prohibited vessels, and risk assessment and multi-scale registration are performed based on multiple factors to accurately identify the presence of prohibited vessels. Warning commands are generated based on the identification results and alarm lights are activated, promptly alerting relevant personnel and improving the accuracy and timeliness of inland waterway safety supervision.

[0013] Preferred options also include:

[0014] During the process of collecting bispectral video of the target inland waterway, environmental data was collected in real time and then...

[0015] The parameters of the dual-spectrum video acquisition device are dynamically optimized, including resolution, exposure parameters, and monitoring temperature threshold.

[0016] By adopting the above scheme, the resolution, exposure parameters, and monitoring temperature threshold of the bispectral video acquisition device are dynamically optimized based on real-time environmental data when acquiring bispectral video, thereby improving the acquisition quality of bispectral video and thus enhancing the accuracy of identifying prohibited vessels entering the target inland waterway, and strengthening the effectiveness of waterway safety supervision.

[0017] Preferably, the step of using real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video as decision factors for the encoding and compression strategy, and obtaining a matching preset encoding and compression scheme and preset multi-stream settings, includes:

[0018] Key feature data of the currently collected river data are extracted and quantified, including: river physical parameters, river hydrological data, and the location and type of fixed obstacles; the target inland waterway area in the bispectral video is determined and divided according to the extracted river physical parameters; the target inland waterway area includes: near-shore area, main channel area, and far-shore area;

[0019] Based on the defined target inland waterway areas, determine the environmental data currently collected in each area, extract key feature data of the environmental data currently collected in each area and quantify the features, including: meteorological data; based on the defined target inland waterway areas, determine the predicted vessel data currently collected in each area, extract key feature data of the vessel data currently collected in each area and quantify the features, including: vessel type and vessel speed.

[0020] A coding compression scheme identification model and a multi-stream setting model were constructed using deep learning algorithms. The quantized values ​​of all key features extracted from each region of a single-track video in a dual-spectrum video were input into the coding compression scheme identification model and the multi-stream setting model, respectively, to obtain corresponding preset coding compression schemes and preset multi-stream setting schemes. The preset coding compression schemes included: H.266 / VVC and ROI encoding, H.265 / HEVC and dynamic ROI, and H.264 and single-track encoding. The multi-stream setting schemes included: a main stream, a secondary stream, and a storage backup stream covering different requirements for resolution, frame rate, and bitrate range.

[0021] By adopting the above scheme, inland waterway areas are accurately divided based on river data, environmental data, and vessel data. Deep learning algorithms are used to match appropriate encoding and compression schemes and multi-stream settings for each area, improving the compression efficiency and storage and transmission flexibility of bispectral video, and better meeting the monitoring needs of different scenarios.

[0022] Preferably, the step of obtaining a preset video transmission matching strategy by combining the real-time acquired transmission bandwidth data includes:

[0023] Set transmission bandwidth data belonging to different ranges, and set a video transmission matching strategy for each range of transmission bandwidth data, including: parallel transmission of dual-spectrum main stream matching data greater than the first transmission bandwidth data, adaptive transmission of dual-spectrum main stream and secondary stream switching matching data less than the first transmission bandwidth data but greater than the second transmission bandwidth data, and transmission of ROI-limited stream matching data less than the second transmission bandwidth data.

[0024] By adopting the above scheme, a matching video transmission strategy can be flexibly selected based on the real-time collected transmission bandwidth data, ensuring the effective transmission of dual-spectrum video under different bandwidth conditions, avoiding transmission problems caused by insufficient bandwidth, and improving the stability and adaptability of dual-spectrum video transmission.

[0025] Preferably, the step of using real-time collected river data, environmental data, and predicted data on prohibited vessels entering the river from the current bispectral video as risk assessment factors, determining the comprehensive risk level according to preset risk assessment rules, and obtaining a matching multi-scale registration strategy includes:

[0026] For each region of a single-track video in a dual-spectrum video, all key features extracted are used as risk assessment factors. A comprehensive risk level is determined according to preset risk assessment rules, and a matching multi-scale registration strategy is obtained. The preset risk assessment rules include matching preset risk assessment quantification values ​​for each key feature. Each multi-scale registration strategy has a preset comprehensive risk level. The system includes: feature extraction methods, registration strategies, scale parameters, and fusion rules. Feature extraction methods include: SIFT and edge gradients, ORB and contour moments, SURF and motion vectors. Registration strategies include: local affine transformation, global projection transformation, and optical flow-guided deformation. Scale parameters include: pyramids with different numbers of layers. The fusion rules include: weighted fusion with different weight parameters.

[0027] By adopting the above scheme, the comprehensive risk level is determined based on the key features of each region of the dual-spectral video, and corresponding multi-scale registration strategies are matched. By utilizing various feature extraction methods, registration strategies, scale parameters and fusion rules, the accuracy and efficiency of identifying prohibited vessels can be improved, thereby enhancing the level of safety supervision of inland waterways.

[0028] Preferred options also include:

[0029] The system monitors and identifies vessels illegally entering the area. If no complete video identification result is received within the preset identification time, the received and transmitted video is cropped according to the unidentified areas. Each cropped unidentified area and the corresponding multi-scale registration strategy obtained for each unidentified area in the dual-spectrum video are sent to an edge node for linkage identification. This allows each edge node to perform feature extraction, registration, and fusion based on each cropped unidentified area and the corresponding multi-scale registration strategy obtained for each unidentified area in the dual-spectrum video, identifying whether there are vessels illegally entering the area in each unidentified area. Finally, all identification results are summarized and displayed.

[0030] By adopting the above scheme, if the complete video recognition is not completed within the preset recognition time, the unrecognized area can be cropped and the edge nodes can be used for linkage recognition, which can improve the recognition efficiency of prohibited vessels and avoid missed judgments due to unrecognized areas, thereby further improving the reliability of inland waterway safety supervision.

[0031] Preferred options also include:

[0032] In the process of generating warning instructions in response to the identification results of prohibited vessels, the location of the identified prohibited vessels in the target inland waterway area is determined simultaneously, and a warning instruction is broadcast to the identified prohibited vessels in the target inland waterway area; at the same time, a linkage warning instruction is sent to the alarm lights, warning voice devices or vessels installed within a preset distance from the identified prohibited vessel location, so as to activate the alarm lights and warning voice devices at the corresponding locations for linkage warning.

[0033] By adopting the above scheme, the location of the waterway area where the prohibited vessel is to enter is simultaneously indicated when the warning instruction is generated. At the same time as broadcasting the warning instruction to the area, a linkage warning instruction is sent to the alarm lights or vessels within a preset distance range. This can accurately locate the position of the prohibited vessel and expand the warning range, improve the early warning effect, and enhance the timeliness and effectiveness of inland waterway safety supervision.

[0034] Secondly, this application provides an inland waterway safety monitoring system, comprising:

[0035] The regulatory data acquisition module is used to acquire bispectral video of the target inland waterway; while acquiring bispectral video, it also acquires river channel data and environmental data of the target inland waterway; and it monitors the bandwidth data of the bispectral video transmission channel in real time.

[0036] The regulatory data processing module is used to obtain a preset encoding compression scheme and a preset multi-stream setting scheme based on real-time collected river data, environmental data, and predicted data of prohibited vessels entering the current bispectral video as encoding compression strategy decision factors; and to obtain a preset video transmission matching strategy based on real-time collected transmission bandwidth data; the predicted data of prohibited vessels entering the current bispectral video is obtained using a prohibited vessel data prediction model built based on deep learning algorithms.

[0037] The monitoring data transmission module is used to complete the compression, transmission and storage of the current dual-spectrum video according to the matching preset encoding compression scheme, preset multi-stream settings and preset video transmission matching strategy;

[0038] The regulatory data analysis module is used to receive and decode transmitted video; based on real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video as risk assessment factors, it determines the comprehensive risk level according to preset risk assessment rules and obtains a matching multi-scale registration strategy; according to the matching multi-scale registration strategy, it performs feature extraction, registration, and fusion on the decoded bispectral video to identify and determine whether there are prohibited vessels entering the bispectral video.

[0039] The regulatory data warning module is used to generate warning commands and trigger alarm lights to issue warnings based on the identification results of vessels entering the area illegally.

[0040] By adopting the above scheme, bispectral video, river data, environmental data, and transmission bandwidth data of inland waterways are collected comprehensively. Deep learning algorithms are used to predict data on prohibited vessels entering the waterway. Based on this, appropriate encoding compression, multi-stream settings, and video transmission strategies are matched to achieve efficient compression, transmission, and storage of bispectral video. At the same time, the system can accurately identify whether there are prohibited vessels entering the waterway and promptly trigger alarms, thereby improving the accuracy, efficiency, and timeliness of inland waterway safety supervision.

[0041] Thirdly, this application provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the method described above.

[0042] Fourthly, this application provides a computer device, the computer device including a memory, a processor and a program stored in the memory and executable thereon, the program being executed by the processor to implement the steps of the method described above.

[0043] In summary, this application has the following beneficial effects:

[0044] 1. Employing dual-spectrum video imaging technology and thermal imaging data fusion technology, this system analyzes river data, environmental data, and data on prohibited vessels. It adaptively matches preset encoding and compression schemes, preset multi-stream settings, preset video transmission matching strategies, and multi-scale registration strategies to optimize video transmission and storage efficiency, thereby improving identification accuracy and all-weather monitoring capabilities. Simultaneously, an event-linked alarm function is implemented, generating warning commands and triggering alarm lights when prohibited vessels are detected, achieving real-time early warning.

[0045] 2. Real-time monitoring of the identification process: If a complete video identification result is not received in time, the video of the unidentified area is cropped and sent to the edge node for linkage identification in different areas. This can improve the efficiency and accuracy of identifying prohibited vessels, ensure that prohibited vessels can be detected in a timely manner, and enhance the reliability of inland waterway safety supervision. Attached Figure Description

[0046] Figure 1 This is a flowchart of the method described in a specific embodiment;

[0047] Figure 2 To apply the method described in the specific embodiments to generate physical images for inland waterway monitoring and linkage warning;

[0048] Figure 3 This is a schematic diagram of the system described in a specific embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] To overcome the problems of low accuracy in identifying incidents at night and in severe weather, and the lack of an effective event linkage mechanism in existing inland waterway safety supervision methods, and to improve the efficiency of waterway safety monitoring and the timeliness of management, thus providing a strong guarantee for the safe operation of inland waterways, this application mainly adopts multi-factor decision-making to complete dual-spectrum video processing and early warning, which achieves the effect of improving the efficiency and timeliness of inland waterway safety supervision. The following is a further detailed description of this application.

[0051] like Figure 1 As shown in the embodiment of this application, a method for safety supervision of inland waterways is disclosed, including: comprehensively considering factors such as waterway, environment, and bandwidth, and carrying out multi-stage supervision such as data collection, strategy matching, video processing, risk assessment, and warning. Each stage cooperates with the others to accurately identify vessels illegally entering the inland waterway and issue timely warnings. Specific steps include:

[0052] S1. Collect bispectral video, river data, and environmental data for the target inland waterway.

[0053] Specifically, considering the identification of actual inland waterways at night and in adverse weather conditions, dual-spectrum video acquisition is achieved using both visible light and thermal imaging cameras. For example, a dual-spectrum tube camera composed of a visible light lens and a thermal imaging lens is selected for dual-spectrum video acquisition. Furthermore, taking into account weather changes, the dual-spectrum camera parameters are dynamically optimized based on real-time environmental perception to more accurately acquire inland waterway video data. Specifically, during the dual-spectrum video acquisition process for the target inland waterway, the parameters of the dual-spectrum video acquisition device are dynamically optimized based on real-time monitored environmental data, including resolution, exposure parameters, and monitoring temperature thresholds. The parameters of the dual-spectrum video acquisition device, which are pre-set to match environmental data combinations, can be determined based on the optimal parameters provided by experts under different historical environmental data combinations.

[0054] For example: in clear daytime conditions, adjust the visible light camera to 1080P resolution and ISO 100; in rainy or foggy weather, enable a polarizing filter to suppress glare for the visible light camera, enable a 30ms long exposure, and set the noise reduction mode, while adjusting the gain of the infrared camera; at night, adjust the visible light camera to 720P resolution, increase the infrared camera to 1080P resolution, and enable infrared fill light.

[0055] Correspondingly, environmental data collection around inland waterways can integrate meteorological data collected by weather stations or be obtained using environmental sensors. The collected environmental data includes meteorological parameters such as wind speed, wind direction, temperature, humidity, air pressure, and visibility.

[0056] Accordingly, in order to more accurately obtain potentially hidden data in the current bispectral video, water level sensors, flow meters and other equipment were selected to collect river data, including river physical parameters (such as river width distribution and river depth distribution), river hydrological data (river surface status, river flow velocity, etc.) and the location and type of fixed obstacles (bridge piers / shoals / vegetation location, etc.).

[0057] In addition, in order to ensure the effective transmission and timely analysis of the collected data, the bandwidth data of the dual-spectrum video transmission channel, including the bandwidth size of the transmission channel, needs to be monitored in real time.

[0058] Through the above data collection process, dual-spectral video is collected to intuitively reflect the situation of ships in the waterway. Collecting river and environmental data helps to understand the actual condition of the waterway, and collecting transmission bandwidth data ensures the stability and efficiency of video transmission.

[0059] S2. Perform comprehensive data decision processing on the collected bispectral video, river data and environmental data, and adaptively match decision schemes for video encoding, transmission and storage.

[0060] To improve video transmission efficiency and storage utilization while ensuring the accuracy of transmitted video, the system adaptively matches the optimal preset encoding compression scheme, preset multi-stream setting scheme, and preset video transmission matching strategy to complete video compression, transmission, and storage. The specific matching decision-making process includes:

[0061] First, it matches the preset encoding compression scheme and the preset multi-stream setting scheme.

[0062] First, considering that the purpose of this application is to identify prohibited vessels, it is necessary to perform differentiated encoding and compression on the parts of the collected video that may contain vessels and the parts that do not contain vessels. Therefore, it is necessary to predict the possible presence of vessels in the collected video in advance. Specifically, a data prediction model for prohibited vessels is pre-built using deep learning algorithms. The model is then used to obtain the predicted data of prohibited vessels. The model is generated by training on data of prohibited vessels in a specific inland waterway under historical environmental conditions. The input of the model is the currently collected environmental data and the current inland waterway data. The output is the predicted location, type, and speed of the prohibited vessels.

[0063] Subsequently, the real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video are used as decision factors for the encoding and compression strategy, and a matching preset encoding and compression scheme and a preset multi-stream setting scheme are obtained; the specific matching process includes:

[0064] Key feature data from the currently collected river channel data are extracted and quantified, including: river channel physical parameters, river hydrological data, and the location and type of fixed obstacles. Based on the extracted river channel physical parameters, the target inland waterway region in the bispectral video is determined and divided. The target inland waterway region includes: near-shore area (e.g., distance from the inland waterway bank ≤ 50m, water depth ≤ 3m, including obstacles such as wharves, shoals, and vegetation), main channel area (e.g., distance from the inland waterway bank 50m-200m, water depth 3-10m), and far-shore area (e.g., distance from the inland waterway bank greater than 200m, water depth ≥ 10m). A corresponding quantization value is set for each type of region.

[0065] For the target inland waterway area in the segmented dual-spectral video, the corresponding area is quantized; based on the extracted river hydrological data, the typical hydrological characteristics of each area of ​​the target inland waterway in the dual-spectral video are determined, such as: based on wave height / flow velocity, it is determined to be a calm water surface, a medium wave surface, or a fast-flowing water surface. A corresponding quantization value is set for each typical hydrological characteristic, and the hydrological characteristics of the corresponding area are quantified for the determined typical hydrological characteristics; corresponding quantization values ​​are set for the location and type of fixed obstacles, and the location and type of fixed obstacles existing in the corresponding area of ​​the target inland waterway are quantified.

[0066] Based on the defined target inland waterway areas, determine the environmental data currently collected for each area (nearshore area, main channel area), extract the key feature data of the environmental data currently collected for each area and quantify the features, including: meteorological data; specific feature quantification can be carried out by pre-dividing the range of each type of meteorological data, determining the typical meteorological conditions involved in each river waterway area within the target in the bispectral video, setting a corresponding quantification value for each typical meteorological condition, and quantifying the meteorological characteristics of the corresponding area for the determined typical meteorological conditions.

[0067] Based on the defined target inland waterway regions, determine the currently predicted vessel data belonging to each region, extract key feature data of the currently collected vessel data in each region and quantify the features, including: vessel type and vessel speed; specific feature quantization can be pre-set with corresponding quantization values ​​for each combination of vessel type and vessel speed, determine the combinations of vessel type and vessel speed involved in each river waterway region in the target in the bispectral video, and quantify the vessel data in the corresponding region for the determined combinations.

[0068] Finally, deep learning algorithms are used to construct a coding compression scheme identification model and a multi-stream setting model, respectively. The quantized values ​​of all key features extracted for each region of the single-track video in the dual-spectrum video are input into the coding compression scheme identification model and the multi-stream setting model, respectively, and the corresponding preset coding compression scheme and preset multi-stream setting scheme are obtained.

[0069] A deep learning algorithm is used to construct a coding compression scheme recognition model. For each region (i.e., each segmented region) of a single-track video (i.e., visible light video image or infrared light video image) in a dual-spectrum video, the quantized values ​​of all key features extracted are input into the coding compression scheme recognition model, and a corresponding preset coding compression scheme is obtained. Specifically, the coding compression scheme recognition model is trained using the quantized values ​​of all key features extracted from each region of a single-track video in historical dual-spectrum videos, as well as coding compression schemes that have been identified by historical experts as meeting the current quantization value coding requirements (e.g., high resolution, medium QP). The preset coding compression schemes include H.266 / VVC and ROI coding (main channel ROI). H.266 encoding, H.265 / HEVC and dynamic ROI encoding (encoding when ship data is present), H.264 and single-track encoding (i.e. infrared single-track video image encoding) are different schemes suitable for different scenarios and requirements. For example, for predicting the presence of large ships traveling at low speeds in the main channel area (encoding requirements of high frame rate across the entire area) and in low visibility weather conditions, the corresponding matching encoding standard is H.266 / VVC and ROI encoding (main channel area ROI priority encoding, QP=22).

[0070] Multi-stream setting models are constructed using deep learning algorithms. For each region (i.e., each divided region) of a single-track video (i.e., visible light video image or infrared light video image) in a dual-spectrum video, the quantized values ​​of all key features extracted are input into the multi-stream setting model, and a corresponding preset multi-stream setting scheme is obtained. Specifically, the multi-stream setting model is trained and generated using the quantized values ​​of all key features extracted from each region of a single-track video in historical dual-spectrum videos and multi-stream setting schemes that have been identified by historical experts as meeting the current application requirements of the quantized values ​​(analysis requirements, preview requirements, and retrospective requirements, etc.). The preset multi-stream setting scheme includes: a main stream, a secondary stream, and a storage backup stream to cover different requirements for resolution, frame rate, and bit rate range; for example, for ship identification analysis and historical backtracking in the main channel area where large ships are predicted to be traveling at low speed and in low visibility weather conditions, a main stream with a resolution of 1080P, a frame rate of 25-30fps, and a bit rate range of 2-4Mbps is set, and a storage backup stream with a resolution of 480P, a frame rate of 10fps, and a bit rate range of no more than 300kbps is set.

[0071] Second, match the preset video transmission matching strategy.

[0072] Secondly, in order to further ensure effective transmission, a preset video transmission matching strategy is obtained by combining the real-time collected transmission bandwidth data.

[0073] Specifically, different ranges of transmission bandwidth data are set, and corresponding video transmission matching strategies are matched for each range. For example, parallel transmission of the dual-spectral main stream is matched with bandwidth greater than the first range (sufficient bandwidth, e.g., ≥5Mbps); adaptive transmission switching between the dual-spectral main and secondary streams is matched with bandwidth less than the first range but greater than the second range (fluctuating bandwidth, e.g., 2-5Mbps); and ROI-limited stream transmission is matched with bandwidth less than the second range (urgent bandwidth, e.g., no greater than 2Mbps). Specifically, parallel transmission of the dual-spectral main stream transmits the main streams of visible light or infrared video images separately when bandwidth is sufficient; adaptive transmission switching between the dual-spectral main and secondary streams adaptively switches between the main and secondary streams for transmitting visible light or infrared video images under fluctuating bandwidth conditions; and ROI-limited stream transmission transmits only the ROI-region stream for transmitting visible light or infrared video images when bandwidth is urgent.

[0074] Therefore, the strategy matching based on the above multiple factors makes video encoding, compression, and transmission more flexible and efficient, adapting to different river channels, environments, and bandwidth conditions, and improving the quality and efficiency of video processing.

[0075] S3. Compress, transmit, and store the current dual-spectrum video according to the matching preset encoding compression scheme, preset multi-stream settings, and preset video transmission matching strategy.

[0076] Specifically, the video processing stage includes compressing, transmitting, and storing the current dual-spectrum video according to the matching preset encoding compression scheme, preset multi-stream settings, and preset video transmission matching strategy.

[0077] Compressing bispectral video using a matched encoding compression scheme can reduce video data volume and lower storage and transmission costs. For example, H.265 encoding compression technology can significantly reduce the bitrate while maintaining the same image quality, thus reducing storage space requirements and bandwidth consumption. Setting up multiple bitstreams allows for the provision of video streams with different resolutions and frame rates to meet the needs of different users and facilitates multi-user monitoring. Video transmission is configured according to a pre-defined video transmission matching strategy, ensuring stable and timely transmission under varying bandwidth conditions. For storage, the compressed video can be stored on a local hard drive or a cloud server for easy retrieval and analysis.

[0078] S4. Receive and decode the transmitted video, analyze and identify the risk of prohibited vessels entering the video image, determine the comprehensive risk level, match a multi-scale registration strategy, and perform feature extraction, registration, and fusion of the bispectral video to identify and obtain whether there are prohibited vessels entering the video.

[0079] Specifically, it receives and decodes transmitted video; through the decoder, it can quickly and accurately restore the compressed video to the original video.

[0080] Considering that the difficulty of identifying whether there are prohibited vessels in the image varies under different scenario conditions, the risk value of identifying prohibited vessels is pre-quantified and assessed. Specifically, all key features extracted from each region of the single-track video in the dual-spectral video are used as risk assessment factors. That is, the real-time collected river data, environmental data, and predicted data of prohibited vessels in the current dual-spectral video are used as risk assessment factors. The comprehensive risk level is determined according to the preset risk assessment rules, and a multi-scale registration strategy matching it is obtained.

[0081] The preset risk assessment rules include matching preset risk assessment quantification values ​​to each key feature. For example, the preset risk assessment values ​​for nearshore areas, main channel areas, and offshore areas gradually increase, corresponding to quantification values ​​of 30, 60, and 80, respectively; the preset risk assessment values ​​for large cargo ships, speedboats, and small fishing boats gradually increase, corresponding to quantification values ​​of 30, 60, and 80; and the preset risk assessment values ​​for calm waters, moderate waves, and heavy rain / rapid currents gradually increase, corresponding to quantification values ​​of 30, 60, and 80. The risk assessment values ​​matched to all key features are comprehensively and weighted to obtain the final risk assessment value, which is then compared with the preset risk assessment threshold to obtain the final comprehensive risk level.

[0082] Each multi-scale matching and registration strategy is pre-defined with a corresponding comprehensive risk level (low, medium, and high). Each multi-scale matching strategy includes: feature extraction method, registration strategy, scale parameter, and fusion rule. Different multi-scale matching strategies have different combinations of feature extraction methods, registration strategies, scale parameters, and fusion rules. The feature extraction methods include: SIFT and edge gradients, ORB and contour moments, SURF and motion vectors. The registration strategies include: local affine transformation, global projection transformation, and optical flow-guided deformation. The scale parameters include: pyramids with different numbers of layers. The fusion rules include: weighted fusion with different weight parameters. For example: if the overall risk level is high, the first multi-scale matching and registration strategy is used, including SIFT and edge gradient, local affine transformation and 5-layer pyramid, with a weighted fusion setting where the weight of the registration data in the previous layer is greater than the weight of the next layer; if the overall risk level is medium, the second multi-scale matching and registration strategy is used, including SURF and motion vector, optical flow guided deformation and 4-layer pyramid, with a weighted fusion setting where the weight of the registration data in each layer is similar; if the overall risk level is low, the third multi-scale matching and registration strategy is used, including ORB + contour moments, global projection transformation and 3-layer pyramid, with a weighted fusion setting where the weight of the registration data in the next layer is greater than the weight of the previous layer.

[0083] Finally, the decoded bispectral video is subjected to feature extraction, registration, and fusion according to the matching multi-scale registration strategy to identify and obtain whether there are any prohibited vessels. Specifically, multiple prohibited vessel identification models are constructed, and each prohibited vessel identification model adopts a multi-scale registration strategy. The corresponding model input is bispectral video. The feature weights after the bispectral video is fused using the multi-scale registration strategy, and the output is whether there are any prohibited vessels and the location and type of the prohibited vessels.

[0084] S5. Based on the identification results of the presence of prohibited vessels, generate warning commands and activate alarm lights to issue warnings.

[0085] Specifically, based on the identification results of the prohibited vessels, a warning instruction is generated accordingly, and the location of the prohibited vessels in the target inland waterway area is determined simultaneously. The warning instruction is then broadcast to the prohibited vessels in the target inland waterway area.

[0086] To further enhance early warning capabilities, a coordinated warning command is simultaneously sent to alarm lights, warning voice devices, or vessels installed within a preset distance of the identified prohibited vessel, activating the corresponding alarm lights for coordinated warning. For example... Figure 2 As shown, taking an inland waterway as an example, the on-site deployment includes installing dual-spectrum tube cameras, network speakers, and red and blue alarm lights on poles set every 60 meters along the river and on bridges, forming a monitoring network covering an area 180 meters long and 20 meters wide. Once a prohibited vessel is detected, a linkage warning command is sent to the alarm lights or vessels installed within a preset distance from the detected vessel's location, based on the vessel's location. The network speakers play a pre-recorded warning voice message, and the red and blue alarm lights flash, visually reminding the driver of the vessel.

[0087] In one specific embodiment, to further improve the timeliness of alerts, joint recognition of video images can be performed using multiple edge nodes, while minimizing the cost of applying multiple edge nodes. Joint recognition of multiple edge nodes is only performed for situations with poor timeliness. The method described in this application includes:

[0088] The system monitors and identifies vessels that have illegally entered the area. If no complete video identification result is received within the preset identification time, the received and transmitted video will be cropped and segmented according to the unidentified area. Specifically, the area where no identification result was obtained will be determined according to the different area ranges. The preset identification time can be set by the user according to their needs.

[0089] Each unidentified region after cropping and each unidentified region in the single-track video of the dual-spectrum video are respectively sent to an edge node for joint recognition. This allows each edge node to perform feature extraction, registration, and fusion based on each unidentified region after cropping and each unidentified region in the single-track video of the dual-spectrum video, and to identify whether there are prohibited vessels entering each unidentified region.

[0090] Finally, all recognition results are summarized and displayed. Images from different regions are stitched together to obtain the final recognition result.

[0091] like Figure 3 As shown, this application also discloses an inland waterway safety monitoring system, specifically including:

[0092] The monitoring data acquisition module 101 is used to acquire bispectral video of the target inland waterway; while acquiring bispectral video, it also acquires river channel data and environmental data of the target inland waterway; and monitors the bandwidth data of the bispectral video transmission channel in real time.

[0093] The monitoring data processing module 102 is used to obtain a preset encoding compression scheme and a preset multi-stream setting scheme that match the real-time collected river data, environmental data, and predicted data of prohibited vessels entering the current bispectral video as encoding compression strategy decision factors; and to obtain a preset video transmission matching strategy that matches the real-time collected transmission bandwidth data; the predicted data of prohibited vessels entering the current bispectral video is obtained using a prohibited vessel data prediction model built based on a deep learning algorithm.

[0094] The monitoring data transmission module 103 is used to complete the compression, transmission and storage of the current dual-spectrum video according to the matching preset encoding compression scheme, preset multi-stream settings and preset video transmission matching strategy;

[0095] The regulatory data analysis module 104 is used to receive and decode transmitted video; based on real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video as risk assessment factors, it determines the comprehensive risk level according to preset risk assessment rules and obtains a matching multi-scale registration strategy; according to the matching multi-scale registration strategy, it performs feature extraction, registration, and fusion on the decoded bispectral video to identify and obtain whether there are prohibited vessels entering the bispectral video.

[0096] The regulatory data warning module 105 is used to generate warning commands and activate alarm lights to issue warnings based on the identification results of the presence of prohibited vessels.

[0097] In a specific embodiment, the monitoring data analysis module 104 in this system is also used to monitor and identify the identification time of prohibited vessels. If no complete video identification result is received within the preset identification time, the received and transmitted video after decoding is cropped according to the unidentified areas. Each cropped unidentified area and the corresponding multi-scale registration strategy obtained for each unidentified area of ​​the single-track video in the dual-spectrum video are respectively transmitted to an edge node for linkage identification. This allows each edge node to perform feature extraction, registration, and fusion based on each cropped unidentified area and the corresponding multi-scale registration strategy obtained for each unidentified area of ​​the single-track video in the dual-spectrum video, identify and obtain whether there are prohibited vessels in each unidentified area, and finally summarize and display all identification results.

[0098] This application also discloses a computer-readable storage medium.

[0099] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed as described above for the inland waterway safety supervision method. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] This application also discloses a computer device.

[0101] Specifically, the computer device includes a memory and a processor, and the memory stores computer programs that can be loaded by the processor and executed inland waterway safety supervision methods.

[0102] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. An inland waterway safety supervision method, characterized in that, include: Bispectral video was collected for the target inland waterway; While acquiring bispectral video, river channel data and environmental data of the target inland waterway are also acquired. Real-time monitoring of bandwidth data for dual-spectrum video transmission channels; Based on real-time collected river data, environmental data, and predicted data of prohibited vessels in the current bispectral video as decision factors for encoding and compression strategies, a preset encoding and compression scheme and a preset multi-stream setting scheme are obtained to match them; combined with real-time collected transmission bandwidth data, a preset video transmission matching strategy is obtained to match it; the predicted data of prohibited vessels in the current bispectral video is obtained using a prohibited vessel data prediction model built based on deep learning algorithms. The current dual-spectrum video is compressed, transmitted, and stored according to the matching preset encoding compression scheme, preset multi-stream settings, and preset video transmission matching strategy. Receive and decode transmitted video; Based on real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video as risk assessment factors, the comprehensive risk level is determined according to preset risk assessment rules, and a matching multi-scale registration strategy is obtained. The decoded dual-spectral video is subjected to feature extraction, registration and fusion according to the matching multi-scale registration strategy to identify and obtain whether there are prohibited vessels entering the video. Based on the identification results of the prohibited vessels, a warning command is generated and the alarm light is activated to issue a warning.

2. The inland waterway safety supervisory method according to claim 1, characterized by, Also includes: During the process of collecting bispectral video of the target inland waterway, environmental data was collected in real time and then... The parameters of the dual-spectrum video acquisition device are dynamically optimized, including resolution, exposure parameters, and monitoring temperature threshold.

3. The inland waterway safety supervisory method according to claim 1, characterized by, The step of using real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video as decision factors for encoding and compression strategies, and obtaining a matching preset encoding and compression scheme and preset multi-stream settings, includes: Key feature data of the currently collected river channel data are extracted and quantified, including: river channel physical parameters, river channel hydrological data, and the location and type of fixed obstacles; based on the extracted river channel physical parameters, the target inland waterway area in the bispectral video is determined and divided; the target inland waterway area includes: near-shore area, main channel area, and far-shore area; combined with the divided target inland waterway area, the environmental data currently collected in each area is determined, and key feature data of the environmental data currently collected in each area is extracted and quantified, including: meteorological data; combined with the divided target inland waterway area, the predicted vessel data currently collected in each area is determined, and key feature data of the vessel data currently collected in each area is extracted and quantified, including: vessel type and vessel speed; A coding compression scheme identification model and a multi-stream setting model were constructed using deep learning algorithms. The quantized values ​​of all key features extracted from each region of a single-track video in a dual-spectrum video were input into the coding compression scheme identification model and the multi-stream setting model, respectively, to obtain corresponding preset coding compression schemes and preset multi-stream setting schemes. The preset coding compression schemes included: H.266 / VVC and ROI encoding, H.265 / HEVC and dynamic ROI, and H.264 and single-track encoding. The multi-stream setting schemes included: a main stream, a secondary stream, and a storage backup stream covering different requirements for resolution, frame rate, and bitrate range.

4. The inland waterway safety supervisory method according to claim 1, characterized by, The step of combining real-time acquired transmission bandwidth data to obtain a pre-defined video transmission matching strategy includes: Set transmission bandwidth data belonging to different ranges, and set a video transmission matching strategy for each range of transmission bandwidth data, including: parallel transmission of dual-spectrum main stream matching data greater than the first transmission bandwidth data, adaptive transmission of dual-spectrum main stream and secondary stream switching matching data less than the first transmission bandwidth data but greater than the second transmission bandwidth data, and transmission of ROI-limited stream matching data less than the second transmission bandwidth data.

5. The inland waterway safety supervisory method according to claim 3, characterized by, The process of using real-time collected river data, environmental data, and predicted data on prohibited vessels entering the river from the current bispectral video as risk assessment factors, determining the comprehensive risk level according to preset risk assessment rules, and obtaining a matching multi-scale registration strategy includes: For each region of a single-track video in a dual-spectrum video, all key features extracted are used as risk assessment factors. A comprehensive risk level is determined according to preset risk assessment rules, and a matching multi-scale registration strategy is obtained. The preset risk assessment rules include matching preset risk assessment quantification values ​​for each key feature. Each multi-scale registration strategy has a preset comprehensive risk level. The system includes: feature extraction methods, registration strategies, scale parameters, and fusion rules. Feature extraction methods include: SIFT and edge gradients, ORB and contour moments, SURF and motion vectors. Registration strategies include: local affine transformation, global projection transformation, and optical flow-guided deformation. Scale parameters include: pyramids with different numbers of layers. The fusion rules include: weighted fusion with different weight parameters.

6. The inland waterway safety supervisory method according to claim 5, characterized by, Also includes: The system monitors and identifies vessels illegally entering the area. If no complete video identification result is received within the preset identification time, the received and transmitted video is cropped according to the unidentified areas. Each cropped unidentified area and the corresponding multi-scale registration strategy obtained for each unidentified area in the dual-spectrum video are sent to an edge node for linkage identification. This allows each edge node to perform feature extraction, registration, and fusion based on each cropped unidentified area and the corresponding multi-scale registration strategy obtained for each unidentified area in the dual-spectrum video, identifying whether there are vessels illegally entering the area in each unidentified area. Finally, all identification results are summarized and displayed.

7. The inland waterway safety supervisory method according to claim 1, characterized by, Also includes: In the process of generating warning instructions in response to the identification results of prohibited vessels, the location of the identified prohibited vessels in the target inland waterway area is determined simultaneously, and a warning instruction is broadcast to the identified prohibited vessels in the target inland waterway area; at the same time, a linkage warning instruction is sent to the alarm lights, warning voice devices or vessels installed within a preset distance from the identified prohibited vessel location, so as to activate the alarm lights and warning voice devices at the corresponding locations for linkage warning.

8. An inland waterway safety supervisory system, characterized by, include: The regulatory data acquisition module is used to collect bispectral video of the target inland waterway. While acquiring bispectral video, river channel data and environmental data of the target inland waterway are also acquired. Real-time monitoring of bandwidth data for dual-spectrum video transmission channels; The regulatory data processing module is used to obtain a preset encoding compression scheme and a preset multi-stream setting scheme based on real-time collected river data, environmental data, and predicted data of prohibited vessels entering the current bispectral video as encoding compression strategy decision factors; and to obtain a preset video transmission matching strategy based on real-time collected transmission bandwidth data; the predicted data of prohibited vessels entering the current bispectral video is obtained using a prohibited vessel data prediction model built based on deep learning algorithms. The monitoring data transmission module is used to complete the compression, transmission and storage of the current dual-spectrum video according to the matching preset encoding compression scheme, preset multi-stream settings and preset video transmission matching strategy; The regulatory data analysis module is used to receive and decode transmitted video. Based on real-time collected river data, environmental data, and predicted data on prohibited vessels entering the current bispectral video as risk assessment factors, the comprehensive risk level is determined according to preset risk assessment rules, and a matching multi-scale registration strategy is obtained. The decoded dual-spectral video is subjected to feature extraction, registration and fusion according to the matching multi-scale registration strategy to identify and obtain whether there are prohibited vessels entering the video. The regulatory data warning module is used to generate warning commands and trigger alarm lights to issue warnings based on the identification results of vessels entering the area illegally.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 7.

10. A computer device, comprising: The computer device includes a memory, a processor, and a program stored in and executable on the memory, the program being executed by the processor to implement the steps of the method as described in any one of claims 1 to 7.

Images