A protection device and system for improving the safety of a wharf bollard

Abstract

The present application belongs to the technical field of improving wharf bollard safety, and discloses a protection device and system for improving wharf bollard safety, which comprises a bollard monitoring module, a main control module, a monitoring video enhancement module, a bollard rust detection module, a corrosion judgment module, a stability test module, an alarm module and a display module. The monitoring video enhancement module considers the wharf bollard monitoring video clip as a frame sequence. For each frame in the frame sequence, there may be a certain time sequence correlation between the frame and the adjacent multiple adjacent frames, such as texture, color, motion trajectory, etc. A high-quality enhanced frame corresponding to the target frame is obtained, effectively improving the picture quality. At the same time, the bollard rust detection module reduces the influence of the background on the image segmentation stage, and realizes classification and positioning of power corrosion at the pixel level.

Description

Technical Field

[0001] This invention belongs to the technical field of improving the safety of dock bollards, and particularly relates to a protective device and system for improving the safety of dock bollards. Background Technology

[0002] Mooring bollards, also known as "mooring posts" or "lined bollards," are stakes fixed to the deck or dockside for securing mooring lines. They are generally made of cast or welded metal. Because they bear significant stress during use, their bases must be extremely sturdy. Types include single-cross bollards, double-cross bollards, upright bollards, inclined bollards, and ram's horn bollards. To prevent the lines from slipping off the bollard, a cap slightly larger than the bollard body is typically placed on top. On ships, bollards are usually installed at the bow, stern, and port and starboard decks. However, existing protection systems for improving dock bollard safety acquire unclear monitoring videos; furthermore, current features extracted from rust detection on dock bollards have very low dimensionality, easily leading to overfitting and unsatisfactory recognition results.

[0003] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0004] (1) The existing protection system for improving the safety of dock bollards has unclear monitoring videos.

[0005] (2) Currently, the features extracted from the detection of rust on dock bollards have very low dimensionality, which can easily lead to overfitting and unsatisfactory recognition results. Summary of the Invention

[0006] In view of the problems existing in the prior art, the present invention provides a protective device and system for improving the safety of dock bollards.

[0007] This invention is implemented as follows: a protection system for improving the safety of dock bollards includes:

[0008] The bollard monitoring module is used to monitor the dock bollards via video using cameras;

[0009] The surveillance video enhancement module is used to enhance the surveillance video of the dock bollards.

[0010] The bollard rust detection module is used to detect rust on dock bollards.

[0011] The corrosion assessment module is used to determine the degree of corrosion of the dock bollards.

[0012] The stability testing module is used to test the stability of the dock bollards;

[0013] The alarm module is used to issue alerts for abnormal conditions of the dock bollards;

[0014] The display module is used to display the monitoring video, detection results, judgment results, and test results of the dock bollards;

[0015] The main control module is connected to the bollard monitoring module, monitoring video enhancement module, bollard rust detection module, corrosion judgment module, stability testing module, alarm module, and display module, and is used to control the normal operation of each module.

[0016] The stability testing module is used to determine the degree of corrosion of the dock bollards, and specifically includes:

[0017] 1) Install a sensor on the cable under test, and excite a guided wave signal on the cable under test through the sensor. The guided wave signal is reflected by the cable and then collected by the sensor. The collected signal is processed by a pre-conditioning circuit to obtain the actual measured signal, and then read into the laptop computer by the data acquisition card.

[0018] 2) Based on the cable's geometric dimensions, material properties, and excitation signal conditions, a detailed numerical model for determining cable corrosion is established using finite element technology;

[0019] 3) Initialize the numerical model for cable corrosion determination described in 2), input the geometric dimensions, material properties, and excitation signal conditions of the cable under test, couple it with the model, and obtain the simulated response signal through analysis and calculation to provide a comparison signal for the actual measured signal;

[0020] 4) The actual measured signal is used as the discrimination signal for cable corrosion, and the simulated response signal obtained by the numerical model coupled with the cable corrosion judgment is used as the comparison signal for the actual measured signal. The analysis is performed using a neural network analysis method. If the damage does not exceed the threshold, the cable is not corroded; otherwise, an alarm is issued and the location of the corrosion is located.

[0021] 5) Generate a record report and record it in the local database.

[0022] Furthermore, the enhancement method of the surveillance video enhancement module is as follows:

[0023] (1) Obtain the monitoring video frame sequence of the dock bollard to be processed through a video processing program. The frame sequence includes the target frame and multiple neighboring frames adjacent to the target frame. Extract the feature information of each frame in the frame sequence.

[0024] (2) Obtain reference feature information for enhancing the target frame based on the feature information of each frame; obtain the target enhanced frame corresponding to the target frame based on the reference feature information and the target frame;

[0025] Wherein, obtaining reference feature information for enhancing the target frame based on the feature information of each frame includes:

[0026] Based on the temporal relationship of the frame sequence, the feature information of each frame is input into a nonlocal convolutional long short-term memory network (NLSM). The reference feature information is obtained through the NLSM, which includes hidden state information and / or unit state information corresponding to the target frame. The NLSM includes a forward NLSM and a backward NLSM. The forward NLSM includes a first nonlocal module and a forward LSTM module. The backward NLSM includes a second nonlocal module and a backward LSTM module. The first nonlocal module is used to determine the weights of the hidden state information and unit state information output in the previous frame based on the inter-frame similarity between two adjacent frames. The second nonlocal module is used to determine the weights of the hidden state information and unit state information output in the subsequent frame based on the inter-frame similarity between two adjacent frames.

[0027] Furthermore, the plurality of neighboring frames includes a plurality of previous neighboring frames and a plurality of subsequent neighboring frames;

[0028] The plurality of preceding neighboring frames include a plurality of consecutive frames that are adjacent to and in front of the target frame, and the plurality of following neighboring frames include a plurality of consecutive frames that are adjacent to and in behind the target frame.

[0029] Furthermore, the extraction of feature information from each frame in the frame sequence includes:

[0030] Feature information of each frame is extracted by an encoder, wherein the encoder includes multiple convolutional layers.

[0031] Furthermore, the detection method of the cable bollard rust detection module is as follows:

[0032] 1) Collect images of rust stains on dock bollards using dock bollard monitoring equipment, and preprocess them to obtain a training set; input the training set into an improved fully convolutional network for training, extract multi-scale features from the rust stain images, and generate feature maps;

[0033] 2) Use a region candidate network to perform region candidate operations on the generated feature map to generate multiple candidate target detection windows; use a neural network to obtain the detection results of each target detection window on the rust images in the training set; fuse the detection results of all target detection windows to obtain the optimal target detection window.

[0034] Furthermore, the step of preprocessing the acquired rust images includes:

[0035] The acquired rust images were preprocessed, including image rotation, vertical and horizontal mirror flipping, scale transformation, and the addition of difficult samples.

[0036] A training set is constructed using the preprocessed image data.

[0037] Furthermore, the improved fully convolutional network includes multiple dilated convolutions at different scales.

[0038] Furthermore, the step of extracting multi-scale features from the rust image includes:

[0039] The rust images in the training set are used as input to the improved fully convolutional network for training.

[0040] After the pooling layer of the improved fully convolutional network, multiple dilated convolutional layers with different sampling rates are used in parallel to scale the input rust image to different degrees, so as to obtain feature information of rust images of dock bollards of different sizes.

[0041] Pooling is used to fuse feature information from multiple dilated convolutional layers to generate a fixed-length feature map.

[0042] Furthermore, after generating multiple candidate target detection windows, the non-maximum suppression algorithm in the regional candidate network is used to eliminate overlapping target detection windows.

[0043] Furthermore, the step of obtaining the detection results of each target detection window for the rust images in the training set through the neural network includes:

[0044] The training set is used as input to the neural network for deep learning training;

[0045] The rust images in the training set are processed through the target detection window to obtain a fractional image;

[0046] In the first layer of the neural network, two parallel layers are set up. The second layer has two channels. The first channel is used to obtain the score of the image in the training set passing through the object detection window, and the score parameter is added to the score map.

[0047] Based on the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solution to be protected by this invention from the following aspects:

[0048] This invention utilizes a video enhancement module to consider that the video clips from dock bollard monitoring are frame sequences. For each frame in this sequence, there may be temporal correlations between it and its multiple neighboring frames, such as texture, color, and motion trajectories. Taking a target frame as an example, its multiple neighboring frames may contain information lost during compression. Therefore, this information can be used to repair the target frame, resulting in a high-quality enhanced frame that effectively improves image quality. Simultaneously, the bollard rust detection module reduces the impact of the background on the image segmentation stage, enabling pixel-level classification and localization of electrical corrosion. Attached Figure Description

[0049] Figure 1 This is a structural block diagram of a protection system for improving the safety of dock bollards, provided in an embodiment of the present invention.

[0050] Figure 2 This is a flowchart of the video enhancement module enhancement method provided in the embodiments of the present invention.

[0051] Figure 3 This is a flowchart of the detection method for the cable bollard rust detection module provided in an embodiment of the present invention.

[0052] Figure 1 The system consists of: 1. Cable bollard monitoring module; 2. Main control module; 3. Monitoring video enhancement module; 4. Cable bollard rust detection module; 5. Corrosion judgment module; 6. Stability testing module; 7. Alarm module; 8. Display module. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] I. Explanatory and Illustrative Embodiments. To enable those skilled in the art to fully understand how the present invention is specifically implemented, this section provides an explanatory and illustrative description of the embodiments described in the claims.

[0055] like Figure 1 As shown, the protection system for improving the safety of dock bollards provided in this embodiment of the invention includes:

[0056] Bollard monitoring module 1 is used to monitor the dock bollards via video using a camera;

[0057] The video enhancement module 3 is used to enhance the video of the dock bollard monitoring.

[0058] Module 4 for detecting rust on dock bollards;

[0059] Corrosion assessment module 5 is used to assess the degree of corrosion of the dock bollards;

[0060] Stability testing module 6 is used to test the stability of the dock bollards;

[0061] Alarm module 7 is used to issue alarms for abnormal conditions of the dock bollards;

[0062] Display module 8 is used to display the monitoring video, detection results, judgment results, and test results of the dock bollards;

[0063] The main control module 2 is connected to the bollard monitoring module 1, the monitoring video enhancement module 3, the bollard rust detection module 4, the corrosion judgment module 5, the stability test module 6, the alarm module 7, and the display module 8, and is used to control the normal operation of each module.

[0064] The stability testing module is used to determine the degree of corrosion of the dock bollards, and specifically includes:

[0065] 1) Install a sensor on the cable under test, and excite a guided wave signal on the cable under test through the sensor. The guided wave signal is reflected by the cable and then collected by the sensor. The collected signal is processed by a pre-conditioning circuit to obtain the actual measured signal, and then read into the laptop computer by the data acquisition card.

[0066] 2) Based on the cable's geometric dimensions, material properties, and excitation signal conditions, a detailed numerical model for determining cable corrosion is established using finite element technology;

[0067] 3) Initialize the numerical model for cable corrosion determination described in 2), input the geometric dimensions, material properties, and excitation signal conditions of the cable under test, couple it with the model, and obtain the simulated response signal through analysis and calculation to provide a comparison signal for the actual measured signal;

[0068] 4) The actual measured signal is used as the discrimination signal for cable corrosion, and the simulated response signal obtained by the numerical model coupled with the cable corrosion judgment is used as the comparison signal for the actual measured signal. The analysis is performed using a neural network analysis method. If the damage does not exceed the threshold, the cable is not corroded; otherwise, an alarm is issued and the location of the corrosion is located.

[0069] 5) Generate a record report and record it in the local database.

[0070] like Figure 2 As shown, the video enhancement module enhancement method provided by this invention is as follows:

[0071] S101, the video processing program obtains a sequence of monitoring video frames of the dock bollards to be processed, the sequence of frames including the target frame and multiple neighboring frames adjacent to the target frame; and extracts the feature information of each frame in the sequence of frames.

[0072] S102, obtain reference feature information for enhancing the target frame based on the feature information of each frame; obtain the target enhanced frame corresponding to the target frame based on the reference feature information and the target frame;

[0073] Wherein, obtaining reference feature information for enhancing the target frame based on the feature information of each frame includes:

[0074] Based on the temporal relationship of the frame sequence, the feature information of each frame is input into a nonlocal convolutional long short-term memory network (NLSM). The reference feature information is obtained through the NLSM, which includes hidden state information and / or unit state information corresponding to the target frame. The NLSM includes a forward NLSM and a backward NLSM. The forward NLSM includes a first nonlocal module and a forward LSTM module. The backward NLSM includes a second nonlocal module and a backward LSTM module. The first nonlocal module is used to determine the weights of the hidden state information and unit state information output in the previous frame based on the inter-frame similarity between two adjacent frames. The second nonlocal module is used to determine the weights of the hidden state information and unit state information output in the subsequent frame based on the inter-frame similarity between two adjacent frames.

[0075] The multiple neighboring frames provided by this invention include multiple previous neighboring frames and multiple subsequent neighboring frames;

[0076] The plurality of preceding neighboring frames include a plurality of consecutive frames that are adjacent to and in front of the target frame, and the plurality of following neighboring frames include a plurality of consecutive frames that are adjacent to and in behind the target frame.

[0077] The present invention provides a method for extracting feature information of each frame in the frame sequence, including:

[0078] Feature information of each frame is extracted by an encoder, wherein the encoder includes multiple convolutional layers.

[0079] like Figure 3 As shown, the detection method of the cable bollard rust detection module provided by the present invention is as follows:

[0080] S201. Images of rust on dock bollards are collected using dock bollard monitoring equipment and preprocessed to obtain a training set. The training set is then input into an improved fully convolutional network for training to extract multi-scale features from the rust images and generate feature maps.

[0081] S202, a region candidate network is used to perform region candidate operations on the generated feature map to generate multiple candidate target detection windows; the detection results of each target detection window on the rust images in the training set are obtained through a neural network; the detection results of all target detection windows are fused to obtain the optimal target detection window.

[0082] The preprocessing steps for the acquired rust images provided by this invention include:

[0083] The acquired rust images were preprocessed, including image rotation, vertical and horizontal mirror flipping, scale transformation, and the addition of difficult samples.

[0084] A training set is constructed using the preprocessed image data.

[0085] The improved fully convolutional network provided by this invention includes multiple dilated convolutions at different scales.

[0086] The steps for extracting multi-scale features from rust images provided by this invention include:

[0087] The rust images in the training set are used as input to the improved fully convolutional network for training.

[0088] After the pooling layer of the improved fully convolutional network, multiple dilated convolutional layers with different sampling rates are used in parallel to scale the input rust image to different degrees, so as to obtain feature information of rust images of dock bollards of different sizes.

[0089] Pooling is used to fuse feature information from multiple dilated convolutional layers to generate a fixed-length feature map.

[0090] The present invention generates multiple candidate target detection windows and then uses a non-maximum suppression algorithm in a regional candidate network to eliminate overlapping target detection windows.

[0091] The steps provided by this invention for obtaining the detection results of each target detection window on the rust images in the training set through a neural network include:

[0092] The training set is used as input to the neural network for deep learning training;

[0093] The rust images in the training set are processed through the target detection window to obtain a fractional image;

[0094] In the first layer of the neural network, two parallel layers are set up. The second layer has two channels. The first channel is used to obtain the score of the image in the training set passing through the object detection window, and the score parameter is added to the score map.

[0095] II. Application Examples. To demonstrate the inventiveness and technical value of the technical solution of this invention, this section provides application examples of the technical solution of the claims on specific products or related technologies.

[0096] In operation, the present invention firstly monitors the dock bollards via video using a camera through the bollard monitoring module 1; secondly, the main control module 2 enhances the monitoring video of the dock bollards through the monitoring video enhancement module 3; the bollard rust detection module 4 detects rust on the dock bollards; the corrosion judgment module 5 judges the degree of corrosion of the dock bollards; the stability test module 6 tests the stability of the dock bollards; then, the alarm module 7 issues an alarm for abnormal conditions of the dock bollards; finally, the display module 8 displays the dock bollard monitoring video, detection results, judgment results, and test results.

[0097] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0098] III. Evidence of the Relevant Effects of the Embodiments. The embodiments of the present invention have achieved some positive effects during research and development or use, and indeed possess significant advantages compared to existing technologies. The following description, in conjunction with data, charts, and other materials from the experimental process, illustrates these advantages.

[0099] This invention utilizes a video enhancement module to consider that the video clips from dock bollard monitoring are frame sequences. For each frame in this sequence, there may be temporal correlations between it and its multiple neighboring frames, such as texture, color, and motion trajectories. Taking a target frame as an example, its multiple neighboring frames may contain information lost during compression. Therefore, this information can be used to repair the target frame, resulting in a high-quality enhanced frame that effectively improves image quality. Simultaneously, the bollard rust detection module reduces the impact of the background on the image segmentation stage, enabling pixel-level classification and localization of electrical corrosion.

[0100] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A protection system for improving the safety of dock bollards, characterized in that, The protection system for improving the safety of dock bollards includes: The bollard monitoring module is used to monitor the dock bollards via video using cameras; The surveillance video enhancement module is used to enhance the surveillance video of the dock bollards. The bollard rust detection module is used to detect rust on dock bollards. The corrosion assessment module is used to determine the degree of corrosion of the dock bollards. The stability testing module is used to test the stability of the dock bollards; The alarm module is used to issue alerts for abnormal conditions of the dock bollards; The display module is used to display the monitoring video, detection results, judgment results, and test results of the dock bollards; The main control module is connected to the bollard monitoring module, monitoring video enhancement module, bollard rust detection module, corrosion judgment module, stability testing module, alarm module, and display module, and is used to control the normal operation of each module. The enhancement method of the surveillance video enhancement module is as follows: (1) Obtain the monitoring video frame sequence of the dock bollard to be processed through a video processing program. The frame sequence includes the target frame and multiple neighboring frames adjacent to the target frame. Extract the feature information of each frame in the frame sequence. (2) Obtain reference feature information for enhancing the target frame based on the feature information of each frame; obtain the target enhanced frame corresponding to the target frame based on the reference feature information and the target frame; The stability testing module is used to determine the degree of corrosion of the dock bollards, and specifically includes: 1) Install a sensor on the cable under test, and excite a guided wave signal on the cable under test through the sensor. The guided wave signal is reflected by the cable and then collected by the sensor. The collected signal is processed by a pre-conditioning circuit to obtain the actual measured signal, and then read into the laptop computer by the data acquisition card. 2) Based on the cable's geometric dimensions, material properties, and excitation signal conditions, a detailed numerical model for determining cable corrosion is established using finite element technology; 3) Initialize the numerical model for cable corrosion determination described in 2), input the geometric dimensions, material properties, and excitation signal conditions of the cable under test, couple it with the model, and obtain the simulated response signal through analysis and calculation to provide a comparison signal for the actual measured signal; 4) The actual measured signal is used as the discrimination signal for cable corrosion, and the simulated response signal obtained by the numerical model coupled with the cable corrosion judgment is used as the comparison signal for the actual measured signal. The analysis is performed using a neural network analysis method. If the damage does not exceed the threshold, the cable is not corroded; otherwise, an alarm is issued and the location of the corrosion is located. 5) Generate a record report and record it in the local database.

2. The protection system for improving the safety of dock bollards as described in claim 1, characterized in that, The step of obtaining reference feature information for enhancing the target frame based on the feature information of each frame includes: Based on the temporal relationship of the frame sequence, the feature information of each frame is input into a nonlocal convolutional long short-term memory network (NLSM). The reference feature information is obtained through the NLSM, which includes hidden state information and / or unit state information corresponding to the target frame. The NLSM includes a forward NLSM and a backward NLSM. The forward NLSM includes a first nonlocal module and a forward LSTM module. The backward NLSM includes a second nonlocal module and a backward LSTM module. The first nonlocal module is used to determine the weights of the hidden state information and unit state information output in the previous frame based on the inter-frame similarity between two adjacent frames. The second nonlocal module is used to determine the weights of the hidden state information and unit state information output in the subsequent frame based on the inter-frame similarity between two adjacent frames.

3. The protection system for improving the safety of dock bollards as described in claim 2, characterized in that, The plurality of neighboring frames includes a plurality of previous neighboring frames and a plurality of subsequent neighboring frames; The plurality of preceding neighboring frames include a plurality of consecutive frames that are adjacent to and in front of the target frame, and the plurality of following neighboring frames include a plurality of consecutive frames that are adjacent to and in behind the target frame.

4. The protection system for improving the safety of dock bollards as described in claim 2, characterized in that, The step of extracting feature information from each frame in the frame sequence includes: Feature information of each frame is extracted by an encoder, wherein the encoder includes multiple convolutional layers.

5. The protection system for improving the safety of dock bollards as described in claim 1, characterized in that, The detection method of the cable bollard rust detection module is as follows: 1) Collect images of rust stains on dock bollards using dock bollard monitoring equipment, and preprocess them to obtain a training set; input the training set into an improved fully convolutional network for training, extract multi-scale features from the rust stain images, and generate feature maps; 2) Use a region candidate network to perform region candidate operations on the generated feature map to generate multiple candidate target detection windows; use a neural network to obtain the detection results of each target detection window on the rust images in the training set; fuse the detection results of all target detection windows to obtain the optimal target detection window.

6. The protection system for improving the safety of dock bollards as described in claim 5, characterized in that, The preprocessing steps for the acquired rust images include: The acquired rust images were preprocessed, including image rotation, vertical and horizontal mirror flipping, scale transformation, and the addition of difficult samples. A training set is constructed using the preprocessed image data.

7. The protection system for improving the safety of dock bollards as described in claim 5, characterized in that, The improved fully convolutional network includes multiple dilated convolutions at different scales.

8. The protection system for improving the safety of dock bollards as described in claim 5, characterized in that, The steps for extracting multi-scale features from rust images include: The rust images in the training set are used as input to the improved fully convolutional network for training. After the pooling layer of the improved fully convolutional network, multiple dilated convolutional layers with different sampling rates are used in parallel to scale the input rust image to different degrees, so as to obtain feature information of rust images of dock bollards of different sizes. Pooling is used to fuse feature information from multiple dilated convolutional layers to generate a fixed-length feature map.

9. The protection system for improving the safety of dock bollards as described in claim 5, characterized in that, After generating multiple candidate target detection windows, the non-maximum suppression algorithm in the regional candidate network is used to eliminate overlapping target detection windows.

10. The protection system for improving the safety of dock bollards as described in claim 5, characterized in that, The step of obtaining the detection results of each target detection window for the rust images in the training set through the neural network includes: The training set is used as input to the neural network for deep learning training; The rust images in the training set are processed through the target detection window to obtain a fractional image; In the first layer of the neural network, two parallel layers are set up. The second layer has two channels. The first channel is used to obtain the score of the image in the training set passing through the object detection window, and this score is added to the score map.

Images