Visual intelligent detection model and system for marine facility attaching organisms

Abstract

The invention discloses a visual intelligent detection model and system for marine facility attaching creatures, and belongs to the field of visual detection.The visual intelligent detection model for marine facility attaching creatures comprises an image enhancement module, a feature extraction and fusion module and a visual intelligent detection result output module; the visual intelligent detection system comprises a main control module, a high-definition imaging module, a seawater environment sensing module and a visual intelligent detection module, and is characterized in that a visual intelligent detection model of the marine facility attaching creatures, which can be carried at an embedded end, is constructed, and the visual intelligent detection system of the marine facility attaching creatures is designed; the real-time intelligent detection and evaluation of the marine attaching organisms on the surface of the marine facility are realized by detecting the type, the attaching position, the size and the number of the attaching organisms.

Description

Technical Field

[0001] The present invention relates to the field of underwater detection technology, and in particular to a visual intelligent detection model and system for organisms attached to marine facilities. Background Art

[0002] The ocean occupies about 70% of the earth's surface and contains rich minerals, organisms and energy. It is of indispensable importance to human survival and economic prosperity, and supports the needs and development of global human society. The surface of marine facilities is vulnerable to the invasion of attached organisms, such as algae and shellfish, which not only affects the service life of the facilities, but also increases the maintenance and cleaning costs. For example, submarine pipelines, as the main transportation carrier of submarine oil and gas resources, play an extremely important role in the development and utilization of marine oil and gas resources. They are an important component of major underwater infrastructure. With the continuous development of the ocean by humans, the scale of laying submarine pipelines is increasing. However, some problems have gradually emerged. In the long-term use of marine pipelines, they are often affected by marine attached organisms. For example, marine organisms such as microorganisms, algae, shellfish, and barnacles will adhere to the surface. Some organisms (such as algae, shellfish and barnacles) will release chemicals, accelerate the corrosion of pipeline materials, and damage the structural integrity of the pipeline. Therefore, it is crucial to discover and identify attached organisms on marine facilities as early as possible and clean them up in time to avoid the above problems.

[0003] With the development of deep learning networks and AI technology, it has become possible to use image recognition technology to identify the types of marine attached organisms. Underwater target detection technology based on neural networks has been widely used in aquaculture, marine garbage detection, deep-sea organism tracking and other fields. However, there is still a lack of a visual intelligent detection model and system for marine facility attached organisms for real-time detection of attached organisms on the surface of marine facilities. Summary of the invention

[0004] The purpose of the present invention is to provide a visual intelligent detection model for marine facility attached organisms to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: a visual intelligent detection model for marine facility attached organisms, an image enhancement module, a feature extraction and fusion module and a visual intelligent detection result output module,

[0006] The image enhancement module is used to pre-process the collected marine facility images to improve the image quality and obtain enhanced images with higher definition;

[0007] The feature extraction and fusion module is used to extract key features from the enhanced image and fuse them to obtain a fused image. The key features include features that can distinguish between attached organisms and non-attached organisms.

[0008] The visual intelligent detection result output module is used to input the enhanced image and the fused image into a pre-trained deep learning-based marine attachment organism detection model to obtain the type of attached organisms as well as the location, size and number of attached organisms, thereby realizing real-time detection of biological attachment to marine facilities.

[0009] This application.

[0010] Preferably, the preprocessing includes color restoration and contrast stretching.

[0011] Preferably, the image enhancement module preprocesses the collected marine facility images including:

[0012] S1. Decompose the collected marine facility images into RGB channels, calculate the grayscale average of each channel, and sort them by size to obtain large, medium, and small color channels I l ,I m ,I s ;

[0013] S2. Multiply the gray value of the large color channel by the compensation coefficient and add it to the medium and small color channels to make the gray average values ​​of the three channels equal. The calculation formula includes:

[0014] Among them, I' m ,I s ' is the transformed color channel, is the grayscale average of the three color channels;

[0015] S3, convert the transformed RGB image into HSI color space, and use linear transformation to stretch the intensity channel. The calculation formula includes:

[0016] Among them, I represents the input pixel value, I new Represents the output pixel value, I max_new and I min_new Respectively represent the maximum and minimum values ​​of the expected conversion range, I max and I min Respectively represent the maximum and minimum values ​​of the current input image.

[0017] S4. Restoring the image to the RGB space, and outputting the image after color restoration and contrast stretching, which is the enhanced image.

[0018] Preferably, the process of constructing a marine attachment organism detection model based on deep learning includes:

[0019] A1: Obtain a set of images of organisms attached to submarine pipelines, and annotate the images of organisms attached to submarine pipelines according to the types of organisms attached to them, to obtain annotated sample images;

[0020] A2: Divide the labeled sample images into training set, test set and validation set;

[0021] A3: Based on deep learning, the training set is used for model training, and the test set and validation set are used for testing and verification to obtain a marine attachment organism detection model based on deep learning.

[0022] Preferably, the feature extraction and fusion module includes several layers of convolutional neural networks, each layer includes a convolutional layer, a nonlinear activation layer and a pooling layer, which are used to extract multi-level features of the image step by step. The enhanced image is first input into the convolutional layer of the convolutional neural network, and the low-level features of the image are extracted through the local receptive field. The pooling layer reduces the spatial resolution of the feature map and suppresses noise, extracts intermediate and high-level features, and the feature maps output by each convolutional layer are weighted fused in the feature fusion unit to finally generate a high-dimensional feature vector for image classification, detection and recognition tasks.

[0023] A visual intelligent detection system for organisms attached to marine facilities, using any of the above-mentioned visual intelligent detection models for organisms attached to marine facilities, including a main control module, a high-definition imaging module, a seawater environment perception module and a visual intelligent detection module.

[0024] Main control module: used to control the work of each module;

[0025] High-definition imaging module: used to collect images of marine facilities to be inspected.

[0026] Seawater environment perception module: used to obtain the marine environment data around the marine facilities to be tested in order to adjust the imaging parameters of the high-definition imaging module;

[0027] Visual intelligent detection module: includes a visual intelligent detection model for organisms attached to marine facilities, which is used to perform real-time intelligent detection on the images of marine facilities to be detected, obtain the type of attached organisms and the location, size and number of attached organisms, and realize real-time detection of biological attachment to marine facilities.

[0028] Storage module: used to store collected data and test results.

[0029] Preferably, the high-definition imaging module includes an underwater high-definition camera and a high-power LED strobe light.

[0030] The underwater high-definition camera's camera lens is a fixed-focus lens with a pixel greater than 12 million, which is used to collect images of marine facilities to be inspected.

[0031] The high-power LED strobe light has a brightness of more than 300,000 lumens and is used as a light source for underwater high-definition imaging, providing sufficient illumination in low-light conditions.

[0032] Preferably, the seawater environment sensing module includes a turbidity sensor, which monitors the turbidity of seawater in real time. The main control module dynamically adjusts the exposure parameters of the underwater high-definition camera in real time according to the data collected by the turbidity sensor, ensuring that the underwater high-definition camera obtains the best imaging effect under different water quality conditions.

[0033] Preferably, the underwater high-definition camera and the high-power LED strobe light are connected through a trigger signal line. The underwater high-definition camera outputs a TTL pulse signal when starting exposure, and the pulse width is equal to the camera exposure time length. The high-power LED strobe light achieves synchronous flash lighting by receiving the TTL pulse signal, ensuring that the flash time is accurately matched with the camera exposure time, providing the best lighting effect.

[0034] Compared with the prior art, the beneficial effects of the present invention are: the present application can automatically identify and detect attached organisms on marine facilities, reducing the time and cost of manual detection. Compared with traditional manual detection methods, the system has a higher level of automation, faster processing speed, and significantly improved efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of the structure of a visual intelligent detection model for organisms attached to marine facilities in the first embodiment of the present invention;

[0036] Figure 2 It is a structural schematic diagram of a visual intelligent detection system for organisms attached to marine facilities in the second embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Embodiment 1

[0039] like Figure 1 As shown, the present invention provides a visual intelligent detection model for marine facility attached organisms, comprising:

[0040] The image enhancement module is used to perform color restoration and contrast stretching on the collected marine facility images to improve image quality and obtain enhanced images with higher definition;

[0041] The image enhancement module pre-processes the collected marine facility images including:

[0042] S1. Decompose the collected marine facility images into RGB channels, calculate the grayscale average of each channel, and sort them by size to obtain large, medium, and small color channels I l ,I m ,I s ;

[0043] S2. Multiply the gray value of the large color channel by the compensation coefficient and add it to the medium and small color channels to make the gray average values ​​of the three channels equal. The calculation formula includes:

[0044] Among them, I' m ,I s ' is the transformed color channel, is the grayscale average of the three color channels;

[0045] S3, convert the transformed RGB image into HSI color space, and use linear transformation to stretch the intensity channel. The calculation formula includes:

[0046] Among them, I represents the input pixel value, I new Represents the output pixel value, I max_new and I min_new Respectively represent the maximum and minimum values ​​of the expected conversion range, I max and I min Respectively represent the maximum and minimum values ​​of the current input image.

[0047] S4. Restoring the image to the RGB space, and outputting the image after color restoration and contrast stretching, which is the enhanced image.

[0048] The feature extraction and fusion module is used to extract key features from the enhanced image and fuse them to obtain a fused image. The key features include features that can distinguish attached organisms from non-attached organisms, such as shape, contour and other information;

[0049] The feature extraction and fusion module consists of a multi-layer convolutional neural network (CNN), each of which contains a convolutional layer, a nonlinear activation layer, and a pooling layer, which is used to extract multi-level features of the image step by step. The preprocessed enhanced image is first input into the initial convolutional layer of the CNN to extract low-level features of the image, such as edges and textures, through the local receptive field; further, after the pooling operation, the spatial resolution of the feature map is reduced and the noise is suppressed. Furthermore, more complex intermediate and high-level features are extracted, such as the semantic information of the shape and object contour. The feature maps output by each convolutional layer are weighted fused in the feature fusion unit to ensure that the key features are retained and the redundant information is minimized. Finally, a high-dimensional feature vector is generated for image classification, detection, and recognition tasks.

[0050] The visual intelligent detection result output module is used to input the enhanced image and the fused image into a pre-trained deep learning-based marine attachment organism detection model to obtain the type of attached organisms (such as barnacles, oysters, hydras, etc.) and the location, size and number of attached organisms, thereby realizing real-time detection of biological attachment to marine facilities.

[0051] The process of building a marine attachment organism detection model based on deep learning includes:

[0052] A1: Obtain an image set of organisms attached to submarine pipelines (such as algae, shellfish, and barnacles), and use annotation tools (such as LabelImg and CVAT) to annotate the organisms attached to submarine pipelines according to the species of the organisms attached to them, to obtain annotated sample images;

[0053] A2: The labeled sample images are randomly divided into training set, test set and validation set according to the ratio of 8:1:1, and each submarine pipeline attached organism is labeled with an independent name label to complete the construction of the submarine pipeline attached organism dataset;

[0054] A3: Based on deep learning, the training set is used for model training, and the test set and validation set are used for testing and verification to obtain a marine attachment organism detection model based on deep learning.

[0055] Specifically, the deep learning-based marine attachment organism detection model is trained by the following method:

[0056] Based on the image extraction and fusion of feature vectors, a deep learning model is constructed, such as Faster R-CNN, YOLO or SSD target detection network, and model compression techniques such as pruning are used to reduce the model size and computational complexity while maintaining detection performance. During the training process, optimization algorithms such as gradient descent are used to continuously adjust the model parameters to minimize the loss function. Cross-validation and other methods are used during training to prevent overfitting of the model and improve its generalization ability. An independent validation set is used to evaluate the performance of the trained model. The main evaluation indicators include accuracy, recall rate, precision rate, etc. According to the evaluation results, the model structure and parameters are repeatedly adjusted to further optimize the model performance, and finally a marine attached organism detection model that can be integrated into an embedded system is obtained.

[0057] In summary, by sequentially connecting the image enhancement module, the feature extraction and fusion module and the visual intelligent detection result output module, a visual intelligent detection model for organisms attached to marine facilities is constructed, which can be used to realize real-time intelligent detection of organisms attached to marine facilities by installing it on an embedded platform.

[0058] Embodiment 2

[0059] like Figure 2 As shown, a deep learning-based marine attachment organism detection system adopts the visual intelligent detection model for marine facility attachment organisms of embodiment 1, including a main control module, a high-definition imaging module, a seawater environment perception module and a visual intelligent detection module.

[0060] Main control module: used to control the work of each module;

[0061] High-definition imaging module: used to collect images of marine facilities to be inspected. The high-definition imaging module includes an underwater high-definition camera and a high-power LED strobe light.

[0062] The underwater high-definition camera has a 6mm fixed-focus lens with a pixel size greater than 12 million. It is used to collect images of marine facilities to be inspected and transmit the collected image data to the main control module in real time.

[0063] The high-power LED strobe light has a brightness of more than 300,000 lumens and is used as a light source for underwater high-definition imaging, providing sufficient illumination in low-light conditions.

[0064] Specifically, the strobe light is lit in the following way: a DC-DC boost circuit is used to charge a large capacitor group. When the LED is lit, the large capacitor group is equivalent to a constant current source that provides a large current drive to the LED, thereby lighting the LED. The circuit design ensures that before the next LED is lit, the circuit charges the capacitor to the previous voltage value to ensure the consistency of the LED brightness.

[0065] Specifically, the underwater high-definition camera and the high-power LED strobe light achieve collaborative work in the following manner: the underwater high-definition camera and the high-power LED strobe light are connected through a trigger signal line, the camera outputs a TTL pulse signal when starting exposure, and the pulse width is equal to the camera exposure time length, and the high-power LED strobe light achieves synchronous flash lighting by receiving the TTL pulse signal, ensuring that the flash time is accurately matched with the camera exposure time, providing the best lighting effect.

[0066] Seawater environment perception module: used to obtain the marine environment data around the marine facilities to be tested to adjust the imaging parameters of the high-definition imaging module. The seawater environment perception module includes a turbidity sensor, which monitors the turbidity of seawater in real time. The main control module dynamically adjusts the exposure parameters of the underwater high-definition camera in real time according to the data collected by the turbidity sensor to ensure that the underwater high-definition camera obtains the best imaging effect under different water quality conditions. The working principle of the turbidity sensor is based on optical scattering technology. Specifically, the built-in light source of the sensor emits a light beam into the water. When the light beam encounters suspended particles, it is scattered. The sensor calculates the turbidity of the water body by measuring the intensity of the scattered light. The intensity of the scattered light is proportional to the concentration of suspended particles in the water, so as to obtain an accurate turbidity value. The turbidity sensor is connected to the main controller through a data transmission interface (such as UART, I2C or SPI), and the data collected by the sensor is transmitted to the main control unit in the form of a digital signal. The main control unit dynamically adjusts the key parameters of the camera, such as exposure time, white balance and contrast, according to the received turbidity value to adapt to different water transparency conditions, thereby ensuring that clear images can still be obtained in a high turbidity environment.

[0067] Visual intelligent detection module: includes a visual intelligent detection model for organisms attached to marine facilities, which is used to perform real-time intelligent detection on the images of marine facilities to be detected, obtain the type of attached organisms and the location, size and number of attached organisms, and realize real-time detection of biological attachment to marine facilities.

[0068] The visual intelligent detection module is composed of an embedded edge computer, and the computing power of the embedded edge computer can meet the workload requirements of running the visual intelligent detection model of attached organisms. The visual intelligent detection model of attached organisms on marine facilities is deployed on the embedded edge computer, and real-time target detection and identification of attached organisms on the surface of marine facilities is achieved by receiving and processing image data from the high-definition imaging module.

[0069] Specifically, the embedded edge computer and the underwater high-definition camera work together in the following ways: the embedded edge computer and the underwater high-definition camera are connected via a USB3.0 high-speed data cable to achieve rapid transmission of collected image data. The I / O port of the embedded edge computer is connected to the trigger signal line of the camera, and the image acquisition mode of the underwater high-definition camera is set to an external trigger mode, which can be triggered to take pictures through an external TTL signal control. A computer program is written to enable the embedded edge computer to output a TTL trigger signal through the I / O port, thereby controlling the underwater high-definition camera to take pictures, and intelligently controlling the camera's shooting.

[0070] Storage module: used to store collected data and test results. The storage module consists of a non-volatile memory and a control unit, and is connected to the main processor in the embedded system through a fast transmission interface. The control unit is responsible for the reading and writing of data. When receiving a write instruction from the main processor, the control unit writes the data into the non-volatile memory and ensures that the data remains intact in the event of a power outage. When receiving a read instruction, the control unit reads the data from the non-volatile memory and returns it to the main processor via the bus. The storage module also supports data verification to improve the reliability of data storage.

[0071] In summary, the marine facility attached organisms visual intelligent detection system uses the collaborative work of high-definition imaging modules, seawater environment perception modules, and visual intelligent detection modules to achieve real-time target detection and identification of organisms attached to the surface of marine facilities. It can detect the attachment of marine facility organisms in real time in water, reduce the maintenance and management costs of marine facilities, improve the efficiency of marine facility detection and maintenance, and ensure the long-term stable operation of marine facilities. The system provides a practical and effective solution for the intelligent management of marine facilities, which helps promote the sustainable use of marine resources.

[0072] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A visual intelligent detection model for marine facility attached organisms, characterized by: It includes image enhancement module, feature extraction and fusion module and visual intelligent detection result output module. The image enhancement module is used to pre-process the collected marine facility images to improve the image quality and obtain enhanced images with higher definition; The feature extraction and fusion module is used to extract key features from the enhanced image and fuse them to obtain a fused image. The key features include features that can distinguish between attached organisms and non-attached organisms. The visual intelligent detection result output module is used to input the enhanced image and the fused image into a pre-trained deep learning-based marine attachment organism detection model to obtain the type of attached organisms as well as the location, size and number of attached organisms, thereby realizing real-time detection of biological attachment to marine facilities.

2. A visual intelligent detection model for marine facility attached organisms according to claim 1, characterized in that: Preprocessing includes color restoration and contrast stretching.

3. A visual intelligent detection model for marine facility attached organisms according to claim 1 or 2, characterized in that: The image enhancement module pre-processes the collected marine facility images including: S1. Decompose the collected marine facility images into RGB channels, calculate the grayscale average of each channel, and sort them by size to obtain large, medium, and small color channels I l ,I m ,I s ; S2. Multiply the gray value of the large color channel by the compensation coefficient and add it to the medium and small color channels to make the gray average values ​​of the three channels equal. The calculation formula includes: Among them, I' m ,I s ' is the transformed color channel, is the grayscale average of the three color channels; S3, convert the transformed RGB image into HSI color space, and use linear transformation to stretch the intensity channel. The calculation formula includes: Among them, I represents the input pixel value, I new Represents the output pixel value, I max_new and I min_new Respectively represent the maximum and minimum values ​​of the expected conversion range, I max and I min Respectively represent the maximum and minimum values ​​of the current input image. S4. Restoring the image to the RGB space, and outputting the image after color restoration and contrast stretching, which is the enhanced image.

4. A visual intelligent detection model for marine facility attached organisms according to claim 1 or 2, characterized in that: The process of building a marine attachment organism detection model based on deep learning includes: A1: Obtain a set of images of organisms attached to submarine pipelines, and annotate the images of organisms attached to submarine pipelines according to the types of organisms attached to them, to obtain annotated sample images; A2: Divide the labeled sample images into training set, test set and validation set; A3: Based on deep learning, the training set is used for model training, and the test set and validation set are used for testing and verification to obtain a marine attachment organism detection model based on deep learning.

5. A visual intelligent detection model for organisms attached to marine facilities according to claim 1 or 2, characterized in that: The feature extraction and fusion module includes several layers of convolutional neural networks, each of which contains a convolutional layer, a nonlinear activation layer and a pooling layer, which is used to extract multi-level features of the image step by step. The enhanced image is first input into the convolutional layer of the convolutional neural network, and the low-level features of the image are extracted through the local receptive field. The pooling layer reduces the spatial resolution of the feature map and suppresses noise, extracts intermediate and high-level features, and the feature maps output by each convolutional layer are weighted fused in the feature fusion unit to finally generate a high-dimensional feature vector for image classification, detection and recognition tasks.

6. A visual intelligent detection system for marine facility attached organisms, using the visual intelligent detection model for marine facility attached organisms as claimed in any one of claims 1 to 5, characterized in that: It includes main control module, high-definition imaging module, seawater environment perception module and visual intelligent detection module. Main control module: used to control the work of each module; High-definition imaging module: used to collect images of marine facilities to be inspected. Seawater environment perception module: used to obtain the marine environment data around the marine facilities to be tested in order to adjust the imaging parameters of the high-definition imaging module; Visual intelligent detection module: includes a visual intelligent detection model for organisms attached to marine facilities, which is used to perform real-time intelligent detection on the images of marine facilities to be detected, obtain the type of attached organisms and the location, size and number of attached organisms, and realize real-time detection of biological attachment to marine facilities. Storage module: used to store collected data and test results.

7. A visual intelligent detection system for marine facility attached organisms according to claim 6, characterized in that: The high-definition imaging module includes an underwater high-definition camera and a high-power LED strobe light. The underwater high-definition camera's camera lens is a fixed-focus lens with a pixel greater than 12 million, which is used to collect images of marine facilities to be inspected. The high-power LED strobe light has a brightness of more than 300,000 lumens and is used as a light source for underwater high-definition imaging, providing sufficient illumination in low-light conditions.

8. A visual intelligent detection system for marine facility attached organisms according to claim 6, characterized in that: The seawater environment perception module includes a turbidity sensor, which monitors the turbidity of seawater in real time. The main control module dynamically adjusts the exposure parameters of the underwater high-definition camera in real time according to the data collected by the turbidity sensor to ensure that the underwater high-definition camera obtains the best imaging effect under different water quality conditions.

9. A visual intelligent detection system for marine facility attached organisms according to claim 7, characterized in that: The underwater HD camera and the high-power LED strobe light are connected through a trigger signal line. The underwater HD camera outputs a TTL pulse signal at the beginning of exposure. The pulse width is equal to the exposure time of the camera. The high-power LED strobe light realizes synchronous flash lighting by receiving the TTL pulse signal, ensuring that the flash time is accurately matched with the exposure time of the camera, providing the best lighting effect.

Images