Method, device, medium and equipment for detecting cleaning condition of hull surface in underwater environment

Abstract

The invention discloses a hull surface cleaning condition detection method and device in an underwater environment, a medium and equipment, and the method comprises the steps: obtaining image data of a hull surface in the underwater environment, and obtaining a detection result based on a soft voting integrated classifier; the construction method comprises the following steps: carrying out fine tuning on a plurality of pre-trained convolutional neural network models by utilizing transfer learning; performing image classification on the image data of the hull surface in the underwater environment by using a plurality of fine-tuned convolutional neural network models; averaging the output probability value to obtain an average probability value; wherein if the average probability values of all the convolutional neural network models exceed a preset threshold value, the output detection result is unclean, and otherwise, the output detection result is clean. According to the invention, the problems of high labor intensity, high safety risk, low efficiency, strong subjectivity, great influence of underwater environment, easy interference of data transmission, lack of autonomous decision-making ability and high operation cost of a traditional hull surface detection method are solved.

Description

Technical Field

[0001] The invention relates to a method, a device, a medium and equipment for detecting the cleanliness condition of a hull surface in an underwater environment, and belongs to the technical field of computer vision and underwater robots. Background Art

[0002] Traditional hull surface inspection methods mainly rely on manual divers to conduct underwater inspections, which has many shortcomings. First, divers need to work underwater for a long time, which is labor-intensive and faces certain safety risks. At the same time, the inspection efficiency is low and it is impossible to quickly cover a large area of ​​the hull surface. Secondly, manual inspection is highly subjective, and the results often rely on the experience and judgment of divers, which are easily interfered by personal factors, resulting in inconsistent inspection results. In addition, harsh underwater environments, such as low visibility or turbulent water flow, will also affect the accuracy and safety of manual inspections. With the advancement of underwater robot technology, although it has been widely used in hull surface inspection, the existing technology still faces some obvious limitations. The quality of underwater images taken by underwater robots is often affected by the underwater environment, such as insufficient light or turbid water, resulting in blurred images and affecting the recognition of the hull surface condition. Although underwater robots have real-time video transmission capabilities, in complex underwater environments, signals are easily interfered with, resulting in data transmission delays and affecting real-time detection efficiency. At the same time, existing underwater robots rely on manual remote operation during the hull surface inspection process, lack autonomous identification and decision-making capabilities, and cannot automatically adjust the inspection strategy according to real-time results, resulting in low overall inspection efficiency. In addition, the research and development, maintenance and operation costs of underwater robots are high, and the wear and tear and maintenance requirements of the equipment increase additional costs, limiting their widespread application in hull inspection.

[0003] In summary, traditional hull surface inspection methods have the problems of high labor intensity, high safety risks, low efficiency, strong subjectivity, great influence of underwater environment, easy interference of data transmission, lack of independent decision-making ability and high operating costs. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device, medium and equipment for detecting the cleanliness condition of a hull surface in an underwater environment. Through underwater automated detection, the problems of traditional hull surface detection methods, such as high labor intensity, high safety risks, low efficiency, strong subjectivity, great influence of the underwater environment, easy interference in data transmission, lack of independent decision-making ability and high operating costs, can be solved. At the same time, the detection effect is more stable, more reliable and more efficient than traditional hull surface detection methods.

[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0006] In a first aspect, the present invention provides a method for detecting the cleanliness condition of a hull surface in an underwater environment, comprising: Acquire image data of the hull surface in an underwater environment; According to the image data, detection is performed based on a soft voting integrated classifier to obtain a detection result; The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

[0007] Furthermore, the method for acquiring image data of the hull surface in an underwater environment includes: In different underwater environments, remotely controlled underwater robots use high-definition cameras to collect video of the hull surface; A frame of image is extracted from the video every 1 second as image data of the hull surface in the underwater environment.

[0008] Furthermore, the training method of the convolutional neural network model includes: Acquire historical image data of the hull surface in an underwater environment over a period of time; Manually annotating the historical images to obtain an annotated image dataset, wherein the annotated contents include two categories: “clean” and “unclean”; The convolutional neural network model is trained using the annotated image dataset, using the Adam optimizer and binary cross entropy loss function, and the parameters of the convolutional neural network model are updated through the back propagation algorithm.

[0009] Furthermore, it also includes, in the initial stage of training the convolutional neural network model, freezing most of the network layers of the convolutional neural network model and only training the newly added network layers; After the initial stage of training, all network layers are unfrozen, training is continued with a smaller learning rate, and the hyperparameters of the convolutional neural network model are tuned.

[0010] Furthermore, it also includes preprocessing the labeled image data set, including adjusting brightness, contrast and saturation, and performing cropping.

[0011] Furthermore, the pre-training method of the convolutional neural network model includes adjusting the hyperparameters of each convolutional neural network model using a hyperparameter optimization technique.

[0012] Furthermore, using transfer learning to fine-tune multiple pre-trained convolutional neural network models includes: Each convolutional neural network model is initialized using the weights pre-trained on the ImageNet dataset, and a fine-tuning process is performed to make each convolutional neural network model more suitable for the underwater hull surface image dataset.

[0013] In a second aspect, the present invention provides a device for detecting the cleanliness condition of a hull surface in an underwater environment, comprising: A data acquisition module, used to acquire image data of the hull surface in an underwater environment; A cleaning detection module, used for performing detection based on the image data and a pre-trained soft voting ensemble classifier; A result output module, used to obtain the test results; The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

[0014] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting the cleanliness condition of a hull surface in an underwater environment as described in the first aspect.

[0015] In a fourth aspect, the present invention provides a computer device, comprising: A memory for storing instructions; The processor is used to execute the instructions so that the device implements the method for detecting the cleanliness condition of the hull surface in an underwater environment as described in the first aspect.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention obtains a soft voting integrated classifier by integrating multiple convolutional neural network models, including DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG, and uses transfer learning for fine-tuning, which significantly improves the detection accuracy and robustness under different underwater lighting conditions. The present invention also averages the output probability values ​​of multiple fine-tuned models, which not only effectively integrates the prediction advantages of each model, reduces the possible misjudgment of a single model, but also makes the final detection result more reliable. In addition, the preset average probability value threshold judgment mechanism can automatically distinguish the clean and unclean states of the hull surface, providing ship maintenance personnel with a fast and intuitive cleanliness assessment, solving the problems of high labor intensity, high safety risks, low efficiency, strong subjectivity, great influence of underwater environment, easy interference in data transmission, lack of independent decision-making ability and high operating costs in traditional hull surface detection methods, and also helps to take necessary cleaning measures in time to extend the service life of the hull, while ensuring navigation safety and environmental protection requirements.

[0017] 2. The present invention uses a high-definition camera to collect videos of the hull surface by remotely controlling an underwater robot, and extracts a frame of image as detection data every 1 second, thereby achieving real-time monitoring of the cleanliness of the underwater hull surface. The present invention also combines a soft voting integrated classifier to quickly process the image data and output the detection results. This efficient real-time detection capability enables ship maintenance personnel to promptly discover and deal with hull surface contamination problems, thereby avoiding potential navigation safety hazards and environmental problems.

[0018] 3. The present invention uses multiple convolutional neural network models for ensemble learning, and fine-tunes the model through transfer learning to make it more suitable for underwater hull surface image datasets. In addition, in the initial stage of training, most of the network layers are frozen to train only the newly added network layers, and then all network layers are unfrozen and continued training is performed at a smaller learning rate, which further improves the adaptability and accuracy of the model. This model training strategy makes the final detection results more reliable, reduces the possibility of misjudgment, and provides accurate cleaning status assessment for ship maintenance personnel.

[0019] 4. The present invention also preprocesses the labeled image dataset, such as adjusting brightness, contrast, and saturation, and performs cropping processing to improve the quality and consistency of image data. At the same time, the Adam optimizer and binary cross-entropy loss function are adopted, and the model parameters of the convolutional neural network model are updated through the backpropagation algorithm to further improve the training efficiency and accuracy of the convolutional neural network model. In addition, hyperparameter optimization technology is used to optimize the convolutional neural network model, so that the convolutional neural network model can exhibit good performance in different scenarios. This flexible data preprocessing and model optimization strategy makes the method have stronger generalization ability and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The figure shows a schematic flowchart of a method for detecting the cleaning status of the hull surface in an underwater environment provided by an embodiment of the present invention; Figure 2 The figure shows a schematic flowchart of the implementation steps of a method for detecting the cleaning status of the hull surface in an underwater environment provided by an embodiment of the present invention.

[0021] Figure 3 The figure shows a schematic physical structure diagram of a computer provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present invention and the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. Without conflict, the technical features in the embodiments of the present invention and the embodiments can be combined with each other.

[0023] The term "and / or" is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " generally represents an "or" relationship between the associated objects before and after.

[0024] Before introducing the embodiments of the present invention, the following interpretations are made for the relevant terms involved in the embodiments of the present invention: Transfer learning: Transfer learning is a machine learning method that improves the learning efficiency and performance of a new task by transferring the model or knowledge trained on one task or dataset to another related task. In transfer learning, the model does not need to learn from scratch, but uses existing knowledge such as the weights of a model pre-trained on a large-scale dataset to accelerate the learning process in a new domain or new task, especially suitable for scenarios with small data volume or limited computing resources.

[0025] Convolutional Neural Network (CNN): A deep learning model specifically designed to process grid data such as images. It automatically extracts local features through convolutional layers and learns more complex high-level features through multi-layer structures. It is widely used in tasks such as image classification and object detection.

[0026] Soft voting ensemble learning: A method to improve model performance by combining the predictions of multiple base learners. In soft voting, each base learner outputs a predicted probability for a category instead of directly giving a classification label. The final prediction result is determined by the average of the output probabilities of all base learners, and the category with the highest probability is usually selected. Soft voting ensemble learning takes into account the prediction confidence of each model, so it can better integrate the information of different models and improve the overall classification accuracy.

[0027] Example 1

[0028] like Figure 1 As shown, this embodiment introduces a method for detecting the cleanliness condition of a hull surface in an underwater environment, comprising: Acquire image data of the hull surface in an underwater environment, According to the image data, detection is performed based on a soft voting ensemble classifier to obtain a detection result.

[0029] The present invention provides original input information for cleaning condition detection by capturing images of the hull surface, wherein the soft voting integrated classifier includes multiple pre-trained convolutional neural network models, which are trained using a large amount of hull surface image data with known cleaning conditions, and can learn and identify features in the image, thereby judging the cleaning condition of the hull surface. The present invention can automatically complete the detection of the cleaning condition of the hull surface, improve the detection efficiency, reduce the subjectivity and uncertainty of manual judgment, and improve the accuracy of detection.

[0030] The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

[0031] Convolutional neural networks are suitable for image classification tasks and can efficiently extract features from images. The convolutional neural network models used in the present invention include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG. These convolutional neural network models perform well in image classification tasks and have different network structures and characteristics. By fine-tuning them as basic models and fusing them with soft voting ensemble classifiers, the accuracy and robustness of detection can be further improved.

[0032] Transfer learning is a method of using existing knowledge, i.e. pre-trained models, to solve new problems. In the present invention, the pre-trained convolutional neural network model is fine-tuned to better adapt to the task of detecting the cleanliness of the underwater hull surface, which reduces the time and data required to train the model from scratch and also improves the generalization ability of the model.

[0033] The soft voting ensemble classifier is a method for fusing the prediction results of multiple models. In the present invention, the soft voting ensemble classifier integrates multiple convolutional neural network models. The soft voting ensemble classifier obtains the final detection result by averaging the output probability values ​​of multiple fine-tuned convolutional neural network models, thereby improving the accuracy and robustness of the detection and reducing the possible misjudgment of a single model.

[0034] Among them, if the average probability values ​​of all convolutional neural network models exceed the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean.

[0035] The present invention sets a predetermined threshold and compares the average probability value of each convolutional neural network model with the threshold, thereby judging the cleanliness condition of the hull surface, making the detection result more objective and quantifiable.

[0036] Example 2

[0037] Based on the same inventive concept as in Example 1, Figure 2 As shown, this embodiment introduces the implementation steps of a method for detecting the cleanliness condition of a hull surface in an underwater environment, including: Step 1: Data collection.

[0038] First, based on the camera equipment carried by the remote-controlled underwater robot, underwater hull video data was collected at different dates and locations, covering the surface of 20 hulls. Each video was processed at one frame per second, and a total of 5,683 images were obtained, including 2,035 clean images and 3,648 unclean images. In order to train the deep learning model, all images were manually annotated and each image was cropped to 512×512 pixels. The effective image area was obtained by extracting the circular area captured by the camera, and these image data will be used for subsequent model training.

[0039] In some embodiments, a method for acquiring image data of a hull surface in an underwater environment includes: In different underwater environments, remotely controlled underwater robots use high-definition cameras to collect video of the hull surface; In order to ensure the quality and diversity of the data, a frame of image is extracted from the video every 1 second as image data of the hull surface in the underwater environment.

[0040] Step 2: Data labeling and processing.

[0041] In some embodiments, during the data set processing stage, clean images and unclean images are randomly selected, 2,000 images of each category are selected, and manual annotation is performed to obtain an annotated image data set, where the annotation content includes two categories: "clean" and "unclean". Step 3: Dataset division.

[0042] The labeled image dataset is divided into training set, validation set and test set in a ratio of 60%:20%:20%. The training set of each category contains 1,200 images, and the validation set and test set each contain 400 images.

[0043] Step 4: Data enhancement.

[0044] Table 1 Datasets for training, validation, and testing In addition, in order to improve the diversity of the data set, this embodiment also preprocesses the labeled image data set, including adjusting the brightness, contrast and saturation, and performing cropping to enhance the robustness of the model under different underwater environmental changes. After the samples are processed, the data set is shown in Table 1.

[0045] Step 5: Model selection and pre-training.

[0046] In some embodiments, convolutional neural network models including DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG are selected to construct the soft voting ensemble classifier, and all convolutional neural network models are pre-trained on the ImageNet dataset, and the obtained pre-training weights are used as initialization weights and applied to the dataset of this embodiment for transfer learning.

[0047] The pre-training method of the convolutional neural network model includes adjusting the hyperparameters of each convolutional neural network model using hyperparameter optimization technology to achieve the best classification performance, thereby improving the overall classification accuracy. The optimization process ensures that the convolutional neural network model can not only accurately classify, but also has strong generalization capabilities.

[0048] Step 6: Transfer learning and fine-tuning.

[0049] In order to enable these convolutional neural network models to process the hull surface images more accurately, this embodiment adopts transfer learning technology to retrain each convolutional neural network model for the hull surface features, thereby improving the recognition ability of each convolutional neural network model for different hull surface states. In the process of transfer learning, the parameters of these convolutional neural network models are fine-tuned to adapt them to the classification task of clean and unclean hull surfaces, further improving the accuracy and efficiency of classification. Table 2 shows the classification results of each model.

[0050] Table 2 Classification results of training and validation sets During the testing phase, the method provided in this embodiment performed excellently, with a classification accuracy rate of 98.13%. The results show that the method provided in this embodiment has high classification performance and stability. In addition, the classification time for a single image is 56.25 milliseconds, which fully meets the needs of underwater robots to perform real-time hull inspection tasks underwater. The method provided in this embodiment not only improves the automation level of underwater hull inspections, but also can provide real-time and accurate hull health assessments in practical applications, which helps to detect potential problems in a timely manner and ensure the safe operation of the ship. In addition, transfer learning not only accelerates the training process of the convolutional neural network model, but also improves the performance of the convolutional neural network model on underwater images, especially when the data samples are relatively limited.

[0051] In some embodiments, fine-tuning multiple pre-trained convolutional neural network models using transfer learning includes: Each convolutional neural network model is initialized using weights pre-trained on the ImageNet dataset, and a fine-tuning process is performed to make each convolutional neural network model more suitable for the underwater hull surface image dataset.

[0052] Step 7: Ensemble learning.

[0053] In order to further improve the detection performance, the present invention combines the above six convolutional neural network models that have been fine-tuned by transfer learning into a soft voting ensemble classifier.

[0054] The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

[0055] Step 8: Model training and optimization.

[0056] In this embodiment, the soft voting integrated classifier includes a plurality of pre-trained convolutional neural network models, the pre-trained convolutional neural network models perform image classification on image data of a hull surface in an underwater environment, the pre-trained convolutional neural network models are fine-tuned using transfer learning, and then the output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability values ​​of all convolutional neural network models exceed the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean.

[0057] In this embodiment, after receiving the image data of the hull surface in the underwater environment, each convolutional neural network model independently classifies the image data of the hull surface in the underwater environment and outputs the probability value of it belonging to the "unclean" category. The output probability values ​​are averaged. If the average probability values ​​of all convolutional neural network models exceed a predetermined threshold value such as 0.5, the detection result is unclean, otherwise the detection result is clean. This integration method effectively utilizes the advantages of multiple convolutional neural network models, significantly improves the classification accuracy, and enhances the adaptability of the soft voting integrated classifier to different underwater environments.

[0058] In some embodiments, the training method of the convolutional neural network model includes: Acquire historical image data of the hull surface in an underwater environment over a period of time; Manually annotating the historical images to obtain an annotated image dataset, wherein the annotated contents include two categories: “clean” and “unclean”; The model is trained using the annotated image dataset, using the Adam optimizer and binary cross entropy loss function, and the parameters of the convolutional neural network model are updated through the back-propagation algorithm.

[0059] In this embodiment, it also includes freezing most of the network layers of the parameters of the multiple convolutional neural network models in the initial stage of training the parameters of the multiple convolutional neural network models, and only training the newly added network layers; After the initial stage of training, all network layers are unfrozen, training is continued with a smaller learning rate, and hyperparameter tuning is performed on the parameters of multiple convolutional neural network models.

[0060] Step 9: Model integration and deployment.

[0061] The trained soft voting integrated classifier of this embodiment is integrated into the underwater robot system to realize real-time detection of the cleaning status of the hull surface. Table 3 shows the time spent on classifying images using the soft voting integration. During navigation, the underwater robot collects images of the hull surface in real time and inputs the images of the hull surface into the soft voting integrated classifier for real-time classification. According to the classification results, the underwater robot system can automatically adjust the cleaning strategy, such as selecting the area to be cleaned, adjusting the cleaning intensity, etc., so as to improve the efficiency and effect of cleaning the hull surface. Through this intelligent real-time detection and control system, the present invention can significantly improve the automation level of the underwater hull cleaning process and realize efficient and accurate cleaning operations.

[0062] Table 3. Time taken to classify images using soft voting ensemble (CPU only)

[0063] Example 3

[0064] Based on the same inventive concept as other embodiments, the present invention provides a device for detecting the cleanliness condition of a hull surface in an underwater environment, comprising: A data acquisition module, used to acquire image data of the hull surface in an underwater environment; A cleaning detection module, used for performing detection based on the image data and a pre-trained soft voting ensemble classifier; A result output module is used to obtain the test results; The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

[0065] The specific functional implementation of each of the above modules can be found in the relevant contents of the method in Example 1 and will not be elaborated here.

[0066] Example 4

[0067] Based on the same inventive concept as other embodiments, Figure 1 The method shown, accordingly, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for detecting the cleaning condition of the hull surface in an underwater environment in any of the above embodiments are implemented.

[0068] Example 5

[0069] Based on the same inventive concept as other embodiments, Figure 1 The method shown in the embodiment of the present invention also provides a physical structure diagram of a computer device, such as Figure 2 As shown, the computer device may include a communication bus, a processor, a memory and a communication interface, and may also include an input / output interface and a display device, wherein each functional unit may communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the steps of the method for detecting the cleanliness of the hull surface in an underwater environment described in the above embodiment.

[0070] In summary, the embodiments of the present invention have significant advantages in the detection of the hull surface, especially in improving the detection accuracy, enhancing the robustness and stability, realizing automated detection, improving the real-time performance, and reducing the operation cost. The application of this technology enables the condition of the hull surface to be identified more accurately, thereby providing more reliable data support for subsequent cleaning and maintenance; integrating convolutional neural network models with different architectures. In practical applications, the underwater environment may encounter various challenges such as uneven illumination and turbid water, and traditional single models are prone to misjudgment under these conditions. Through the integration strategy, not only the adaptability of the model to the complex underwater environment is improved, but also the robustness and stability in the detection process are enhanced; realizing the automated detection of the hull surface condition without manual intervention. After detecting the hull surface condition, the underwater robot can automatically adjust the cleaning strategy according to the results output by the model, such as selecting the area to be cleaned and adjusting the cleaning intensity. This intelligent decision-making process improves the efficiency and quality of the hull cleaning operation, reduces the need for manual divers, and reduces the manual labor intensity and safety risks. This not only optimizes the work process but also brings a higher level of automation to the cleaning operation; the real-time performance of the detection is excellent, and the processing time for single-image classification is 56.25 milliseconds, ensuring that the underwater robot can obtain the information of the hull surface condition in real time during the task execution and make timely responses. Even under the condition of relatively limited computing resources, the system can maintain a high processing speed to meet the requirements of real-time detection, avoiding delaying decisions and affecting the efficiency of hull maintenance work; using automated detection and intelligent decision-making to effectively reduce the operation cost. By reducing the usage frequency and time of manual divers, the labor cost is significantly reduced. At the same time, the improvement of the efficiency and effect of the hull cleaning operation also reduces the additional maintenance costs caused by hull pollution, significantly reducing the operation expenses.

[0071] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for detecting the cleanliness of a hull surface in an underwater environment, characterized in that: include: Acquire image data of the hull surface in an underwater environment; According to the image data, detection is performed based on a soft voting integrated classifier to obtain a detection result; The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

2. The method for detecting the cleanliness of a hull surface in an underwater environment according to claim 1, characterized in that: The method for acquiring image data of the hull surface in an underwater environment includes: In different underwater environments, remotely controlled underwater robots use high-definition cameras to collect video of the hull surface; A frame of image is extracted from the video every 1 second as image data of the hull surface in the underwater environment.

3. The method for detecting the cleanliness of a hull surface in an underwater environment according to claim 2, characterized in that: The training method of the convolutional neural network model includes: Acquire historical image data of the hull surface in an underwater environment over a period of time; Manually annotating the historical images to obtain an annotated image dataset, wherein the annotated contents include two categories: "clean" and "unclean"; The convolutional neural network model is trained using the annotated image dataset, using the Adam optimizer and binary cross entropy loss function, and the parameters of the convolutional neural network model are updated through the back propagation algorithm.

4. The method for detecting the cleanliness of a hull surface in an underwater environment according to claim 3, characterized in that: It also includes freezing most of the network layers of the convolutional neural network model in the early stages of training the convolutional neural network model, and only training the newly added network layers; After the initial stage of training, all network layers are unfrozen, training is continued with a smaller learning rate, and the hyperparameters of the convolutional neural network model are tuned.

5. The method for detecting the cleanliness of a hull surface in an underwater environment according to claim 3, characterized in that: It also includes preprocessing the labeled image data set, including adjusting brightness, contrast and saturation, and cropping.

6. The method for detecting the cleanliness of a ship surface in an underwater environment according to claim 1, characterized in that: The pre-training method of the convolutional neural network model includes adjusting the hyperparameters of each convolutional neural network model using a hyperparameter optimization technique.

7. The method for detecting the cleanliness of a hull surface in an underwater environment according to claim 1, characterized in that: Using transfer learning to fine-tune multiple pre-trained convolutional neural network models includes: Each convolutional neural network model is initialized using the weights pre-trained on the ImageNet dataset, and a fine-tuning process is performed to make each convolutional neural network model more suitable for the underwater hull surface image dataset.

8. A device for detecting the cleanliness of a ship surface in an underwater environment, characterized in that: include: A data acquisition module, used to acquire image data of the hull surface in an underwater environment; A cleaning detection module, used for performing detection based on the image data and a pre-trained soft voting ensemble classifier; A result output module, used to obtain the test results; The method for constructing the soft voting ensemble classifier includes: Use transfer learning to fine-tune multiple pre-trained convolutional neural network models; Then, multiple fine-tuned convolutional neural network models are used to classify the image data of the hull surface in the underwater environment; The output probability values ​​of each fine-tuned convolutional neural network model are averaged to obtain an average probability value; Among them, if the average probability value of all convolutional neural network models exceeds the predetermined threshold, the output detection result is unclean, otherwise the output detection result is clean; The convolutional neural network models include DenseNet, EfficientNet, Inception, MobileNet, ResNet and VGG.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting the cleanliness condition of a hull surface in an underwater environment as described in any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: include: A memory for storing instructions; The processor is used to execute the instructions so that the device performs the operation of implementing the method for detecting the cleanliness condition of the hull surface in an underwater environment as described in any one of claims 1 to 7.

Images