Coral reef detection and classification method, device, equipment, medium and product

Abstract

The invention discloses a coral reef detection and classification method and device, equipment, a medium and a product, and relates to the field of coral reef detection and classification, and the method comprises the steps: obtaining a coral reef image; processing the coral reef image through a detection model to obtain a coral reef detection image; performing color information restoration on the coral reef detection image through a restoration model to obtain a coral reef restored image; segmenting the coral reef restored image to obtain a coral reef segmented image; and classifying the coral reef segmented images through a classification model to obtain a classification result of the coral reef images, and judging a coral reef type according to the classification result of the coral reef images. According to the invention, high-precision detection and classification requirements of coral reef images are realized.

Description

Technical Field

[0001] The present application relates to the field of coral reef detection and classification, and particularly to a method, device, equipment, medium and product for coral reef detection and classification. Background Art

[0002] As one of the oldest and most biodiverse ecosystems on Earth, coral reefs not only provide habitats and shelters for numerous marine organisms but also play a crucial role in coastal protection, fishery resources, and tourism. However, due to the impact of global climate change, especially the increasing seawater temperature and acidification, coral reefs are facing an unprecedented survival crisis. According to research predictions, in the next two decades, the coral reef ecosystem may degrade rapidly, which will have a profound impact on the marine organisms dependent on it and the related human communities.

[0003] To effectively address this challenge, scientists are exploring various methods to monitor and protect coral reefs. However, traditional coral reef detection methods usually rely on divers' on-site inspections or underwater photography video analysis, which is time-consuming and laborious and cannot meet the requirements of high-precision detection and classification of coral reef images. Summary of the Invention

[0004] The purpose of the present application is to provide a method, device, equipment, medium and product for coral reef detection and classification, which can meet the requirements of high-precision detection and classification of coral reef images.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In the first aspect, the present application provides a method for coral reef detection and classification, including:

[0007] Obtain a coral reef image;

[0008] Process the coral reef image through a detection model to obtain a coral reef detection image;

[0009] Restore the color information of the coral reef detection image through a restoration model to obtain a coral reef restoration image;

[0010] Segment the coral reef restoration image to obtain a coral reef segmentation image;

[0011] Classify the coral reef segmentation image through a classification model to obtain the classification result of the coral reef image, and judge the coral reef type according to the classification result of the coral reef image.

[0012] In the second aspect, the present application provides a device for coral reef detection and classification, including:

[0013] An acquisition module for acquiring coral reef images;

[0014] A detection module for processing the coral reef image through a detection model to obtain a coral reef detection image;

[0015] A restoration module for restoring color information of the coral reef detection image through a restoration model to obtain a coral reef restored image;

[0016] A segmentation module for segmenting the coral reef restored image to obtain a coral reef segmented image;

[0017] A classification module for classifying the coral reef segmented image through a classification model to obtain a classification result of the coral reef image, and judging the type of coral reef according to the classification result of the coral reef image.

[0018] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the steps of the coral reef detection and classification method described in any one of the above.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the coral reef detection and classification method described in any one of the above are implemented.

[0020] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the coral reef detection and classification method described in any one of the above are implemented.

[0021] According to the specific embodiments provided by the present application, the following technical effects are disclosed in the present application:

[0022] The present application provides a coral reef detection and classification method, device, equipment, medium and product. The method includes acquiring a coral reef image; processing the coral reef image through a detection model to obtain a coral reef detection image; restoring color information of the coral reef detection image through a restoration model to obtain a coral reef restored image; segmenting the coral reef restored image to obtain a coral reef segmented image; classifying the coral reef segmented image through a classification model to obtain a classification result of the coral reef image, and judging the type of coral reef according to the classification result of the coral reef image. By integrating the detection model, the restoration model and the classification model together, the present application realizes the detection and classification requirements of coral reef images with relatively high accuracy. Description of the Drawings

[0023] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0024] Figure 1 It is an application environment diagram of a coral reef detection and classification method in an embodiment of the present application;

[0025] Figure 2 It is a schematic flowchart of a coral reef detection and classification method provided in an embodiment of the present application;

[0026] Figure 3 It is a schematic structural diagram of a detection model provided in an embodiment of the present application;

[0027] Figure 4 It is a schematic structural diagram of a classification model provided in an embodiment of the present application;

[0028] Figure 5 It is a schematic diagram of the test results of a classification model provided in an embodiment of the present application.

[0029] Figure 6 It is a schematic diagram of the functional modules of a coral reef detection and classification device provided in an embodiment of the present application;

[0030] Figure 7 It is a schematic structural diagram of a computer device provided in an embodiment of the present application. Detailed implementation manners

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0032] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the drawings and specific implementation manners.

[0033] The coral reef detection and classification method provided in the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set separately, integrated on the server 104, placed on the cloud or other servers. The terminal 102 can send the coral reef image to be processed to the server 104. After receiving the coral reef image to be processed, for the coral reef image to be processed, the server 104 processes the coral reef image through a detection model to obtain a coral reef detection image, restores the color information of the coral reef detection image through a restoration model to obtain a coral reef restoration image; segments the coral reef restoration image to obtain a coral reef segmentation image; classifies the coral reef segmentation image through a classification model to obtain a classification result of the coral reef image. The server 104 can feedback the obtained classification result of the coral reef image to the terminal 102. In addition, in some embodiments, the coral reef detection and classification method can also be implemented separately by the server 104 or the terminal 102. For example, it can be directly processed by the terminal 102 for the coral reef image to be processed, or the server 104 can obtain the coral reef image to be processed from the data storage system and process the coral reef image to be processed.

[0034] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0035] In an exemplary embodiment, as Figure 2 shown, a coral reef detection and classification method is provided. This method is executed by a computer device, and can be specifically executed alone by a computer device such as a terminal or a server, or jointly executed by a terminal and a server. In the embodiments of the present application, taking this method applied to Figure 1 the server 104 in it as an example for illustration, it includes the following steps S201 to step S205. Among them:

[0036] In step S201, obtain a coral reef image.

[0037] Specifically, a coral reef image can be obtained through an online database. For example, through platforms such as Google Earth Engine, which contains a large number of coral reef images. Or, directly take photos in the coral reef environment through underwater photography equipment. Or, for the coral reefs in shallow waters, drones can be used to take coral reef images.

[0038] In step S202, the coral reef image is processed by the detection model to obtain a coral reef detection image.

[0039] Specifically, the detection model includes a backbone network, a neck network, and a head network. The backbone network is used to extract features from the coral reef image and output feature maps of multiple different scales; the neck network is used to fuse the feature maps of multiple different scales to obtain multiple fused maps; the head network makes predictions on the fused maps according to the loss function to obtain the detection results.

[0040] It should be noted that the backbone is one of the core components of the detection model. Its main task is to extract discriminative features from the input coral reef image. These features can help identify and classify different elements in the image, such as different coral species or health conditions. The backbone usually consists of multiple convolutional layers, which gradually reduce the spatial dimensions (width and height) of the image while increasing the number of channels (depth) to capture higher-level abstract features. To adapt to the different sizes of the targets that may appear in the image, the backbone is designed to output feature maps of multiple different scales. For example, the shallow feature maps retain more detailed information, while the deep feature maps contain more extensive context information. This multi-scale characteristic is very important for detecting targets of different sizes, such as small coral individuals and large coral communities.

[0041] The neck network is mainly used to effectively fuse the multi-scale feature maps from the backbone network to generate a series of enhanced feature maps, namely the so-called "fused maps". This process aims to improve the quality of the features and ensure that the features at each scale can be fully utilized. After being processed by the neck network, multiple fused maps can be obtained, each of which represents the feature representation at a different scale, which helps to improve the detection accuracy of various size targets.

[0042] The head network is mainly used to make the final prediction based on the fused maps. The head network makes predictions on the fused maps according to the loss function to obtain the detection results.

[0043] In one embodiment, before processing the coral reef image by the detection model, it is necessary to preprocess the coral reef image by the Mosaic data augmentation method to increase the data diversity and help improve the robustness of the model under different scenarios and conditions. Among them, the preprocessing of the coral reef image by the Mosaic data augmentation method includes the following sub-steps:

[0044] A1. Randomly select four pictures from the coral reef images containing annotation information.

[0045] Coral reef images should cover coral reef scenes under various conditions, including different lighting conditions, changes in underwater visibility, and different distributions of coral species and marine organisms. Randomly select four pictures from the coral reef image dataset, ensuring that each picture has its own characteristics, which can increase the complexity of the final composite image and help improve the generalization ability of the model.

[0046] For example, four pictures were randomly selected from a dataset of over 3,000 labeled coral reef images, labeled as Img_1, Img_2, Img_3, and Img_4 respectively. Each picture contains corals of different colors, shapes, and textures, as well as various marine organisms and seabed terrains.

[0047] Annotation information refers to the marking of specific objects or regions in the image. For coral reef images, annotation information may include but is not limited to classification labels, bounding boxes, timestamp information, and location information. Each annotated coral reef may have one or more classification labels, such as different coral species (e.g., staghorn coral, brain coral, etc.), abiotic structures (e.g., rocks), or other marine organisms (e.g., fish, seaweed, etc.). A bounding box is a rectangular area around the object. Timestamp information contains the shooting time of the coral reef image, and location information contains the geographical location where the coral reef image was taken. Timestamp information and location information are used for long-term monitoring of the health of coral reefs.

[0048] A2. Randomly scale the four pictures to different sizes.

[0049] To simulate the observation effects at different distances and perspectives, randomly scale each picture to a different size while keeping the aspect ratio of the image unchanged to avoid distorting the image content.

[0050] For example, scale Img_1 to 800×600 pixels, Img_2 to 1024×768 pixels, Img_3 to 1280×960 pixels, and Img_4 to 640×480 pixels.

[0051] A3. Randomly crop the four scaled pictures.

[0052] To further enhance the diversity of the coral reef image dataset, a cropping window can be defined. This window can randomly move within the coral reef picture, but it must ensure that at least a part of the original picture is covered. Each cropped area will become a part of the final stitched image. Since even the same picture may produce completely different cropping results, step A3 also further increases the variability of the data.

[0053] For example:

[0054] Img_1: Crop from the upper left corner (100, 150) to the lower right corner (700, 550).

[0055] Img_2: Crop from the upper left corner (200, 250) to the lower right corner (824, 518).

[0056] Img_3: Crop from the upper left corner (300, 350) to the lower right corner (980, 610).

[0057] Img_4: Crop from the upper left corner (50, 70) to the lower right corner (590, 410).

[0058] A4. Piece together the four processed coral reef pictures according to certain rules to form a new composite image.

[0059] Normally, these pictures will be arranged in a grid form (such as 2×2), and the boundaries between them may be randomly distributed and can even overlap in some places. The piecing rules can also be designed according to actual needs, such as horizontal piecing, vertical piecing or more complex mosaic methods.

[0060] For example, the method of horizontal piecing in pairs first and then vertical combination can be selected, that is, first piece Img_1 and Img_2 horizontally, then do the same for Img_3 and Img_4, and finally piece the two results vertically together to form a new composite image.

[0061] A5. Adjust the annotation information of the four processed coral reef pictures.

[0062] Since the original pictures are modified, the annotation information of each processed coral reef picture needs to be adjusted accordingly, such as classification labels, position information and bounding box coordinates. Ensure that all objects in the newly generated composite image can correctly reflect their positions and categories.

[0063] Since the original pictures are randomly scaled and randomly cropped, the original annotation positions are no longer accurate. Therefore, new bounding box coordinates need to be calculated according to the transformation.

[0064] For example, assume there is a piece of coral in Img_1, and its original bounding box is (200, 250, 300, 350). After it is scaled and cropped, the new bounding box position needs to be recalculated. If the cropped area starts from (100, 150), the new bounding box may become (100, 100, 200, 200), and the specific values depend on the actual scaling ratio and cropping range.

[0065] Through the above steps A1 - A5, a pre - processed new coral reef image is obtained. The new composite image contains different parts from four original images, and the annotation information has been updated. Optionally, the new composite image can be input into the backbone network of the deep - learning model to extract features and perform subsequent object - detection tasks. Through the above steps A1 - A5, not only is the training dataset enriched, but also the adaptability of the detection model to different scene changes is enhanced, thereby improving the performance and robustness of the model in practical applications. Due to the diverse distribution of objects and background changes in coral reefs, the above - mentioned method is applicable to complex natural environments such as coral reefs.

[0066] Specifically, "processing the coral reef image through the detection model to obtain the coral reef detection image" in step S202 includes the following sub - steps S2021 - S2023:

[0067] S2021. Extract scales of the coral reef image through the backbone network to obtain coral reef feature maps of different scales.

[0068] Specifically, the backbone network includes a first extraction module, a second extraction module, and a third extraction module. Step S2021 includes the following sub - steps S20211 - S20213:

[0069] S20211. Input the coral reef image into the first extraction module and output the first - layer feature map.

[0070] S20212. Input the first - layer feature map into the second extraction module and output the second - layer feature map.

[0071] S20213. Input the second - layer feature map into the third extraction module and output the third - layer feature map.

[0072] Specifically, the first extraction module includes a first convolutional layer, a second convolutional layer, a first residual layer, a third convolutional layer, and a second residual layer arranged in sequence. The first - layer feature map f1 is output through the second residual layer.

[0073] The second extraction module includes a fourth convolutional layer and a third residual layer arranged in sequence. The second - layer feature map f2 is output through the third residual layer.

[0074] The third extraction module includes a fifth convolutional layer, a fourth residual layer, a polarization self - attention layer, and a pooling layer arranged in sequence. The third - layer feature map f3 is output through the pooling layer.

[0075] It should be noted that through the preliminary feature extraction of the coral reef image in sub-step S20211, selecting the first extraction module is similar to looking at the coral reef picture with a microscope at a lower magnification. The first extraction module includes a series of convolutional layers and residual layers: The first convolutional layer is equivalent to applying a filter to the picture, which can identify basic visual elements such as picture edges and picture color boundaries. The second convolutional layer is to further refine the image edge features and image color boundary features. For example, it can identify shapes composed of multiple edges. The first residual layer is similar to maintaining the focus under the microscope, aiming to ensure that important features of the image will not be lost due to the deeper network. After the convolutional processing of the first two layers, the basic features of the image, such as edges and color boundaries, have been preliminarily extracted. At this time, through the third convolutional layer, more complex and abstract shape features are continuously identified. The first feature map finally output through the second residual layer contains obvious structural information, such as large pieces of coral or rocks. Through this step, the coral reef structure within a large range can be identified from the macroscopic perspective of the entire coral reef.

[0076] For example, suppose there is a photo of an underwater landscape containing various corals. After passing through the first extraction module, an image highlighting the general outline of the coral reef, the distribution of rocks, and large marine organisms may be obtained. This step may also highlight some obvious color changes or texture differences. If the color of the coral reef in certain areas is significantly different from other places, it may indicate the presence of diseased corals or excessive algae growth.

[0077] Through the intermediate feature extraction of the coral reef image in sub-step S20212, selecting the second extraction module is similar to increasing the magnification of the microscope. The second extraction module also includes a series of convolutional layers and residual layers. Through the fourth convolutional layer, the features in the first feature map are deeply analyzed to identify specific types of coral shapes. The third residual layer ensures that important features are not lost, and finally, the second feature map is output. Some medium-scale features, such as certain coral colony features, are shown through the second feature map. Based on the above steps, different coral species and their relative positions can be distinguished more clearly, and subtle changes in specific types of corals or their surfaces may also be discovered. These changes may be signs of bleaching or other diseases.

[0078] Advanced feature extraction of the coral reef image is performed through sub-step S20213. The third extraction module is similar to further increasing the magnification of the microscope, aiming to capture very fine features of the coral reef image. The third extraction module not only has convolutional layers and residual layers, but also includes a polarization self-attention layer and a pooling layer. The features of the coral reef image are further abstracted through the fifth convolutional layer. The stability and integrity of the coral reef image features are maintained through the fourth residual layer. The coral reef image is enhanced through the polarization self-attention layer, highlighting the important features of the coral reef image and suppressing noise, and improving the detection accuracy under the condition of ensuring the detection speed. The coral reef image is downsampled through the pooling layer, reducing the spatial size of the coral reef image, reducing the computational complexity, and retaining the most important coral reef image feature information.

[0079] For example, through sub-step S20213, small-area anomalies occurring on a certain rare coral species can be located, which helps to take timely protection measures.

[0080] S2022. The coral reef feature maps of different scales are fused through the neck network to obtain a coral reef fusion map.

[0081] In one embodiment, the neck network includes a first fusion module, a second fusion module, a third fusion module, and a fourth fusion module;

[0082] Fusing the coral reef feature maps of different scales through the neck network to obtain a coral reef fusion map includes the following sub-steps S20221 - S20224:

[0083] S20221. The first fusion module includes a sixth convolutional layer, a first upsampling layer, a first fusion layer, a fifth residual layer, and a seventh convolutional layer arranged in sequence. After the third-layer feature map passes through the first upsampling layer, it is concatenated with the second-layer feature map through the first fusion layer, and then passes through the fifth residual layer and the seventh convolutional layer to obtain the fourth-layer feature map.

[0084] Specifically, the sixth convolutional layer performs preliminary feature extraction or transformation on the third feature map. The sixth convolutional layer uses a smaller convolutional kernel (such as 3×3) to keep the spatial size unchanged. In order to make the third-layer feature map match the second-layer feature map, in the first upsampling layer, the spatial size of the third-layer feature map is enlarged through an interpolation method, that is, the width and height of the third feature map are increased. When the third-layer feature map after the first upsampling layer matches the original second-layer feature map in size, the third feature map and the second feature map can be directly concatenated in the same dimension. Through the fifth residual layer, when the number of network layers increases, the problem of key features being diluted or lost as the network depth increases is avoided. The fused feature map is further processed through the seventh convolutional layer to refine the feature expression and prepare a suitable output format for the subsequent step S20222.

[0085] For example, for a photo of a coral reef on the sea floor, it contains various coral structures of different sizes. Currently, different feature maps from the shallow layer to the deep layer have been obtained. The third-layer feature map may capture the coral community patterns in a larger range, while the second-layer feature map records more local details, such as the colors and textures of individual polyps or small pieces of coral. First, the third-layer feature map passes through the sixth convolutional layer, and its feature representation may be adjusted to better fuse with other layers. Then, the third-layer feature map is upsampled to the same resolution as the second-layer feature map. This means that the features that originally covered a larger area but had less detail can now correspond to the finer local features. These two feature maps are concatenated together in the first fusion layer to obtain a new feature map that contains information about both the macroscopic structure (such as the layout of the coral community) and the microscopic details (such as the morphology of the polyps). Through the fifth residual layer, the newly generated feature map is further optimized, ensuring that the key features are not lost and helping to converge faster during the training process. After passing through the seventh convolutional layer, the fourth-layer feature map f4 is obtained. The fourth-layer feature map f4 synthesizes high-quality feature representations of multi-level information and can be used not only to identify coral species but also to detect health conditions or discover anomalies, such as bleaching phenomena.

[0086] S20222. The second fusion module includes a second upsampling layer, a second fusion layer, and a sixth residual layer arranged in sequence. After the fourth-layer feature map passes through the second upsampling layer, it is concatenated with the first-layer feature map through the second fusion layer, and then passes through the sixth residual layer to obtain the fifth-layer feature map.

[0087] Specifically, the fourth-layer feature map f4 is generated by the first fusion module and combines the information of the third-layer feature map and the second-layer feature map. The fourth-layer feature map is first enlarged through the second upsampling layer, increasing the spatial size of the fourth feature map to match that of the first-layer feature map so that the fourth-layer feature map and the first-layer feature map can be concatenated at the same scale. The upsampled fourth-layer feature map and the first-layer feature map are concatenated in the second fusion layer, combining the fine-grained local features with the large-range global background information to form a comprehensive new feature map. The concatenated feature map is processed through the sixth residual layer, and the role of the sixth residual layer is to maintain and strengthen the feature information to ensure that these feature information are not lost due to the deepening of the network, and finally the fifth-layer feature map f5 is obtained.

[0088] For example, for a high-resolution image covering a large area of coral reefs, it contains various coral species and their health conditions. The fourth-layer feature map has been obtained from the previous steps, which combines relatively detailed local structures and some larger patterns. However, the spatial resolution of these feature maps is still relatively low. Through the second upsampling layer, the fourth-layer feature map is enlarged to the same spatial size as the first-layer feature map, locating the specific coral positions and retaining the important details extracted previously. The upsampled fourth-layer feature map is concatenated with the first-layer feature map. The first-layer feature map contains overview information of the entire image, such as the layout of the entire coral reef, the underwater terrain, and the presence of large marine organisms, enabling the fifth-layer feature map to not only show the large-scale coral distribution patterns but also point out the health problems or anomalies of the coral reefs at certain specific positions. After being processed by the sixth residual layer, key feature information can be better retained, such as the health status of polyps in a specific area or the exact location of a certain rare coral. The specific position of a certain coral community and its surrounding environment can be seen, while noting the possible bleaching or other health problems of some of the coral individuals among them.

[0089] S20223. The third fusion module includes an eighth convolutional layer, a third fusion layer, a seventh residual layer, and a ninth convolutional layer arranged in sequence. After passing through the eighth convolutional layer, the fifth-layer feature map is concatenated with the fourth-layer feature map through the third fusion layer, and then passes through the seventh residual layer and the ninth convolutional layer to obtain the sixth-layer feature map.

[0090] Specifically, the fifth-layer feature map f5 is first processed by the eighth convolutional layer, whose role is to further extract and strengthen important features, such as specific coral morphologies, color changes, or anomalies. The processed fifth-layer feature map f5 is then concatenated with the non-upsampled fourth-layer feature map in the third fusion layer. The concatenated feature map is processed by the seventh residual layer, which helps to retain and enhance key features while reducing the risk of information loss caused by the increase in network depth. The feature map processed by the seventh residual layer passes through the ninth convolutional layer again to further optimize the features, and finally the sixth-layer feature map f6 is obtained.

[0091] S20224. The fourth fusion module includes a fourth fusion layer and an eighth residual layer arranged in sequence. The sixth-layer feature map and the third-layer feature map are concatenated through the fourth fusion layer, and then pass through the eighth residual layer to obtain the seventh-layer feature map.

[0092] Specifically, the sixth-layer feature map f6 contains the feature representation after multi-scale integration, while the third-layer feature map retains more original and detailed local features. In the fourth fusion layer, the third-layer feature map and the sixth-layer feature map are concatenated to combine high-level abstract features with low-level specific details, ensuring that no important local information is lost. The concatenated feature map is processed through the eighth residual layer to maintain key features without loss and reduce the information attenuation problem that may be caused by the deep network, finally obtaining the seventh-layer feature map f7.

[0093] S2023. Feature prediction is performed on the coral reef fusion map through the head network to obtain the coral reef detection image.

[0094] Specifically, the head network includes a two-dimensional convolutional layer. After passing the fifth-layer feature map, the sixth-layer feature map, and the seventh-layer feature map through the two-dimensional convolutional layer respectively, the coral reef detection image is output. The two-dimensional convolutional layer in the head network processes the above-mentioned fifth-layer feature map, sixth-layer feature map, and seventh-layer feature map to identify and mark the key features in the coral reef, such as specific types of corals, potential disease points, or other important features. The final coral reef detection image may be annotated with different colors or bounding boxes to mark various discoveries, helping with localization and analysis. For example, the location of bleached corals is highlighted with a red box, healthy coral communities are identified with a green box, and suspected diseased areas are circled with a yellow box, etc. In this way, through the prediction process, the head network can effectively extract information from feature maps at different scales and generate a comprehensive coral reef detection image.

[0095] In some embodiments, the prediction result of the head network is optimized through a loss function to improve the regression accuracy of the bounding box in the coral reef detection image.

[0096] The loss function L Focal-EIOU is expressed as:

[0097] L Focal-EIOU = IOU γ L EIOU ;

[0098] where L EIOU represents the basic EIOU loss function, which is used to measure the similarity between the predicted box and the ground truth box; IOU γ represents an adjustment factor, and IOU represents the intersection over union ratio of the predicted box and the ground truth box; γ represents a hyperparameter used to control the intensity of the adjustment. The ground truth box refers to the bounding box marked in the training data, which indicates the position and size of the actual target in the image. The predicted box is the bounding box generated by the object detection model based on the input image, representing the model's estimation of the target position and size.

[0099] When processing coral reef images, due to the significant differences in target sizes (such as very small newly born coral polyps or extremely large coral communities) and the problem of unbalanced class distribution (the proportion of rare species or diseased corals is very low), it may be difficult for the model to accurately identify key features. To address these issues, an adjusted loss function (such as the Focal-EIoU loss function) can be introduced, which adjusts the parameters to assign smaller weights to samples with high IOU values and larger weights to samples with low IOU values, enabling the model to focus more on difficult-to-classify samples and outliers. This not only enhances the model's detection ability for targets of different scales and shapes but also improves its robustness and accuracy in the case of unbalanced dataset distribution. In the coral reef detection task, even when faced with extremely rare but crucial rare species or diseased corals, the adjusted loss function can guide the model to improve the detection accuracy of these minority classes, ensuring that even a small number of important samples will not be overlooked.

[0100] L EIOU It is represented by the following formula:

[0101]

[0102]

[0103] where L IOU represents the Intersection over Union (IoU) loss function, L dis represents the center point distance loss, L asp represents the orientation loss, w c represents the width of the smallest bounding box containing the target box and the predicted box, B represents the predicted box, B gt represents the target box, h c represents the height of the smallest bounding box containing the target box and the predicted box, b represents the center point coordinates of the predicted box B, b gt represents the center point coordinates of the target box Bg t of, represents the Euclidean distance, w represents the width of the smallest bounding box of the predicted box B, w gt represents the width of the smallest bounding box of the target box B gt of, h represents the height of the smallest bounding box of the predicted box B, h gt represents the height of the smallest bounding box of the target box B gt of. L EIOU minimizes the differences in width and height between the target box and the predicted box, thus accelerating the convergence speed of model training and improving the localization accuracy for different scales, especially extremely small or large targets (such as rare species or diseased corals).

[0104] It should be noted that in the above content, the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer, the eighth convolutional layer, the ninth convolutional layer, and the tenth convolutional layer are all convolutional layers. The first residual layer, the second residual layer, the third residual layer, the fourth residual layer, the fifth residual layer, the sixth residual layer, the seventh residual layer, and the eighth residual layer are all residual layers. The first fusion layer, the second fusion layer, the third fusion layer, and the fourth fusion layer are all fusion layers. The first upsampling layer and the second upsampling layer are both upsampling layers.

[0105] As Figure 3 described, Figure 3 shown in the structural schematic diagram of the detection model, the backbone network includes a first extraction module, a second extraction module, and a third extraction module. The input of the first extraction module is a coral reef image, and the output of the first extraction module is the first-layer feature map f1; the input of the second extraction module is the first-layer feature map f1, and the output of the second extraction module is the second-layer feature map f2; the input of the third extraction module is the second-layer feature map f2, and the output of the third extraction module is the third-layer feature map f3. The first extraction module includes a Conv layer (convolutional layer), a Conv layer (convolutional layer), a C3Res2 layer (residual layer), a Conv layer (convolutional layer), and a C3Res2 layer (residual layer) arranged in sequence. The coral reef image is input through the first-set Conv layer, and the first-layer feature map f1 is output by the last-set C3Res2 layer (residual layer). The second extraction module includes a Conv layer and a C3Res2 layer (residual layer) arranged in sequence to output the second-layer feature map f2. The third extraction module includes a Conv layer (convolutional layer), a C3Res2 layer (residual layer), a PSA layer (polarization self-attention layer), and an SPPF layer (pooling layer) arranged in sequence. The SPPF layer (pooling layer) outputs the third-layer feature map f3.

[0106] The neck network includes a first fusion module, a second fusion module, a third fusion module, and a fourth fusion module. The first fusion module includes a Conv layer (convolution layer), an Upsample layer (upsampling layer), a Concat layer (fusion layer), a C3Res2 layer (residual layer), and a Conv layer (convolution layer) arranged in sequence. After the third-layer feature map f3 passes through the Upsample layer (upsampling layer), it is concatenated with the second-layer feature map f2 through the Concat layer, and then passes through the C3Res2 layer (residual layer) and the last Conv layer (convolution layer) to obtain the fourth-layer feature map f4. The second fusion module includes an Upsample layer (upsampling layer), a Concat layer, and a C3Res2 layer arranged in sequence. After the fourth-layer feature map f4 passes through the Upsample layer (upsampling layer), it is concatenated with the first-layer feature map f1 through the Concat layer (fusion layer), and then passes through the C3Res2 layer to obtain the fifth-layer feature map f5. The third fusion module includes a Conv layer (convolution layer), a Concat layer (fusion layer), a C3Res2 layer (residual layer), and a Conv layer (convolution layer) arranged in sequence. After the fifth-layer feature map f5 passes through the Conv layer (convolution layer), it is concatenated with the fourth-layer feature map f4 through the Concat layer (fusion layer), and then passes through the C3Res2 layer and the Conv layer (convolution layer) to obtain the sixth-layer feature map f6. The fourth fusion module includes a Concat layer (fusion layer) and a C3Res2 layer (residual layer) arranged in sequence. The sixth-layer feature map f6 and the third-layer feature map f3 are concatenated through the Concat layer (fusion layer), and then pass through the C3Res2 layer (residual layer) to obtain the seventh-layer feature map f7.

[0107] The head network includes a two-dimensional convolution layer (Conv2d layer). After the fifth-layer feature map f5, the sixth-layer feature map f6, and the seventh-layer feature map f7 pass through the Conv2d layer (two-dimensional convolution layer) of the head network respectively, detection results of different scales are output.

[0108] The backbone network, neck network, and head network of the detection model are used to process the coral reef image, fusing multi-scale features to ensure that information at different scales is fully utilized. The loss function combines the IOU loss, distance loss, and orientation loss to ensure the accuracy of the prediction box is evaluated from multiple angles. Through efficient loss calculation and gradient update, it is ensured that the model can converge quickly to meet the requirements of real-time detection. The regression accuracy of the coral reef bounding box is improved by the loss function. The problem of extremely unbalanced dataset distribution is improved. Through multi-scale feature fusion and loss function L Focal-EIOU , not only the detection ability of small targets is improved, but also the problem of large-size targets being interfered by the background is reduced, and the requirements of real-time detection are met. In addition, through IOU γIn this form, it effectively solves the problem of extremely unbalanced dataset distribution and further improves the overall performance of the model.

[0109] In addition, the model also meets the requirements of real-time detection and improves the problem of low accuracy caused by attenuation in actual underwater photos. Specifically, the seventh-layer feature map integrates multi-scale information. Especially after being processed by the fourth fusion module, the ability to capture fine structures is enhanced. This means that even very small targets (such as newly born corals or small marine organisms) can be accurately identified. Through multi-layer feature fusion, the model can better understand the context relationship, thereby reducing the risk of misclassifying non-target objects as targets. The detection model not only focuses on the target itself but also takes into account the surrounding environmental factors, enabling it to more accurately distinguish the target and the background when facing large coral reefs and avoiding interference caused by background complexity.

[0110] In step S203, the color information of the coral reef detection image is restored through the restoration model to obtain the coral reef restored image.

[0111] In one embodiment, the restoration model includes the range-Doppler algorithm or the chirp scaling algorithm, etc. This can improve the saturation, contrast, and sharpness of the coral reef image; it is beneficial to improve the accuracy of the classification model for coral reef classification.

[0112] Specifically, in the coral reef detection task, the color information of the image is crucial for accurate identification and classification. Due to the complexity of the underwater environment (such as light scattering, absorption, etc.), the originally captured coral reef images often have problems such as color distortion and low contrast. To improve the accuracy of the subsequent classification model, the color information of the coral reef detection image can be restored through the restoration model to obtain a higher-quality coral reef restored image.

[0113] Exemplarily, in the coral reef image, the color offset caused by water depth changes is restored through the range-Doppler algorithm, making the colors of each region in the image closer to those in the real world. It is also possible to choose that under the action of the chirp scaling algorithm, the saturation, contrast, and sharpness of the coral reef image will be significantly improved, thereby improving the visual effect and feature extraction ability.

[0114] In step S204, the coral reef restored image is segmented to obtain the coral reef segmented image.

[0115] Specifically, the goal of image segmentation of the coral reef restored image is to divide the image into multiple regions, and each region represents a specific object or background. For coral reef images, segmentation can be used to identify different coral species, distinguish between living corals and dead corals, and mark other biological or non-biological structures (such as sand, rocks, etc.) on the coral reef.

[0116] Exemplarily, gradient information can be utilized to find the object boundary and further perform image segmentation on the restored coral reef image.

[0117] In step S205, the classified model classifies the segmented coral reef image to obtain the classification result of the coral reef image, and determines the type of the coral reef according to the classification result of the coral reef image.

[0118] Specifically, as Figure 4 shown, the classified model includes a tenth convolutional layer (Conv layer), a max pooling layer (Maxpool layer), a first convolutional attention layer (CBAM layer), a stacked residual module (ResNeSt Block module), a second convolutional attention layer (CBAM layer), an average pooling layer (avgpool layer), and a fully connected layer (FC layer) which are sequentially arranged; after the segmented coral reef image passes through the tenth convolutional layer, the max pooling layer, the first convolutional attention layer, the stacked residual module, the second convolutional attention layer, the average pooling layer, and the fully connected layer in sequence, the classification result of the coral reef image is output. The convolutional attention layer improves the feature extraction ability of the model.

[0119] The tenth convolutional layer detects local features in the image, such as edges, textures, etc. The tenth convolutional layer means that a convolutional layer at a deeper position in the pre-trained deep network is used at this stage, which can usually capture higher-level semantic information, such as the shape and structure of the coral reef. The max pooling layer can reduce the data volume and computational complexity while retaining the most important features. The max pooling layer shrinks the image size by taking the maximum value of each small area, which helps to enhance the translational invariance of the model. The first convolutional attention layer allows the model to adaptively focus on different parts of the input, thereby better handling the object recognition problem in complex backgrounds. The convolutional attention layer emphasizes key features and suppresses irrelevant information by calculating the weight matrix, further improving the quality of feature representation. By stacking multiple residual modules, the network structure can be deepened without significantly increasing the computational burden, and the model performance can be improved. The second convolutional attention layer strengthens the deep features to ensure that the final output can focus on reflecting the most important visual elements. The average pooling layer is used for further dimensionality reduction, and the average value within each small block is calculated as a representative. The fully connected layer connects all the neurons in the previous layer and is used to generate the final classification prediction. The activation function (such as Softmax) after the fully connected layer gives the probability distribution of each class, indicating which class the input image is most likely to belong to.

[0120] In one embodiment, through a detection model and a classification model, coral reefs on an image can be detected and classified. The detection model and the classification model are trained, and the trained detection model and classification model are used to detect and classify coral reef images and classification data sets respectively, so as to further improve the accuracy of detection and classification.

[0121] Exemplarily, when training the detection model, real underwater coral reef photos taken and coral reef videos recorded during a day are used as the detection data set. The shooting device is a Canon EOS 80D, the photo resolution is 6000×4000, the time is from 14:00 to 9:00 the next day, and there is a corresponding photo at each whole hour. The coral reef video resolution is 1920×1080, the sampling interval is 1 hour, the sampling time is 6 minutes, the video is frame-extracted, and one frame is extracted every 50 frames, which together with all the photos form the detection data set. The positions of the coral reefs are marked on each picture with the LabelImg program (target detection annotation tool) and divided into a training set and a validation set according to a ratio of 7:3. The YOLOv5 framework is selected, and transformations such as random cropping, flipping, and color jittering are applied to expand the training set to increase the robustness of the model. The key parameters such as the learning rate and batch size are optimized through grid search. The finally selected learning rate is 0.01, and the batch size is 16.

[0122] The intersection over union (IoU) loss combined with the categorical cross-entropy loss is used to guide the model to learn correct bounding box predictions. The mean average precision (mAP) is calculated regularly, and the model performance is evaluated on the validation set to prevent overfitting.

[0123] After the training is completed, the training results of the detection model are shown in Table 1. The mAP of the detection model on the validation set reaches 98.3%, and the frame rate reaches 200, indicating that the model has good generalization ability.

[0124] Table 1: Training Results of the Detection Model

[0125] mAP (%) <![CDATA[Frame rate (frames·s -1 )]]> 98.3 200

[0126] When training the classification model, the collected coral reef picture library is used as the classification training set. This data set contains 22 families, and each family represents a class. In the classification model, the classification data set can be enhanced first to expand the classification data set, thereby enhancing the generalization ability of the model. And it is divided into a training set, a validation set and a test set according to 7:1:2. ResNet50 is selected as the basic network structure, which includes the tenth convolutional layer, the max pooling layer, the first convolutional attention layer, the stacked residual modules, the second convolutional attention layer, the average pooling layer and the fully connected layer. This architecture can make full use of the powerful feature extraction ability of deep learning, and at the same time improve the attention to important features through the attention mechanism. After training, the training results of the classification model are shown in Table 2. The training accuracy of the classification model on the test set reaches 99.8%, and the test accuracy reaches 88.9%, showing strong generalization ability and classification accuracy.

[0127] Table 2: Training results of the classification model

[0128] Training Accuracy (%) Testing Accuracy (%) 99.8 88.9

[0129] After the detection model and the classification model are trained, the corresponding trained weight files can be loaded respectively to detect and classify real underwater coral reefs. By integrating the workflows of the two models, the system can effectively identify the positions of coral reefs in the image and further determine their specific types. And the detection results and classification results are output.

[0130] The test results of the classification model test set are as Figure 5 shown. The classification model test set is tested through the coral reef prediction confusion matrix. The performance of the coral reef species classification model is evaluated through the confusion matrix, which shows the matching situation between the predicted values and the actual values of the model. Among them, the horizontal axis represents the coral reef species predicted by the model, and the vertical axis represents the actual coral reef species.

[0131] Figure 5 Both the horizontal axis and the vertical axis of contain the following coral species:

[0132] Acroporidae: Stony coral family, a common hard coral;

[0133] Actinernidae: Sea anemone order, including various sea anemones;

[0134] Agariciidae: Mushroom coral family, a hard coral;

[0135] Aleyoniidae: Soft coral family;

[0136] Antipathidae: Black coral family, a deep-sea coral;

[0137] Astrocoeniidae: a family of hard corals, the star corals;

[0138] Coscinaraecidae: a family of hard corals;

[0139] Dendrophylliidae: a family of hard corals, the tree corals;

[0140] Euphyllidae: a family of hard corals, the large polyp stony corals;

[0141] Faviidae: a family of hard corals, the brain corals;

[0142] Fungiidae: a family of hard corals, the mushroom corals;

[0143] Halcuriidae: a family of green algae;

[0144] Merulinidae: a family of hard corals;

[0145] Massidae: a family of soft corals;

[0146] Nephtheidae: a family of soft corals, the sea fans;

[0147] Nidaliidae: a family of soft corals;

[0148] Octilinidae: the subclass of octocorals;

[0149] Paracylomiidae: a family of hard corals;

[0150] Pectiniidae: a family of hard corals;

[0151] Pocilloporidae: a family of hard corals, the cauliflower corals;

[0152] Poritidae: a family of hard corals, the pore corals;

[0153] Xeniidae: a family of soft corals, the whips;

[0154] The number in each cell represents the number of samples for the corresponding row (true value) and column (predicted value). If a cell is on the diagonal (i.e., the row and column correspond to the same species), the number represents the number of correctly classified samples. If a cell is not on the diagonal (i.e., the row and column correspond to different species), the number represents the number of misclassified samples. Figure 5 The color bar on the right represents the magnitude of the values, from black (high value) to white (low value). Black indicates a high value, meaning that the prediction and the true value for that category are highly consistent. White indicates a low value, meaning that few samples are correctly classified.

[0155] For example, in the cell corresponding to Acroporidae, the diagonal number is 0.700, indicating that 70% of Acroporidae are correctly classified; in the cell corresponding to Actinernidae, the diagonal number is 1.00, indicating that 100% are correctly classified; in the cell corresponding to Agariciidae, the diagonal number is 0.750, indicating that 75% of Agariciidae are correctly classified. By observing this matrix, it can be evaluated that the model has a high precision in the detection and classification of coral reefs.

[0156] The present invention integrates three algorithms for the detection of coral reefs, underwater image restoration, and classification, and finally realizes a relatively accurate classification of coral reefs; the detection model improves the detection ability of small targets of coral reefs, and makes large-sized coral reefs less susceptible to background interference, improving the detection accuracy. The restoration model meets the requirements of real-time detection and improves the problem of low accuracy caused by the attenuation of actual underwater photos; the classification model improves the problem of extremely unbalanced dataset distribution, improves the feature extraction ability of the classification model, and finally enables the classification model to obtain a high classification accuracy.

[0157] Based on the same inventive concept, the embodiments of the present application also provide a device for realizing the above-mentioned coral reef detection and classification. The implementation solutions provided by this device for solving problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the following coral reef detection and classification devices can refer to the limitations on the coral reef detection and classification method in the above text, and will not be repeated here.

[0158] In an exemplary embodiment, as Figure 6 shown, a coral reef detection and classification device is provided, including:

[0159] An acquisition module 610, configured to acquire coral reef images;

[0160] A detection module 620, configured to process the coral reef images through a detection model to obtain coral reef detection images;

[0161] A restoration module 630, configured to restore the color information of the coral reef detection images through a restoration model to obtain coral reef restoration images;

[0162] A segmentation module 640, configured to segment the coral reef restoration images to obtain coral reef segmentation images;

[0163] A classification module 650, configured to classify the coral reef segmentation images through a classification model to obtain the classification results of the coral reef images, and judge the types of coral reefs according to the classification results of the coral reef images.

[0164] As an alternative implementation, the detection model includes a backbone network, a neck network, and a head network. The detection module is specifically configured to:

[0165] Extract scales from the coral reef image through the backbone network to obtain coral reef feature maps of different scales;

[0166] Fuse the coral reef feature maps of different scales through the neck network to obtain a coral reef fusion map;

[0167] Predict the coral reef fusion map through the head network to obtain a coral reef detection image.

[0168] As an alternative implementation, the backbone network includes a first extraction module, a second extraction module, and a third extraction module. In terms of extracting scales from the coral reef image through the backbone network to obtain coral reef feature maps of different scales, the detection module 620 is specifically configured to:

[0169] Input the coral reef image into the first extraction module and output a first-layer feature map;

[0170] Input the first-layer feature map into the second extraction module and output a second-layer feature map;

[0171] Input the second-layer feature map into the third extraction module and output a third-layer feature map.

[0172] As an alternative implementation, the first extraction module includes a first convolutional layer, a second convolutional layer, a first residual layer, a third convolutional layer, and a second residual layer arranged in sequence, and outputs the first-layer feature map through the second residual layer;

[0173] The second extraction module includes a fourth convolutional layer, a third residual layer, and a fourth residual layer arranged in sequence, and outputs the second-layer feature map through the fourth residual layer;

[0174] The third extraction module includes a fifth convolutional layer, a fourth residual layer, a polarization self-attention layer, and a pooling layer arranged in sequence, and outputs the third-layer feature map through the pooling layer.

[0175] As an alternative implementation, the neck network includes a first fusion module, a second fusion module, a third fusion module, and a fourth fusion module; in terms of fusing the coral reef feature maps of different scales through the neck network to obtain a coral reef fusion map, the detection module is specifically configured to:

[0176] The first fusion module includes a sixth convolutional layer, a first upsampling layer, a first fusion layer, a fifth residual layer, and a seventh convolutional layer arranged in sequence. After passing through the sixth convolutional layer and the first upsampling layer, the third-layer feature map is concatenated with the second-layer feature map through the first fusion layer, and then passes through the sixth residual layer and the seventh convolutional layer to obtain the fourth-layer feature map;

[0177] The second fusion module includes a second upsampling layer, a second fusion layer, and a sixth residual layer arranged in sequence. After passing through the second upsampling layer, the fourth-layer feature map is concatenated with the first-layer feature map through the second fusion layer, and then passes through the sixth residual layer to obtain the fifth-layer feature map;

[0178] The third fusion module includes an eighth convolutional layer, a third fusion layer, a seventh residual layer, and a ninth convolutional layer arranged in sequence. After passing through the eighth convolutional layer, the fifth-layer feature map is concatenated with the fourth-layer feature map through the third fusion layer, and then passes through the seventh residual layer and the ninth convolutional layer to obtain the sixth-layer feature map;

[0179] The fourth fusion module includes a fourth fusion layer and an eighth residual layer arranged in sequence. The sixth-layer feature map is concatenated with the third-layer feature map through the fourth fusion layer, and then passes through the eighth residual layer to obtain the seventh-layer feature map;

[0180] The prediction of the coral reef fusion map by the head network to obtain the coral reef detection image includes:

[0181] The head network includes a two-dimensional convolutional layer. After passing the fifth-layer feature map, the sixth-layer feature map, and the seventh-layer feature map through the two-dimensional convolutional layer respectively, the coral reef detection image is output.

[0182] As an optional implementation manner, the classification model includes a tenth convolutional layer, a max pooling layer, a first convolutional attention layer, a stacked residual module, a second convolutional attention layer, an average pooling layer, and a fully connected layer arranged in sequence;

[0183] In terms of classifying the coral reef segmentation image by the classification model to obtain the classification result of the coral reef image, the classification module is used for:

[0184] After passing the coral reef segmentation image through the tenth convolutional layer, the max pooling layer, the first convolutional attention layer, the stacked residual module, the second convolutional attention layer, the average pooling layer, and the fully connected layer in sequence, the classification result of the coral reef image is output.

[0185] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be asFigure 7 As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store coral reef images. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for coral reef detection and classification.

[0186] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0187] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0188] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0189] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0190] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0191] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memories (RAMs) or external caches, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0192] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0193] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0194] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for detecting and classifying coral reefs, characterized in that: The coral reef detection and classification method comprises: Acquire coral reef images; Processing the coral reef image by using a detection model to obtain a coral reef detection image; Restoring the color information of the coral reef detection image by using a restoration model to obtain a coral reef restoration image; Segmenting the coral reef restored image to obtain a coral reef segmented image; The coral reef segmentation image is classified by a classification model to obtain a classification result of the coral reef image, and the type of the coral reef is determined according to the classification result of the coral reef image.

2. The method for detecting and classifying coral reefs according to claim 1, characterized in that: The detection model includes a backbone network, a neck network and a head network. The coral reef image is processed by the detection model to obtain a coral reef detection image, including: Performing scale extraction on the coral reef image through the backbone network to obtain coral reef feature maps of different scales; The coral reef feature maps of different scales are fused through the neck network to obtain a coral reef fusion map; The coral reef detection image is obtained by performing feature prediction on the coral reef fusion image through the head network.

3. The method for detecting and classifying coral reefs according to claim 2, characterized in that: The backbone network includes a first extraction module, a second extraction module and a third extraction module. The backbone network is used to perform scale extraction on the coral reef image to obtain coral reef feature maps of different scales, including: Inputting the coral reef image into the first extraction module and outputting a first layer feature map; Input the first layer feature map into the second extraction module, and output the second layer feature map; The second layer feature map is input into the third extraction module, and the third layer feature map is output.

4. The method for detecting and classifying coral reefs according to claim 3, characterized in that: The first extraction module includes a first convolutional layer, a second convolutional layer, a first residual layer, a third convolutional layer, and a second residual layer, which are arranged in sequence, and outputs the first layer feature map through the second residual layer; The second extraction module includes a fourth convolutional layer and a third residual layer which are arranged in sequence, and outputs the second layer feature map through the third residual layer; The third extraction module includes a fifth convolutional layer, a fourth residual layer, a polarization self-attention layer and a pooling layer which are arranged in sequence, and the third layer feature map is output through the pooling layer.

5. The method for detecting and classifying coral reefs according to claim 3, characterized in that: The neck network includes a first fusion module, a second fusion module, a third fusion module and a fourth fusion module; The coral reef feature maps of different scales are fused through the neck network to obtain a coral reef fusion map, including: The first fusion module includes a sixth convolutional layer, a first upsampling layer, a first fusion layer, a fifth residual layer and a seventh convolutional layer, wherein the third-layer feature map passes through the sixth convolutional layer and the first upsampling layer, and is concatenated with the second-layer feature map through the first fusion layer, and then passes through the sixth residual layer and the seventh convolutional layer to obtain a fourth-layer feature map; The second fusion module includes a second upsampling layer, a second fusion layer and a sixth residual layer which are sequentially arranged, and the fourth layer feature map is spliced ​​with the first layer feature map through the second fusion layer after passing through the second upsampling layer, and then passes through the sixth residual layer to obtain a fifth layer feature map; The third fusion module includes an eighth convolutional layer, a third fusion layer, a seventh residual layer and a ninth convolutional layer which are sequentially arranged, and the fifth-layer feature map is spliced ​​with the fourth-layer feature map through the third fusion layer after passing through the eighth convolutional layer, and then passes through the seventh residual layer and the ninth convolutional layer to obtain a sixth-layer feature map; The fourth fusion module includes a fourth fusion layer and an eighth residual layer which are sequentially arranged, the sixth-layer feature map and the third-layer feature map are spliced ​​through the fourth fusion layer, and then pass through the eighth residual layer to obtain a seventh-layer feature map; The step of predicting the coral reef fusion image through the head network to obtain the coral reef detection image includes: The head network includes a two-dimensional convolution layer, and after the fifth layer feature map, the sixth layer feature map and the seventh layer feature map pass through the two-dimensional convolution layer respectively, the coral reef detection image is output.

6. The method for detecting and classifying coral reefs according to claim 1, characterized in that: The classification model includes a tenth convolutional layer, a maximum pooling layer, a first convolutional attention layer, a stacked residual module, a second convolutional attention layer, an average pooling layer and a fully connected layer arranged in sequence; The classifying the coral reef segmented image by using a classification model to obtain a classification result of the coral reef image includes: After the coral reef segmentation image passes through the tenth convolutional layer, the maximum pooling layer, the first convolutional attention layer, the stacked residual module, the second convolutional attention layer, the average pooling layer and the fully connected layer in sequence, the classification result of the coral reef image is output.

7. A coral reef detection and classification device, characterized in that: The coral reef detection and classification device comprises: An acquisition module, used to acquire coral reef images; A detection module, used to detect the coral reef image through a detection model to obtain a coral reef detection image, wherein the detection model includes a backbone network, a neck network and a head network; A restoration module, used for restoring color information of the coral reef detection image through a restoration model to obtain a coral reef restoration image; A segmentation module, used for segmenting the coral reef restoration image to obtain a coral reef segmentation image; The classification module is used to classify the coral reef segmentation image through a classification model to obtain a classification result of the coral reef image, and determine the type of the coral reef according to the classification result of the coral reef image.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the coral reef detection and classification method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the coral reef detection and classification method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the coral reef detection and classification method according to any one of claims 1 to 6 are implemented.

Images