Coral Health Status Diagnosis Method, System, Device and Medium Based on Deep Learning Algorithm

Abstract

The present invention relates to a method, system, device and medium for diagnosing the health status of corals based on deep learning algorithms. The method includes: collecting image data of the same coral at different angles for screening, obtaining each image data to be detected for image preprocessing, using the YOLOv5 instance segmentation task model for image segmentation and recognition, and obtaining the area data of each coral region; based on the weighted average formula, calculating the respective coral state data corresponding to the area data of each coral region, and calculating and displaying the coral health index. By collecting images of the same coral at different angles, after screening, preprocessing, segmentation and recognition, the coral health index is calculated and displayed. There is no need to collect coral samples for chemical reagent extraction and detection, which can quickly and accurately diagnose the health status of a coral, objectively reflect the health status of a single coral, avoid the influence of subjectivity on the evaluation results, and improve the efficiency and accuracy of coral health status diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of coral health diagnosis, and particularly to a method, system, device and medium for diagnosing the health status of corals based on a deep learning algorithm. Background Art

[0002] Coral reefs are extremely important components of marine ecosystems. Coral reefs are also one of the most fragile ecosystems on Earth and have an important impact on the stability and biodiversity of marine ecosystems. However, due to the impacts of climate change and human activities, coral reefs worldwide are facing serious threats. For example, problems such as coral bleaching, diseases and pollution are becoming increasingly serious, leading to the continuous deterioration of coral health. Currently, the greatest threat to global coral reefs is the increase in sea surface temperature caused by global climate change. Coral bleaching occurs when its symbiotic dinoflagellates disappear from the host coral tissue or the symbiont pigments degrade, resulting in the coral presenting a white appearance. Without effective protection measures, the coral reef ecosystem may disappear completely by 2070. Therefore, carrying out scientific monitoring and diagnosis of the health status of corals is of great significance for protecting the coral reef ecosystem.

[0003] Coral health status indicators are an effective way to evaluate the health status of coral reef ecosystems. Currently, there are mainly five calculation methods for coral health status indicators, namely RHI-HMRW, CRHI-Ind, EHI, CHI and 2D-CHI. Among them, the RHI-HMRW method is the most complex calculation method, which uses more parameters related to the social dimension; the EHI index is mainly used to calculate the ecosystem health index, and the coral reef health index is one of the parameters for EHI calculation (EHI = coral health index + water quality index); the 2D-CHI and CRHI-Ind indices use the fewest parameters, only the biomass of coral reefs and fish. However, "coral health status indicators" is a general term, and usually the health status of the entire coral reef ecosystem is evaluated through the combination of the above multiple parameters and indicators. In actual situations, these calculation methods not only require a large amount of manpower and time. The health status of a single coral can be accurately reflected by physiological indicators such as chlorophyll a, protein, lipid, and symbiotic dinoflagellate density. However, the damage caused to corals by obtaining physiological indicators is irreversible and cannot play a role in dynamic monitoring. This kind of evaluation usually requires professional coral taxonomists or ecologists to observe and analyze. The quantitative indicators of the health status of a single coral can provide objective and standardized data to monitor and evaluate the health status of each coral, help researchers compare and analyze the differences in the health status of corals under different regions, species and environmental conditions, help identify important factors affecting coral health, provide guidance for the protection and management of corals, and help scientists and researchers analyze and compare the health status of corals.

[0004] Deep learning (DL) technology is a machine learning technology based on artificial neural networks. Through a multi-layer neural network structure, it can achieve efficient processing and feature extraction of complex data, and has played an important role in the research of coral species target detection and health status monitoring. For example, by extracting features and classifying coral images in the sea area taken by remote sensing (satellites, drones, etc.), the coral bleaching areas can be identified to help researchers monitor and evaluate the health status of coral reefs in this water area in a timely manner. Although this method of non-contact monitoring of coral health status based on computer vision technology effectively solves some problems existing in traditional methods, this method still has certain limitations. First, it is difficult to obtain remote sensing data sets based on devices such as satellites and drones (limited to publicly available data sets), which results in low practicality of this method; in addition, remote sensing images are generally top views, while the coral reef ecosystem is a three-dimensional structure. It is impossible to accurately evaluate the health status of a single coral only through a single picture perspective, which leads to deviations in the evaluation of the health status of coral reefs in this water area.

[0005] However, the traditional method of diagnosing the health status of a single coral often relies on manual experience, with large subjective factors, and has problems of low diagnosis efficiency and low diagnosis accuracy. Summary of the Invention

[0006] Based on this, in order to solve the above technical problems, a method for diagnosing the health status of corals based on a deep learning algorithm is provided, which can improve the diagnosis efficiency and accuracy of the health status of corals.

[0007] A method for diagnosing the health status of corals based on a deep learning algorithm, the method includes:

[0008] Collect image data of the same coral from different angles, and use the YOLOv5 object detection model to screen the image data to obtain each image data to be detected;

[0009] Perform image preprocessing on each image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on the preprocessed image data to be detected to obtain each coral area data;

[0010] Based on the weighted average formula, calculate each coral state data corresponding to the coral area data, and calculate the coral health index according to each coral state data;

[0011] Display the coral health index.

[0012] In one embodiment, using the YOLOv5 object detection model to screen the image data to obtain each image data to be detected includes:

[0013] Perform object detection on the corals in each of the image data to obtain a detection result;

[0014] When the detection result indicates that the image is unqualified, delete the unqualified image data and display a prompt message;

[0015] When the detection result indicates that the image is qualified, use the qualified image data as the image data to be detected.

[0016] In one embodiment, performing object detection on the corals in each of the image data to obtain a detection result includes:

[0017] Based on the YOLOv5 object detection model, use the YOLOv5s convolutional neural network structure as the basic network to extract feature maps of different scales of the input image data;

[0018] Perform convolutional operations on the feature maps of different scales to obtain the class confidence and bounding box regression results corresponding to each position;

[0019] Judge whether there are corals in the image data according to the class confidence, and determine the position and size of the corals according to the bounding box regression results;

[0020] Determine that the detection result of the image data with corals in the central area is qualified, and determine that the detection result of the image data without corals or with corals not in the central area is unqualified.

[0021] In one embodiment, the image preprocessing of each of the image data to be detected includes:

[0022] Use the segmentation polygon tool in Labelme to outline the shapes of the corals and the diseased areas in the image data to be detected, and annotate the image data to be detected to obtain the annotated coral image data;

[0023] Perform size adjustment, grayscale processing, and denoising operations on the annotated coral image data to obtain the processed image data.

[0024] In one embodiment, the use of the YOLOv5 instance segmentation task model to perform image segmentation and recognition on each of the preprocessed image data to be detected to obtain the area data of each coral region includes:

[0025] Use the YOLOv5 instance segmentation task model to perform coral segmentation on each of the preprocessed image data to be detected to obtain several coral region image data;

[0026] Identify the coral bleaching areas from several of the coral region image data;

[0027] Calculate the area of the coral bleaching area and the non-bleaching area respectively.

[0028] In one embodiment, based on the weighted average formula, calculate the respective coral status data corresponding to the area data of each coral region, and calculate the coral health index according to each of the coral status data, including:

[0029] Calculate the coral bleaching rate of each of the to-be-detected image data according to the area of the coral bleaching area and the non-bleaching area.

[0030] Based on the weighted average formula, calculate the weight coefficient of each of the to-be-detected image data.

[0031] Calculate the coral health index according to the bleaching rate and the weight coefficient of the coral.

[0032] In one embodiment, the method further includes:

[0033] Determine the reference coral health index, and perform a correlation analysis on the reference coral health index and the coral health index using GraphPad to obtain an analysis result;

[0034] Perform reliability verification according to the analysis result.

[0035] A coral health status diagnosis system based on a deep learning algorithm, the system includes:

[0036] A quality control module, configured to collect each image data of the same coral from different angles, and use the YOLOv5 object detection model to screen the image data to obtain each to-be-detected image data;

[0037] A detection module, configured to perform image preprocessing on each of the to-be-detected image data, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on the preprocessed to-be-detected image data to obtain each coral region area data;

[0038] A calculation module, configured to calculate the respective coral status data corresponding to the area data of each coral region based on the weighted average formula, and calculate the coral health index according to each of the coral status data;

[0039] A display module, configured to display the coral health index.

[0040] A computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0041] Collect image data of different angles of the same coral, and use the YOLOv5 object detection model to screen each piece of the image data to obtain each piece of image data to be detected;

[0042] Perform image preprocessing on each piece of the image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on each piece of the preprocessed image data to be detected to obtain each piece of coral area data;

[0043] Based on the weighted average formula, calculate each coral status data corresponding to the coral area data of each region respectively, and calculate the coral health index according to each coral status data;

[0044] Display the coral health index.

[0045] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0046] Collect image data of different angles of the same coral, and use the YOLOv5 object detection model to screen each piece of the image data to obtain each piece of image data to be detected;

[0047] Perform image preprocessing on each piece of the image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on each piece of the preprocessed image data to be detected to obtain each piece of coral area data;

[0048] Based on the weighted average formula, calculate each coral status data corresponding to the coral area data of each region respectively, and calculate the coral health index according to each coral status data;

[0049] Display the coral health index.

[0050] The above coral health status diagnosis method, system, device and medium based on deep learning algorithms collect images of the same coral from different angles, and after screening, preprocessing, segmentation and recognition, calculate and display the coral health index. There is no need to collect coral samples for chemical reagent extraction and detection, which can quickly and accurately diagnose the health status of a coral, objectively reflect the health status of a single coral, avoid the influence of subjectivity on the evaluation results, and improve the efficiency and accuracy of coral health status diagnosis; ordinary members of the public can all participate in the collection of coral health data, based on a new model of national scientific monitoring and diagnosis of coral health status, which can not only help researchers in the coral field quickly and conveniently obtain a large amount of coral health data, reduce the workload of on-site inspections and manual measurements, and improve research efficiency. At the same time, it also provides good technical support for ordinary members of the public to participate in coral protection and ecological restoration work. Description of the Drawings

[0051] Figure 1 It is an application environment diagram of a coral health status diagnosis method based on a deep learning algorithm in an embodiment;

[0052] Figure 2 It is a schematic flow diagram of a coral health status diagnosis method based on a deep learning algorithm in an embodiment;

[0053] Figure 3 It is a schematic diagram of taking images of different angles of a single coral in an embodiment;

[0054] Figure 4 It is a block diagram of a YOLOv5 object detection model for determining whether the position of a coral is in the central area in an embodiment;

[0055] Figure 5 It is a block diagram of a YOLOv5 instance segmentation task model for performing object detection and instance segmentation in an embodiment;

[0056] Figure 6 It is a schematic diagram of calculating the health index of a single coral in an embodiment;

[0057] Figure 7 It is a schematic diagram of screening out 12 specimens of Pocillopora damicornis with different health statuses in an embodiment;

[0058] Figure 8 It is a schematic diagram of an obvious positive correlation between the health index and the visual health degree of corals in an embodiment;

[0059] Figure 9 It is a schematic diagram of a very strong positive correlation between the health index and the content of chlorophyll a in an embodiment;

[0060] Figure 10 It is a block diagram of a coral health status diagnosis system based on a deep learning algorithm in an embodiment;

[0061] Figure 11 It is a schematic diagram of the process of obtaining training weights by segmenting and training a YOLOv5 deep learning model in an embodiment;

[0062] Figure 12 It is a schematic diagram of the AUC curve of the object detection probability and object detection for the internal validation set images in an embodiment;

[0063] Figure 13 It is a schematic diagram of the reliability analysis of the vSCHM system in an embodiment;

[0064] Figure 14Variance analysis comparison chart of the artificial counting results of the coral health index in different health states and the detection and calculation results of the qualified coral perspective images input into the vSCHM system in an embodiment;

[0065] Figure 15 Comparison chart of the linear regression operation between the detection and calculation results of the health index and the artificial counting results, and the residual operation between the detection and calculation results of the health index and the artificial counting results in an embodiment;

[0066] Figure 16 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0067] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] The coral health status diagnosis method based on deep learning algorithm provided by the embodiment of the present application can be applied to, for example, Figure 1 the application environment shown. As Figure 1 shown, the application environment includes a computer device 110. The computer device 110 can collect various image data of the same coral at different angles, use the YOLOv5 object detection model to screen the various image data to obtain various image data to be detected; the computer device 110 can perform image preprocessing on the various image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on the preprocessed various image data to be detected to obtain various coral area data; the computer device 110 can calculate various coral status data corresponding to the coral area data based on the weighted average formula, and calculate the coral health index according to the various coral status data; the computer device 110 can display the coral health index. Among them, the computer device 110 can be but is not limited to various personal computers, laptop computers, smart phones, robots, unmanned aerial vehicles, tablet computers and portable wearable devices, etc.

[0069] In one embodiment, as Figure 2 shown, a coral health status diagnosis method based on deep learning algorithm is provided, including the following steps:

[0070] Step 202, collect various image data of the same coral at different angles, use the YOLOv5 object detection model to screen the various image data to obtain various image data to be detected.

[0071] The computer device can be a mobile phone terminal, and the user can collect various image data of the same coral from different angles through a small program on the mobile phone terminal. Specifically, the various image data of the same coral from different angles can be obtained by the user manually collecting in real time through the camera on the mobile phone terminal, or can be pre-collected.

[0072] Among them, a small program system, namely the versatile smartphone-based coral health monitor (vSCHM system), can be set on the mobile phone terminal. In this embodiment, 5 image data of a single coral can be captured from different angles based on the vSCHM system. Specifically, the user can use the built-in rear camera of the mobile phone terminal to take pictures of different angles of a single coral, and the number of photos ≥ 5. These images capture the external phenotypic characteristics of the coral in a week, such as Figure 3 As shown, during the process of taking pictures, the vSCHM system guides the user to familiarize with the system and complete the standardized preparation in an interactive manner. If the captured images do not meet the requirements, the vSCHM system will give prompt information, such as Figure 3 As shown.

[0073] Next, the YOLOv5 object detection model can be used to screen the various image data. Specifically, based on the YOLOv5 object detection model, a convolutional neural network can be used to screen the various image data, screen out the image data that does not meet the specifications and the image data that meets the specifications, and use the image data that meets the specifications as the image data to be detected and enter the next step of processing of the vSCHM system.

[0074] Step 204: Perform image preprocessing on each image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on the preprocessed image data to be detected to obtain the area data of each coral region.

[0075] After screening out each image data to be detected, image preprocessing can be further performed for subsequent processing. The computer device can be divided into two parallel tasks based on the YOLOv5 instance segmentation task model, with object detection and instance segmentation performing parallel computing to achieve image segmentation and recognition of the image data to be detected, thereby obtaining the area data of each coral region. Specifically, the coral region area data can include the area data of the coral bleaching region and the area data of the non-bleaching region of the coral.

[0076] Step 206: Based on the weighted average formula, calculate the respective coral state data corresponding to the area data of each coral region, and calculate the coral health index according to the respective coral state data.

[0077] The computer device can calculate the health index of a single coral by combining the instance segmentation results of images of the same coral from different angles output by the vSCHM system based on the weighted average formula.

[0078] Step 208, display the coral health index.

[0079] Among them, the main scientific basis of the coral health index is the visual health degree of the coral, that is, the coverage rate of polyps and the content of chlorophyll a. The coverage rate of polyps intuitively shows the health degree of the coral, the content of chlorophyll a is directly reflected as the color of the coral, and the content of chlorophyll a is positively correlated with the coral health state. Therefore, it is scientific and reasonable to use the coral health index as a quantitative indicator to measure the health state of a single coral.

[0080] In this embodiment, by collecting images of the same coral from different angles, after screening, preprocessing, segmentation and recognition, the coral health index is calculated and displayed. There is no need to collect coral samples for chemical reagent extraction and detection, which can quickly and accurately diagnose the health state of a coral, objectively reflect the health condition of a single coral, avoid the influence of subjectivity on the evaluation results, and improve the efficiency and accuracy of coral health state diagnosis; ordinary members of the public can all participate in the collection of coral health data, based on a new model of national scientific monitoring and diagnosis of coral health state, which can not only help coral researchers quickly and conveniently obtain a large amount of coral health data, reduce the workload of field investigations and manual measurements, improve the research efficiency, but also provide good technical support for ordinary members of the public to participate in coral protection and ecological restoration work.

[0081] In one embodiment, a method for diagnosing the health state of corals based on a deep learning algorithm may further include a process of image screening. The specific process includes: using the YOLOv5 object detection model to perform object detection on corals in each image data to obtain detection results; when the detection result is that the image is unqualified, delete the unqualified image data and display a prompt message; when the detection result is that the image is qualified, use the qualified image data as the image data to be detected.

[0082] Specifically, in one embodiment, the process of image screening includes: based on the YOLOv5 object detection model, using the YOLOv5s convolutional neural network structure as the basic network to extract feature maps of different scales of the input image data; passing the feature maps of different scales through convolutional operations to obtain the class confidence and bounding box regression results corresponding to each position; judging whether there is a coral in the image data according to the class confidence, and determining the position and size of the coral according to the bounding box regression results; determining the detection result of the image data with the coral in the central area as the image being qualified, and determining the detection result of the image data without a coral or with the coral not in the central area as the image being unqualified.

[0083] Based on the YOLOv5 object detection model, the lightweight convolutional neural network structure of YOLOv5s is used as the basic network, and predefined anchor boxes are used as prior boxes to predict the position and size of corals. There will be anchor boxes of different sizes on feature maps of different scales.

[0084] In this embodiment, as Figure 4 shown, after the image data is input into the vSCHM system, the size is gradually reduced through the convolutional layer, and feature maps of different scales are extracted at the same time. The feature map of each scale undergoes a convolutional operation to obtain the class confidence and bounding box regression results corresponding to each position. Among them, the class confidence represents the confidence that this position is a coral, so as to distinguish whether the image uploaded by the user in the vSCHM system contains corals; the bounding box regression results are used to predict the position and size of the corals, and finally judge whether the coral position in the image uploaded by the user in the vSCHM system is in the central area. The specific judgment process is as Figure 4 shown. After the prediction is completed, non-maximum suppression (NMS) is performed by the quality control module of the vSCHM system to remove overlapping bounding boxes, and the bounding box with the highest confidence is selected as the final detection result. Finally, the images that do not meet the specifications are screened out and fed back to the display interface of the mobile terminal, and the images that meet the specifications enter the next operation.

[0085] In one embodiment, a method for diagnosing the health status of corals based on a deep learning algorithm may further include the process of image preprocessing. The specific process includes: using the segmentation polygon tool in Labelme to outline the coral shape and lesion area in the image data to be detected, and annotating the image data to be detected to obtain the annotated coral image data; performing size adjustment, grayscale processing, and denoising operations on the annotated coral image data to obtain the processed image data.

[0086] Among them, the computer device can use the segmentation polygon tool in Labelme to annotate the image data to be detected to outline the coral shape and the disease lesion area. These masks from the annotation are the content for the algorithm to learn and simulate. Then, the annotated coral image data can be preprocessed, including operations such as image size adjustment, grayscale processing, and denoising, so as to obtain the processed image data.

[0087] Preprocess the labeled coral image data, including operations such as image resizing, grayscale processing, and denoising; select a classic object detection model for comparative analysis with the segmentation model, select a suitable deep learning algorithm, i.e., the segmentation model, and design a deep learning model for coral health status diagnosis, i.e., the coral health status diagnosis model; use the preprocessed coral image data to train the deep learning model and adjust the model parameters to improve the accuracy of the model; use the validation set to evaluate the trained model and evaluate the accuracy and robustness of the model; calculate the weight ratio of coral photos taken at different angles through weighted average, and calculate the formula

[0088] The weight coefficient, HI represents the health index of the coral, and S Fi represents the image area, and S Ci represents the coral area in the image, and n represents the total number of images taken of each type of coral.

[0089] In one embodiment, a method for diagnosing the health status of corals based on a deep learning algorithm may further include a process of performing image segmentation and recognition. The specific process includes: using the YOLOv5 instance segmentation task model to segment corals from each preprocessed image data to be detected, obtaining several coral area image data; identifying the coral bleaching areas from the several coral area image data; calculating the area of the coral bleaching areas and the non-bleaching areas respectively.

[0090] When testing and evaluating the YOLOv5 instance segmentation task model, the weights of the trained coral health status diagnosis model can be tested on the validation set to evaluate the accuracy and robustness of the model. The final model after testing and evaluation can be applied to the actual scenario to diagnose and evaluate the health status of corals.

[0091] Among them, such as Figure 5As shown in the figure, in the vSCHM system, based on the YOLOv5 instance segmentation task model, the instance segmentation task can be divided into two parallel tasks, where object detection and instance segmentation are computed in parallel. The object detection branch mainly inputs the qualified image data to be detected through the vSCHM system, extracts features of the image through the backbone network, and fuses feature maps of different sizes through the FPN feature pyramid. The detection branch outputs the category, bounding box information (x, y, w, h), and k Mask coefficients (the confidence value of the mask takes 1 or -1) for each target object in the image. The segmentation branch outputs k Prototypes (mask prototype maps) for the currently input image. The Prototypes output for different images are different, but the number is also k. Finally, for each target object, multiply the k Mask coefficients, that is, the confidence of the mask, by the k Prototypes, that is, the mask prototype maps, and sum all the results to finally obtain the instance segmentation result of the target object, as Figure 5 shown. Finally, output the areas of the non-bleached and bleached regions of the coral in the instance segmentation result.

[0092] In one embodiment, a coral health status diagnosis method based on a deep learning algorithm may further include a process of calculating a coral health index. The specific process includes: calculating the coral bleaching rate of each image data to be detected according to the area of the coral bleached region and the non-bleached region; calculating the weight coefficient of each image data to be detected based on the weighted average formula; and calculating the coral health index according to the bleaching rate and weight coefficient of the coral.

[0093] Specifically, as Figure 6 shown, in the vSCHM system, based on the weighted average formula, combined with the instance segmentation results of images of the same coral at different angles in the vSCHM system, calculate the coral bleaching rate of each image. The calculation formula is: Hi = S Wi / S Ci , where S Wi represents the total area of the coral bleached region in the instance segmentation of the image, and S Ci represents the area of the coral region in the instance segmentation of the image. It is also necessary to calculate the weight coefficient of each image. The calculation formula is: where S Fi represents the total area of the input image. Finally, calculate the health index of a single coral. The formula is:

[0094] In one embodiment, a method for diagnosing the health status of corals based on a deep learning algorithm may further include a process of reliability verification. The specific process includes: determining a reference coral health index, and performing a correlation analysis on the reference coral health index and the coral health index using GraphPad to obtain an analysis result; and performing reliability verification based on the analysis result.

[0095] Among them, the reference coral health index can be calculated manually. By manually counting and statistically analyzing the coverage rate of polyps of each coral as the health index to measure the health status of a single coral.

[0096] When establishing the standard quantitative index for the health status of corals, the method of counting and statistics can be preferably used to count the coverage rate of polyps on the surface of a coral, and it is used as the quantitative index for the health status of corals. Among them, the coverage rate of polyps on the surface of a coral refers to the proportion of the number of polyps of a coral to the coral wormholes of the whole coral. By regularly monitoring the coverage rate of polyps of the coral to be measured, the overall health status of the coral is evaluated. In addition, in order to verify the accuracy of the quantitative index, it is necessary to extract and detect the coral biomass of a certain sample size with different health statuses, especially pay attention to detecting the chlorophyll a content in different samples, analyze the correlation between the chlorophyll a content of each sample and the polyp coverage rate, and further verify that the polyp coverage rate on the surface of a coral can be used as the quantitative index for the health status of corals. Among them, the coral biomass refers to the mass or biomass of corals per unit area. By detecting the coral biomass, the biological productivity of coral reefs and the stability of the ecosystem can be understood.

[0097] Specifically, in this embodiment, the manual counting and statistical method can be used to calculate each reference coral health index. The calculation formula is: HI i =a i / (a i +b i )×100%, where a represents the number of polyps in the sample, b represents the number of wormholes in the sample, i represents the sample, and HI i represents the reference coral health index. Among them, for each coral sample, at least three people should repeat the technical statistics to ensure the accuracy of the counting and statistical data. After calculating the coral health index, it is necessary to take photos of each sample from different angles. Each coral sample should have at least 30 photos for use in subsequent verification experiments. Grind the above-mentioned counted and photographed samples by quick freezing in liquid nitrogen, take an appropriate amount of the sample for chlorophyll a content detection, and place the remaining samples in a -80°C refrigerator for later use.

[0098] Taking Pocillopora damicornis as an example, 12 Pocillopora damicornis with different health statuses are selected, such as Figure 7As shown, they are respectively named c1 - 12 and are used for the counting statistics of the hydroid coverage rate (health index), the acquisition of image datasets, the detection of chlorophyll a content, and the observation and identification by coral experts. The results show that there is an obvious positive correlation between the health index and the visual health of corals. As Figure 7 , Figure 8 shown, and there is a very strong positive correlation between the health index and the content of chlorophyll a (R 2 = 0.9443, P < 0.0001) as Figure 9 shown. Therefore, the health index can be used as a quantitative indicator to measure the health status of a single coral. According to the health index and the observation and identification results of 12 specimens of Pocillopora damicornis by coral experts, the health index is divided into 5 levels. Among them, Level 1: "Healthy", health index ≥ 0.95; Level 2: "Sub - healthy", 0.95 > health index ≥ 0.85; Level 3: "Partial bleaching", 0.85 > health index ≥ 0.75; Level 4: "Moderate bleaching", 0.75 > health index ≥ 0.65; Level 5: "Severe bleaching", 0.65 > health index.

[0099] GraphPad was used to analyze the correlation between the manually counted coral health index and the coral health index calculated by the coral health status diagnosis method based on the deep - learning algorithm in this embodiment, further verifying the reliability of the deep - learning diagnosis and evaluation of the coral health status by the coral health status diagnosis method based on the deep - learning algorithm.

[0100] It should be understood that although the steps in the above flowchart are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the above flowchart may include multiple sub - steps or multiple stages. These sub - steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these sub - steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or sub - steps or stages of other steps.

[0101] In one embodiment, as Figure 10 shown, a coral health status diagnosis system based on a deep - learning algorithm is provided, including: a quality control module 1010, a detection module 1020, a calculation module 1030, and a display module 1040, where:

[0102] The quality control module 1010 is used to collect various image data of the same coral from different angles, and use the YOLOv5 object detection model to screen the various image data to obtain each image data to be detected;

[0103] The detection module 1020 is used to perform image preprocessing on each image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on the preprocessed image data to be detected, so as to obtain the area data of each coral region;

[0104] The calculation module 1030 is used to calculate the respective coral state data corresponding to the area data of each coral region based on the weighted average formula, and calculate the coral health index according to the respective coral state data;

[0105] The display module 1040 is used to display the coral health index.

[0106] Among them, in the vSCHM system of the computer device, there may be a quality control module 1010, a detection module 1020, and a calculation module 1030.

[0107] The computer device can randomly collect a dataset of 1000 images of Acropora cervicornis with different health states. Among them, 200 images are used for the development of the quality control module 1010 of the vSCHM system, and are divided into a training set and an internal test set according to a ratio of 9:1; the remaining 800 images are used for the development of the detection module 1020 of the vSCHM system, and are divided into a training set and an internal test set according to a ratio of 9:1.

[0108] The principle of weighted average is introduced into the calculation module 1030 of the vSCHM system, and finally the vSCHM system outputs the health index of a single coral. In order to verify the generality of the vSCHM system, 12 Acropora cervicornis with different health states can be used. The 12 corals are divided into 4 groups according to the health index, and a total of 330 images are used as an independent external validation set for external validation; in order to verify the accuracy of the output results of the vSCHM system, the detection values of the health indexes of 12 Acropora cervicornis with different health states by the vSCHM system can be analyzed for phototropism and residual analysis with the artificial statistical counting results.

[0109] In one embodiment, the quality control module 1010 is developed based on the YOLOv5 object detection model to ensure the performance of the vSCHM system in a complex environment. Among them, the complex environment can be an environment with various interference factors or deviations that may affect the system performance. Taking Acropora cervicornis as an example, the quality control module 1010 can include the detection of Acropora cervicornis targets, and the maximum margin object detection (MMOD) convolutional neural network (CNN) can be used; for the localization of key points on the surface of Acropora cervicornis, a set of regression trees can be used. The quality control module 1010 of the vSCHM system achieved an area under the receiver operating characteristic curve (AUC) of 0.963 for coral targets in 50 internal validation set images, including a 95% confidence interval (95% CI). After the automatic quality control of the quality control module 1010, 700 out of 1000 original Acropora cervicornis images were retained as the development dataset; before training, the development dataset was randomly stratified into a training set and an internal validation set at a ratio of 9:1, and the images used for the development of the quality control module 1010 were excluded from the validation set developed by the detection module 1020.

[0110] In one embodiment, the quality control module 1010 is further configured to perform object detection on the corals in each of the image data to obtain a detection result; when the detection result indicates that the image is unqualified, the unqualified image data is deleted and a prompt message is displayed; when the detection result indicates that the image is qualified, the qualified image data is used as the image data to be detected.

[0111] In one embodiment, the quality control module 1010 is further configured to, based on the YOLOv5 object detection model, use the YOLOv5s convolutional neural network structure as the basic network to extract feature maps of different scales of the input image data; perform convolutional operations on the feature maps of different scales to obtain the class confidence and bounding box regression results corresponding to each position; determine whether there is a coral in the image data according to the class confidence, and determine the position and size of the coral according to the bounding box regression result; determine that the detection result of the image data with the coral in the central area is qualified, and determine that the detection result of the image data without a coral or with the coral not in the central area is unqualified.

[0112] In one embodiment, the detection module 1020 is further configured to use the segmentation polygon tool in Labelme to outline the shape and lesion area of the corals in the image data to be detected, and annotate the image data to be detected to obtain the annotated coral image data; perform size adjustment, grayscale processing, and denoising operations on the annotated coral image data to obtain the processed image data.

[0113] Among them, the detection module 1020 can use the labelme annotation tool to segment and annotate the coral images in the development dataset, mark the polyp region and the bleached region of the coral, and preprocess the annotated coral image data, such as size adjustment, grayscale processing, denoising, etc.

[0114] In one embodiment, the detection module 1020 is further configured to use the YOLOv5 instance segmentation task model to perform coral segmentation on each preprocessed image data to be detected, obtaining a number of coral region image data; identifying the coral bleached regions from the number of coral region image data; calculating the area of the coral bleached regions and the area of the non-bleached regions respectively.

[0115] Among them, the preprocessed coral image data is used to perform segmentation training on the YOLOv5 deep learning model to obtain training weights with higher accuracy, as Figure 11 shown. The results show that the detection module 1020 performs segmentation probability on 70 internal validation set images, the corals in the images are all successfully segmented, and a total of 126 bleached regions are segmented and identified. The detection module of the vSCHM system achieves an area under the receiver operating characteristic curve (AUC) of 0.995, 95% confidence interval (95% CI) for the segmented regions of the corals in the images, and an AUC (95% CI) of 0.697 for the segmented regions of the coral bleaching, as Figure 12 shown in b and c of. In addition, there is an overfitting phenomenon in the detection module 1020 of the vSCHM system for the segmented regions of the coral bleaching in the images, and the non-coral bleached regions in the image background are also segmented. Post-processing and control are performed on the segmentation model of the detection module 1020, and the detection box pixel area less than the threshold is not included in the coral bleached region, finally eliminating the problem of over-segmentation of the detection module 1020 for the coral bleached region.

[0116] In one embodiment, the calculation module 1030 is further configured to calculate the coral bleaching rate of each image data to be detected according to the area of the coral bleached regions and the area of the non-bleached regions; calculate the weight coefficient of each image data to be detected based on the weighted average formula; calculate the coral health index according to the coral bleaching rate and the weight coefficient.

[0117] Among them, for the instance segmentation results of images of the same coral at different angles output by the detection module 1020, the calculation module 1030 can calculate the coral bleaching rate Hi of each image and the weight coefficient Ki of each image, and finally calculate the health index H of a single coral.

[0118] In one embodiment, reliability analysis can also be performed on the vSCHM system. Specifically, 12 images of Pocillopora damicornis with different health states can be selected as the external validation set. After screening by the quality control module 1010, a total of 330 qualified images are obtained for the reliability analysis of the vSCHM system. To verify the reliability of the vSCHM system in evaluating the health index of Pocillopora damicornis with different health degrees, 330 qualified images of 12 Pocillopora damicornis are divided into 4 categories according to the health index of each coral. There are 71 images (21.51%) of Pocillopora damicornis in the "healthy" state, 181 images (54.85%) of Pocillopora damicornis in the "sub-healthy" state, 25 images (7.58%) of Pocillopora damicornis in the "partial bleaching" state, and 53 images (16.06%) of Pocillopora damicornis in the "severe bleaching" state. In the vSCHM system, the detection model 1020 achieved an AUC (95% CI) of 0.995 for the coral area in the total external validation set and an AUC (95% CI) of 0.969 for the bleached area, as Figure 13 shown in a, b, c in Figure 13 The validation set of Pocillopora damicornis in the "healthy" state achieved an AUC (95% CI) of 0.995 for the coral area in the image, as Figure 13 shown in d, h, i in Figure 13 The validation set of Pocillopora damicornis in the "sub-healthy" state achieved an AUC (95% CI) of 0.995 for the coral area in the image and an AUC (95% CI) of 0.960 for the bleached area, as Figure 13 shown in e, j, k in

[0119] In one embodiment, it can also include performing reliability analysis on the vSCHM system. Specifically, 330 qualified images can be selected from the images of 12 Pocillopora damicornis with different health states for verifying the reliability of the vSCHM system.

[0120] The prediction probability of the detection module 1010 in the vSCHM system for segmenting 330 qualified images, and the AUC curve (95% CI) for segmenting 330 qualified images. As Figure 13As shown in d, e, f, and g in the figure: 330 qualified images of 12 Acropora cervicornis corals were grouped by the health index into 4 groups: "healthy", "sub-healthy", "partial bleaching", and "severe bleaching". One image was selected from each group as a representative for display. As Figure 13 As shown in h, j, l, and n in the figure: The prediction probabilities of the detection module 1010 in the vSCHM system for segmenting 71, 181, 25, and 53 qualified images of "healthy", "sub-healthy", "partial bleaching", and "severe bleaching" respectively. As Figure 13 As shown in I, k, m, and o in the figure: The AUC curves (95% CI) of the detection module 1010 in the vSCHM system for segmenting qualified images of "healthy", "sub-healthy", "partial bleaching", and "severe bleaching".

[0121] In one embodiment, the true performance of the vSCHM system in the ART environment can be evaluated by comparing the cosine similarity between the detected value and the true value. Specifically, the performance of the vSCHM system was evaluated through independent external verification, which was carried out by trained volunteers using smartphones in the ART environment. At this stage, quality inspection was embedded in the data collection and upload process. When the quality of the captured or uploaded images was low or did not meet the requirements, the quality control module 1010 would automatically remind the volunteers to check the data. The final verification used 180 qualified images of 12 Acropora cervicornis corals in different health states, 3 corals (25%) in the healthy state, 6 corals (50%) in the sub-healthy state, 1 coral (8.3%) in the moderate bleaching state, and 2 corals (16.7%) in the severe bleaching state. Qualified images of 5 different perspectives, 10 different perspectives, and 15 different perspectives of the 12 corals were respectively input into the vSCHM system (vSCHM-5, vSCHM-10, vSCHM-15). After detection and calculation, the health index was output, and there was no significant difference compared with manual counting (except for the results of c2 vSCHM-5 and vSCHM-10, and c4 vSCHM-15), as Figure 14 As shown. Subsequently, a correlation analysis and a residual analysis were performed on the results of vSCHM-5, vSCHM-10, and vSCHM-15 of the health index of the 12 corals and the manual counting results. The results showed that there was a very strong positive correlation between the results of vSCHM-5, vSCHM-10, and vSCHM-15 and the manual counting results (RvSCHM-52 = 0.9866, RvSCHM-102 = 0.9863, RvSCHM-152 = 0.9868), as Figure 15 As shown in b in the figure, the residual values were all between -0.1 and 0.1, and most of the values were between -0.05 and 0.05, indicating that the difference between the estimated value and the true value was relatively small, and the accuracy of the estimated result was relatively high, as Figure 15 As shown in c in the figure.

[0122] In one embodiment, it is also possible to verify the performance in the real environment of the vSCHM system. Specifically, the artificial counting results of the health indices of 12 strains of Pocillopora damicornis with different health states can be subjected to an analysis of variance with the detection and calculation results of qualified images of 12 strains of Pocillopora damicornis input into the vSCHM system from 5 different perspectives, 10 different perspectives, and 15 different perspectives (vSCHM-5, vSCHM-10, vSCHM-15). The results are expressed as mean ± s.d.*p<0.001. Among them, linear regression operations are performed on the detection and calculation results of the health indices of vSCHM-5, vSCHM-10, and vSCHM-15 and the artificial counting results. There is a very strong positive correlation between the results of vSCHM-5, vSCHM-10, and vSCHM-15 and the artificial counting results, with RvSCHM-52 = 0.9866, RvSCHM-102 = 0.9863, and RvSCHM-152 = 0.9868 (95% confidence bands). When residual operations are performed on the detection and calculation results of the health indices of vSCHM-5, vSCHM-10, and vSCHM-15 and the artificial counting results, the residual values are all between -0.1 and 0.1, and most of the values are between -0.05 and 0.05. Taking the artificial counting results as the true values and the monitoring and calculation results of the vSCHM system as the observed values.

[0123] In one embodiment, a computer device is provided. This computer device can be a terminal, and its internal structure diagram can be as Figure 16 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for diagnosing the health state of corals based on a deep learning algorithm. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0124] Those skilled in the art can understand, Figure 16The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0125] In one embodiment, a system applet can be loaded in a provided computer device. Specifically, an applet can be developed based on a target segmentation method, and the established vSCHM system can be integrated into the background system of the applet to establish an API interface of the model, so that the applet can call the vSCHM system to diagnose the health status of corals. In terms of user interface design, design the user interface of the applet, including function modules for uploading coral images and videos, a diagnostic result display module, a health status explanation module, etc., so that users can conveniently upload images and videos and view the diagnostic results. In terms of function development, develop a function module for diagnosing the health status of corals in the applet, including an image and video upload module, a module for calling the model for diagnosis, a module for displaying diagnostic results, etc. The function of diagnosing the health status of corals integrated into the applet can also be tested to ensure its stability and accuracy on the computer device side, and the function can be optimized and improved according to the test results to improve the user experience and diagnostic accuracy.

[0126] When using the applet in the computer device of this embodiment to diagnose the health status of corals, as Figure 6 shown, the applet can open the camera of the computer device to collect image data containing corals from various angles, which can be pictures or videos. The applet can run the vSCHM system and display the diagnostic results on the interface. Integrate the vSCHM system into the background system of the applet and establish an API interface of the model, so that the applet can call the model to diagnose the health status of corals, and use the applet developer mode to design the user interface, function development, testing and optimization to improve the user experience. Due to the setting of the applet, more members of the public can participate in coral protection work, providing more opportunities and channels for the public to participate in environmental protection undertakings and enhancing the public's environmental awareness and sense of responsibility.

[0127] The method, system, device and medium for diagnosing the health status of corals based on deep learning algorithms provided by this application scientifically formulates and demonstrates the quantitative indicators of coral health status by collecting coral samples; obtains a coral picture data set and uses image segmentation to successfully identify the coral polyp area and the bleached area; successfully develops an applet and successfully applies the coral health status diagnosis model to the applet to realize real-time monitoring and diagnosis of the health status of single corals, which is of great significance for coral protection and the protection of the marine ecological environment.

[0128] In one embodiment, the process of using the coral health status diagnosis method based on deep learning algorithms is as follows Figure 7 As shown, by introducing advanced deep learning technologies, the automated monitoring and diagnosis of the health status of individual corals are achieved, greatly improving the efficiency and accuracy of monitoring. By guiding the general public to participate in the monitoring of coral health status, and by using artificial intelligence devices, the general public can collect coral image and video data and upload them to the central database for analysis through a specially designed application program. This can not only expand the monitoring scope but also increase the public's participation in environmental protection, having high social participation and promotion value.

[0129] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps of the coral health status diagnosis method based on deep learning algorithms are implemented.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps of the coral health status diagnosis method based on deep learning algorithms are implemented.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-described method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0133] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for diagnosing coral health status based on a deep learning algorithm, characterized in that: The method comprises: Collect image data of the same coral at different angles, use the YOLOv5 target detection model to perform image screening on each of the image data, and obtain each image data to be detected, including: based on the YOLOv5 target detection model, use the YOLOv5 convolutional neural network structure as the basic network to extract feature maps of different scales of the input image data; perform convolution operations on the feature maps of different scales to obtain the category confidence and bounding box regression results corresponding to each position; determine whether there is a coral in the image data according to the category confidence, and determine the position and size of the coral according to the bounding box regression result; determine the detection result of the image data in which the coral is located in the central area as a qualified image, and determine the detection result of the image data without the coral or the coral is not located in the central area as an unqualified image; when the detection result is an unqualified image, delete the unqualified image data and display a prompt message; when the detection result is a qualified image, use the qualified image data as the image data to be detected; Perform image preprocessing on each of the image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on the preprocessed image data to be detected to obtain the area data of each coral region; Based on the weighted average formula, the coral status data corresponding to the area data of each region of the coral are calculated respectively, and the coral health index is calculated according to the coral status data; the reference coral health index is determined, and the correlation analysis between the reference coral health index and the coral health index is performed using GraphPad to obtain the analysis result; the reliability is verified according to the analysis result; wherein, the reference coral health index is a health index that measures the health status of a single coral by manually counting the coverage rate of each coral polyp; Displays the coral health index.

2. The method for diagnosing coral health status based on deep learning algorithm according to claim 1, characterized in that: The performing image preprocessing on each of the image data to be detected comprises: Use the segmentation polygon tool in Labelme to outline the coral shape and the lesion area in the image data to be detected, and annotate the image data to be detected to obtain the labeled coral image data; The labeled coral image data is resized, gray-scaled, and denoised to obtain processed image data.

3. The method for diagnosing coral health status based on deep learning algorithm according to claim 1, characterized in that: The YOLOv5 instance segmentation task model is used to perform image segmentation and recognition on each pre-processed image data to be detected, and the area data of each coral region is obtained, including: Use the YOLOv5 instance segmentation task model to perform coral segmentation on each pre-processed image data to obtain several coral area image data; identifying a coral bleaching area from the plurality of coral area image data; The areas of coral bleaching and non-bleaching were calculated respectively.

4. The method for diagnosing coral health status based on deep learning algorithm according to claim 3, characterized in that: The weighted average formula is used to calculate the coral status data corresponding to the area data of each coral region, and the coral health index is calculated according to the coral status data, including: Calculate the coral bleaching rate of each of the image data to be detected according to the coral bleaching area and the non-bleaching area; Calculate the weight coefficient of each image data to be detected based on the weighted average formula; The coral health index is calculated according to the bleaching rate of the coral and the weight coefficient.

5. A coral health status diagnosis system based on deep learning algorithm, characterized in that: The system comprises: The quality control module is used to collect image data of the same coral at different angles, and use the YOLOv5 target detection model to perform image screening on each of the image data to obtain each image data to be detected, including: based on the YOLOv5 target detection model, using the YOLOv5 convolutional neural network structure as the basic network, extracting feature maps of different scales of the input image data; performing convolution operations on the feature maps of different scales to obtain the category confidence and bounding box regression results corresponding to each position; judging whether there is a coral in the image data according to the category confidence, and determining the position and size of the coral according to the bounding box regression result; determining the detection result of the image data in which the coral is located in the central area as a qualified image, and determining the detection result of the image data in which there is no coral or the coral is not located in the central area as an unqualified image; when the detection result is an unqualified image, deleting the unqualified image data and displaying a prompt message; when the detection result is a qualified image, using the qualified image data as the image data to be detected; A detection module is used to perform image preprocessing on each of the image data to be detected, and use the YOLOv5 instance segmentation task model to perform image segmentation and recognition on each of the preprocessed image data to be detected to obtain the area data of each coral region; A calculation module, for respectively calculating the coral status data corresponding to the area data of each region of the coral based on a weighted average formula, and calculating the coral health index according to each coral status data, including: calculating the coral bleaching rate of each image data to be detected based on the area of ​​the coral bleaching region and the area of ​​the non-bleaching region; calculating the weight coefficient of each image data to be detected based on the weighted average formula; calculating the coral health index according to the coral bleaching rate and the weight coefficient; The display module is used to display the coral health index.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images