Benthic organism two-stage automatic identification method based on computer vision model

Abstract

The invention discloses a benthic organism two-stage automatic identification method based on a computer vision model, and the method comprises the following steps: S1, building a simple monitoring system; s2, screening out a benthic organism image library in a specific area in combination with river network data, and taking the benthic organism image library as a dynamic reference library for subsequent comparison; s3, the collected monitoring images are input into the trained Dinov2 model; s4, constructing a comprehensive rule set for final identification; and S5, adopting a plurality of pre-trained computer vision models. S6, realizing accurate classification of'family / genus' levels; and S7, formulating a new classification rule into the classification rule general library. The system supports dynamic loading of benthic organism databases and detection models of different rivers according to the belonging and longitude and latitude collected and returned by equipment, a reliable and credible benthic organism identification result is obtained through an adjustable step-by-step screening mode, and maintenance is facilitated when new species in the region are found. And different rivers can be conveniently and quickly deployed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of biological identification based on visual models, and in particular relates to a two-stage automatic identification method for benthic organisms based on a computer visual model. BACKGROUND

[0002] Traditional benthic organism identification methods usually rely on manual sampling and laboratory analysis, which is not only time-consuming and labor-intensive, but also difficult to achieve large-scale, real-time data collection and processing. In recent years, with the development of information technology, especially the computer vision technology represented by artificial neural networks, relatively complex image target recognition tasks can be completed. However, in the identification of benthic organisms, it is difficult to achieve accurate classification at the genus / species level due to the similarity between different categories and the lack of high-quality labeled image datasets. Moreover, due to the black-box nature of the model, retraining the model is often the only way to implement compatibility with new species and eliminate false results, which is a high-cost method. Therefore, a more efficient and easy-to-adjust method is needed to improve the accuracy of identification and reduce the cost of deployment and maintenance.

[0003] Therefore, the present application is proposed. SUMMARY

[0004] To solve the above technical problems, the basic idea of the technical solution of the present application is:

[0005] A two-stage automatic identification method for benthic organisms based on a computer visual model, comprising the following steps:

[0006] Step S1: Build a simple monitoring system to return geographic location information and image information.

[0007] Step S2: Use river network data to filter out a benthic organism image library in a specific area as a dynamic library for subsequent comparison, and train a visual model based on the Dinov2 architecture. The model is pre-trained on a large number of labeled benthic organism images and obtains high-level visual features with semantic information.

[0008] Step S3: Input the collected monitoring images into the trained Dinov2 model and perform similarity matching with the dynamic library to preliminarily classify the benthic organisms to the class / order level.

[0009] Wherein, the similarity uses cosine similarity, and the calculation formula is:

[0010]

[0011] Step S4: According to the rough classification result in S3, filter the relevant classification rules from the existing benthic organism classification rule library to construct a comprehensive rule set for final identification.

[0012] Step S5: using multiple pre-trained visual models, detailed biological trait features are extracted from the monitoring images in terms of morphology, color distribution, texture features, etc.

[0013] Each visual model is trained individually through a large number of labeled tags, and each training model includes but is not limited to benthic organism pictures on the training data set.

[0014] Step S6: combining the comprehensive rule set in S4 with the biological trait feature summary information obtained in S5 to make a comprehensive decision and realize accurate classification at the genus / species level.

[0015] Step S7: for samples that are difficult to classify, after manual review, new pictures are added to the total picture library, and new classification rules are developed in the total classification rule library.

[0016] As a preferred embodiment of the present application, the range screened in step S2 is the river section where the monitoring point is located and the upstream part thereof; the large number of labeled categories include the classification results of multiple levels of the pictures of the organisms obtained according to expert identification methods and DNA molecular identification methods; the high-level visual features of semantic information refer to the output of the Dinov2 model after image semantic segmentation.

[0017] As a preferred embodiment of the present application, in step S4, the classification rules in the benthic organism classification rule library are obtained according to existing literature and classification practices, and are continuously supplemented and modified according to actual needs; the comprehensive rule set refers to the biological trait classification rules for distinguishing species at a finer level under a specific classification level after integrating species.

[0018] As a preferred embodiment of the present application, in step S1, the reference dynamic picture library is screened out according to the geographical position, and the similarity matching is completed by using a powerful computer vision model as a black box model to obtain a rough classification result; in the second step, the comprehensive rules are obtained in the benthic organism classification rules according to the rough classification result, a plurality of biological traits are detected by using a plurality of simple computer vision models, and finally the biological class is determined by comprehensive judgment.

[0019] As a preferred embodiment of the present application, the comprehensive rule set integrates expert knowledge and can be dynamically adjusted to adapt to new biological trait features and classification standards; the system supports real-time updating and online learning functions, allows users to upload newly discovered benthic organism images and related information, and continuously optimizes the model performance.

[0020] Compared with the prior art, the present application has the following beneficial effects:

[0021] 1. The present application focuses on the precision and efficiency of image-based automatic identification method, which can greatly save the time and cost of manual sampling and laboratory analysis method.

[0022] 2. Through the two-stage design of "rough classification + fine classification", redundant calculation is reduced to meet the needs of near real-time processing.

[0023] 3. The method has dynamic adaptability and scalability. When new species or classification standards change, the entire model does not need to be retrained. Only new images and rules need to be added. If necessary, only new biological trait feature recognition models need to be added. At the same time, each model can be replaced by the latest pre-trained model. Through this two-stage automatic identification method, the efficiency, accuracy and adaptability of benthic organism identification are significantly improved, providing a feasible automated solution for large-scale ecological monitoring.

[0024] The specific embodiments of the present application will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0025] In the drawings:

[0026] Fig. 1 is a step diagram of the method of the present application;

[0027] Fig. 2 is a flowchart of the method of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments will be described clearly and completely below with reference to the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application.

[0029] Embodiment 1:

[0030] As shown in Figs. 1-2 , a two-stage automatic identification method for benthic organisms based on a computer vision model:

[0031] The first stage is the construction of a dynamic reference library under geographical constraints and rough classification. Specifically, it includes:

[0032] S1, the monitoring terminal should be packaged with a three-proof (waterproof, dustproof, and corrosion-proof) shell, integrated. It needs to include a high-definition underwater imaging module, an RTK-GPS positioning unit (positioning error ≤10 cm). And adjust the shooting frequency (default 30 minutes / once) according to the hydrological period (such as wet season / dry season). The original image data is wirelessly transmitted to the cloud after being encoded by H.265.

[0033] S2, get the region associated with the monitoring point by a geographic screening algorithm. Input the coordinates (x, y) of the monitoring point, and the screening range is the river section where the monitoring point is located and the upstream part thereof. Extract relevant images from the benthic organism image library as the dynamic reference library according to the monitoring point associated region.

[0034] S3, perform similarity matching. The cosine similarity is used in the specific matching algorithm. The cosine similarity of the monitoring image and each image in the screened dynamic reference library is calculated. The specific formula is as follows:

[0035]

[0036] wherein, A i represents the i-th element of the monitoring image, B i represents the i-th element of the image in the dynamic reference library.

[0037] If there is no category with a similarity higher than the set threshold, the candidate list is all categories.

[0038] The second stage is the identification of biological characteristics under the rule driving and fine classification.

[0039] S4, generate a dynamic rule set. According to the coarse classification result in S3, the possible species are regarded as usually multiple. The related classification rules in the existing benthic organism classification rule library are screened, and the biological characteristic feature model that needs to be identified subsequently is activated according to the screened related rules. If it is marked as “unknown”, all biological characteristic feature identification models are activated.

[0040] The classification rules in the benthic organism classification rule library are obtained according to the existing literature and classification practice, and are continuously supplemented and modified according to actual needs. The comprehensive rule set refers to the biological characteristic classification rules that can distinguish the species to a more fine level under a specific classification level after the possible species are comprehensively classified. In some cases, the picture may cause the visual model to make wrong judgments due to blurring or angle problems, so the “unknown” label is included in the specific rule as a fault-tolerant, which should be considered in the classification rule.

[0041] Step S5: adopt multiple pre-trained visual models to extract detailed biological characteristics from the monitoring image in terms of morphology, color distribution, and texture features, etc.

[0042] The visual model adopts the open source YOLO11. Each YOLO11 is trained separately by a large number of labeled tags. Since each model is only responsible for the identification of a single biological characteristic, it can be trained using benthic organism pictures, including but not limited to.

[0043] Step S6: Combine the comprehensive rule set in S4 and the biological trait feature summary information obtained in S5 to make a comprehensive decision, realizing accurate classification at the "genus / species" level.

[0044] The comprehensive decision is made by weighted scoring. Each category is weighted and the top N categories with scores greater than a certain threshold are selected as the final result.

[0045] Step S7: For samples that are difficult to classify, i.e., there is no category with a score greater than a certain threshold. Then trigger the manual review mechanism. After manual review, add new pictures to the total picture library and develop new classification rules to the total classification rule library.

[0046] Embodiment 2:

[0047] Based on the above embodiment, the difference between this embodiment is: a two-stage automatic identification method of benthic organisms based on computer vision model:

[0048] I. Dynamic benchmark library construction and coarse classification under geographical constraints.

[0049] 1. Determine the geographical location of the monitoring point.

[0050] Return the geographical coordinates from the monitoring system.

[0051] 2. Determine the associated area of the monitoring point.

[0052] Determined according to the river network information.

[0053] 3. Determine the dynamic benchmark library of benthic pictures.

[0054] Determined according to the associated area of the monitoring point.

[0055] 4. Use the Dinov2 model to perform similarity matching to determine a high-confidence coarse classification candidate list. If there is no category with a similarity higher than the set threshold, the candidate list is all categories.

[0056] II. Biological trait feature recognition and fine classification under rule-driven.

[0057] 1. According to the high-confidence coarse classification candidate list, filter the classification rules to determine the comprehensive rules for benthic organism classification.

[0058] The rule base stores classification rules in a tree structure, and a node contains: a conditional expression (such as "abdominal segment number ≥ 8 AND gill filament presents comb-like"); logical operators (AND / OR / NOT); pointer to child node; confidence weight (determined by expert score and historical accuracy).

[0059] 2. Input the image from the detection system to the biological trait detection module to obtain the biological trait feature summary.

[0060] The biological trait detection module is mainly based on YOLO11, and each model is responsible for the identification of a single biological trait feature.

[0061] After comprehensive decision-making according to the biological trait feature summary and the comprehensive rules of biological classification, a fine classification candidate list is obtained as the final identification result.

[0062] The implementation principle of the two-stage automatic identification method of benthic organisms based on computer vision model in this embodiment is as follows: in the first stage, dynamic reference library construction and coarse classification under geographical constraints. Specifically, it includes:

[0063] S1, the monitoring terminal should be packaged with a three-proof (waterproof, dustproof, and corrosion-proof) shell and integrated. It needs to contain a high-definition underwater imaging module, an RTK-GPS positioning unit (positioning error ≤10 cm). And adjust the shooting frequency (default 30 minutes / time) according to the hydrological period (such as the wet season / dry season). The original image data is wirelessly transmitted to the cloud after being encoded by H.265.

[0064] S2, obtain the region associated with the monitoring point through the geographical screening algorithm. Input the monitoring point coordinates (x, y), and the screening range is the river section where the monitoring point is located and the upstream part. According to the monitoring point associated region, extract relevant images from the benthic organism image library as the dynamic reference library.

[0065] S3, similarity matching, the specific matching algorithm uses cosine similarity, calculates the cosine similarity of each image in the dynamic reference library filtered in S3 and the monitoring image, if there is no category with similarity higher than the set threshold, then the candidate list is all categories.

[0066] The second stage is biological trait feature identification and fine classification under the rule driving.

[0067] S4, generate a dynamic rule set. According to the coarse classification result in S3, it is regarded as possible species, usually multiple. In the existing benthic organism classification rule library, filter the related classification rules, and activate the biological trait feature model that needs to be identified according to the filtered related rules. If it is marked as "unknown", activate all biological trait feature identification models.

[0068] The classification rules in the benthic organism classification rule library are obtained according to existing literature and classification practice, and are constantly supplemented and modified according to actual needs. The comprehensive rule set refers to the biological trait feature classification rules that can distinguish species to a more detailed level after considering possible species under a specific classification level. In some cases, pictures may be blurred or have angle problems, causing the visual model to make wrong judgments, so in specific rules, the "unknown" label is included as a fault-tolerant, which should be considered in the classification rules.

[0069] Step S5: Using multiple pre-trained visual models, detailed biological trait features are extracted from the monitoring images in terms of morphology, color distribution, texture features, etc.

[0070] The visual model uses open source YOLO11. Each YOLO11 is trained separately by a large number of labeled tags. Since each model is only responsible for the identification of a single biological trait feature, it can be trained using images of benthic organisms, including but not limited to.

[0071] Step S6: Comprehensive decision-making is made by combining the comprehensive rule set in S4 and the biological trait feature summary information obtained in S5, realizing accurate classification at the "family / genus" level.

[0072] The comprehensive decision-making is made by using a weighted score method. Each category is weighted and the final result is obtained by adding the top N categories greater than a certain threshold.

[0073] Step S7: For samples that are difficult to classify, i.e. there is no category greater than a certain threshold. Then trigger the manual review mechanism. After manual review, new pictures are added to the total picture library and new classification rules are added to the total classification rule library.

Claims

1. A two-stage automatic benthic organism identification method based on a computer vision model, characterized in that, The method comprises the following steps: Step S1: Establish a simple monitoring system to return geographic location information and image information. Step S2: Use river network data to screen out a benthic organism image library in a specific area as a dynamic library for subsequent comparison, and train a visual model based on the Dinov2 architecture, which is pre-trained through a large number of annotated benthic organism images and obtains high-level visual features with semantic information. Step S3: Input the collected monitoring images into the trained Dinov2 model and perform similarity matching with the dynamic library to preliminarily classify the benthic organisms to the class / order level. Wherein, the similarity uses cosine similarity, and the calculation formula is: Step S4: According to the rough classification result in S3, screen the related classification rules in the existing benthic organism classification rule library to construct a comprehensive rule set for final identification. Step S5: Use multiple pre-trained visual models to extract detailed biological trait features from the monitoring images in terms of morphology, color distribution, and texture features. Wherein, each visual model is trained separately through a large number of annotated labels, and each training model includes but is not limited to benthic organism pictures on the training data set. Step S6: Combine the comprehensive rule set in S4 with the biological trait feature summary information obtained in S5 to make a comprehensive decision and achieve accurate classification at the family / genus level. Step S7: For samples that are difficult to classify, manually review, add new pictures to the picture library, and develop new classification rules to the classification rule library.

2. The two-stage automatic benthic organism identification method based on a computer vision model according to claim 1, wherein, The range screened in step S2 is the river section where the monitoring point is located and the upstream part; the large number of annotated categories include the classification results of multiple levels obtained by expert identification methods and DNA molecular identification methods; the high-level visual features with semantic information refer to the output of the Dinov2 model after image semantic segmentation belonging to a single biological instance.

3. The two-stage automatic benthic organism identification method based on a computer vision model according to claim 1, wherein, In step S4, the classification rules in the benthic organism classification rule library are obtained according to existing literature and classification practice, and are continuously supplemented and modified according to actual needs; the comprehensive rule set refers to the biological trait classification rules for distinguishing species to a more detailed level after integrating species under a specific classification level.

4. The two-stage automatic benthic organism identification method based on a computer vision model according to claim 1, wherein, In step S1, the reference dynamic picture library is screened out from the geographic location, and the similarity matching is completed by a powerful computer vision model as a black box model to obtain the rough classification result; in the second step, the comprehensive rules are obtained from the benthic organism classification rules according to the rough classification result, and multiple biological traits are detected through multiple simple computer vision models to finally determine the biological class.

Images