Aquatic species identification method and system integrating species text information and machine vision model

Abstract

The present application relates to the field of data processing, and in particular to a method and system for identifying aquatic species that integrates species text information and machine vision models, including: training a neural network model based on a species atlas data set of the same category to obtain a picture classification model; inputting the picture to be identified into the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model; inputting the picture to be identified into a local picture matching model to obtain the species probability of the picture to be identified based on the local picture matching model; calculating the fusion probability based on the species probability of the picture to be identified based on the picture classification model and the species probability of the picture to be identified based on the local picture matching model; and selecting a predetermined number of species corresponding to the fusion probability as the species identification result of the picture to be identified according to the fusion probability from large to small. The present application ensures the accuracy of aquatic species identification when there are fewer aquatic pictures.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and system for identifying aquatic species by integrating species text information and machine vision models. Background Art

[0002] At present, the country attaches great importance to the water ecological environment. As an important indicator to measure the health of water ecology, the evaluation of aquatic biodiversity has become an important task for all front-line environmental monitoring departments. Species identification, as a basis for calculating aquatic biodiversity, mainly relies on the experience of professionals and combines biological maps to achieve. The identification of algae in aquatic organisms has been achieved through the preliminary identification of images through machine vision-based target detection models or classification models, and then professionals review the model results. However, due to its high identification error rate, the later review still requires a lot of manpower and time costs.

[0003] Taking into account the differences between the same species in different river basins, the same water body including multiple categories of aquatic organisms, the small differences between different species in the same category, and the different appearances of the same species in different growth stages, experts currently need to look through a large amount of atlas data when identifying species. At the same time, different experts are needed to identify different categories of organisms, which is time-consuming, inefficient, and has a certain error rate.

[0004] The use of machine vision models is the preferred solution for automatic, fast and accurate species identification. Traditional visual models rely on building different model structures to extract species feature information. However, in order to obtain key features that can identify species, a dataset with a sufficient number of images and good image quality is required. However, for the current situation of aquatic organisms, such a dataset will be difficult to build for a long time in the future. Because there are fewer images of each species in the aquatic organism atlas, there are fewer image samples that can be trained. In addition, the postures of aquatic organisms in different images are different, and the subtle differences between species due to different resolutions cannot be reflected. For various reasons, the error rate of species identification based on machine vision models is high, making it difficult to apply in practice. In short, the current method of species identification can no longer meet the needs of monitoring aquatic organisms in major water bodies across the country.

[0005] In addition, the images of aquatic organisms used for species identification are basically standardized, so even after expanding the data set using data augmentation, the improvement in identification accuracy is not ideal. In short, when there are few images, even the most reasonable model structure is difficult to extract the key features of the species, which leads to a high identification error rate.

[0006] Therefore, how to ensure the accuracy of aquatic organism species identification when there are few aquatic organism pictures is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0007] The present application provides a method and system for identifying aquatic species by integrating species text information and machine vision models, which ensures the accuracy of aquatic species identification when there are few aquatic organism images.

[0008] In order to solve the above technical problems, this application provides the following technical solutions:

[0009] A method for identifying aquatic species by integrating species text information and machine vision models comprises the following steps: step S110, based on the text information describing the key features of the species, intercepting the image area corresponding to the key features of the species in the species picture to establish a species key feature picture data set, and based on the key feature picture data set of all species in the same category, forming a species atlas data set of the same category; step S120, constructing a neural network model, and training the neural network model based on the species atlas data set of the same category to obtain an image classification model; step S130, Input the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model; step S140, establish a local picture matching model; step S150, input the picture to be identified into the local picture matching model to obtain the species probability of the picture to be identified based on the local picture matching model; step S160, calculate the fusion probability according to the species probability of the picture to be identified based on the picture classification model and the species probability of the picture to be identified based on the local picture matching model; step S170, according to the fusion probability from large to small, select a predetermined number of species corresponding to the fusion probability as the species identification result of the picture to be identified.

[0010] In the above-mentioned aquatic species identification method integrating species text information and machine vision model, preferably, based on the text information describing the key features of the species, an image area corresponding to the key features in a certain species image is cut out, and the cut out key feature image is defined as the jth image of the key feature f of species i. All key feature images of species i are collected together to form the key feature image dataset D of species i i .

[0011] As described above, the aquatic species identification method that integrates species text information and machine vision models, wherein preferably, the key feature image datasets of all species in the same category are grouped together to form a species atlas dataset of the same category.

[0012] The aquatic species identification method integrating species text information and machine vision models as described above, wherein preferably, the species atlas data set of the same category is randomly divided into two categories, one is a support set and the other is a query set; a neural network model is constructed as a prototype network, and the key feature images in the support set are input into the neural network model to obtain the mapping of each key feature image; the mean of the mappings of all key feature images belonging to the same species is calculated, and is used as the mapping of the species through the prototype network; the key feature images in the query set are input into the neural network model to obtain the mapping of each key feature image, and the distance between the mapping of each key feature image and all species mappings is calculated; the probability of belonging to the species in the image is determined based on the distance between the mapping of the image and the species mapping in the image, and a loss function is defined based on the probability; the gradient is calculated based on the loss function, and the parameters of the neural network model are optimized using the gradient descent method, and the training is completed to obtain an image classification model.

[0013] In the above-mentioned aquatic species identification method integrating species text information and machine vision model, preferably, the image to be identified is input into the local image matching model, and the local image matching model traverses the key feature image dataset D of species i. i Each picture in Calculate each image The probability of matching the image to be identified In addition, the matching probabilities of all images representing the same key feature f in the species are averaged, and the mean probability is used as the probability of the key feature f of the species. The probability of the key feature f of the species Calculate the species probability s between species i and the image to be identified based on the local image matching model i .

[0014] A system for identifying aquatic species that integrates species text information and machine vision models, comprising: a data set forming module, a model building module, a probability calculation unit, a probability fusion module and an identification result selection module; the data set forming module intercepts the image area corresponding to the key features of the species in the species picture according to the text information describing the key features of the species to establish the species key feature picture data set, and forms the species atlas data set of the same category according to the key feature picture data set of all species in the same category; the model building module constructs a neural network model, and trains the neural network model according to the species atlas data set of the same category to obtain the picture key feature picture data set; Classification model; the probability calculation unit inputs the picture to be identified into the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model; the model construction module establishes a local picture matching model; the probability calculation unit inputs the picture to be identified into the local picture matching model to obtain the species probability of the picture to be identified based on the local picture matching model; the probability fusion module calculates the fusion probability according to the species probability of the picture to be identified based on the picture classification model and the species probability of the picture to be identified based on the local picture matching model; the identification result selection module selects a predetermined number of species corresponding to the fusion probability as the species identification result of the picture to be identified according to the fusion probability from large to small.

[0015] In the aquatic species identification system integrating species text information and machine vision models as described above, preferably, based on the text information describing the key features of the species, an image area corresponding to the key features in a certain species image is intercepted, and the intercepted key feature image is defined as the jth image of the key feature f of species i All key feature images of species i are collected together to form the key feature image dataset D of species i i .

[0016] In the aquatic species identification system that integrates species text information and machine vision models as described above, it is preferred that the key feature image datasets of all species in the same category are grouped together to form a species atlas dataset of the same category.

[0017] The aquatic species identification system that integrates species text information and machine vision models as described above, wherein, preferably, the species atlas data set of the same category is randomly divided into two categories, one is a support set, and the other is a query set; a neural network model is constructed as a prototype network, and the key feature images in the support set are input into the neural network model to obtain the mapping of each key feature image; the mean of the mappings of all key feature images belonging to the same species is calculated, and is used as the mapping of the species through the prototype network; the key feature images in the query set are input into the neural network model to obtain the mapping of each key feature image, and the distance between the mapping of each key feature image and all species mappings is calculated; the probability of belonging to the species in the image is determined based on the distance between the mapping of the image and the species mapping in the image, and the loss function is defined based on the probability; the gradient is calculated based on the loss function, and the parameters of the neural network model are optimized using the gradient descent method, and the training is completed to obtain the image classification model.

[0018] In the above-mentioned aquatic species identification system integrating species text information and machine vision model, preferably, the image to be identified is input into the local image matching model, and the local image matching model traverses the key feature image dataset D of species i. i Each picture in Calculate each image The probability of matching the image to be identified In addition, the matching probabilities of all images representing the same key feature f in the species are averaged, and the mean probability is used as the probability of the key feature f of the species. The probability of the key feature f of the species Calculate the species probability s between species i and the image to be identified based on the local image matching model i .

[0019] Compared with the above-mentioned background technology, the present application does not increase the amount of training image sample data. The key features of the species have been summarized in the species information description in text form. Therefore, the key features of the species can be extracted from the text information, and this information and the features extracted by machine vision can be integrated. In this way, there is no need to rely solely on the features extracted by the model to make judgments, thereby greatly improving the accuracy of identification; and the combined species description text information is open source, ensuring the feasibility of the technical route and reaching the level of practical application in business; in addition, compared with the time taken for manual identification, the present application greatly shortens the time for identification of aquatic species, while also reducing the time and difficulty of manual secondary review, and improving work efficiency; in addition, the technical route of the present application is applicable to species identification of all aquatic biological categories and has strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a flow chart of a method for identifying aquatic organism species by integrating species text information and a machine vision model provided in an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of forming a key feature image data set provided in an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of an aquatic species identification system that integrates species text information and a machine vision model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0025] Embodiment 1

[0026] like Figure 1 As shown, the present application provides a method for identifying aquatic species by integrating species text information and machine vision models, comprising the following steps:

[0027] Step S110: based on the text information describing the key features of the species, the image area corresponding to the key features of the species in the species picture is intercepted to establish a species key feature picture dataset, and based on the key feature picture datasets of all species in the same category, a species atlas dataset of the same category is formed;

[0028] like Figure 2 As shown in the figure, based on the text information describing the key features of the species (key features refer to certain unique features of the species, that is, features that are easily distinguished from other similar species), the image area corresponding to the key features in a certain species image is cut out, and the cut key feature image is defined as the jth image of the key feature f of species i All key feature images of species i are collected together to form the key feature image dataset D of species i i In addition, the key feature image datasets of all species in the same category are combined together to form a species atlas dataset of the same category.

[0029] Step S120, constructing a neural network model, and training the neural network model based on a species atlas dataset of the same category to obtain an image classification model;

[0030] This scheme uses the prototype network to implement a small sample classification model for aquatic species identification, so the species atlas dataset of the same category is randomly divided into two categories, one is the support set and the other is the query set. Both the support set and the query set must include all species of the same category.

[0031] A neural network model is constructed as a prototype network, and the key feature images in the support set are input into the neural network model to obtain the mapping of each key feature image. Then, the mean of the mapping of all key feature images belonging to the same species is calculated and used as the mapping of the species through the prototype network.

[0032] The key feature images in the query set are input into the neural network model to obtain the mapping of each key feature image, and the distance between the mapping of each key feature image and the mapping of all species is calculated. Specifically, the distance formula is as follows:

[0033]

[0034] in, represents the distance between the mapping of image j and the mapping of species i; F(j) represents the mapping of image j after the prototype network; Ci represents the mapping of species i after the prototype network; a, b represent the rows and columns of the mapping matrix respectively.

[0035] In order to ensure that the distance between the mapping of image j and the mapping of species i in the image is small, the probability of belonging to the species in the image is large; the distance between the mapping of other species is large, and the probability of belonging to other species is small, the loss function is defined according to the following formula:

[0036]

[0037] Among them, loss(j) represents the loss function; P(j,i) represents the probability that image j belongs to species i in the image; P(j,k) represents the probability that image j belongs to other species k; r represents the number of species in the category.

[0038] The gradient is calculated according to the loss function, and the parameters of the neural network model are optimized using the gradient descent method. After the training is completed, the image classification model is obtained.

[0039] Step S130, input the picture to be identified into the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model;

[0040] The picture K that needs to be identified is input into the established picture classification model F, and the probability of obtaining the species based on the picture classification model is:

[0041]

[0042] Among them, P(K,i) represents the probability that the species in image K belongs to species i; represents the distance between the mapping of image K and the mapping of species i; represents the distance between the mapping of image K and the mapping of species h; r represents the number of species in the category.

[0043] Step S140: establishing a local image matching model;

[0044] A picture matching model A based on SIFT (Scale-invariant feature transform) is established as a local picture matching model.

[0045] Through the local image matching model, traverse the key feature image dataset D i Pictures in The key feature image dataset D can be calculated i Each picture in The Euclidean distance between the key point descriptor of the image to be identified is converted into a matching probability.

[0046] Step S150, inputting the image to be identified into the local image matching model to obtain the species probability of the image to be identified based on the local image matching model;

[0047] The image to be identified is input into the local image matching model, and the local image matching model traverses the key feature image dataset D of species i. i Each picture in Calculate each image The probability of matching the image to be identified And the matching probabilities of all pictures representing the same key feature f in the species are averaged, and the mean probability is used as the probability of the key feature f of the species

[0048]

[0049] Where m is the number of images representing key feature f of species i.

[0050] The probability of the key feature f of the species Calculate the species probability s between species i and the image to be identified based on the local image matching model i , as follows:

[0051]

[0052] Among them, max(f) represents the maximum value of the elements in set f, min(f) represents the minimum value of the elements in set f, and median(f) represents the median value of the elements in set f; t1 represents the weight corresponding to max(f), t2 represents the weight corresponding to min(f), and t3 represents the weight corresponding to median(f), generally t1=0.4, t2=0.2, t3=0.4; k is the number of key characteristics of species i.

[0053] Step S160, calculating the fusion probability according to the species probability of the image to be identified based on the image classification model and the species probability of the image to be identified based on the local image matching model;

[0054] The specific calculation is based on the following formula:

[0055] Z i =a l *s i +(1-a l )*p i ; Among them, a l Represents the probability weight of the local image matching model, 0≤a l ≤1, the parameter value varies according to the different aquatic organisms; p i is the species probability of the image to be identified based on the image classification model.

[0056] Generally:

[0057] Aquatic organisms <![CDATA[a l ]]> Benthic animals 0.6 Aquatic plants 0.7 Fish 0.5 Phytoplankton 0.7 Zooplankton 0.4

[0058] Step S170: Select a predetermined number of species corresponding to the fusion probabilities from large to small as the species identification results of the image to be identified.

[0059] Specifically, N species with the largest fusion probability are selected as the identification results of the species in the image to be identified, where N=1, 5, 10.

[0060] Embodiment 2

[0061] like Figure 3 As shown, the present application provides an aquatic species identification system 300 that integrates species text information and machine vision models, including: a data set formation module 310, a model construction module 320, a probability calculation unit 330, a probability fusion module 340 and an identification result selection module 350.

[0062] The data set forming module 310 captures the image area corresponding to the key characteristics of the species in the species picture based on the text information describing the key characteristics of the species to establish a species key characteristic picture data set, and forms a species atlas data set of the same category based on the key characteristic picture data set of all species in the same category.

[0063] like Figure 2 As shown in the figure, based on the text information describing the key features of the species (key features refer to certain unique features of the species, that is, features that are easily distinguished from other similar species), the image area corresponding to the key features in a certain species image is cut out, and the cut key feature image is defined as the jth image of the key feature f of species i All key feature images of species i are collected together to form the key feature image dataset D of species i i In addition, the key feature image datasets of all species in the same category are combined together to form a species atlas dataset of the same category.

[0064] The model building module 320 builds a neural network model, and trains the neural network model based on a species atlas dataset of the same category to obtain an image classification model.

[0065] This scheme uses the prototype network to implement a small sample classification model for aquatic species identification, so the species atlas dataset of the same category is randomly divided into two categories, one is the support set and the other is the query set. Both the support set and the query set must include all species of the same category.

[0066] A neural network model is constructed as a prototype network, and the key feature images in the support set are input into the neural network model to obtain the mapping of each key feature image. Then, the mean of the mapping of all key feature images belonging to the same species is calculated and used as the mapping of the species through the prototype network.

[0067] The key feature images in the query set are input into the neural network model to obtain the mapping of each key feature image, and the distance between the mapping of each key feature image and the mapping of all species is calculated. Specifically, the distance formula is as follows:

[0068]

[0069] in, represents the distance between the mapping of image j and the mapping of species i; F(j) represents the mapping of image j after the prototype network; Ci represents the mapping of species i after the prototype network; a, b represent the rows and columns of the mapping matrix respectively.

[0070] In order to ensure that the distance between the mapping of image j and the mapping of species i in the image is small, the probability of belonging to the species in the image is large; the distance between the mapping of other species is large, and the probability of belonging to other species is small, the loss function is defined according to the following formula:

[0071]

[0072] Among them, loss(j) represents the loss function; P(j,i) represents the probability that image j belongs to species i in the image; P(j,k) represents the probability that image j belongs to other species k; r represents the number of species in the category.

[0073] The gradient is calculated according to the loss function, and the parameters of the neural network model are optimized using the gradient descent method. After the training is completed, the image classification model is obtained.

[0074] The probability calculation unit 330 inputs the picture to be identified into the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model.

[0075] The picture K that needs to be identified is input into the established picture classification model F, and the probability of obtaining the species based on the picture classification model is:

[0076]

[0077] Among them, P(K,i) represents the probability that the species in image K belongs to species i; represents the distance between the mapping of image K and the mapping of species i; represents the distance between the mapping of image K and the mapping of species h; r represents the number of species in the category.

[0078] The model building module 320 builds a local image matching model.

[0079] A picture matching model A based on SIFT (Scale-invariant feature transform) is established as a local picture matching model.

[0080] Through the local image matching model, traverse the key feature image dataset D i Pictures in The key feature image dataset D can be calculated i Each picture in The Euclidean distance between the key point descriptor of the image to be identified is converted into a matching probability.

[0081] The probability calculation unit 330 inputs the picture to be identified into the local picture matching model to obtain the species probability of the picture to be identified based on the local picture matching model.

[0082] The image to be identified is input into the local image matching model, and the local image matching model traverses the key feature image dataset D of species i. i Each picture in Calculate each image The probability of matching the image to be identified And the matching probabilities of all pictures representing the same key feature f in the species are averaged, and the mean probability is used as the probability of the key feature f of the species

[0083]

[0084] Where m is the number of images representing key feature f of species i.

[0085] The probability of the key feature f of the species Calculate the species probability s between species i and the image to be identified based on the local image matching model i , as follows:

[0086]

[0087] Among them, max(f) represents the maximum value of the elements in set f, min(f) represents the minimum value of the elements in set f, and median(f) represents the median value of the elements in set f; t1 represents the weight corresponding to max(f), t2 represents the weight corresponding to min(f), and t3 represents the weight corresponding to median(f), generally t1=0.4, t2=0.2, t3=0.4; k is the number of key characteristics of species i.

[0088] The probability fusion module 340 calculates the fusion probability according to the species probability of the image to be identified based on the image classification model and the species probability of the image to be identified based on the local image matching model.

[0089] The specific calculation is based on the following formula:

[0090] Z i =a l *s i +(1-a l )*p i ; Among them, a l Represents the probability weight of the local image matching model, 0≤a l ≤1, the parameter value varies according to the different aquatic organisms; p i is the species probability of the image to be identified based on the image classification model.

[0091] Generally:

[0092] Aquatic organisms <![CDATA[a l ]]> Benthic animals 0.6 Aquatic plants 0.7 Fish 0.5 Phytoplankton 0.7 Zooplankton 0.4

[0093] The identification result selection module 350 selects a predetermined number of species corresponding to the fusion probabilities from large to small as the species identification results of the image to be identified.

[0094] Specifically, N species with the largest fusion probability are selected as the identification results of the species in the image to be identified, where N=1, 5, 10.

[0095] The aquatic species identification method in the embodiment of the present application is a program segment executed by a CPU; the module in the embodiment of the present application is a program segment executed by a CPU, or a hardware unit with data processing capabilities.

[0096] This application does not increase the amount of training image sample data. The key features of the species have been summarized in text form in the species information description. Therefore, the key features of the species can be extracted from the text information, and this information and the features extracted by machine vision can be integrated. In this way, there is no need to rely solely on the features extracted by the model to make judgments, thereby greatly improving the accuracy of identification; and the combined species description text information is open source, ensuring the feasibility of the technical route and reaching the level of practical application in business; in addition, compared with the time taken for manual identification, this application greatly shortens the time for aquatic species identification, while also reducing the time and difficulty of manual secondary review, and improving work efficiency; in addition, the technical route of this application is applicable to species identification of all aquatic biological categories and has strong versatility.

[0097] Comparison of the accuracy and time of identification based on manual experience and machine models:

[0098]

[0099] Note: TopN means that the model gives N possible identification results for the species in the image. If the actual species in the image is among these N results, the model's identification is considered correct.

[0100] From the results, the recognition results of this application are basically close to the accuracy of manual work, but the time is greatly shortened. Compared with the ordinary machine classification model based on image features, the accuracy is greatly improved, which greatly reduces the time and manpower cost of manual secondary review.

[0101] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0102] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for identifying aquatic species by integrating species text information and machine vision models, characterized in that: The steps include: Step S110: based on the text information describing the key features of the species, the image area corresponding to the key features of the species in the species picture is intercepted to establish a species key feature picture dataset, and based on the key feature picture datasets of all species in the same category, a species atlas dataset of the same category is formed; Step S120, constructing a neural network model, and training the neural network model based on a species atlas dataset of the same category to obtain an image classification model; Step S130, input the picture to be identified into the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model; Step S140: establishing a local image matching model; Step S150, inputting the image to be identified into the local image matching model to obtain the species probability of the image to be identified based on the local image matching model; Step S160, calculating the fusion probability according to the species probability of the image to be identified based on the image classification model and the species probability of the image to be identified based on the local image matching model; Step S170: Select a predetermined number of species corresponding to the fusion probabilities from large to small as the species identification results of the image to be identified.

2. The aquatic organism species identification method integrating species text information and machine vision model according to claim 1 is characterized in that: Based on the text information describing the key features of the species, the image area corresponding to the key feature in a certain species image is intercepted, and the intercepted key feature image is defined as the jth image of the key feature f of species i All key feature images of species i are collected together to form the key feature image dataset D of species i i .

3. The aquatic species identification method integrating species text information and machine vision model according to claim 2 is characterized in that: The key feature image datasets of all species in the same category are combined together to form a species atlas dataset of the same category.

4. The aquatic organism species identification method integrating species text information and machine vision model according to any one of claims 1 to 3, characterized in that: The species atlas dataset of the same phylum is randomly divided into two categories, one is the support set and the other is the query set; Construct a neural network model as a prototype network, input the key feature images in the support set into the neural network model, and obtain the mapping of each key feature image; Calculate the mean of all key feature image mappings belonging to the same species and use it as the mapping of the species through the prototype network; Input the key feature images in the query set into the neural network model, obtain the mapping of each key feature image, and calculate the distance between the mapping of each key feature image and the mapping of all species; Determine the probability of belonging to the species in the image based on the distance between the image's mapping and the species mapping in the image, and define the loss function based on the probability; The gradient is calculated according to the loss function, and the parameters of the neural network model are optimized using the gradient descent method. After the training is completed, the image classification model is obtained.

5. The aquatic organism species identification method integrating species text information and machine vision model according to any one of claims 1 to 3, characterized in that: The image to be identified is input into the local image matching model, and the local image matching model traverses the key feature image dataset D of species i. i Each picture in Calculate each image The probability of matching the image to be identified In addition, the matching probabilities of all images representing the same key feature f in the species are averaged, and the mean probability is used as the probability of the key feature f of the species. The probability of the key feature f of the species Calculate the species probability s between species i and the image to be identified based on the local image matching model i .

6. An aquatic species identification system integrating species text information and machine vision models, characterized in that: include: Data set formation module, model building module, probability calculation unit, probability fusion module and identification result selection module; The data set forming module intercepts the image area corresponding to the key features of the species in the species picture based on the text information describing the key features of the species to establish the species key feature picture data set, and forms the species atlas data set of the same category based on the key feature picture data set of all species in the same category; The model building module builds a neural network model and trains the neural network model based on the species atlas dataset of the same category to obtain an image classification model; The probability calculation unit inputs the picture to be identified into the picture classification model to obtain the species probability of the picture to be identified based on the picture classification model; The model building module builds a local image matching model; The probability calculation unit inputs the image to be identified into the local image matching model to obtain the species probability of the image to be identified based on the local image matching model; The probability fusion module calculates the fusion probability based on the species probability of the image to be identified based on the image classification model and the species probability of the image to be identified based on the local image matching model; The identification result selection module selects a predetermined number of species corresponding to the fusion probabilities from large to small as the species identification results of the image to be identified.

7. The aquatic species identification system integrating species text information and machine vision models according to claim 6 is characterized in that: Based on the text information describing the key features of the species, the image area corresponding to the key feature in a certain species image is intercepted, and the intercepted key feature image is defined as the jth image of the key feature f of species i All key feature images of species i are collected together to form the key feature image dataset D of species i i .

8. The aquatic species identification system integrating species text information and machine vision models according to claim 7 is characterized in that: The key feature image datasets of all species in the same category are combined together to form a species atlas dataset of the same category.

9. The aquatic species identification system integrating species text information and machine vision models according to any one of claims 6 to 8, characterized in that: The species atlas dataset of the same phylum is randomly divided into two categories, one is the support set and the other is the query set; Construct a neural network model as a prototype network, input the key feature images in the support set into the neural network model, and obtain the mapping of each key feature image; Calculate the mean of all key feature image mappings belonging to the same species and use it as the mapping of the species through the prototype network; Input the key feature images in the query set into the neural network model, obtain the mapping of each key feature image, and calculate the distance between the mapping of each key feature image and the mapping of all species; Determine the probability of belonging to the species in the image based on the distance between the image's mapping and the species mapping in the image, and define the loss function based on the probability; The gradient is calculated according to the loss function, and the parameters of the neural network model are optimized using the gradient descent method. After the training is completed, the image classification model is obtained.

10. The aquatic species identification system integrating species text information and machine vision models according to any one of claims 6 to 8, characterized in that: The image to be identified is input into the local image matching model, and the local image matching model traverses the key feature image dataset D of species i. i Each picture in Calculate each image The probability of matching the image to be identified In addition, the matching probabilities of all images representing the same key feature f in the species are averaged, and the mean probability is used as the probability of the key feature f of the species. The probability of the key feature f of the species Calculate the species probability s between species i and the image to be identified based on the local image matching model i .

Images