Fish image semantic description model training method and system and electronic equipment
By constructing and transforming clustering dictionaries to determine the feature vectors of fish images, the problems of few data sets, categories imbalance and missed detection in fish recognition technology are solved, the learning and generalization ability of fish semantic description models are improved, and the identification and description accuracy of new samples is enhanced.
Patent Information
- Application Number
- CN202510276037.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-10
- Publication Date
- 2025-06-13
AI Technical Summary
In the existing fish recognition technology, there are many types of fish and diverse living environments, resulting in a small number of data sets and an imbalance in categories. The neural network model lacks feature extraction capabilities when processing small targets, which is prone to missed detection, which limits the learning ability and generalization ability of fish image recognition models.
A training method for the semantic description model of fish image is proposed. By obtaining the fish name corresponding to the target fish image and the fish name corresponding to the fish image in the training set, a clustering dictionary is constructed and converted into a collection dictionary, the target feature vector of the target fish image is determined, and the descriptive words are determined and the model training is realized.
In the absence of directly related images, effective model training is carried out with the help of information about the approximate species, increasing the amount of data, alleviating the problem of category imbalance, avoiding missed detection, improving the learning ability and generalization performance of fish semantic description models, and improving the accuracy of identification and description of new samples.
Smart Images
Figure CN120147780A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image technology, and in particular, to a method, a system and an electronic device for training a semantic description model of fish images. Background Art
[0002] In current fish recognition technology, artificial intelligence image semantic description and training based on neural networks are the main technical means of current image semantic description. It can automatically extract features from a large amount of image data and convert these features into natural language descriptions, improving the efficiency and accuracy of image processing.
[0003] However, due to the large variety of fish species, with different morphologies and diverse living environments, it increases the difficulty of data collection, resulting in a small dataset. At the same time, due to the large number of images of some common or easily accessible fish, while the number of images of some rare or difficult-to-obtain fish is relatively small, there is a problem of class imbalance. At the same time, when existing neural network models process small targets, due to insufficient feature extraction ability or limitations of target detection algorithms, missed detection phenomena often occur.
[0004] The above problems will all limit the learning ability and generalization ability of the fish image recognition model, making it difficult to accurately identify and describe new samples. Summary of the Invention
[0005] The present invention aims to solve at least one of the technical problems existing in the prior art.
[0006] To this end, one object of the present invention is to provide a method for training a semantic description model of fish images, which can solve the problems of small dataset size, dataset class imbalance and missed detection phenomena faced in the training of existing fish semantic description models, improve the learning ability and generalization performance of the fish semantic description model, and further improve the accuracy of identifying and describing new samples.
[0007] To this end, a second object of the present invention is to provide a training system for a semantic description model of fish images.
[0008] To this end, a third object of the present invention is to provide an electronic device.
[0009] To this end, a fourth object of the present invention is to provide a computer-readable storage medium.
[0010] To achieve the above object, an embodiment of the first aspect of the present invention provides a method for training a semantic description model of fish images, the method comprising the following steps: obtaining a first fish name corresponding to a target fish image and a second fish name corresponding to the fish images included in the training set of the fish image semantic description model; constructing a clustering dictionary based on the first fish name and the second fish name; converting the clustering dictionary into a set dictionary; determining a target feature vector corresponding to the target fish image based on the first fish name and the set dictionary; and determining a description word corresponding to the target fish image based on the target feature vector, so as to complete the training process of the fish image semantic description model.
[0011] According to the method for training a semantic description model of fish images of the embodiments of the present invention, by constructing a clustering dictionary based on the fish name corresponding to the target fish image and the fish names corresponding to the fish images included in the training set and converting it into a set dictionary, a target feature vector corresponding to the target fish image can be determined, and then a description word corresponding to the target fish image can be determined, realizing effective model training by means of information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of unbalanced dataset categories. At the same time, the occurrence of missed detection can be effectively avoided, the learning ability and generalization performance of the fish semantic description model can be improved, and further the accuracy of recognizing and describing new samples can be improved.
[0012] In addition, the method for training a semantic description model of fish images according to the embodiments of the present invention may further have the following additional technical features: In some examples, the constructing a clustering dictionary based on the first fish name and the second fish name includes: determining a list of apparent features corresponding to the second fish name; grouping the second fish name based on the list of apparent features; and constructing the clustering dictionary based on the first fish name and the grouped second fish name.
[0013] In some examples, the constructing the clustering dictionary based on the first fish name and the grouped second fish name includes: using the grouped second fish name as the primary key and the normalized value of the distance between the grouped second fish name and the first fish name in a preset evolutionary classification tree as the content to construct the clustering dictionary.
[0014] In some examples, the converting the clustering dictionary into a set dictionary includes: taking the complement of the list of apparent features to obtain a list of apparent feature groups; and converting the clustering dictionary into a set dictionary based on the list of apparent feature groups.
[0015] In some examples, converting the clustering dictionary into a set dictionary based on the apparent feature grouping list includes: constructing the set dictionary with the feature names in the apparent feature grouping list as the primary keys and the clustering dictionaries within the groups corresponding to the feature names as the content.
[0016] In some examples, determining the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary includes: obtaining the target feature name corresponding to the first fish name; querying the set dictionary based on the target feature name to determine the clustering dictionary corresponding to the target feature name; and determining the target feature vector based on the clustering dictionary corresponding to the target feature name.
[0017] In some examples, determining the target feature vector based on the clustering dictionary corresponding to the target feature name includes: obtaining the fish image and the feature confidence corresponding to the clustering dictionary corresponding to the target feature name; determining the corresponding edge feature vector based on the fish image; and obtaining the target feature vector based on the inner product of the edge feature vector and the feature confidence.
[0018] To achieve the above object, an embodiment of the second aspect of the present invention provides a training system for a fish image semantic description model. The training system for the fish image semantic description model includes: an acquisition module, configured to acquire the first fish name corresponding to the target fish image and the second fish name corresponding to the fish images included in the training set of the fish image semantic description model; a construction module, configured to construct a clustering dictionary based on the first fish name and the second fish name; a conversion module, configured to convert the clustering dictionary into a set dictionary; a first determination module, configured to determine the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary; and a second determination module, configured to determine the description word corresponding to the target fish image based on the target feature vector, so as to complete the training process of the fish image semantic description model.
[0019] According to the training system for the fish image semantic description model of the present invention, by constructing a clustering dictionary based on the fish name corresponding to the target fish image and the fish names corresponding to the fish images included in the training set and converting it into a set dictionary, the target feature vector corresponding to the target fish image can be determined, and then the description word corresponding to the target fish image can be determined, realizing effective model training by means of the information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of dataset category imbalance. At the same time, the occurrence of missed detection can be effectively avoided, the learning ability and generalization performance of the fish semantic description model can be improved, and further the accuracy of identifying and describing new samples can be improved.
[0020] To achieve the above object, an embodiment of the third aspect of the present invention discloses an electronic device, which includes: the disciplinary classification system of fishery literature described in the embodiment of the second aspect of the present invention; or, a processor, a memory, and a training program of a fish image semantic description model stored on the memory and executable on the processor. When the training program of the fish image semantic description model is executed by the processor, it implements the training method of the fish image semantic description model described in the embodiment of the first aspect of the present invention.
[0021] According to the electronic device of the embodiment of the present invention, a clustering dictionary is constructed by using the fish names corresponding to the target fish images and the fish names corresponding to the fish images included in the training set, and it is converted into a set dictionary, so that the target feature vector corresponding to the target fish image can be determined, and then the description words corresponding to the target fish image can be determined. It realizes effective model training by means of the information of approximate species in the case of lack of directly relevant images, increases the available data volume and alleviates the effect of class imbalance of the data set. At the same time, it can effectively avoid the occurrence of missed detection phenomena, improve the learning ability and generalization performance of the fish semantic description model, and then improve the accuracy of recognition and description of new samples.
[0022] To achieve the above object, an embodiment of the fourth aspect of the present invention discloses a computer-readable storage medium, on which a training program of a fish image semantic description model is stored. When the training program of the fish image semantic description model is executed by the processor, it implements the training method of the fish image semantic description model described in the embodiment of the first aspect of the present invention.
[0023] According to the computer-readable storage medium of the embodiment of the present invention, when the disciplinary classification program of fishery literature stored thereon is executed by the processor, a clustering dictionary is constructed by using the fish names corresponding to the target fish images and the fish names corresponding to the fish images included in the training set, and it is converted into a set dictionary, so that the target feature vector corresponding to the target fish image can be determined, and then the description words corresponding to the target fish image can be determined. It realizes effective model training by means of the information of approximate species in the case of lack of directly relevant images, increases the available data volume and alleviates the effect of class imbalance of the data set. At the same time, it can effectively avoid the occurrence of missed detection phenomena, improve the learning ability and generalization performance of the fish semantic description model, and then improve the accuracy of recognition and description of new samples.
[0024] The additional aspects and advantages of the present invention will be partly given in the following description, partly will become obvious from the following description, or will be understood through the practice of the present invention. Description of the Drawings
[0025] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, wherein: Figure 1 is a schematic flowchart of a method for training a fish image semantic description model according to an embodiment of the present invention; Figure 2 is a schematic structural diagram of a training system for a fish image semantic description model according to an embodiment of the present invention.
[0026] Reference numerals: Training system for fish image semantic description model - 100; Acquisition module - 110; Construction module - 120; Conversion module - 130; First determination module - 140; Second determination module - 150. Detailed implementation manners
[0027] In order to be able to understand the features and technical content of the embodiments of the present invention in more detail, the implementation of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The accompanying drawings are only for reference and illustration purposes and are not used to limit the embodiments of the present invention. In the following technical description, for the sake of explanation, numerous details are provided to give a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be shown in a simplified manner to simplify the drawings.
[0028] Next, refer to Figure 1 - Figure 2 to describe a method and device for training a fish image semantic description model according to an embodiment of the present invention.
[0029] Figure 1 is a schematic flowchart of a method for training a fish image semantic description model according to an embodiment of the present invention. With reference to Figure 1 as shown, the method for training a fish image semantic description model includes the following steps: Step S1: Obtain the first fish name corresponding to the target fish image and the second fish name corresponding to the fish images included in the training set of the fish image semantic description model.
[0030] Specifically, when performing semantic description training on fish images, the name of the fish in the image to be processed that needs to be semantically described, that is, the first fish name corresponding to the target fish image, can be obtained. At the same time, the second fish name corresponding to the fish images included in the training set can be obtained. The obtaining method includes, but is not limited to, querying the fish image annotation database according to the fish image.
[0031] Step S2: Construct a clustering dictionary based on the first fish name and the second fish name.
[0032] Specifically, after determining the first fish name and the second fish name, a suitable clustering algorithm can be selected. For example, the clustering algorithm includes K-means, hierarchical clustering, etc. The first fish name and the second fish name are clustered, and according to the clustering result, a clustering dictionary is generated. Among them, the primary key of the clustering dictionary includes but is not limited to the first fish name or the second fish name, and the content of the clustering dictionary includes but is not limited to the similarity information between the two fish names.
[0033] Step S3: Convert the clustering dictionary into a set dictionary.
[0034] Specifically, after constructing the clustering dictionary, the clustering dictionary can be further abstracted and organized, that is, the clustering dictionary is converted into a set dictionary, including but not limited to organizing fish names and their related information according to specific apparent features or classification criteria (such as body shape, scale shape, etc.). For example, the criteria for classification can be determined first. For example, "body shape", "scale shape", etc. can be selected as the classification criteria. Then, the classification criteria can be combined with the clustering dictionary correspondingly to convert the clustering dictionary into a set dictionary.
[0035] Step S4: Determine the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary.
[0036] Specifically, after obtaining the set dictionary, the target feature vector corresponding to the target fish image can be determined based on the first fish name and the set dictionary. For example, the set dictionary can be queried according to the first fish name to determine the information matching the first fish name. For example, the feature vector corresponding to the first fish name and the feature confidence corresponding to the first fish name. The target feature vector corresponding to the target fish image is determined based on the above information.
[0037] Step S5: Determine the description word corresponding to the target fish image based on the target feature vector to complete the training process of the fish image semantic description model.
[0038] Specifically, since the target feature vector can be mapped to corresponding features, after determining the target feature vector, the description word matching the target fish image can be determined based on the target feature vector, that is, the target fish image is associated with the description word corresponding to the target feature vector to form a semantic description mapping, thereby completing the training process of the fish image semantic description model.
[0039] Therefore, in the above training method of the fish image semantic description model, a clustering dictionary is constructed based on the fish name corresponding to the target fish image and the fish names corresponding to the fish images included in the training set, and it is converted into a set dictionary, so that the target feature vector corresponding to the target fish image can be determined. Furthermore, the description words corresponding to the target fish image can be determined, realizing effective model training by means of the information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of class imbalance in the data set. At the same time, the occurrence of missed detection phenomena can be effectively avoided, the learning ability and generalization performance of the fish semantic description model can be improved, and further the accuracy of recognizing and describing new samples can be improved.
[0040] In an embodiment of the present invention, constructing a clustering dictionary based on the first fish name and the second fish name includes: determining the corresponding apparent feature list based on the second fish name; grouping the second fish names based on the apparent feature list; constructing a clustering dictionary based on the first fish name and the grouped second fish names.
[0041] Specifically, when constructing the clustering dictionary, based on the second fish name, the descriptive words of the apparent features corresponding to the second fish name can be retrieved from fish literature such as fish catalogs, and an apparent feature list can be established based on the obtained descriptive words. For example, body shape, scale shape, color pattern, etc.
[0042] Furthermore, the second fish names can be classified based on the apparent feature list to form different groups. For example, by comparing the apparent feature lists of each fish, fish with similar features can be found. For example, all fish with the same apparent body shape can be grouped into one group, and all fish with the same apparent scale shape can be grouped into one group.
[0043] Furthermore, a clustering dictionary can be constructed by combining the first fish name corresponding to the target fish image and the grouped second fish names. For example, for the fish in each group, the similarity scores between them and the first fish name corresponding to the target fish image can be calculated, and a clustering dictionary can be constructed based on the similarity scores.
[0044] In an embodiment of the present invention, constructing a clustering dictionary based on the first fish name and the grouped second fish names includes: using the grouped second fish names as the primary key and the normalized values of the distances between the grouped second fish names and the first fish name in a preset evolutionary classification tree as the content to construct the clustering dictionary.
[0045] Specifically, after grouping the second fish names, clustering can be performed on each group respectively based on the first fish name and the grouped second fish names to construct a clustering dictionary. For example, for the fish in each group, the normalized values of the distances between each fish (the grouped second fish name) and the target fish (the first fish name) in the five-level evolutionary classification tree of phylum, class, order, genus, and species can be calculated respectively, and these values are used as the content of the clustering dictionary, with the second fish name within its corresponding group as the primary key, to complete the construction of the clustering dictionary.
[0046] In an embodiment of the present invention, converting the clustering dictionary into a set dictionary includes: taking the complement of the apparent feature list to obtain an apparent feature grouping list; Converting the clustering dictionary into a set dictionary based on the apparent feature grouping list.
[0047] Specifically, in the process of converting the clustering dictionary into a set dictionary, the complement of the apparent feature list can be taken, that is, the apparent feature list is combined and duplicates are removed to obtain an apparent feature grouping list. For example, assume there are the following second fish names and their apparent feature lists: fish names = [Nibea albiflora, Larimichthys crocea, Larimichthys polyactis, Sardina pilchardus], apparent feature lists = [Nibea albiflora: body shape, scale shape], [Larimichthys crocea: body shape, color pattern], [Larimichthys polyactis: scale shape, color pattern], [Sardina pilchardus: scale shape, tail shape], then the apparent feature grouping list is: [body shape, scale shape, color pattern, tail shape].
[0048] Furthermore, the clustering dictionary can be converted into a set dictionary based on the apparent feature grouping list, that is, the apparent feature grouping list is combined with the clustering dictionary correspondingly to convert the clustering dictionary into a set dictionary.
[0049] In an embodiment of the present invention, converting the clustering dictionary into a set dictionary based on the apparent feature grouping list includes: using the feature names in the apparent feature grouping list as the primary key and the clustering dictionary within the group corresponding to the feature name as the content to construct a set dictionary.
[0050] Specifically, after determining the apparent feature grouping list, the clustering dictionary can be converted into a set dictionary based on the apparent feature grouping list. For example, the apparent feature grouping list is the result of combining and removing duplicates from the apparent feature list, that is, the feature names in the apparent feature grouping list exist in the apparent feature list. Since the clustering basis of the clustering dictionary is the grouped second fish names, that is, each clustering dictionary corresponds to a group, and the name corresponding to the group is the feature name in the apparent feature list, that is, each feature name corresponds to a clustering dictionary. Therefore, the feature names in the apparent feature grouping list can be used as the primary key and the clustering dictionary within the group corresponding to the feature name as the content to complete the construction of the set dictionary.
[0051] In a specific embodiment, its manifestation is as follows: If the clustering dictionary within the group corresponding to body shape is R1 = [{"small yellow croaker", 0.7}, {"yellow drum", 0.85},...], the clustering dictionary within the group corresponding to scale shape is R2 = [{"yellow drum", 0.7}, {"large yellow croaker", 0.85},...],..., then the set dictionary can be Q = [{"body shape": R1}, {"scale shape": R2},...].
[0052] In an embodiment of the present invention, determining the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary includes: obtaining the target feature name corresponding to the first fish name; Querying the set dictionary based on the target feature name to determine the clustering dictionary corresponding to the target feature name; Determining the target feature vector based on the clustering dictionary corresponding to the target feature name.
[0053] Specifically, when determining the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary, the target feature name corresponding to the first fish name can be obtained, and the obtaining method includes but is not limited to performing edge detection on the fish image corresponding to the first fish name to determine its corresponding target feature name.
[0054] Furthermore, the set dictionary can be queried based on the target feature name, that is, the target feature name is gradually matched with the set dictionary, and the corresponding clustering dictionary part is extracted, that is, the content of the set dictionary is extracted to obtain the clustering dictionary corresponding to the target feature name.
[0055] Furthermore, the target feature vector can be determined based on the clustering dictionary corresponding to the target feature name. For example, the corresponding multiple second fish names can be determined according to the primary key of the clustering dictionary, multiple fish images can be determined according to the multiple second fish names, and the target feature vector can be determined according to the vector representation of the multiple fish images in the two-dimensional space and the clustering dictionary corresponding to the target feature name.
[0056] In an embodiment of the present invention, determining the target feature vector based on the clustering dictionary corresponding to the target feature name includes: obtaining the fish image and the feature confidence corresponding to the clustering dictionary corresponding to the target feature name; Determining the corresponding edge feature vector based on the fish image; Obtaining the target feature vector based on the inner product of the edge feature vector and the feature confidence.
[0057] Specifically, when determining the target feature vector based on the clustering dictionary corresponding to the target feature name, multiple second fish names corresponding to the primary key in the clustering dictionary corresponding to the target feature name can be obtained, and then multiple fish images corresponding to the multiple second fish names can be determined. At the same time, according to the content in the clustering dictionary corresponding to the target feature name, that is, the normalized value of the distance between the grouped second fish names and the first fish names in the preset evolutionary classification tree, the feature confidence can be determined.
[0058] Furthermore, the corresponding edge feature vector can be determined based on the fish image, including but not limited to extracting the features of the outer edge of the fish image using an edge detection algorithm (such as the Canny algorithm, Yolo algorithm) to determine the edge feature vector.
[0059] Furthermore, for each fish image corresponding to the clustering dictionary corresponding to the target feature name, the inner product between its edge feature vector and its corresponding feature confidence can be calculated, so as to obtain the target feature vector.
[0060] In summary, in a specific embodiment, when obtaining the target feature name, multiple can be obtained at one time. Multiple clustering dictionaries are determined according to multiple target features, and multiple target feature vectors are determined according to multiple clustering dictionaries, so as to realize the multi-faceted description of the target fish image.
[0061] In summary, according to the training method of the fish image semantic description model of the embodiment of the present invention, by constructing a clustering dictionary through the fish names corresponding to the target fish image and the fish names corresponding to the fish images included in the training set, and converting it into a set dictionary, and calculating the feature confidence and determining the target feature vector, the description words corresponding to the target fish image can be determined, realizing effective model training with the help of the information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of dataset category imbalance. At the same time, the occurrence of missed detection phenomena can be effectively avoided, the learning ability and generalization performance of the fish semantic description model can be improved, and then the accuracy of new sample recognition and description can be improved.
[0062] A further embodiment of the present invention proposes a training system 100 for a fish image semantic description model, as Figure 2 shown. The training system 100 for the fish image semantic description model includes: an acquisition module 110, a construction module 120, a conversion module 130, a first determination module 140, and a second determination module 150, where The acquisition module 110 is used to acquire the first fish name corresponding to the target fish image and the second fish name corresponding to the fish image included in the training set in the fish image semantic description model.
[0063] The construction module 120 is used to construct a clustering dictionary based on the first fish name and the second fish name.
[0064] The conversion module 130 is used to convert the clustering dictionary into a set dictionary.
[0065] The first determination module 140 is used to determine the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary.
[0066] The second determination module 150 is used to determine the description word corresponding to the target fish image based on the target feature vector, so as to complete the training process of the fish image semantic description model.
[0067] In some embodiments, when constructing the clustering dictionary based on the first fish name and the second fish name, the construction module 120 is specifically configured to: use the grouped second fish name as the primary key, and use the normalized value of the distance between the grouped second fish name and the first fish name in the preset evolutionary classification tree as the content to construct the clustering dictionary.
[0068] In some embodiments, when constructing the clustering dictionary based on the first fish name and the grouped second fish name, the construction module 120 is specifically configured to: obtain the normalized value of the distance between the clustered vocabulary and the clustering center point to which its corresponding discipline belongs; draw the corresponding clustering image based on the clustered vocabulary, and use the normalized value as the transparency value of the clustering image; keep the transparency value unchanged, and convert the clustering image into a grayscale image to obtain a thesaurus mask image.
[0069] In some embodiments, when converting the clustering dictionary into a set dictionary, the conversion module 130 is specifically configured to: take the complement of the apparent feature list to obtain an apparent feature grouping list; convert the clustering dictionary into a set dictionary based on the apparent feature grouping list.
[0070] In some embodiments, when converting the clustering dictionary into a set dictionary based on the apparent feature grouping list, the conversion module 130 is specifically configured to: use the feature name in the apparent feature grouping list as the primary key, and use the clustering dictionary within the group corresponding to the feature name as the content to construct the set dictionary.
[0071] In some embodiments, when determining the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary, the first determination module 140 is specifically configured to: obtain the target feature name corresponding to the first fish name; query the set dictionary based on the target feature name to determine the clustering dictionary corresponding to the target feature name; determine the target feature vector based on the clustering dictionary corresponding to the target feature name.
[0072] In some embodiments, when determining the target feature vector based on the clustering dictionary corresponding to the target feature name, the second determination module 150 is specifically configured to: obtain the fish image and the feature confidence corresponding to the clustering dictionary corresponding to the target feature name; determine the corresponding edge feature vector based on the fish image; and obtain the target feature vector based on the inner product of the edge feature vector and the feature confidence.
[0073] According to the training system 100 of the fish image semantic description model of the present invention, a clustering dictionary is constructed by the fish name corresponding to the target fish image and the fish names corresponding to the fish images included in the training set, and it is converted into a set dictionary, so that the target feature vector corresponding to the target fish image can be determined. Furthermore, the description word corresponding to the target fish image can be determined, realizing effective model training by means of the information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of dataset category imbalance. At the same time, the occurrence of missed detection can be effectively avoided, the learning ability and generalization performance of the fish semantic description model are improved, and further the accuracy of identifying and describing new samples is improved.
[0074] To achieve the above object, an embodiment of the third aspect of the present invention discloses an electronic device, which includes: the subject classification system of fishery literature in the embodiment of the second aspect of the present invention; or, a processor, a memory, and a training program of the fish image semantic description model stored on the memory and executable on the processor. When the training program of the fish image semantic description model is executed by the processor, it implements the training method of the fish image semantic description model in the embodiment of the first aspect of the present invention.
[0075] According to the electronic device of the embodiment of the present invention, a clustering dictionary is constructed by the fish name corresponding to the target fish image and the fish names corresponding to the fish images included in the training set, and it is converted into a set dictionary, so that the target feature vector corresponding to the target fish image can be determined. Furthermore, the description word corresponding to the target fish image can be determined, realizing effective model training by means of the information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of dataset category imbalance. At the same time, the occurrence of missed detection can be effectively avoided, the learning ability and generalization performance of the fish semantic description model are improved, and further the accuracy of identifying and describing new samples is improved.
[0076] To achieve the above object, an embodiment of the fourth aspect of the present invention discloses a computer-readable storage medium, on which a training program of the fish image semantic description model is stored. When the training program of the fish image semantic description model is executed by the processor, it implements the training method of the fish image semantic description model in the embodiment of the first aspect of the present invention.
[0077] When the subject classification program of fishery literature stored on a computer-readable storage medium according to an embodiment of the present invention is executed by a processor, a clustering dictionary is constructed by using the fish names corresponding to the target fish images and the fish names corresponding to the fish images included in the training set, and it is converted into a set dictionary, so that the target feature vector corresponding to the target fish image can be determined. Furthermore, the description words corresponding to the target fish image can be determined, realizing effective model training by means of the information of approximate species in the case of lack of directly relevant images, increasing the available data volume and alleviating the effect of class imbalance in the data set. At the same time, the occurrence of missed detection phenomena can be effectively avoided, the learning ability and generalization performance of the fish semantic description model are improved, and further the accuracy of recognition and description of new samples is improved.
[0078] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example.
[0079] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and purposes of the present invention. The scope of the present invention is defined by the claims and their equivalents.
Claims
1. A method for training a semantic description model for fish images, characterized in that: The following steps are involved: Obtaining a first fish name corresponding to the target fish image and a second fish name corresponding to the fish image included in the training set in the fish image semantic description model; Building a clustering dictionary based on the first fish name and the second fish name; Converting the cluster dictionary into a set dictionary; Determine a target feature vector corresponding to the target fish image based on the first fish name and the set dictionary; The descriptive word corresponding to the target fish image is determined based on the target feature vector to complete the training process of the fish image semantic description model.
2. The method for training a fish image semantic description model according to claim 1, characterized in that: The step of constructing a clustering dictionary based on the first fish name and the second fish name includes: Determine a list of corresponding appearance characteristics based on the second fish name; grouping the second fish names based on the list of apparent features; The clustering dictionary is constructed based on the first fish name and the grouped second fish name.
3. The method for training a fish image semantic description model according to claim 2, characterized in that: The step of constructing the clustering dictionary based on the first fish name and the grouped second fish name includes: The clustering dictionary is constructed using the second fish name after the grouping as the primary key and the normalized value of the distance between the second fish name after the grouping and the first fish name in the preset evolutionary classification tree as the content.
4. The method for training a fish image semantic description model according to claim 2, characterized in that: The step of converting the cluster dictionary into a set dictionary comprises: Taking a complement based on the apparent feature list to obtain an apparent feature grouping list; The cluster dictionary is converted into a collection dictionary based on the list of apparent feature groups.
5. The method for training a fish image semantic description model according to claim 4, characterized in that: The step of converting the cluster dictionary into a set dictionary based on the apparent feature grouping list comprises: The set dictionary is constructed by taking the feature name in the apparent feature grouping list as the primary key and the clustering dictionary in the group corresponding to the feature name as the content.
6. The method for training a fish image semantic description model according to claim 1, characterized in that: The step of determining the target feature vector corresponding to the target fish image based on the first fish name and the set dictionary includes: Obtain a target feature name corresponding to the first fish name; Querying the set dictionary based on the target feature name to determine a clustering dictionary corresponding to the target feature name; The target feature vector is determined based on a clustering dictionary corresponding to the target feature name.
7. The method for training a fish image semantic description model according to claim 6, characterized in that: The determining the target feature vector based on the clustering dictionary corresponding to the target feature name includes: Obtaining the fish image and feature confidence corresponding to the cluster dictionary corresponding to the target feature name; Determine the edge feature vector corresponding to the fish image based on the fish image; The target feature vector is obtained based on the inner product of the edge feature vector and the feature confidence.
8. A training system for a fish image semantic description model, the system comprising: An acquisition module, used to acquire a first fish name corresponding to the target fish image and a second fish name corresponding to the fish image included in the training set in the fish image semantic description model; A construction module, used for constructing a clustering dictionary based on the first fish name and the second fish name; A conversion module, used for converting the cluster dictionary into a set dictionary; A first determination module, used for determining a target feature vector corresponding to the target fish image based on the first fish name and the set dictionary; The second determination module is used to determine the description word corresponding to the target fish image based on the target feature vector to complete the training process of the fish image semantic description model.
9. An electronic device comprising a training system for a fish image semantic description model as described in claim 8; or, a processor, a memory, and a training program for a fish image semantic description model stored in the memory and executable on the processor, wherein the training program for the fish image semantic description model implements the training method for the fish image semantic description model as described in claims 1-7 when executed by the processor.
10. A computer-readable storage medium, on which a training program for a fish image semantic description model is stored, wherein when the training program for the fish image semantic description model is executed by a processor, the training method for the fish image semantic description model as described in any one of claims 1 to 7 is implemented.