Transformer-based mosquito species detection method and system
By constructing a transformer-based mosquito species detection network and utilizing a multi-head self-attention mechanism and encoder-decoder structure, the problems of time-consuming, labor-intensive, and low-accuracy mosquito species identification were solved, achieving rapid and accurate mosquito species identification and subspecies identification, and supporting the effective prevention and control of mosquito-borne diseases.
Patent Information
- Application Number
- PCT/CN2024/084664
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-11
- Filing Date
- 2024-03-29
- Publication Date
- 2024-11-14
AI Technical Summary
Existing mosquito species identification methods rely on morphological feature classification, which is time-consuming, labor-intensive, and easily affected by genetic variability and phenotypic plasticity. Deep learning technology has low accuracy in mosquito species identification and cannot meet the needs of speed and accuracy.
A transformer-based mosquito species detection method is adopted. By constructing a mosquito image dataset, a transformer-based mosquito species detection network is trained. The multi-head self-attention mechanism and forward propagation encoder-decoder structure are used to achieve fast and accurate identification of mosquito species.
It enables rapid and accurate identification of mosquito species, distinguishes differences between subspecies, assists biologists in classification work, and effectively prevents mosquito-borne infectious diseases.
Smart Images

Figure CN2024084664_14112024_PF_FP_ABST
Abstract
Description
Transformer-based mosquito species detection method and system Technical Field
[0001] The present invention relates to a transformer-based mosquito species detection method, system, equipment and medium, and relates to the field of medical insect classification. Background Art
[0002] Mosquitoes belong to the family Culicidae, order Diptera, class Insecta. They transmit diseases that infect hundreds of millions of people and kill nearly a million worldwide each year. Consequently, mosquitoes are known as the number one "animal killer" and are one of the most important groups of insects used in medicine. Different mosquito species have different biological characteristics, such as their breeding habitats and habitats, and thus transmit different diseases. For example, Anopheles is the primary vector of malaria; Culex pipiens is the primary vector of West Nile virus disease, Japanese encephalitis, and lymphatic filariasis; and Aedes is the primary vector of dengue fever, Zika virus disease, and chikungunya fever. Since most mosquito-borne diseases currently lack vaccines or effective treatments, surveillance and control of vector mosquitoes remains a fundamental measure for preventing and controlling mosquito-borne disease epidemics. Identifying mosquito species is a prerequisite and foundation for effective prevention and control. Only by accurately identifying mosquito species and understanding their breeding characteristics and behavioral habits can we develop appropriate prevention and control strategies and implement targeted measures for rapid control.
[0003] Currently, mosquito species identification relies primarily on morphological classification. After centuries of development, mosquito morphological classification has become a comprehensive technical framework. However, morphological classification, which primarily relies on external features, is time-consuming and labor-intensive, susceptible to factors such as genetic variability and phenotypic plasticity. Accurate classification often requires years of specialized training for taxonomists. Even experienced taxonomists struggle with subtle external morphological differences in mosquito complexes. For example, the Culex pipiens complex requires dissection of male genitalia to identify the morphological characteristics of the median lobe of the penis before accurate identification of subspecies within the complex is possible. Furthermore, due to the difficulty of collecting male mosquitoes in the wild and the time-consuming preparation of specimens, traditional morphological classification methods are no longer sufficient to meet current mosquito species identification needs.
[0004] Deep learning technology, a new field in machine learning research, aims to use computers to build neural networks that can simulate and implement the learning and analysis problems of the human brain. This allows computers to autonomously learn the model's characteristic information and analyze and learn images, text, audio, and other objects, meeting the needs of computers to simulate the human brain's cognitive problems. In some cases, the recognition accuracy of deep learning can far exceed that of humans. Therefore, deep learning technology has been widely applied in the field of artificial intelligence. Although deep learning technology has the advantages of being simple, fast, accurate, and fully automated for mosquito species identification compared to previous classification methods, it also suffers from low prediction accuracy of recognition results, and recognition efficiency still needs to be improved.
[0005] Summary of the Invention
[0006] In response to the above problems, the purpose of the present invention is to provide a transformer-based mosquito species detection method, system, device and medium, which can improve the accuracy of mosquito species identification.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] In a first aspect, the present invention provides a transformer-based mosquito species identification and detection method, comprising:
[0009] Obtain the mosquito image to be identified;
[0010] Input the mosquito image to be identified into the trained transformer-based mosquito species detection network and output the species information feature vector;
[0011] The species information of the mosquitoes in the mosquito image to be identified is determined according to the species information feature vector.
[0012] Furthermore, the training process of the transformer-based mosquito species detection network is as follows:
[0013] Build a mosquito image dataset;
[0014] Construct a transformer-based mosquito species detection network;
[0015] The mosquito image dataset is passed into the transformer-based mosquito species detection network. The initial learning rate and final learning rate are set, and the convolutional neural network weights are updated for training. The trained transformer-based mosquito species detection network is obtained.
[0016] Furthermore, a mosquito image dataset is constructed, including:
[0017] Images of mosquitoes were taken. The mosquitoes belonged to 14 species, including Anopheles sinensis, Aedes aegypti, Aedes albopictus, Culex pipiens, Culex quinquefasciatus, Culex pipiens quinquefasciatus, Culex pipiens pallens, Aedes aggravating, Aedes koreana, Aedes spinulosa, Aedes marginata, Culex flavus, Culex pipiens albus, and Culex tritaeniorhynchus. The mosquito breeding environment was 26±1°C, 75±5% relative humidity, and a light cycle of L:D = 14:10. Adult mosquitoes were reared in 8% sucrose water.
[0018] Processing the captured mosquito images into mosquito images to be identified with the same size;
[0019] The mosquito images are labeled separately to obtain their specific location information and species information, and the mosquito images are enhanced.
[0020] Furthermore, the transformer-based mosquito species detection network is constructed, including a front image segmentation layer, a first transformer layer, a first region merging layer, a second transformer layer, a second region merging layer, a third transformer layer, a third region merging layer and a fourth transformer layer connected front and back. The front image segmentation layer processes the mosquito image data to be identified into serialized data X∈R that can be used by the transformer layer through linear and serialization processing. N,D , where N represents N data, D represents the feature dimension of each data, and the first to third region merging layers each contain a downsampling layer and a linear layer.
[0021] Furthermore, the first to fourth transformer layers contain transformer structures with the numbers of 2, 2, 18, and 2 respectively, wherein the transformer structure is an encoder-decoder structure with a multi-head self-attention mechanism and forward propagation. The multi-head self-attention mechanism of the transformer structure processes the serialized data output by the previous image segmentation layer and outputs a category information feature vector.
[0022] Furthermore, the category information feature vector output by the multi-head self-attention mechanism is expressed as:
[0023] M head =Concat(head1,head i ,...,head m )·W o
[0024] Among them, M head Represents the output of the multi-head self-attention mechanism, Concat represents the serial operation, and W o Represents the weight relationship of the multi-head self-attention mechanism, headi =Attention(Q i ,K i ,V i ), where Q i Queries that produce predictions for the weight matrix, K i Keys for producing prediction results for the weight matrix, V i The weight matrix produces the Values of the prediction results.
[0025] Furthermore, determining the species information of the mosquitoes in the mosquito image to be identified based on the species information feature vector includes:
[0026] The species information with the highest probability corresponding to the output of the feature vector is used as the probability value of the mosquito image to be identified. If the probability value is higher than the preset mosquito species confidence threshold, the mosquito image to be identified is recorded as a known mosquito species and its species information is recorded; if this probability value is lower than the preset mosquito species confidence threshold, it is recorded as an unknown mosquito species and the genus information corresponding to the highest possible output probability is recorded.
[0027] In a second aspect, the present invention further provides a mosquito species identification and detection system based on a transformer, comprising:
[0028] An image acquisition module is configured to acquire images of mosquitoes to be identified;
[0029] The feature recognition module is configured to input the mosquito image to be identified into the trained transformer-based mosquito species detection network and output a species information feature vector;
[0030] The mosquito detection module is configured to determine the species information of the mosquitoes in the mosquito image to be identified based on the species information feature vector.
[0031] In a third aspect, the present invention also provides an electronic device, wherein the processing device includes at least a processor and a memory, and a computer program is stored in the memory. When the processor runs the computer program, it executes the steps of any transformer-based mosquito species identification and detection method.
[0032] In a fourth aspect, the present invention also provides a computer storage medium having computer-readable instructions stored thereon, which can be executed by a processor to implement any step of the transformer-based mosquito species identification and detection method.
[0033] Due to the adoption of the above technical solution, the present invention has the following advantages: the transformer-based mosquito species detection method of the present invention has excellent performance, can quickly and accurately identify mosquito species, and realizes identification between subspecies. The detection method for unknown mosquito species can assist biologists in classification work, achieve effective prevention of mosquito-borne infectious diseases, and can be widely used in mosquito species detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. Throughout the drawings, the same reference numerals are used to denote the same components. In the drawings:
[0035] FIG1 is a flow chart of a transformer-based mosquito species detection method according to an embodiment of the present invention;
[0036] FIG2 is a diagram of a mosquito species detection network model based on a transformer according to an embodiment of the present invention;
[0037] FIG3 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0038] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" as used in the text may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0039] Although the terms first, second, third, etc. can be used in the text to describe multiple elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms can only be used to distinguish an element, component, region, layer or section from another region, layer or section. Unless the context clearly indicates otherwise, terms such as "first", "second" and other numerical terms do not imply order or sequence when used in the text. Therefore, the first element, component, region, layer or section discussed below can be referred to as the second element, component, region, layer or section without departing from the teaching of the example embodiments.
[0040] For ease of description, spatially relative terms may be used herein to describe the relationship of one element or feature relative to another element or feature as shown in the figures, such as "inside," "outside," "inner side," "outer side," "lower," "upper," etc. Such spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures.
[0041] Compared with previous classification methods, deep learning technology has the advantages of simple, fast, accurate, and fully automated identification for mosquito species identification. However, it also has the problem of low prediction accuracy of the identification effect. The transformer-based mosquito species identification and detection method, system, equipment, and medium provided by the present invention include obtaining a mosquito image to be identified; inputting the mosquito image to be identified into a trained transformer-based mosquito species detection network, outputting a species information feature vector; and determining the species information of the mosquito in the mosquito image to be identified based on the species information feature vector. Therefore, the transformer-based mosquito species detection method of the present invention has excellent performance, can quickly and accurately identify mosquito species, and realizes identification between subspecies.
[0042] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0043] Example 1: As shown in Figure 1, the transformer-based mosquito species identification and detection method provided in this embodiment includes the following steps:
[0044] Step S1: Obtain mosquito images and construct a mosquito image dataset, including:
[0045] The mosquito species selected in this embodiment are 14 types, including Anopheles sinensis, Aedes aegypti, Aedes albopictus, Culex pipiens, Culex quinquefasciatus, Culex pipiens harassing and Culex pipiens pallens, Aedes harassing, Aedes korean, Aedes spinulosa, Aedes marginatus, Culex fulvidraco, Culex albus, and Culex tritaeniorhynchus.
[0046] Mosquito breeding environmental conditions: room temperature 26±1°C, relative humidity 75±5%, photoperiod L:D=14:10, and adult mosquitoes were raised in 8% sucrose water.
[0047] The above mosquito pictures are taken and processed into mosquito pictures to be identified with the same size. In this embodiment, 8,400 mosquito picture information of 14 types are obtained, and the picture size is 4,464×2,976×3.
[0048] The mosquito images are labeled separately to obtain their specific location information and species information, the captured mosquito images are classified and stored, and the mosquito images are enhanced, including random image rotation and flipping, and adding noise for data enhancement.
[0049] Step S2: construct a transformer-based mosquito species detection network;
[0050] In this embodiment, as shown in Figure 2, a transformer-based mosquito species detection network is constructed. The transformer-based mosquito recognition network is a four-layer transformer layer network, including a front image segmentation layer, a first transformer layer, a first region merging layer, a second transformer layer, a second region merging layer, a third transformer layer, a third region merging layer and a fourth transformer layer.
[0051] Among them, the front image segmentation layer processes the image data into serialized data that can be used by the transformer through linear and serial processing, which is represented as X∈R N,D , where N represents N data and D represents the feature dimension of each data.
[0052] The first to fourth transformer layers contain transformer structures with the numbers of 2, 2, 18, and 2, respectively; among them, the transformer structure is an encoder-decoder structure with a multi-head self-attention mechanism and forward propagation.
[0053] The first to third region merging layers each contain a downsampling layer and a linear layer.
[0054] Furthermore, the transformer’s multi-head self-attention mechanism processes the serialized data output by the previous image segmentation layer and generates a global representation by performing contextual interactions on the data from multiple aspects, including:
[0055] First, define three sets of weight matrices: Where m represents the number of heads in the multi-head self-attention. Input the input sequence into the above three sets of weight matrices and generate prediction results, which are: i , Keys K i , Values V i , specifically:
[0056] Due to the use of multi-head self-attention mechanism, X=Y.
[0057] Attention score S i It can be calculated by the following formula: S i =Q i ·K i T (1)
[0058] Attention score S i Determines the degree of attention given to other elements when updating and encoding elements at the current position in the sequence data. Attention score S i Scaling is performed according to the following formula to improve the stability of gradient iteration.
[0059] Among them, D K K i dimension.
[0060] Then, the normalized attention score is converted into Converted into probability Pi, it can be calculated by the following formula:
[0061] Finally, the output of the multi-head self-attention mechanism can be expressed as follows: Mh ead =Concat(head1,headi,...,head m )·W o (5)
[0062] Among them, M head Represents the output of the multi-head self-attention mechanism, Concat represents the serial operation, and W o Represents the weight relationship of the multi-head self-attention mechanism.
[0063] After building the transformer-based mosquito recognition network, you need to train it: 70% of each image in the dataset is used as the training set, 15% as the validation set, and 15% as the test set. The training set from the dataset is fed into the transformer-based mosquito recognition network, with an initial learning rate of 0.0001 and a final learning rate of 0.1. The convolutional neural network weights are updated 300 times to obtain the new transformer-based mosquito recognition network weights and the trained transformer-based mosquito recognition network weights.
[0064] After training is complete, the transformer-based mosquito recognition network is verified. The validation set images from the aforementioned dataset are fed into the transformer-based mosquito recognition network. The number of images in the validation set is recorded as Lenth. A feature vector containing mosquito species information is output, and the species with the highest probability in the feature vector is used as the species information for the image being verified. To determine the accuracy of mosquito species information, the species information output by the transformer-based mosquito recognition network is compared with the manually labeled mosquito species information. If the species information matches the manually labeled species information, the probability of this species information is retained as the valid probability and recorded as TP in subsequent statistical work. If the species information does not match the manually labeled species information, the probability of this species information is considered invalid and recorded as 0 probability in subsequent statistical work.
[0065] The method for determining the accuracy of mosquito species information is as follows: obtain the prediction matrix M∈R output by the transformer-based mosquito recognition network N,L , where N is the number of images input into the transformer-based mosquito recognition network each time, L is the number of species in the transformer-based mosquito recognition network, and the prediction vector N generated for each image i Perform softmax calculation and obtain the index of its maximum value, which is used as the mosquito species for this verification.
[0066] The average probability of species information in the validation set of the transformer-based mosquito recognition network is calculated. If the accuracy of the average probability of species information in this validation set is greater than 75%, the trained transformer-based mosquito recognition network is considered to be effective. The average probability of species information (P) can be expressed as follows:
[0067] Testing mosquito image information: After obtaining the above-mentioned effective transformer-based mosquito recognition network, retain the effective transformer-based mosquito recognition network weights and model for testing. Pass the test set images from the above dataset into the above-mentioned effective transformer-based mosquito recognition network. The mosquito species information and location information output by the transformer-based mosquito recognition network are statistically analyzed to obtain the final correct probability of this transformer-based mosquito recognition network.
[0068] Step S3: Input the mosquito image to be identified into the transformer-based mosquito species detection network to obtain the output species information feature vector, and determine the species information of the mosquito image to be identified based on the species information feature vector.
[0069] In this embodiment, a mosquito image to be identified is input into the above-mentioned effective transformer-based mosquito identification network, so that it outputs a feature vector of valid species information. Based on the feature vector of the output species information, the species information with the highest probability output by formula (3) is used as the species information of the mosquito image. If this probability is higher than a preset mosquito species confidence threshold (since the mosquito image input to the network may have never appeared in the dataset constructed in this embodiment, misclassification may occur. This confidence threshold is an evaluation indicator for judging whether the network has made an effective classification), it will be recorded as a known mosquito species and its species information will be recorded. If this probability is lower than the preset mosquito species confidence threshold, it will be recorded as an unknown mosquito species, and the genus information corresponding to the highest probability will be output (indicating the superior species of the mosquito to be identified. The dataset constructed in this embodiment uses species as the smallest classification unit, and a genus may correspond to multiple similar but different species).
[0070] Example 2: The above-mentioned Example 1 provides a transformer-based mosquito species detection method. Correspondingly, this embodiment provides a transformer-based mosquito species detection system. The system provided in this embodiment can implement the transformer-based mosquito species detection method of Example 1. The system can be implemented through software, hardware, or a combination of software and hardware. For the convenience of description, this embodiment is described by dividing the functions into various units and describing them separately. Of course, during implementation, the functions of each unit can be implemented in the same or multiple software and / or hardware. For example, the system can include integrated or separate functional modules or functional units to perform the corresponding steps in each method of Example 1. Since the system of this embodiment is basically similar to the method embodiment, the description process of this embodiment is relatively simple. For relevant points, please refer to the partial description of Example 1. The embodiment of the transformer-based mosquito species detection system provided by the present invention is only illustrative.
[0071] Specifically, the transformer-based mosquito species detection system provided in this embodiment includes:
[0072] An image acquisition module is configured to acquire images of mosquitoes to be identified;
[0073] The feature recognition module is configured to input the mosquito image to be identified into the trained transformer-based mosquito species detection network and output a species information feature vector;
[0074] The mosquito detection module is configured to determine the species information of the mosquitoes in the mosquito image to be identified based on the species information feature vector.
[0075] Example 3: This example provides an electronic device corresponding to the transformer-based mosquito species detection method provided in Example 1. The electronic device may be an electronic device used for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of Example 1.
[0076] As shown in Figure 3, the electronic device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected via a bus to complete communication with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Component (EISA) bus, etc. A computer program that can be run on the processor is stored in the memory, and the processor executes the method of embodiment 1 when running the computer program. Its implementation principle and technical effect are similar to those of embodiment 1 and are not described in detail here. It will be understood by those skilled in the art that the structure shown in Figure 3 is only a block diagram of a partial structure related to the present application solution, and does not constitute a limitation on the computing device to which the present application solution is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0077] In a preferred embodiment, the logic instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), and an optical disk.
[0078] In a preferred embodiment, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.
[0079] Embodiment 4: This embodiment provides a computer program product, which may include a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method provided in the above-mentioned embodiment 1. Its implementation principle and technical effects are similar to those of embodiment 1 and will not be repeated here.
[0080] In a preferred embodiment, a computer-readable storage medium may be a tangible device that retains and stores instructions used by an instruction execution device, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. The computer-readable storage medium stores computer program instructions that cause a computer to execute the method provided in the first embodiment.
[0081] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In the description of this specification, the reference terms "a preferred embodiment", "further", "specifically", "in the present embodiment", etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0082] The present application is described with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a system for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.
[0083] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction system that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A mosquito species identification and detection method based on transformer, characterized in that: include: Obtain the mosquito picture to be identified; Input the mosquito image to be identified into the trained transformer-based mosquito species detection network, and output the species information feature vector; The species information of the mosquitoes in the mosquito image to be identified is determined according to the species information feature vector.
2. The transformer-based mosquito species identification and detection method according to claim 1, characterized in that: The training process of the transformer-based mosquito species detection network is: Build a mosquito image dataset; Construct a transformer-based mosquito species detection network; The mosquito image dataset is passed into the transformer-based mosquito species detection network, the initial learning rate and final learning rate are set, and the convolutional neural network weights are updated for training to obtain a trained transformer-based mosquito species detection network.
3. The transformer-based mosquito species identification and detection method according to claim 1, characterized in that: Construct a mosquito image dataset, including: Take pictures of mosquitoes, there are 14 types of mosquitoes, including Anopheles sinensis, Aedes aegypti, Aedes albopictus, Culex pipiens, Culex pipiens quinquefasciatus, Culex pipiens harassing and Culex pipiens pallens, Aedes irritans, Aedes koreaiensis, Aedes spinulosa, Aedes marginata, Culex flavus, Culex pipiens alba and Culex tritaeniorhynchus, the mosquito breeding environment conditions are room temperature 26±1℃, relative humidity 75±5%, light duration L:D=14:10, and the adult mosquitoes are raised with 8% sucrose water; Processing the captured mosquito images into mosquito images to be identified with the same size; The mosquito images are labeled separately to obtain their determined location information and species information, and the mosquito images are enhanced.
4. The transformer-based mosquito species identification and detection method according to claim 1, characterized in that: The transformer-based mosquito species detection network constructed includes a front image segmentation layer, a first transformer layer, a first region merging layer, a second transformer layer, a second region merging layer, a third transformer layer, a third region merging layer and a fourth transformer layer connected to each other. The front image segmentation layer processes the mosquito image data to be identified into serialized data X∈R that can be used by the transformer layer through linear and serialization processing. N,D , where N represents N data, D represents the feature dimension of each data, and the first to third region merging layers each contain a downsampling layer and a linear layer.
5. The transformer-based mosquito species identification and detection method according to claim 4, characterized in that: The first to fourth transformer layers contain transformer structures with numbers of 2, 2, 18, and 2 respectively, where the transformer structure is an encoder-decoder structure with a multi-head self-attention mechanism and forward propagation. The multi-head self-attention mechanism of the transformer structure processes the serialized data output by the previous image segmentation layer and outputs a category information feature vector.
6. The transformer-based mosquito species identification and detection method according to claim 5, characterized in that: The type information feature vector output by the multi-head self-attention mechanism is expressed as: head =Concat(head1,head i ,...,head m )·W o Among them, M head Represents the output of the multi-head self-attention mechanism, Concat represents the concatenation operation, and W o Represents the weight relationship of the multi-head self-attention mechanism, head i =Attention(Q i ,K i ,V i ), where Q i Queries that produce prediction results for the weight matrix, K i Keys for producing prediction results for the weight matrix, V i The weight matrix produces the Values of the prediction results.
7. The transformer-based mosquito species identification and detection method according to claim 1, characterized in that: Determine the type information of the mosquito in the mosquito image to be identified according to the type information feature vector, including: The highest probability of the species information corresponding to the feature vector output is used as the probability value of the mosquito image to be identified. If the probability value is higher than the preset mosquito species confidence threshold, the mosquito image to be identified is recorded as a known mosquito species. Species, and record its species information; if this probability value is lower than the preset mosquito species confidence threshold, it will be recorded as an unknown mosquito species, and the genus information corresponding to the maximum possible output probability will be recorded.
8. A mosquito species identification and detection system based on transformer, characterized in that: include: An image acquisition module is configured to acquire an image of a mosquito to be identified; A feature recognition module is configured to input the mosquito image to be identified into the trained transformer-based mosquito species detection network and output a species information feature vector; The mosquito detection module is configured to determine the species information of the mosquitoes in the mosquito image to be identified according to the species information feature vector.
9. An electronic device, wherein the processing device comprises at least a processor and a memory, wherein a computer program is stored in the memory, wherein: When the processor runs the computer program, the steps of any one of claims 1 to 7 of the transformer-based mosquito species identification and detection method are implemented.
10. A computer storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the steps of the transformer-based mosquito species identification and detection method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Mosquito recognition method based on convolutional neural network
CN110245604A
Mosquito identification method based on convolutional neural network
CN113505651A
Mosquito type identification method and system based on deep learning
CN114463581A
Transform-based mosquito species detection method and system
CN116612323A
Leveraging smart-phone cameras and image processing techniques to classify mosquito genus and species
US20200143202A1
Cited By
Transform-based mosquito species detection method and system
CN116612323A
