A method and device for identifying fungi

Abstract

The present application discloses a fungus category recognition method and device, which relates to the field of image recognition technology. The fungus category recognition method includes: obtaining a first fungus culture dish image and a second fungus culture dish image, wherein the first fungus culture dish image and the second fungus culture dish image are respectively a front image of a culture dish cultured with fungi and a back image of a culture dish cultured with fungi; segmenting fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image; performing image registration on the first fungus image and the second fungus image; fusing the two registered fungus images to obtain a fused fungus image; performing fungus category recognition on the fused fungus image to obtain a fungus category recognition result. The fungus category recognition method disclosed in the present application can automatically identify the category of fungi, and the method has a high recognition efficiency and can ensure the stability of the recognition result.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and device for identifying fungi categories. Background Art

[0002] Fungal classification is of great significance in biological research, agricultural disease control and environmental monitoring. Traditional fungal classification methods are based on manual methods, that is, people observe fungi under a microscope to determine the fungal classification.

[0003] Although the artificial fungus category recognition method can realize fungus category recognition, the recognition efficiency is low and the recognition results are easily affected by human subjective factors. Summary of the invention

[0004] In view of this, the present application provides a fungus category identification method and device to solve the problem that the existing fungus category identification method has low identification efficiency and the identification results are easily affected by human subjective factors. The technical solution is as follows:

[0005] The first aspect of the present application provides a method for identifying fungal species, comprising:

[0006] Acquire a first fungus culture dish image and a second fungus culture dish image, wherein the first fungus culture dish image and the second fungus culture dish image are respectively a front image of a culture dish in which fungi are cultured and a back image of the culture dish in which fungi are cultured;

[0007] segmenting fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image;

[0008] Performing image registration on the first fungus image and the second fungus image to obtain two registered fungus images;

[0009] fusing the two registered fungus images to obtain a fused fungus image;

[0010] The fused fungus image is subjected to fungus category recognition to obtain a fungus category recognition result.

[0011] In a possible implementation, segmenting fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image includes:

[0012] Based on the pre-trained fungus segmentation model, segmenting fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image;

[0013] The fungus segmentation model is trained using training fungus culture dish images with real fungus segmentation results marked.

[0014] In a possible implementation, segmenting fungi from any one of the first fungi culture dish image and the second fungi culture dish image based on a pre-trained fungi segmentation model includes:

[0015] Based on the pre-trained fungus segmentation model, the fungus culture dish image is encoded to obtain features of multiple scales of the fungus culture dish image;

[0016] Based on the fungus segmentation model, the features at the multiple scales are fused to obtain fused features;

[0017] Based on the fungus segmentation model and taking the fused features as a basis, the fungus segmentation result of the fungus culture dish image is predicted.

[0018] In a possible implementation, based on the fungus segmentation model, the features at multiple scales are fused to obtain fused features, including:

[0019] Based on the fungus segmentation model, the features of each scale are centralized to obtain the features after the centralization of each scale;

[0020] Based on the fungus segmentation model, aligning the features after the centralization processing at each scale to the same spatial size to obtain the aligned features at each scale;

[0021] Based on the fungus segmentation model, the aligned features of each scale are fused in the channel dimension to obtain fused features.

[0022] In a possible implementation, performing fungus category recognition on the fused fungus image to obtain a fungus category recognition result includes:

[0023] Based on a plurality of pre-trained target fungus image classification models, the fungus categories corresponding to the fused fungus images are predicted respectively to obtain a plurality of fungus category prediction results, wherein the plurality of target fungus image classification models are trained using different training data sets, and each training data in each training data set is a training fungus image labeled with a fungus category;

[0024] The final fungus category is determined according to the multiple fungus category prediction results.

[0025] In a possible implementation, any training data set includes a first part of training data and a second part of training data, where the second part of training data is obtained by performing data enhancement on the first part of training data;

[0026] The data enhancement method includes one or more of the following methods: contrast-limited adaptive histogram equalization, flipping, translation, scaling, cropping, adding noise, brightness adjustment, and contrast adjustment.

[0027] In one possible implementation, the process of training a target fungus image classification model using a training dataset includes:

[0028] Dividing the training data in the training data set into N parts to obtain N training data subsets, where N is an integer greater than 1;

[0029] Using the N training data subsets to respectively train the fungus image classification models to be trained, to obtain N trained fungus image classification models;

[0030] From the N trained fungus image classification models, a fungus image classification model with the best performance is selected as the target fungus image classification model.

[0031] In one possible implementation, the fungus image classification model to be trained is trained using any subset of the training data, including:

[0032] Acquire a training fungus image from the training data subset as a first training fungus image;

[0033] Inputting the first training fungus image into the fungus image classification model to obtain a fungus category prediction result of the first training fungus image;

[0034] Determining a fungus category prediction loss according to a fungus category prediction result of the first training fungus image and a fungus category annotated by the first training fungus image;

[0035] According to the fungus category prediction loss, parameters of the fungus image classification model are updated.

[0036] In a possible implementation, the training data subset includes training fungus images at different growth stages, and the training fungus images in the training data subset are also labeled with the growth stages;

[0037] Train the fungus image classification model using any of the training data subsets, including:

[0038] Acquire a plurality of training fungus images at different growth stages from the training data subset as second training fungus images, wherein the fungus category annotated on each second training fungus image is consistent with the fungus category annotated on the first training fungus image;

[0039] encoding the plurality of second training fungus images respectively to obtain image features respectively corresponding to the plurality of second training fungus images;

[0040] Extracting features from image features respectively corresponding to the plurality of second training fungus images based on a fungus image classification model;

[0041] The growth periods respectively corresponding to the plurality of second training fungi images are predicted based on the features extracted from the image features respectively corresponding to the plurality of second training fungi images based on the fungi image classification model, so as to obtain prediction results of the growth periods respectively corresponding to the plurality of second training fungi images;

[0042] determining a growth period prediction loss according to the growth period prediction results respectively corresponding to the plurality of second training fungus images and the growth periods respectively marked on the plurality of second training fungus images;

[0043] The updating of parameters of the fungus image classification model according to the fungus category prediction loss includes:

[0044] The parameters of the fungus image classification model are updated according to the fungus category prediction loss and the growth period prediction loss.

[0045] A second aspect of the present application provides a fungus category identification device, comprising: a fungus culture dish image acquisition module, a fungus segmentation module, an image registration module, an image fusion module and a fungus category identification module;

[0046] The fungus culture dish image acquisition module is used to acquire a first fungus culture dish image and a second fungus culture dish image, wherein the first fungus culture dish image and the second fungus culture dish image are respectively a front image of a culture dish in which fungi are cultured and a back image of the culture dish in which fungi are cultured;

[0047] The fungus segmentation module is used to segment fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image;

[0048] The image registration module is used to perform image registration on the first fungus image and the second fungus image to obtain two registered fungus images;

[0049] The image fusion module is used to fuse the two registered fungus images to obtain a fused fungus image;

[0050] The fungus category recognition module is used to perform fungus category recognition on the fused fungus image to obtain a fungus category recognition result.

[0051] By means of the above technical solution, the fungus category recognition method provided by the present application first obtains a first fungus culture dish image (a front image of a culture dish with fungi cultured) and a second fungus culture dish image (a back image of a culture dish with fungi cultured), and then respectively segments the fungi from the first fungus culture dish image and the second fungus culture dish image to obtain a first fungus image and a second fungus image, and then performs image registration on the first fungus image and the second fungus image to obtain two registered fungus images, and then fuses the two registered fungus images to obtain a fused fungus image, and finally performs fungus category recognition on the fused fungus image to obtain a fungus category recognition result. The fungus category recognition method provided by the present application can automatically identify the category of fungi and has a high recognition efficiency. Since no human participation is required in the recognition process, the influence of human subjective factors on the recognition result is avoided, thereby ensuring the stability of the recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0053] Figure 1 A schematic diagram of a system architecture involved in this application;

[0054] Figure 2 A schematic diagram of a hardware structure of a terminal provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a hardware structure of a server provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the process of the fungus classification identification method provided in the embodiment of the present application;

[0057] Figure 5 A schematic diagram of determining fungal categories based on multiple pre-trained target fungal image classification models provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a fungus segmentation model provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of encoding, multi-scale feature fusion, and decoding of the fungus segmentation model provided in an embodiment of the present application;

[0060] Figure 8A schematic diagram of a training method for a fungus image classification model provided in an embodiment of the present application;

[0061] Fig. 9 A schematic diagram of the structure of a fungus category identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0063] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0064] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0065] Before introducing the fungus category identification method provided by the present application, the system architecture involved in the present application is first described.

[0066] In one possible implementation, Figure 1 As shown, the system architecture involved in the present application may include a terminal 101 and a server 102, and the terminal 101 may interact with the server 102 via a network (wired network or wireless network). The server 102 may include one or more servers ( Figure 1 The example includes a server. The terminal 101 can obtain the front and back images of the culture dish with fungi cultured, and transmit the obtained images to the server 102 through the network. The server 102 uses the fungus category recognition method provided by the present application to perform fungus category recognition on the front and back images of the culture dish with fungi cultured, and transmits the fungus category recognition result to the terminal 101 through the network. The terminal 101 can output the fungus category recognition result.

[0067] In another possible implementation, the system architecture involved in the present application may include a terminal. The terminal has strong data processing capabilities. The terminal may use the fungus category recognition method provided in the present application to perform fungus category recognition on the front and back images of a culture dish in which fungi are cultured, and then output the fungus category recognition result.

[0068] Next, the product form of the above-mentioned terminal is described.

[0069] The above-mentioned terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiments of the present application do not impose any restrictions on this.

[0070] Figure 2 An optional hardware structure diagram of a terminal is shown.

[0071] refer to Figure 2 As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), an earphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290 and other components. Those skilled in the art will appreciate that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0072] The input unit 230 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 270, and can receive and execute the command sent by the processor 270; the touch signal at least includes the contact point coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch screen 231, the input unit 230 can also include other input devices. Specifically, other input devices 232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0073] The display unit 240 may be used to display information input by a user or information provided to a user, various menus of the terminal, an interactive interface, file display, and / or playback of any multimedia file.

[0074] The memory 220 can be used to store instructions and data. The memory 220 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; it provides the processor 270 with hardware, software, and data resources including management of computing and processing equipment, and supports control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0075] The processor 270 is the control center of the terminal. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing instructions stored in the memory 220 and calling data stored in the memory 220, it executes various functions of the terminal and processes data, thereby controlling the terminal as a whole. Optionally, the processor 270 may include one or more processing units; preferably, the processor 270 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 270. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 270 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 220, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0076] The memory 220 may be used to store software codes related to the fungus category identification method, and the processor 270 may execute the software codes in the memory 220 or schedule other units (such as the input unit 230 and the display unit 240) to implement corresponding functions.

[0077] The radio frequency unit 210 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 270 for processing; in addition, the designed uplink data is sent to the base station. Generally, the radio frequency unit 210 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LowNoiseAmplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 210 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General PacketRadio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0078] In the embodiment of the present application, the radio frequency unit 210 can send data to other devices, and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional, and can be replaced by other communication interfaces, such as a network port.

[0079] The terminal also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0080] The terminal also includes an external interface 280, which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal to communicate with other devices, or to connect a charger to charge the terminal.

[0081] Although not shown, the terminal may further include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here.

[0082] Next, the product form of the above server is described.

[0083] Figure 3 A structural diagram of the above server is provided, such as Figure 3 As shown, the server may include a bus 301, a processor 302, a communication interface 303, and a memory 304. The processor 302, the memory 304, and the communication interface 303 communicate with each other via the bus 301.

[0084] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0085] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0086] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0087] The memory 304 may be used to store software codes related to the fungus category identification method. The processor 302 may call the software codes stored in the memory 304 or schedule other units to implement corresponding functions.

[0088] The processors in the above-mentioned terminals and servers (for example, processor 270 and processor 302) may be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (DSP), microprocessors or microcontrollers, etc.), or combinations of these hardware circuits. For example, the processor may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0089] Next, the fungus category identification method provided by the present application is introduced through the following examples.

[0090] See also Figure 4 , shows a schematic diagram of the process of the fungus category identification method provided in an embodiment of the present application, the method may include:

[0091] Step S401: acquiring a first fungus culture dish image and a second fungus culture dish image.

[0092] Among them, the first fungus culture dish image is a front image of a culture dish in which fungi (such as one of Aspergillus fumigatus, Aspergillus flavus, Aspergillus terreus, Aspergillus niger, Aspergillus nidulans, Penicillium, etc.) are cultured, and the second fungus culture dish image is a back image of the culture dish in which fungi are cultured.

[0093] Step S402: segmenting fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image.

[0094] In a possible implementation, fungi may be segmented from the first fungus culture dish image and the second fungus culture dish image respectively based on a pre-trained fungus segmentation model to obtain a first fungus image and a second fungus image.

[0095] Among them, the fungus segmentation model is trained using the training data in the training data set. Each piece of training data in the training data set of the fungus segmentation model is a training fungus petri dish image with the real fungus segmentation result marked. It should be noted that the training data set of the fungus segmentation model contains the front image of the training fungus petri dish (the front image of the petri dish with fungi cultured) and the back image of the training fungus petri dish (the back image of the petri dish with fungi cultured). During training, the Dice loss function can be used to evaluate the accuracy of the segmentation result.

[0096] Step S403: performing image registration on the first fungus image and the second fungus image to obtain two registered fungus images.

[0097] In a possible implementation, the process of performing image registration of the first fungus image and the second fungus image may include: extracting multiple matching feature pairs from the first fungus image and the second fungus image; determining a transformation matrix based on the multiple matching feature pairs; and registering the first fungus image and the second fungus image using the transformation matrix.

[0098] When determining the transformation matrix based on multiple matching feature pairs, the transformation matrix from the first fungus image to the second fungus image can be determined based on the multiple matching feature pairs, and then the first fungus image is transformed based on the determined transformation matrix. Of course, the transformation matrix from the second fungus image to the first fungus image can also be determined based on the multiple matching feature pairs, and then the second fungus image is transformed based on the determined transformation matrix.

[0099] Step S404: fusing the two registered fungus images to obtain a fused fungus image.

[0100] In a possible implementation, the two registered fungus images may be fused by weighted averaging to obtain a fused fungus image.

[0101] The weighted average method is as follows:

[0102] (1)

[0103] Among them, I1(x, y) and I2(x, y) represent the two registered fungus images, w1 represents the weight corresponding to I1(x, y), and w2 represents the weight corresponding to I2(x, y). The specific values ​​of w1 and w2 can be determined according to the actual application scenario. fused (x,y) represents the fused fungus image.

[0104] It should be noted that this embodiment is not limited to the weighted average method for fusing the two registered fungus images. Other methods may be used to fuse the two registered fungus images. For example, the two registered fungus images may be fused by direct summation or weighted summation.

[0105] Step S405: performing fungus category recognition on the fused fungus image to obtain a fungus category recognition result.

[0106] In one possible implementation, Figure 5 As shown, based on the pre-trained multiple target fungus image classification models, the fungus categories corresponding to the fused fungus images can be predicted respectively to obtain multiple fungus category prediction results, and then the final fungus category can be determined based on the multiple fungus category prediction results. The fungus category recognition based on the pre-trained multiple target fungus image classification models can ensure the reliability and stability of the recognition results.

[0107] Among them, multiple target fungus image classification models are trained using different training data sets, and each training data in each training data set is a training fungus image labeled with a fungus category. Among them, any training fungus image can be obtained by aligning and fusing a training fungus front image and a corresponding training fungus back image, and the training fungus front image and the corresponding training fungus back image can be obtained by performing fungus segmentation on the training fungus culture dish front image and the corresponding training fungus culture dish back image.

[0108] Any training data set includes a first part of training data and a second part of training data, and the second part of training data is obtained by performing data enhancement on the first part of training data (such as directly collected data), wherein the data enhancement method includes one or more of the following methods: contrast-limited adaptive histogram equalization, flipping, translation, scaling, cropping, adding noise, brightness adjustment, contrast adjustment, etc. When performing data enhancement processing on the first part of training data, for each piece of training data contained in the first part of training data, any one of the above-mentioned data enhancement methods or a combination of any multiple data enhancement methods can be used to process the training data. The introduction of data enhancement can, on the one hand, increase the amount of training data in the training data set, and on the other hand, increase the diversity of training data in the training data set. In addition, in order to further expand the training data set, some training data can be artificially synthesized and added to the training data set.

[0109] When determining the final fungus category according to multiple fungus category prediction results, the final fungus category can be determined based on a voting mechanism, that is, the category with the largest number in the predicted categories is determined as the final fungus category, that is, the category of fungi in the first fungus culture dish image and the second fungus culture dish image.

[0110] Exemplarily, five target fungus image classification models can be trained in advance using five different training data sets, and the fungus category corresponding to the fused fungus image is predicted based on each target fungus image classification model. In this way, five fungus category prediction results can be obtained. After obtaining the five fungus category prediction results, the final fungus category can be determined based on the five fungus category prediction results. It is assumed that the fungus category predicted by the first target fungus image classification model is A, the fungus category predicted by the second target fungus image classification model is B, the fungus category predicted by the third target fungus image classification model is A, the fungus category predicted by the fourth target fungus image classification model is C, and the fungus category predicted by the fifth target fungus image classification model is A. Since category A is the largest among the five categories, it is finally determined that the category of the fungi in the first fungus culture dish image and the second fungus culture dish image is A.

[0111] It should be noted that the present embodiment is not limited to realizing the identification of fungus categories based on multiple pre-trained target fungus image classification models. For example, the identification of fungus categories may also be realized based on one pre-trained target fungus image classification model.

[0112] In addition, the target fungus image classification model in this embodiment can be but is not limited to a classification model based on EfficientNet.

[0113] The fungus category recognition method provided in the embodiment of the present application first obtains a first fungus culture dish image (a front image of a culture dish with fungi cultured) and a second fungus culture dish image (a back image of a culture dish with fungi cultured), and then segments the fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image, and then performs image registration on the first fungus image and the second fungus image to obtain two registered fungus images, and then fuses the two registered fungus images to obtain a fused fungus image, and finally performs fungus category recognition on the fused fungus image to obtain a fungus category recognition result. The fungus category recognition method provided in the embodiment of the present application can automatically identify the category of fungi and has a high recognition efficiency. Since no human participation is required in the recognition process, the influence of human subjective factors on the recognition result is avoided, thereby ensuring the stability of the recognition result.

[0114] In another embodiment of the present application, the specific implementation process of "step S402: segmenting fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain the first fungus image and the second fungus image" in the above embodiment is introduced.

[0115] The above embodiment mentioned that fungi can be segmented from the first fungi culture dish image and the second fungi culture dish image respectively based on the pre-trained fungi segmentation model to obtain the first fungi image and the second fungi image.

[0116] In a possible implementation, based on the pre-trained fungus segmentation model, the process of segmenting fungi from any one of the first fungus culture dish image and the second fungus culture dish image may include:

[0117] Step a1: Based on the pre-trained fungus segmentation model, the fungus culture dish image is encoded to obtain features of multiple scales of the fungus culture dish image.

[0118] In one possible implementation, the fungus segmentation model in this embodiment may be an improved U-Net model. The conventional U-Net model includes an encoder and a decoder. The present application improves the conventional U-Net model and introduces a multi-scale feature fusion module between the encoder and the decoder.

[0119] like Figure 6 As shown, the fungus segmentation model in this embodiment may include an encoder 601 , a multi-scale feature fusion module 602 , and a decoder 603 .

[0120] The fungus culture dish image can be input into the encoder 601 of the fungus segmentation model, and the encoder 601 encodes the input fungus culture dish image and outputs features of multiple scales of the fungus culture dish image. Figure 7 As shown, the process of encoding the input fungus culture dish image by the encoder 601 is a process of extracting image features from the input fungus culture dish image and gradually reducing the spatial resolution.

[0121] Step a2: Based on the fungus segmentation model, features at multiple scales are fused to obtain fused features.

[0122] The features of multiple scales output by the encoder 601 of the fungus segmentation model are input into the multi-scale feature fusion module 602 . The multi-scale feature fusion module 602 fuses the input features of multiple scales and outputs the fused features.

[0123] In a possible implementation, the process of the multi-scale feature fusion module 602 of the fungus segmentation model fusing the features of multiple scales output by the encoder 601 may include:

[0124] Step a21: Centralize the features of each scale to obtain the centralized features of each scale.

[0125] The features after centralization at the i-th scale can be expressed as:

[0126] F centered,si = F si -mean(F si ) (2)

[0127] Among them, F si represents the feature of the i-th scale, mean(F si ) represents the feature F of the i-th scale si The average value, F centered,si Represents the features after centralization at the i-th scale.

[0128] Step a22: align the features after centralization at each scale to the same spatial size to obtain aligned features at each scale.

[0129] Downsampling can be used to align the centralized features of each scale to the same spatial size.

[0130] Step a23: Fuse the aligned features of each scale in the channel dimension to obtain fused features.

[0131] In a possible implementation, the aligned features of each scale can be fused in the channel dimension by splicing, that is, the aligned features of each scale are spliced ​​in the channel dimension, and the spliced ​​features (i.e., fused features) F fused It can be expressed as:

[0132] F fused =Concat(F′centered,s1 , F′ centered,s2 ,…, F′ centered,sn ) (3)

[0133] Among them, Concat means concatenating features in the channel dimension, F′ centered,s1 represents the aligned features of the first scale, F′ centered,s2 represents the aligned features of the second scale, ..., F′ centered,sn Represents the aligned features of the nth scale.

[0134] In a possible implementation, the multi-scale feature fusion module 602 may implement multi-scale feature fusion based on a centralized feature pyramid (CFP).

[0135] It should be noted that the introduction of the multi-scale feature fusion module can improve the model's detail capture capability and segmentation accuracy, so that the model can ultimately output accurate segmentation results.

[0136] Step a3: Based on the fungus segmentation model and the fused features, predict the fungus segmentation result of the fungus culture dish image.

[0137] The fused features output by the multi-scale feature fusion module 602 of the fungus segmentation model are input into the decoder 603 of the fungus segmentation model for decoding, and the decoder 603 outputs the fungus segmentation result.

[0138] like Figure 7 As shown, the decoding process of the decoder 603 is a process of gradually restoring the spatial resolution of the image and generating the fungus segmentation result. It should be noted that in the process of gradually restoring the spatial resolution of the image and generating the fungus segmentation result, the decoder 603, in addition to using the fused features output by the multi-scale feature fusion module 602, also uses the features of the non-minimum scales among the features of multiple scales obtained by the encoder 601. Assuming that the encoder 601 gradually obtains features of 5 scales, the multi-scale feature fusion module 602 will fuse the features of the 5 scales, and the decoder 603 generates the fungus segmentation result based on the fused features and the features of the first 4 scales among the 5 scales.

[0139] In another embodiment of the present application, the specific implementation process of "step S405: performing fungus category recognition on the fused fungus image to obtain a fungus category recognition result" in the above embodiment is introduced.

[0140] The above embodiment mentioned that based on multiple pre-trained target fungus image classification models, the fungus categories corresponding to the fused fungus images can be predicted respectively to obtain multiple fungus category prediction results, and then the final fungus category can be determined based on the multiple fungus category prediction results.

[0141] Next, the process of training to obtain multiple target fungus image classification models is introduced. Since the process of training to obtain each target fungus image classification model is similar, this embodiment introduces the process of training to obtain one target fungus image classification model using one training data set.

[0142] The process of training a target fungus image classification model with a training data set may include: dividing the training data in the training data set into N parts (may be evenly divided into N parts) to obtain N training data subsets; using the N training data subsets to train the fungus image classification models to be trained respectively to obtain N trained fungus image classification models; and selecting the fungus image classification model with the best performance from the N trained fungus image classification models as the target fungus image classification model. Wherein, N is an integer greater than 1.

[0143] There are many implementation methods for training the fungus image classification model to be trained using any training data subset. This embodiment provides two optional implementation methods.

[0144] The first implementation method of using any training data subset to train the fungus image classification model to be trained is: obtain a training fungus image from the training data subset; input the obtained training fungus image into the fungus image classification model to obtain the fungus category prediction result of the training fungus image; determine the fungus category prediction loss (such as cross entropy loss) according to the fungus category prediction result of the training fungus image and the fungus category annotated by the training fungus image; update the parameters of the fungus image classification model according to the fungus category prediction loss. The fungus image classification model is trained multiple times according to the above process until the training end condition is met.

[0145] In order to train a fungus image classification model with higher classification accuracy, this embodiment provides a second implementation method of using a training data subset to train the fungus image classification model to be trained:

[0146] Step b1: obtaining a training fungus image from the training data subset as the first training fungus image.

[0147] Step b2, inputting the first training fungus image into the fungus image classification model to obtain a fungus category prediction result of the first training fungus image;

[0148] like Figure 8As shown, the fungus image classification model may include a feature extraction module 801 and a fungus category prediction module 802, wherein the feature extraction module 801 includes a plurality of cascaded feature extraction submodules. The first training fungus image is input into the feature extraction module 801 of the fungus image classification model for feature extraction, and the features extracted by the feature extraction module 801 are input into the fungus category prediction module 802 for fungus category prediction, so as to obtain the fungus category prediction result of the first training fungus image.

[0149] Step b3: determining the fungus category prediction loss according to the fungus category prediction result of the first training fungus image and the fungus category annotated in the first training fungus image.

[0150] The fungus category prediction loss can use cross entropy loss.

[0151] Step b4: Acquire multiple training fungus images at different growth periods from the training data subset as second training fungus images.

[0152] The training data subset contains training fungus images at different growth stages. In addition to being labeled with fungus categories, the training fungus images in the training data subset are also labeled with growth stages.

[0153] The fungus category annotated on each second training fungus image is consistent with the fungus category annotated on the first training fungus image.

[0154] Step b5: Encode the plurality of second training fungus images respectively to obtain image features corresponding to the plurality of second training fungus images respectively.

[0155] like Figure 8 As shown, the plurality of second training fungus images may be encoded respectively based on the encoder to obtain image features respectively corresponding to the plurality of second training fungus images.

[0156] Step b6: extracting features from image features corresponding to the plurality of second training fungus images respectively based on the fungus image classification model.

[0157] like Figure 8 As shown, the image features respectively corresponding to the plurality of second training fungus images are input into the feature extraction module 801 of the fungus image classification model, and the feature extraction module 801 extracts high-dimensional features from the image features respectively corresponding to the plurality of second training fungus images.

[0158] Step b7: using the features extracted from the image features corresponding to the plurality of second training fungi images based on the fungi image classification model as a prediction basis, predicting the growth periods corresponding to the plurality of second training fungi images, and obtaining prediction results of the growth periods corresponding to the plurality of second training fungi images.

[0159] like Figure 8 As shown, the high-dimensional features extracted by the feature extraction module 801 from the image features corresponding to the plurality of second training fungus images are input into the decoder for decoding to obtain the growth period prediction results corresponding to the plurality of second training fungus images.

[0160] Step b8: determining the growth period prediction loss according to the growth period prediction results respectively corresponding to the plurality of second training fungus images and the growth periods respectively marked on the plurality of second training fungus images.

[0161] Step b9: update the parameters of the fungus image classification model according to the category prediction loss and the growth period prediction loss.

[0162] The category prediction loss and the growth period prediction loss can be fused, and then the parameters of the fungus image classification model can be updated according to the fused loss.

[0163] The fungus image classification model is trained multiple times according to the process of steps b1 to b9 above until the training end condition is met.

[0164] This application additionally introduces a growth period prediction task when training the fungus image classification model. The introduction of the growth period prediction task enables the fungus image classification model to extract more distinctive fungus features, thereby being able to predict accurate categories.

[0165] By using N training data subsets obtained by dividing a training data set and training the fungus image classification models to be trained respectively according to the above-mentioned training method, N trained fungus image classification models can be obtained. Then, the fungus image classification model with the best performance can be selected from the N trained fungus image classification models as the target fungus image classification model.

[0166] In order to ensure the stability and reliability of fungus category recognition, the present application may use different training data sets to train multiple target fungus image classification models, and then perform category recognition on the fused fungus image obtained in step S404 based on the multiple target fungus image classification models.

[0167] The fungal category identification method provided in this application has broad application prospects in the fields of agricultural disease monitoring, biological research, and environmental protection, and can effectively improve the recognition efficiency and accuracy of common fungal categories.

[0168] The fungus category identification method provided in the embodiment of the present application is introduced above, and the device corresponding to the fungus category identification method will be introduced below.

[0169] See also Fig. 9 , Fig. 9This is a structural schematic diagram of a fungus category identification device provided in an embodiment of the present application. The fungus category identification device may include: a fungus culture dish image acquisition module 901, a fungus segmentation module 902, an image registration module 903, an image fusion module 904 and a fungus category identification module 905.

[0170] The fungus culture dish image acquisition module 901 is used to acquire a first fungus culture dish image and a second fungus culture dish image.

[0171] The first fungus culture dish image and the second fungus culture dish image are respectively a front image of the culture dish in which fungi are cultured and a back image of the culture dish in which fungi are cultured;

[0172] The fungus segmentation module 902 is used to segment fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image.

[0173] The image registration module 903 is used to perform image registration on the first fungus image and the second fungus image to obtain two registered fungus images.

[0174] The image fusion module 904 is used to fuse the two registered fungus images to obtain a fused fungus image.

[0175] The fungus category recognition module 905 is used to perform fungus category recognition on the fused fungus image to obtain a fungus category recognition result.

[0176] In a possible implementation, when the fungus segmentation module 902 segments the fungus from the first fungus culture dish image and the second fungus culture dish image to obtain the first fungus image and the second fungus image, it is specifically used to:

[0177] Based on a pre-trained fungus segmentation model, fungi are segmented from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image; wherein the fungus segmentation model is trained using training fungus culture dish images with real fungus segmentation results annotated.

[0178] In a possible implementation, when the fungus segmentation module 902 segments the fungus from any one of the first fungus culture dish image and the second fungus culture dish image based on the pre-trained fungus segmentation model, it is specifically used to:

[0179] Based on the pre-trained fungus segmentation model, the fungus culture dish image is encoded to obtain features of multiple scales of the fungus culture dish image;

[0180] Based on the fungus segmentation model, features at multiple scales are fused to obtain fused features;

[0181] Based on the fungus segmentation model and taking the fused features as a basis, the fungus segmentation result of the fungus culture dish image is predicted.

[0182] In a possible implementation, when the fungus segmentation module 902 fuses features of multiple scales based on the fungus segmentation model to obtain fused features, it is specifically used to:

[0183] Based on the fungus segmentation model, the features of each scale are centralized to obtain the centralized features of each scale;

[0184] Based on the fungus segmentation model, the centralized features of each scale are aligned to the same spatial size to obtain the aligned features of each scale;

[0185] Based on the fungus segmentation model, the aligned features of each scale are fused in the channel dimension to obtain the fused features.

[0186] In a possible implementation, when the fungus category recognition module 905 performs fungus category recognition on the fused fungus image to obtain the fungus category recognition result, it is specifically used to:

[0187] Based on the pre-trained multiple target fungus image classification models, the fungus categories corresponding to the fused fungus images are predicted respectively to obtain multiple fungus category prediction results, wherein the multiple target fungus image classification models are trained using different training data sets, and each training data in each training data set is a training fungus image labeled with a fungus category;

[0188] According to the prediction results of multiple fungal categories, the final fungal category is determined.

[0189] In one possible implementation, any training data set includes a first part of training data and a second part of training data, wherein the second part of training data is obtained by performing data enhancement on the first part of training data, wherein the data enhancement method includes one or more of the following methods: contrast-limited adaptive histogram equalization, flipping, translation, scaling, cropping, adding noise, brightness adjustment, and contrast adjustment.

[0190] The fungus category identification device provided in the embodiment of the present application further includes a model training module. The model training module is used to train multiple target fungus image classification models using multiple different training data sets.

[0191] In a possible implementation, when a model training module is trained with a training data set to obtain a target fungus image classification model, it is specifically used to:

[0192] Dividing the training data in the training data set into N parts to obtain N training data subsets, where N is an integer greater than 1;

[0193] Using N training data subsets to respectively train the fungus image classification models to be trained, and obtaining N trained fungus image classification models;

[0194] From the N trained fungus image classification models, the fungus image classification model with the best performance is selected as the target fungus image classification model.

[0195] In a possible implementation, when the model training module uses any training data subset to train the fungus image classification model to be trained, it is specifically used to:

[0196] Acquire a training fungus image from the training data subset as a first training fungus image;

[0197] Inputting the first training fungus image into the fungus image classification model to obtain a fungus category prediction result of the first training fungus image;

[0198] Determining a fungus category prediction loss according to a fungus category prediction result of the first training fungus image and a fungus category annotated by the first training fungus image;

[0199] According to the prediction loss of fungal categories, the parameters of the fungal image classification model are updated.

[0200] In a possible implementation, the training data subset includes training fungus images at different growth stages, and the training fungus images in the training data subset are also labeled with the growth stages.

[0201] The model training module, when training the fungus image classification model to be trained with any of the training data subsets, is also used to:

[0202] Acquire a plurality of training fungus images at different growth stages from the training data subset as second training fungus images, wherein the fungus category annotated on each second training fungus image is consistent with the fungus category annotated on the first training fungus image;

[0203] encoding the plurality of second training fungus images respectively to obtain image features respectively corresponding to the plurality of second training fungus images;

[0204] Extracting features from image features respectively corresponding to the plurality of second training fungus images based on the fungus image classification model;

[0205] The growth periods respectively corresponding to the plurality of second training fungi images are predicted based on the features extracted from the image features respectively corresponding to the plurality of second training fungi images based on the fungi image classification model, and the growth period prediction results respectively corresponding to the plurality of second training fungi images are obtained;

[0206] Determining a growth period prediction loss according to the growth period prediction results respectively corresponding to the plurality of second training fungus images and the growth periods respectively marked on the plurality of second training fungus images;

[0207] According to the prediction loss of fungal categories, the parameters of the fungal image classification model are updated, including:

[0208] According to the prediction loss of fungal category and the prediction loss of growth period, the parameters of the fungal image classification model are updated.

[0209] The fungus category identification device provided in the embodiment of the present application has high identification efficiency, high identification accuracy and high identification stability.

[0210] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0211] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0212] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0213] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0214] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the fungus category identification method provided in the above embodiment.

[0215] An embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the fungus category identification method provided in the above embodiment.

[0216] The embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the fungus category identification method provided in the above embodiment.

[0217] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0218] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0219] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0220] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for identifying fungal species, characterized in that: include: Acquire a first fungus culture dish image and a second fungus culture dish image, wherein the first fungus culture dish image and the second fungus culture dish image are respectively a front image of a culture dish in which fungi are cultured and a back image of the culture dish in which fungi are cultured; Based on the pre-trained fungus segmentation model, the fungus culture dish image is encoded to obtain features of multiple scales of the fungus culture dish image; Based on the fungus segmentation model, the features at the multiple scales are fused to obtain fused features; Based on the fungus segmentation model and the fused features, the fungus segmentation result of the fungus culture dish image is predicted, and fungi are segmented from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image; wherein the fungus segmentation model is trained using a training fungus culture dish image with real fungus segmentation results marked; Performing image registration on the first fungus image and the second fungus image to obtain two registered fungus images; fusing the two registered fungus images to obtain a fused fungus image; Performing fungus category recognition on the fused fungus image to obtain a fungus category recognition result; The performing fungus category recognition on the fused fungus image to obtain a fungus category recognition result includes: Based on a plurality of pre-trained target fungus image classification models, the fungus categories corresponding to the fused fungus images are predicted respectively to obtain a plurality of fungus category prediction results, wherein the plurality of target fungus image classification models are trained using different training data sets, and each training data in each training data set is a training fungus image labeled with a fungus category; The final fungus category is determined according to the multiple fungus category prediction results.

2. The fungus classification identification method according to claim 1, characterized in that: Based on the fungus segmentation model, the features of the multiple scales are fused to obtain fused features, including: Based on the fungus segmentation model, the features of each scale are centralized to obtain the features after the centralization of each scale; Based on the fungus segmentation model, aligning the features after the centralization processing at each scale to the same spatial size to obtain the aligned features at each scale; Based on the fungus segmentation model, the aligned features of each scale are fused in the channel dimension to obtain fused features.

3. The fungus classification identification method according to claim 1, characterized in that: Any training data set includes a first part of training data and a second part of training data, where the second part of training data is obtained by performing data enhancement on the first part of training data; The data enhancement method includes one or more of the following methods: contrast-limited adaptive histogram equalization, flipping, translation, scaling, cropping, adding noise, brightness adjustment, and contrast adjustment.

4. The fungus classification identification method according to claim 1, characterized in that: The process of training a target fungus image classification model using a training dataset includes: Dividing the training data in the training data set into N parts to obtain N training data subsets, where N is an integer greater than 1; Using the N training data subsets to respectively train the fungus image classification models to be trained, to obtain N trained fungus image classification models; From the N trained fungus image classification models, a fungus image classification model with the best performance is selected as the target fungus image classification model.

5. The fungus type identification method according to claim 4, characterized in that: Train the fungus image classification model using any subset of the training data, including: Acquire a training fungus image from the training data subset as a first training fungus image; Inputting the first training fungus image into the fungus image classification model to obtain a fungus category prediction result of the first training fungus image; Determining a fungus category prediction loss according to a fungus category prediction result of the first training fungus image and a fungus category annotated by the first training fungus image; According to the fungus category prediction loss, parameters of the fungus image classification model are updated.

6. The fungus type identification method according to claim 5, characterized in that: The training data subset contains training fungus images at different growth stages, and the training fungus images in the training data subset are also annotated with the growth stages; Train the fungus image classification model using any of the training data subsets, including: Acquire a plurality of training fungus images at different growth stages from the training data subset as second training fungus images, wherein the fungus category annotated on each second training fungus image is consistent with the fungus category annotated on the first training fungus image; encoding the plurality of second training fungus images respectively to obtain image features respectively corresponding to the plurality of second training fungus images; Extracting features from image features respectively corresponding to the plurality of second training fungus images based on a fungus image classification model; The growth periods respectively corresponding to the plurality of second training fungi images are predicted based on the features extracted from the image features respectively corresponding to the plurality of second training fungi images based on the fungi image classification model, so as to obtain the prediction results of the growth periods respectively corresponding to the plurality of second training fungi images; determining a growth period prediction loss according to the growth period prediction results respectively corresponding to the plurality of second training fungus images and the growth periods respectively marked on the plurality of second training fungus images; The updating of parameters of the fungus image classification model according to the fungus category prediction loss includes: The parameters of the fungus image classification model are updated according to the fungus category prediction loss and the growth period prediction loss.

7. A fungus type identification device, characterized in that: include: Fungal culture dish image acquisition module, fungal segmentation module, image registration module, image fusion module and fungal category recognition module; The fungus culture dish image acquisition module is used to acquire a first fungus culture dish image and a second fungus culture dish image, wherein the first fungus culture dish image and the second fungus culture dish image are respectively a front image of a culture dish in which fungi are cultured and a back image of the culture dish in which fungi are cultured; The fungus segmentation module is used to encode the fungus culture dish image based on a pre-trained fungus segmentation model to obtain features of multiple scales of the fungus culture dish image; based on the fungus segmentation model, fuse the features of the multiple scales to obtain fused features; based on the fungus segmentation model and based on the fused features, predict the fungus segmentation result of the fungus culture dish image, and segment fungi from the first fungus culture dish image and the second fungus culture dish image respectively to obtain a first fungus image and a second fungus image; wherein the fungus segmentation model is trained using a training fungus culture dish image with a real fungus segmentation result annotated; The image registration module is used to perform image registration on the first fungus image and the second fungus image to obtain two registered fungus images; The image fusion module is used to fuse the two registered fungus images to obtain a fused fungus image; The fungus category recognition module is used to perform fungus category recognition on the fused fungus image to obtain a fungus category recognition result; When the fungus category recognition module performs fungus category recognition on the fused fungus image to obtain a fungus category recognition result, it is specifically used to: Based on the pre-trained multiple target fungus image classification models, the fungus categories corresponding to the fused fungus images are predicted respectively to obtain multiple fungus category prediction results, wherein the multiple target fungus image classification models are trained using different training data sets, and each training data in each training data set is a training fungus image labeled with a fungus category; According to the prediction results of multiple fungal categories, the final fungal category is determined.

Images