Image processing method, electronic device, and recording medium

Abstract

As an aspect of the present disclosure, an image processing method is proposed, which is executed on an electronic device including one or more processors and one or more memories having instructions executed by the one or more processors, and includes obtaining a first unseparated image of a sample including a first biomolecule labeled with a first fluorescent substance and an unlabeled second biomolecule, obtaining a second unseparated image of the sample including the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with a second fluorescent substance, and generating a separated image of the second biomolecule based on the first unseparated image and the second unseparated image.

Description

[Technical Field]

[0001] This disclosure relates to a technology for processing images. [Background technology]

[0002] Fluorescence imaging of biological samples is a technique for indirectly observing biological materials inside a sample by labeling biomolecules contained in the sample with a fluorescent substance and then photographing the light emitted from this fluorescent substance. When light is shone on a fluorescent substance, it absorbs the light and enters an excited state, and then emits light again, but this time it emits light with a longer wavelength than the light it absorbed. For example, a fluorescent substance absorbs light in a specific wavelength range (e.g., 350-400 nm) and emits light in a specific wavelength range (e.g., 400-600 nm). At this time, the degree of change in the excitation of the fluorescent substance when the wavelength is changed is called the excitation spectrum, and the intensity of the emitted light when the wavelength is changed is called the emission spectrum.

[0003] Conventional techniques have a limitation in that, in order to observe multiple biomolecules contained in a sample, the emission spectra must be kept from overlapping as much as possible. This limits the number of fluorescent substances that can be observed. [Overview of the project] [Problems that the invention aims to solve]

[0004] This disclosure provides technology for processing images. [Means for solving the problem]

[0005] As one aspect of this disclosure, an image processing method is proposed. The method is an image processing method performed in an electronic device including one or more processors and one or more memories on which instructions executed by the one or more processors are recorded, and includes: acquiring a first unseparated image of a sample containing a first biomolecule labeled with a first fluorescent substance and a second biomolecule that is not labeled; acquiring a second unseparated image of the sample containing the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with a second fluorescent substance; and generating a separated image of the second biomolecule based on the first unseparated image and the second unseparated image.

[0006] In one embodiment, the first unseparated image and the second unseparated image may each be images captured by detecting light of the same specific wavelength band from the sample.

[0007] In one embodiment, the first unseparated image and the second unseparated image are images acquired based on the same emission filter, and the emission filter may be a filter that allows light in a specific wavelength band to pass through.

[0008] In one embodiment, the first fluorescent substance and the second fluorescent substance may be the same fluorescent substance.

[0009] In one embodiment, the first fluorescent substance and the second fluorescent substance may be determined such that a first wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of the first fluorescent substance and a second wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of the second fluorescent substance satisfy predetermined conditions.

[0010] In one embodiment, the predetermined condition may be a condition that is satisfied when the difference between the first wavelength value and the second wavelength value is less than or equal to a predetermined threshold.

[0011] In one embodiment, the predetermined condition may be a condition that is satisfied when the ratio of the smaller wavelength value to the larger wavelength value among the first wavelength value and the second wavelength value is greater than or equal to a predetermined threshold.

[0012] In one embodiment, the second unseparated image may be obtained by taking a first unseparated image of the sample, then labeling the second biomolecule contained in the sample with the second fluorescent substance, and then taking an image of the sample.

[0013] In one embodiment, generating the separated image may include using an unmixing matrix to compute the first unseparated image and the second unseparated image.

[0014] In one embodiment, the value of at least one element included in the unmixing matrix may be determined based on a trained artificial neural network model.

[0015] In one embodiment, the process further includes obtaining a third unseparated image of the sample comprising the first biomolecule labeled with the first fluorescent substance, the second biomolecule labeled with the second fluorescent substance, and the third biomolecule labeled with the third fluorescent substance, wherein the generation may further include generating a separated image of the third biomolecule based on the third unseparated image. The first unseparated image is an image obtained by photographing the sample comprising the unlabeled second biomolecule and the unlabeled third biomolecule, and the second unseparated image may be an image obtained by photographing the sample comprising the unlabeled third biomolecule.

[0016] In one embodiment, the second unseparated image is obtained by photographing the sample after labeling the second biomolecule contained in the sample with the second fluorescent substance after photographing the first unseparated image of the sample, and the third unseparated image is obtained by photographing the sample after labeling the third biomolecule contained in the sample with the third fluorescent substance after photographing the second unseparated image of the sample.

[0017] According to another aspect of the present disclosure, an electronic device for image processing is proposed. The electronic device includes one or more processors and one or more memories in which instructions executed by the one or more processors are recorded. The one or more processors obtain a first unseparated image of a sample including a first biomolecule labeled with a first fluorescent substance and a second biomolecule not labeled, and obtain a second unseparated image of the sample including the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with a second fluorescent substance, and may generate a separated image of the second biomolecule based on the first unseparated image and the second unseparated image.

[0018] In one embodiment, the first unseparated image and the second unseparated image may each be an image photographed by detecting light in the same specific wavelength band with respect to the sample.

[0019] In one embodiment, the first unseparated image and the second unseparated image are each an image obtained based on the same emission filter, and the emission filter may be a filter that passes light in a specific wavelength band.

[0020] In one embodiment, the first fluorescent substance and the second fluorescent substance may be the same fluorescent substance as each other.

[0021] In one embodiment, the first fluorescent substance and the second fluorescent substance may be determined such that a first wavelength value at which the intensity of the emission signal is maximum within the emission spectrum of the first fluorescent substance, and a second wavelength value at which the intensity of the emission signal is maximum within the emission spectrum of the second fluorescent substance satisfy a predetermined condition.

[0022] In one embodiment, the second unseparated image may be obtained by photographing the sample after photographing the first unseparated image of the sample, and then photographing the sample after labeling the second biomolecule contained in the sample with the second fluorescent substance.

[0023] In one embodiment, the electronic device further includes a photographing unit, and the one or more processors obtain the first unseparated image by photographing the sample including the first biomolecule labeled with the first fluorescent substance and the unlabeled second biomolecule with the photographing unit, and obtain the second unseparated image by photographing the sample including the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with the second fluorescent substance with the photographing unit.

[0024] In one embodiment, the one or more processors may generate the separated image based on a dependency evaluation value calculated between the first unseparated image and the second unseparated image.

[0025] In one embodiment, the dependency evaluation value may be at least one of a mutual information amount, a Kullback-Leibler Divergence value, a cross entropy value, or a Rand Index.

[0026] In one embodiment, the one or more processors may generate the separated image based on an output value calculated by a learned artificial neural network based on the first unseparated image and the second unseparated image.

[0027] Another aspect of the present disclosure proposes a non-temporary computer-readable recording medium that records instructions for image processing. The non-temporary computer-readable recording medium, when executed by one or more processors, records instructions that cause one or more processors to perform an operation, wherein the instructions may cause one or more processors to acquire a first unseparated image of a sample containing a first biomolecule labeled with a first fluorescent substance and a second unlabeled biomolecule; acquire a second unseparated image of the sample containing the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with the second fluorescent substance; and generate a separated image of the second biomolecule based on the first and second unseparated images. [Effects of the Invention]

[0028] The image processing method described herein does not require the inactivation or removal of fluorescent substances that was required in conventional methods, thus reducing the time required for image processing. [Brief explanation of the drawing]

[0029] [Figure 1] This figure shows a system including a server, user terminals, and a communication network in one embodiment of the present disclosure. [Figure 2] This is a block diagram showing a server in one embodiment of the contents disclosed herein. [Figure 3] This is a block diagram showing a user terminal in one embodiment of the contents disclosed herein. [Figure 4] This is an illustrative diagram illustrating the properties of fluorescent substances used to label biomolecules in conventional image processing methods. [Figure 5] This figure conceptually illustrates the process of generating multiple separated images from multiple unseparated images in one embodiment of the present disclosure. [Figure 6]This flowchart illustrates the operation of a server in one embodiment of the present disclosure that generates separated images for each of several biomolecules from multiple unseparated images of a sample using an unmixing matrix. [Figure 7] This is an illustrative diagram showing histograms determined in different ways from a multichannel image in one embodiment of the present disclosure. [Figure 8] This is a flowchart illustrating the operation of a server in one embodiment of the present disclosure that determines a histogram based on the values ​​of pixels at the same position in each of a plurality of single-channel images. [Figure 9] This is a flowchart illustrating the operation of a server in one embodiment of the present disclosure that determines a histogram based on the values ​​of pixels at different positions contained in each of a plurality of single-channel images. [Figure 10] This is an illustrative diagram showing histograms determined in different ways from a multichannel image in another embodiment of the present disclosure. [Figure 11] This is a flowchart illustrating the operation of a server that updates the parameters of an unmixing matrix in one embodiment of the present disclosure. [Figure 12] This figure conceptually illustrates a process for evaluating dependencies between multiple separated images based on an artificial neural network model in one embodiment of the present disclosure. [Figure 13] This figure conceptually illustrates a process in another embodiment of the present disclosure for evaluating dependencies between multiple separated images based on an artificial neural network model. [Figure 14] This is an illustrative diagram showing the process of sequentially acquiring multiple images in one embodiment of the present disclosure. [Figure 15] This is an illustrative diagram illustrating a method in one embodiment of the present disclosure for obtaining a separated image of at least one biomolecule from two consecutive unseparated images among a plurality of sequentially acquired unseparated images. [Figure 16]This is an illustrative diagram illustrating a method in one embodiment of the present disclosure for obtaining separated images for at least two biomolecules from three consecutive unseparated images among a plurality of unseparated images acquired sequentially. [Figure 17] This is a flowchart illustrating the operation of a server in one embodiment of the present disclosure that generates a separated image of at least one biomolecule based on two consecutive unseparated images. [Figure 18] This is an illustrative diagram showing the emission spectra of multiple fluorescent substances and the wavelength ranges that pass through a specific emission filter. [Figure 19] This is an illustrative diagram showing the emission spectra of multiple fluorescent substances and the wavelength values ​​at which the signal intensity is maximized for each emission spectrum. [Modes for carrying out the invention]

[0030] The various embodiments described herein are illustrative for the purpose of clearly illustrating the technical concept of this disclosure, and this disclosure should not be limited by any particular embodiment. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and selective combinations of all or part of each embodiment described herein. Furthermore, the scope of rights relating to the technical concept of this disclosure should not be limited to the various embodiments presented below or their specific descriptions.

[0031] Unless otherwise defined, terms used herein, including technical or scientific terms, have the meanings generally understood by those skilled in the art to which this disclosure pertains.

[0032] The expressions “includes,” “may include,” “equip,” “may be equipped,” “have,” and “may have” as used herein mean that the feature in question (e.g., function, operation, or component) exists, but do not exclude the existence of other additional features. In other words, such expressions should be understood as open-ended terms that imply the possibility of including other embodiments.

[0033] As used herein, singular expressions also include plural meanings unless otherwise stated in the context. This also applies to singular expressions used in the claims.

[0034] The terms "first," "second," or "first," "second," etc., used herein, unless otherwise stated in context, are used to distinguish one object from another when referring to multiple similar objects, and do not limit the order or importance of these objects. For example, the multiple fluorescent substances relating to this disclosure are distinguished from each other by being referred to as "first fluorescent substance," "second fluorescent substance," etc. Similarly, the multiple input images relating to this disclosure are distinguished from each other by being referred to as "first input image," "second input image," etc. In the same way, terms used in this disclosure such as "biomolecule," "separation image," and "probability distribution" are distinguished from each other by expressions such as "first," "second," etc.

[0035] Expressions used herein such as “A, B, and C,” “A, B, or C,” “at least one of A, B, and C,” or “at least one of A, B, or C” may mean each item listed or all possible combinations of the listed items. For example, “at least one of A or B” may mean (1) at least one A, (2) at least one B, or (3) all of at least one A and at least one B.

[0036] As used herein, the term “part” may mean software or hardware components such as FPGAs (Field-Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits). However, “part” is not limited to hardware and software. A “part” may be configured to be recorded on an addressable recording medium and may be configured to run one or more processors. In one embodiment, a “part” may include components such as software components, object-oriented software components, class components, and task components, processors, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0037] As used herein, the expression "based on" is used to describe one or more factors that influence an act of decision, judgment, or action described in the phrase or sentence containing the expression, but the expression does not exclude any additional factors that influence the act of decision, judgment, or action in question.

[0038] As used herein, the expression that one component (e.g., the first component) is “connected” or “linked” to another component (e.g., the second component) means not only that the component is directly connected or linked to the other component, but also that it is connected or linked indirectly through a new other component (e.g., the third component).

[0039] As used herein, the expression "configured to" may, depending on the context, mean "set to do," "capable to do," "modified to do," "generated to do," or "capable to do." Such expressions are not limited to meaning "specifically designed in hardware." For example, a processor configured to perform a particular operation may mean a generic-purpose processor capable of performing a particular operation by running software, or a special-purpose computer structured by programming to perform a particular operation.

[0040] In this disclosure, artificial intelligence (AI) refers to technologies that mimic human learning, reasoning, and perceptual abilities and realize them using computers, and may include the concepts of machine learning and symbolic logic. Machine learning (ML) is an algorithmic technology that classifies and learns the features of input data. Artificial intelligence technology analyzes input data using machine learning algorithms, learns the results of this analysis, and makes judgments and predictions based on these learning results. Furthermore, technologies that utilize machine learning algorithms to mimic the cognitive and judgment functions of the human brain are also included in the scope of artificial intelligence. For example, this may include technologies such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0041] In this disclosure, machine learning may mean the process of training a neural network model using experience in processing data, or it may mean that computer software improves its data processing capabilities through machine learning. A neural network model is constructed by modeling the correlations between data, and these correlations may be represented by multiple parameters. An artificial neural network model extracts features from given data, analyzes them, and derives correlations between the data. Machine learning can be said to be the process of continuously optimizing the parameters of the neural network model by repeating this process. For example, an artificial neural network model may learn the mapping (correlation) between inputs and outputs for data given as input-output pairs. Alternatively, even if only input data is given, an artificial neural network model may learn relationships by deriving regularities from the given data.

[0042] In this disclosure, an artificial neural network, artificial intelligence learning model, machine learning model, or artificial neural network model may be designed to realize the structure of the human brain on a computer, and may include multiple network nodes that mimic neurons in a human nervous system and have weighted values. The multiple network nodes may have interconnections with each other by mimicking the synaptic activity of neurons, in which neurons send and receive signals using synapses. In an artificial neural network, the multiple network nodes send and receive data according to convolutional connectivity, while being located in layers of different depths. The artificial neural network may be, for example, an artificial neural network model or a convolutional neural network model.

[0043] Various embodiments of this disclosure will be described below with reference to the attached drawings. In the attached drawings and their descriptions, identical or substantially equivalent components will be given the same reference numerals. In the following descriptions of various embodiments, redundant descriptions of identical or corresponding components may be omitted, but this does not mean that the component is not included in that embodiment.

[0044] Figure 1 shows a system in one embodiment of the present disclosure, including a server 100, a user terminal 200, and a communication network 300. The server 100 and the user terminal 200 may send and receive information from each other via the communication network 300.

[0045] Server 100 may be an electronic device that performs the image processing operations related to this disclosure. Server 100 is an electronic device that transmits information or image processing results to a user terminal 200 connected by wire or wireless, and may be, for example, an application server, a proxy server, a cloud server, etc.

[0046] The user terminal 200 may be the terminal of a user attempting to receive the image processing results. The user terminal 200 may be at least one of the following: a smartphone, a tablet computer, a personal computer, a mobile phone, a personal digital assistant (PDA), an audio player, or a wearable device. The communication network 300 may include both wired and wireless networks.

[0047] The communication network 300 may be configured to enable data exchange between the server 100 and the user terminal 200. The wired communication network may include, for example, a communication network using methods such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), RS-232 (Recommended Standard-232), or POTS (Plain Old Telephone Service). The wireless communication network may include, for example, communication networks using methods such as eMBB (enhanced Mobile Broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (Long-Term Evolution), LTE-A (LTE Advance), NR (New Radio), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), WiBro (Wireless Broadband), WiFi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). The communication network 300 in this specification is not limited to the examples given above, and may include a wide variety of communication networks without limitation, as long as they enable data exchange between multiple entities and devices.

[0048] In the disclosures of this specification, when describing the configuration or operation of a device, the term "device" may be used to refer to the device being described, and the term "external device" may be used to refer to a device that exists externally from the perspective of the device being described. For example, if server 100 is described as a "device," then from the perspective of server 100, user terminal 200 may be an "external device." Also, for example, if user terminal 200 is described as a "device," then from the perspective of user terminal 200, server 100 may be an "external device." In other words, server 100 and user terminal 200 may be referred to as a "device" and an "external device," respectively, depending on the perspective of the operating entity, or they may be referred to as an "external device" and a "device," respectively.

[0049] Figure 2 is a block diagram showing a server 100 in one embodiment of what is disclosed herein. The server 100 may include one or more processors 110, a communication interface 120, or memory 130 as components. In one embodiment, at least one of these components of the server 100 may be omitted, or other components may be added to the server 100. In one embodiment, in addition or alternative to, some components may be implemented as an integrated system, or as one or more objects. At least some of the internal or external components of the server 100 may be connected to each other via a bus, GPIO (General Purpose Input / Output), SPI (Serial Peripheral Interface), or MIPI (Mobile Industry Processor Interface), etc., to send and receive data or signals.

[0050] One or more processors 110 may be referred to as processor 110. The term processor 110 may mean a collection of one or more processors unless the context clearly indicates otherwise. A processor 110 may execute software (e.g., instructions or programs) and control at least one component of a server 100 connected to the processor 110. A processor 110 may also perform various operations such as calculations, processing, data generation, or manipulation. Furthermore, a processor 110 may load data, etc., from or record data, etc., into memory 130.

[0051] The communication interface 120 may perform wireless or wired communication between the server 100 and other devices (e.g., a user terminal 200 or another server). For example, the communication interface 120 may perform wireless communication using methods such as eMBB, URLLC, MMTC, LTE, LTE-A, NR, UMTS, GSM, CDMA, WCDMA, WiBro, WiFi, Bluetooth, NFC, GPS, or GNSS. Alternatively, the communication interface 120 may perform wired communication using methods such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), RS-232 (Recommended Standard-232), or POTS (Plain Old Telephone Service).

[0052] Memory 130 may record a variety of data. The data recorded in memory 130 is data that is acquired, processed, or used by at least one component of server 100, and may include software (e.g., instructions or programs). Memory 130 may include volatile or non-volatile memory. The term memory 130 may mean a set of one or more memories unless the context clearly indicates otherwise. The expressions “set of instructions recorded in memory 130” or “programs recorded in memory 130” as described herein may be used to refer to an operating system, application, or middleware that provides a variety of functions to an application to enable the application to take advantage of the resources of server 100 for control of server 100. In one embodiment, when processor 110 performs a particular operation, memory 130 may be executed by processor 110 to record instructions corresponding to that particular operation.

[0053] In one embodiment, the server 100 may transmit data resulting from calculations performed by the processor 110, data received via the communication interface 120, or data recorded in the memory 130 to an external device. The external device may be a device for displaying, showing, or outputting the received data.

[0054] In one embodiment, the server 100 may further include an input unit 140. The input unit 140 may transmit data received from an external source to at least one component included in the server 100. For example, the input unit 140 may include at least one of a mouse, a keyboard, or a touchpad.

[0055] In one embodiment, the server 100 may further include an output unit 150. The output unit 150 may display (output) or transmit (distribute) information processed by the server 100 to an external party. For example, the output unit 150 may visually display the information processed by the server 100. The output unit 150 may display UI (User Interface) information or GUI (Graphic User Interface) information, etc. In this case, the output unit 150 may include at least one of the following: a liquid crystal display (LCD), a thin-film transistor-liquid crystal display (TFT-LCD), an organic light-emitting diode (OLED), a flexible display, a three-dimensional display (3D display), or an e-ink display. Alternatively, for example, the output unit 150 may audibly display the information processed by the server 100. The output unit 150 may display audio data in any audio file format (e.g., MP3, FLAC, WAV, etc.) using an audio device. In this case, the output unit 150 may include at least one of speakers, a headset, or headphones. Alternatively, the output unit 150 may transmit information processed by the server 100 to an external output device. The output unit 150 may transmit or distribute information processed by the server 100 to an external output device using the communication interface 120. The output unit 150 may transmit or distribute information processed by the server 100 to an external output device using a separate output communication interface.

[0056] In one embodiment, the server 100 may further include an imaging unit (not shown). The imaging unit may be, for example, a camera or a camera equipped with a microscope. The processor 110 may control the imaging unit to capture an image of an object (e.g., a sample), acquire the captured image, or record it in the memory 130. Alternatively, if the server 100 includes an output unit 150, the processor 110 may control the imaging unit to capture an image of an object and display the captured image on the output unit 150. In another embodiment, the server 100 may acquire the captured image from an external imaging device.

[0057] Figure 3 is a block diagram showing a user terminal 200 in one embodiment of the subject matter disclosed herein. The user terminal 200 may include one or more processors 210, a communication interface 220, or a memory 230 as components. The user terminal 200 may further include at least one input unit 240 or an output unit 250.

[0058] The processor 210 may execute software (e.g., instructions or programs) and control at least one component of the user terminal 200 connected to the processor 110. The processor 210 may also perform various operations such as calculations, processing, data generation, or manipulation. Furthermore, the processor 210 may load data from or record data in the memory 230.

[0059] The communication interface 220 may perform wireless or wired communication between the user terminal 200 and other devices (e.g., server 100 or other user terminals). For example, the communication interface 220 may perform wireless communication using methods such as eMBB, URLLC, MMTC, LTE, LTE-A, NR, UMTS, GSM, CDMA, WCDMA, WiBro, WiFi, Bluetooth, NFC, GPS, or GNSS. Alternatively, the communication interface 220 may perform wired communication using methods such as USB, HDMI, RS-232, or POTS.

[0060] Memory 230 may record a variety of data. The data recorded in memory 230 is data acquired, processed, or used by at least one component of the user terminal 200, and may include software (e.g., instructions or programs). Memory 230 may include volatile or non-volatile memory. The term memory 230 may mean a set of one or more memories unless the context clearly indicates otherwise. The expressions “set of instructions recorded in memory 230” or “programs recorded in memory 230” as described herein may be used to refer to an operating system, application, or middleware that provides a variety of functions to an application to enable the application to take advantage of the resources of the user terminal 200, etc. In one embodiment, when the processor 210 performs a particular operation, memory 230 may record instructions executed by the processor 210 that correspond to that particular operation.

[0061] In one embodiment, the user terminal 200 may further include an input unit 240. The input unit 240 may transmit data received from an external source to at least one component included in the user terminal 200. For example, the input unit 240 may include at least one of a mouse, a keyboard, or a touchpad.

[0062] In one embodiment, the user terminal 200 may further include an output unit 250. The output unit 250 may display (output) or transmit (distribute) information processed by the user terminal 200 to an external party. For example, the output unit 250 may visually display the information processed by the user terminal 200. The output unit 250 may display UI (User Interface) information or GUI (Graphic User Interface) information, etc. In this case, the output unit 250 may include at least one of the following: liquid crystal display (LCD), thin-film transistor liquid crystal display (TFT-LCD), organic light-emitting diode (OLED), flexible display, three-dimensional display (3D display), or electronic paper display (E-ink display). Alternatively, for example, the output unit 250 may audibly display the information processed by the user terminal 200. The output unit 250 may display audio data in any audio file format (e.g., MP3, FLAC, WAV, etc.) using an audio device. In this case, the output unit 250 may include at least one of a speaker, a headset, or headphones. Alternatively, for example, the output unit 250 may transmit information processed by the user terminal 200 to an external output device. The output unit 250 may transmit or distribute information processed by the user terminal 200 to an external output device using the communication interface 220. The output unit 250 may also transmit or distribute information processed by the user terminal 200 to an external output device using a separate output communication interface.

[0063] In one embodiment, the user terminal 200 may further include an imaging unit (not shown). The imaging unit may be, for example, a camera or a camera equipped with a microscope. The processor 210 may capture an image of an object (e.g., a sample) by controlling the imaging unit, acquire the captured image, or record it in the memory 230. The user terminal 200 may display the captured image on the output unit 250. The user terminal 200 may transmit the captured image to the server 100. The user terminal 200 may also acquire an image captured from an external imaging device.

[0064] In the following explanation, for the sake of clarity, the entity performing the operation may be omitted. In this case, each operation may be understood to be performed by the server 100. However, the method relating to this disclosure may be performed by the user terminal 200, or some of the operations included in the method may be performed by the user terminal 200 and the remainder by the server 100.

[0065] Figure 4 is an illustrative diagram of the properties of fluorescent substances used to label biomolecules using conventional image processing methods. Generally, in order to observe multiple biomolecules in a single biological sample, it is necessary to label the multiple biomolecules with different fluorescent substances and then acquire an image of each biomolecule individually. In this disclosure, a biological sample is also referred to as a "sample." An image of each biomolecule contained in the sample may be acquired, for example, by irradiating the fluorescent substance with light of a specific wavelength to which the fluorescent substance reacts, filtering the light emitted by the excited fluorescent substance with a corresponding emission filter, and photographing the light that has passed through the emission filter. In this disclosure, an "emission filter" may be a filter that allows light in a specific wavelength band to pass through.

[0066] Graph 401, the first graph relating to the emission spectra of light emitted by fluorescent materials, exemplifies the emission spectra of several fluorescent materials (Alexa405, Alexa488, Alexa546, Alexa647).

[0067] The fluorescent substance "Alexa405" may be a fluorescent substance that absorbs light in a specific wavelength range and then emits light having a wavelength between approximately 400 nm and 500 nm. An image of a biomolecule stained with the fluorescent substance "Alexa405" may be obtained by filtering the light emitted by the fluorescent substance "Alexa405" with a first emission filter and capturing the light in the wavelength range that passes through the first emission filter. The wavelength range 410 that passes through the first emission filter may be, for example, between 470 nm and 520 nm.

[0068] The fluorescent substance "Alexa488" may be a fluorescent substance that absorbs light in a specific wavelength range and then emits light having a wavelength between approximately 500 nm and 600 nm. In this case, an image of a biomolecule stained with the fluorescent substance "Alexa488" may be obtained by filtering the light emitted by the fluorescent substance "Alexa488" with a second emission filter and photographing the light in the wavelength band 430 that has passed through the second emission filter. The wavelength band 430 that passes through the second emission filter may be, for example, between 500 nm and 560 nm.

[0069] The fluorescent substance "Alexa546" may be a fluorescent substance that absorbs light in a specific wavelength range and then emits light with wavelengths between approximately 550 nm and 650 nm. In this case, an image of a biomolecule stained with the fluorescent substance "Alexa546" may be obtained by filtering the light emitted by the fluorescent substance "Alexa546" with a third emission filter and photographing the light in wavelength band 450 that has passed through the third emission filter. The wavelength band 450 that passes through the third emission filter may be, for example, between 565 nm and 590 nm.

[0070] The fluorescent substance "Alexa647" may be a fluorescent substance that absorbs light in a specific wavelength range and then emits light with a wavelength between approximately 650 nm and 750 nm. In this case, an image of a biomolecule stained with the fluorescent substance "Alexa647" may be obtained by filtering the light emitted by the fluorescent substance "Alexa647" with a fourth emission filter and capturing the light in wavelength band 470 that passes through the fourth emission filter. The wavelength band 470 that passes through the fourth emission filter may be, for example, between 660 nm and 740 nm.

[0071] The wavelength ranges of light absorbed by each fluorescent substance for excitation, as shown in Graph 401, may differ. Furthermore, the specific numerical ranges for the emission spectra of each fluorescent substance described above with reference to Graph 401 are merely illustrative examples and should not limit this disclosure.

[0072] In one embodiment, it is assumed that a sample contains multiple biomolecules, and that fluorescent substances that label each of the multiple biomolecules (i.e., fluorescent substances that bind to each biomolecule) react similarly to light of a specific wavelength (e.g., 350 nm to 400 nm). In this case, in order to obtain an image of each of the multiple biomolecules contained in the sample, it is usually necessary that the emission spectra of the fluorescent substances used to label each of the multiple biomolecules do not overlap at all, or overlap very little. This is because if the spectra of light emitted from different fluorescent substances overlap significantly, the different biomolecules may not be separated and may be included in the same single image.

[0073] For example, in the second graph 403 of Figure 4, which shows the emission spectra of light emitted by fluorescent substances, we assume that the fluorescent substance "Alexa546" emits light with wavelengths between approximately 550 nm and 650 nm, and the fluorescent substance "CF594" emits light with wavelengths between approximately 575 nm and 700 nm. Under this assumption, the emission spectra of the two fluorescent substances both include the interval between 575 nm and 650 nm. In this case, when an image is taken using a third emission filter to acquire an image of a biomolecule labeled with the fluorescent substance "Alexa546", the wavelength band 450 passing through the third emission filter is between approximately 565 nm and 590 nm, so at least a portion of the biomolecule labeled with the fluorescent substance "CF594" will be included in the captured image. More specifically, in the process of capturing an image of a biomolecule labeled with the fluorescent substance "Alexa546" by acquiring light in the wavelength band 450 that passes through the third emission filter from the fluorescent substance "Alexa46", the imaging device can also acquire at least a portion of the light signal emitted from the fluorescent substance "CF594". As a result, the image of the biomolecule labeled with the fluorescent substance "Alexa546" will include at least a portion of other biomolecules labeled with the fluorescent substance "CF594". Reference numeral 451 in Figure 4 indicates the light signal of another fluorescent substance (e.g., "CF594") acquired when the imaging device uses the third emission filter to capture an image of a biomolecule labeled with the fluorescent substance "Alexa546". Furthermore, even when capturing an image of a biomolecule labeled with the fluorescent substance "CF594" using a fifth emission filter that allows light with wavelengths between approximately 610 nm and 630 nm to pass through, the captured image will also include at least a portion of the biomolecule labeled with the fluorescent substance "Alexa546".In other words, when the imaging device acquires light in the wavelength range that passes through the fifth emission filter from the light emitted by the fluorescent substance "CF594", at least a portion of the light signal emitted from the fluorescent substance "Alexa546" is included in the process of capturing an image of a biomolecule labeled with the fluorescent substance "CF594". As a result, the image of the biomolecule labeled with the fluorescent substance "CF594" will include at least a portion of other biomolecules labeled with the fluorescent substance "Alexa546". Reference numeral 491 in Figure 5 indicates the light signal of another fluorescent substance (e.g., "Alexa546") that the imaging device acquires when capturing an image of a biomolecule labeled with the fluorescent substance "CF594" using the fifth emission filter.

[0074] According to conventional techniques, in order to observe multiple biomolecules contained in a sample, there is a constraint that the emission spectra must be kept as non-overlapping as possible, which limits the number of fluorescent substances that can be used simultaneously to a maximum of four.

[0075] Figure 5 is a conceptual diagram illustrating the process of generating multiple separated images from multiple unseparated images in one embodiment of the present disclosure.

[0076] The unseparated image 510 according to this disclosure may include one or more unseparated images. In embodiments of this disclosure, if the unseparated image 510 includes two or more unseparated images, each unseparated image may be divided into a first unseparated image 510-1, a second unseparated image 510-2, ..., the nth unseparated image 510-n, etc. (where n is a natural number of 2 or more).

[0077] The unseparated image 510 may be an image obtained by the server 100 photographing the sample after the biomolecules contained in the sample have been labeled with a fluorescent substance. As described above, after staining the biomolecules contained in the biological sample with a fluorescent substance (i.e., the biomolecules and the fluorescent substance are physically or chemically bound), when the stained biological sample is irradiated with light, the fluorescent substance contained in the biological sample is excited by absorbing light in a specific wavelength band and emits light in a specific wavelength band. At this time, an unseparated image 510 of the biological sample can be obtained by photographing the light emitted by the fluorescent substance. The unseparated image 510 is an image to which the image processing method according to this disclosure is to be performed, and is distinguished from the "separated image 530" generated by the image processing method according to this disclosure. That is, the unseparated image 510 is an image in which the image processing method according to this disclosure has not been performed, and may also display other biomolecules in addition to the target biomolecules (e.g., biomolecules labeled with other fluorescent substances having similar emission spectra, or biomolecules previously stained in a staining round). In this disclosure, the term "unseparated image" may be used interchangeably with "input image".

[0078] The separation image 530 relating to this disclosure may include one or more separation images. In embodiments relating to this disclosure, if the separation image 530 includes two or more separation images, each separation image may be distinguished as the first separation image 530-1, the second separation image 530-2, ..., the nth separation image 530-n, etc. (where n is a natural number of 2 or more).

[0079] The separated image 530 may be an image obtained as a result of performing the image processing method of the Disclosed on the unseparated image 510. The separated image 530 may be an image that displays a target biomolecule. In this disclosure, “separated image of a specific biomolecule” may mean an image that represents only that biomolecule. For example, the separated image for biomolecule “A” may be an image that shows the shape, size, form, or color of biomolecule “A” contained in the sample. The separated image 530 may be generated to correspond to each biomolecule.

[0080] In various embodiments of this disclosure, each of the multiple unseparated images 510-1, 510-2, ..., 510-n, or the multiple separated images 530-1, 530-2, ..., 530-n, may be a single-channel image having one channel. A single-channel image may mean an image having a single value (for example, a constant between 0 and 255) for each pixel. The pixel value of each pixel in an unseparated image corresponding to a single-channel image may indicate the intensity of light emitted by a fluorescent substance when the imaging unit acquires light to capture an unseparated image. The pixel value of each pixel in a separated image corresponding to a single-channel image may indicate the intensity of light that each pixel has to represent an image of a specific biomolecule as a result of executing the image processing method according to this disclosure. Furthermore, in this disclosure, when a plurality of unseparated images 510-1, 510-2, ..., 510-n or a plurality of separated images 530-1, 530-2, ..., 530-n are described as being included in a multi-channel image, each channel of the multi-channel image may correspond to each of the plurality of unseparated images 510-1, 510-2, ..., 510-n or the plurality of separated images 530-1, 530-2, ..., 530-n. For example, when a plurality of unseparated images containing three single-channel unseparated images are referred to as a "multi-channel unseparated image," each channel of this multi-channel unseparated image may correspond to each of the plurality of unseparated images. Also, when this disclosure displays a plurality of individually acquired unseparated images or plurality of separated images as a multi-channel unseparated image or multi-channel separated image by associating each of them with one channel, the plurality of unseparated images or plurality of separated images may be displayed simultaneously on a single multi-channel image.For example, if three unseparated images are to be displayed as an RGB image with three channels, corresponding to the Red, Green, and Blue channels respectively, the three unseparated images may be displayed simultaneously on the RGB image.

[0081] The image processing method disclosed herein can generate a separated image 530 from an unseparated image 510.

[0082] In one embodiment, the server 100 may generate a plurality of separated images 530-1, 530-2, ..., 530-n by separating the plurality of unseparated images 510-1, 510-2, ..., 510-n based on at least one parameter for separating the plurality of unseparated images 510-1, 510-2, ..., 510-n. For illustrative purposes, assuming that the plurality of unseparated images include two unseparated images, the two separated images generated from the plurality of unseparated images based on at least one parameter may be expressed as shown in the following equation (1).

[0083]

number

[0084] In the present disclosure, the operation of "separating a plurality of unseparated images based on at least one parameter" may be represented based on a matrix. Such a matrix is referred to as an "Unmixing Matrix" in the present disclosure and may include at least one element for generating a plurality of separated images for each biomolecule from a plurality of unseparated images. That is, the unmixing matrix may include at least one element for determining the linear superposition rate between a plurality of unseparated images. When expressing the formula (1) based on a matrix, it may be exemplified as the following formula (2).

[0085]

Equation

[0086] In one embodiment, the server 100 may generate a plurality of separated images 530-1, 530-2,..., 530-n by weighted superposing a plurality of unseparated images in which the value of each pixel is subtracted by a predetermined constant (or also referred to as "constant offset") after subtracting the predetermined constant from each of the plurality of unseparated images 510-1, 510-2,..., 510-n.

[0087]

Equation

[0088] In the above-mentioned formulas (1) to (3), we have assumed that multiple unseparated images include two unseparated images, but this disclosure shall not be limited thereto, and multiple unseparated images may include three or more unseparated images.

[0089] Figure 6 is a flowchart illustrating the operation of a server 100 in one embodiment of the present disclosure, which generates separate images of multiple biomolecules from multiple unseparated images of a sample using an unmixing matrix.

[0090] Server 100 may acquire multiple unseparated images of a sample containing multiple biomolecules (S610). More specifically, processor 110 may acquire multiple unseparated images based on an action in which a user of server 100 inputs multiple unseparated images via input unit 140. Alternatively, processor 110 may acquire multiple unseparated images by capturing an image of the sample via an imaging unit (not shown) of server 100. Furthermore, processor 110 may acquire multiple unseparated images from an external device or user terminal 200 via communication interface 120. Multiple unseparated images (input images) may be represented as a matrix, for example, as shown in equation (4).

[0091]

number

[0092] Next, the server 100 may use the unmixing matrix to generate multiple separated images corresponding to multiple biomolecules from multiple unseparated images (S620). The unmixing matrix according to this disclosure is a square matrix, and may be a square matrix with the same row and column sizes, or a rectangular matrix with different row and column sizes. The unmixing matrix may be expressed, for example, as shown in the following formula (5).

[0093]

number

[0094] The processor 110 may generate multiple separated images by performing calculations on multiple unseparated images using an unmixing matrix. The multiple separated images generated based on the unseparated images according to equation (4) and the unmixing matrix according to equation (5) may be expressed as shown in the following equation (6).

[0095]

number

[0096] As described above, the processor 110 can acquire multiple unseparated images and generate multiple separated images corresponding to each of the multiple biomolecules by performing matrix operations on the multiple unseparated images using an unmixing matrix.

[0097] In one embodiment, the unmixing matrix may be a square matrix with the same row and column sizes. For example, an unmixing matrix that is a square matrix may be expressed as shown in the following equation (7).

[0098]

number

[0099] The following section describes how to determine a histogram or probability distribution function from a multichannel image, referring to Figures 7-10.

[0100] Figure 7 is an illustrative diagram showing histograms determined in different ways from a multichannel image in one embodiment of the present disclosure. The first image 710 may be, for example, a multichannel image containing a plurality of unseparated images. Each channel of the first image 710 may correspond to each of the plurality of unseparated images. Hereinafter, in the present disclosure, the first image 710 will be described as a multichannel image having two channels, but this is merely an assumption for the sake of explanation and the present disclosure shall not be limited thereto. For example, the first image (or the first channel of the first image) included in the first image 710 may be an image (or channel) of light emitted by a fluorescent substance "A2" after staining a biomolecule "A1" contained in a sample with a fluorescent substance "A2". Furthermore, for example, the second image (or the second channel of the first image) included in the first image 710 may be an image (or channel) in which biomolecules "B1" contained in the sample are stained with the fluorescent substance "B2" and the light emitted by the fluorescent substance "B2" is captured, or it may be an image (or channel) in which biomolecules "A1" and "B1" contained in the sample are stained with the fluorescent substances "A2" and "B2" respectively and the light emitted by both the fluorescent substance "A2" and the fluorescent substance "B2" is captured.

[0101] The processor 110 may determine two or more histograms in different ways based on a multichannel image containing multiple single-channel images. For example, the processor 110 may determine one histogram based on the values ​​of pixels at the same positions in the multiple single-channel images, and another histogram based on the values ​​of pixels at different positions in these multiple single-channel images. For example, as shown in Figure 7, the first-1 histogram 730 or the first-2 histogram 750 may each be histograms determined in different ways with respect to the first image 710. In one embodiment, the first-1 histogram 730 may be a histogram determined based on the values ​​of pixels at the same positions in multiple unseparated images contained in the first image 710, and the first-2 histogram 750 may be a histogram determined based on the values ​​of pixels at different positions in multiple unseparated images contained in the first image 710. The histogram determination method according to this disclosure will be described in detail below with reference to Figures 8 and 9.

[0102] Figure 8 is a flowchart illustrating the operation of a server 100 in one embodiment of the present disclosure, which determines a histogram based on the values ​​of pixels at the same positions in each of a plurality of single-channel images. In Figure 8, the first image 710 in Figure 7 is used as an example to illustrate a multi-channel image that includes a plurality of single-channel images.

[0103] The processor 110 may obtain the pixel values ​​of pixels at the same positions in each of the multiple single-channel images (S810). For example, the processor 110 may obtain a pixel value (e.g., 0.9) from the first position 711 in the first image (the first channel of the first image) contained in the first image 710. Alternatively, the processor 110 may obtain a pixel value (e.g., 0.8) from the first position 711 in the second image (the second channel of the first image) contained in the first image 710. As described above, the processor 110 may obtain the pixel values ​​of pixels at the same positions (i.e., the first positions) in each of the multiple unseparated images contained in the first image 710.

[0104] Next, the processor 110 may generate channel value sequence pairs based on the values ​​of pixels at the same positions obtained from each of the multiple single-channel images (S820). A channel value sequence pair may contain multiple elements. For example, a channel value sequence pair for the first image 710 may contain two elements. The channel value sequence pair for the first image 710 may be expressed as, for example, (v1, v2). In this case, the first element (v1) in the channel value sequence pair (v1, v2) may be the value of a pixel included in the first channel of the first image 710, and the second element (i.e., v2) may be the value of a pixel included in the second channel of the first image 710. The value of each element in the channel value sequence pair is a value representing the intensity of light, and may be a real number included in a predetermined interval (e.g., between 0 and 1). If the pixel value obtained from the first position 711 in the first channel of the first image 710 is 0.9, and the pixel value obtained from the first position 711 in the second channel of the first image 710 is 0.8, the processor 110 may generate a channel value sequence pair having the values ​​(0.9, 0.8).

[0105] Next, the processor 110 may generate multiple channel value sequence pairs by repeating the operation of generating channel value sequence pairs a predetermined number of times (S830). For example, the processor 110 may generate channel value sequence pairs equal to the number of repetitions by sequentially repeating the above-described S810 and S820 a predetermined number of times.

[0106] Next, the processor 110 may determine the number of channel value sequence pairs that have the same value among the multiple channel value sequence pairs (S840). For example, assuming that the generated channel value sequence pairs are [(0.9,0.8), (0.8,0.2), (0.6,0.0), (0.9,0.8), (0.9,0.8), (0.9,0.8), (0.8,0.2), (0.6,0.0), (0.8,0.2), (0.6,0.0)], the processor 110 may determine the number of channel value sequence pairs that have the same value, such as four sequence pairs (0.9,0.8), three sequence pairs (0.8,0.2), and three sequence pairs (0.6,0.0).

[0107] Next, the processor 110 may generate a histogram based on the number of channel value sequence pairs (S850). In one embodiment, the histogram may be represented on a two-dimensional coordinate axis. That is, the histogram may have a horizontal axis (x axis) corresponding to the first element of the channel value sequence pair and a vertical axis (y axis) corresponding to the second element of the channel value sequence pair. On the histogram, pixels corresponding to each channel value sequence pair may be represented with different colors or different brightnesses based on the number of channel value sequence pairs. For example, pixels corresponding to each channel value sequence pair on the histogram may be represented brighter as the number of channel value sequence pairs increases, or they may be represented to move from the first color (e.g., blue) towards the second color (e.g., red). Referring to Figure 7, the first-1 histogram 730 may be a histogram generated as a result of the processor 110 executing S810 to S850 described above on the first image 710. Furthermore, the pixel indicated by reference numeral 731 is a pixel on the first-first histogram 730 with an x ​​value of 0.3 and a y value of 0.2, and may correspond to the channel value sequence pair (0.3, 0.2). The pixel indicated by reference numeral 733 is a pixel on the first-first histogram 730 with an x ​​value of 0.6 and a y value of 0.6, and may correspond to the channel value sequence pair (0.6, 0.6). Assuming that the number of channel value sequence pairs with the value (0.3, 0.2) among the channel value sequence pairs generated from the first image 710 is greater than the number of channel value sequence pairs with the value (0.6, 0.6), then on the first-first histogram 730, the pixel 731 corresponding to the channel value sequence pair (0.3, 0.2) may be represented in a brighter color than the pixel 733 corresponding to the channel value sequence pair (0.6, 0.6).

[0108] In another embodiment, the histogram may be represented on a three-dimensional coordinate system. In this case, the histogram may include a first axis (x-axis) corresponding to the first element of the channel value sequence pair, a second axis (y-axis) corresponding to the second element of the channel value sequence pair, and a third axis (z-axis) corresponding to the number of channel value sequence pairs.

[0109] After executing S850, the processor 110 may further perform an operation to determine a probability distribution function from the generated histogram. Hereinafter, in this disclosure, "probability distribution function" will also be referred to as "probability distribution". For example, the processor 110 may approximately determine the probability distribution by normalizing the histogram generated by the execution result of S850. Referring to Figure 7, the probability distribution determined from the first-1 histogram 730 may be a continuous probability distribution having a data distribution similar to the first-1 histogram 730.

[0110] Figure 9 is a flowchart illustrating the operation of a server 100 in one embodiment of the present disclosure, which determines a histogram based on the values ​​of pixels at different positions contained in each of a plurality of single-channel images. In Figure 9, the first image 710 in Figure 7 is used as an example to illustrate a multi-channel image containing a plurality of single-channel images.

[0111] The processor 110 may obtain the pixel values ​​of pixels at different positions contained in each of the multiple single-channel images (S910). For example, the processor 110 may obtain a pixel value (e.g., 0.5) from the 2-1 position 713a in the first image (first channel of the first image) contained in the first image 710. The processor 110 may also obtain a pixel value (e.g., 0.1) from the 2-2 position 713b in the first image (first channel of the first image) contained in the first image 710. The 2-1 position 713a and the 2-2 position 713b may be positions with different coordinate values. As described above, the processor 110 may obtain the pixel values ​​of pixels at different positions contained in each of the multiple unseparated images contained in the first image 710.

[0112] Next, the processor 110 may generate channel value sequence pairs based on the values ​​of pixels at different positions obtained from each of the multiple single-channel images (S920). For example, if the value of the pixel obtained from the 2-1 position 713a in the first channel of the first image 710 is 0.5, and the value of the pixel obtained from the 2-2 position 713b in the second channel of the first image 710 is 0.1, the processor 110 may generate a channel value sequence pair having the values ​​(0.5, 0.1).

[0113] Next, the processor 110 may generate multiple channel value sequence pairs by repeating the operation of generating channel value sequence pairs (i.e., S910 and S920) a predetermined number of times (S930), determine the number of channel value sequence pairs that have the same value among the multiple channel value sequence pairs (S940), and generate a histogram based on the number of channel value sequence pairs (S950). Since S930 to S950 are executed by the processor 110 in the same or similar manner as S830 to S850 described above, the explanation of the overlapping content will be omitted. The first-2 histogram 750 in Figure 7 may be a histogram generated as a result of the processor 110 executing S910 to S950 described above on the first image 710.

[0114] After executing S950, the processor 110 may further perform an operation to determine the probability distribution function from the generated histogram. For example, the processor 110 may approximately determine the probability distribution by normalizing the histogram generated by the execution result of S950. Referring to Figure 7, the probability distribution determined from the first-second histogram 750 may be a continuous probability distribution function having a data distribution similar to that of the first-second histogram 750.

[0115] As described with reference to Figures 8 and 9, the server 100 relating to this disclosure can determine the histogram or probability distribution of a given image in a different manner based on a multichannel image that includes multiple single-channel images.

[0116] In this disclosure, the terms “dependency” or “similarity” between multiple images refer to the degree to which the information contained in multiple images is related to one another, and may be used in contrast to probabilistic independence. When multiple images are highly dependent on each other, pixels corresponding to the same position in each image may have a certain tendency between their pixel values. For example, if the pixel value of the first channel is low among the pixel values ​​corresponding to the same position in each image, there is a high probability that the pixel value of the second channel will be high, and conversely, if the pixel value of the first channel is high, there is a high probability that the pixel value of the second channel will be high. On the other hand, when multiple images are lowly dependent on each other, there may be no particular tendency between the pixel values ​​of pixels corresponding to the same position in each image. In this disclosure, “no particular tendency” between multiple pixel values ​​means that the multiple pixel values ​​do not influence each other, and the magnitude relationship between the multiple pixel values ​​is determined randomly. Also in this disclosure, the term “independence” may be used to refer to the degree to which the information contained in multiple images is independent of each other. In other words, dependency and independence are opposing concepts; the higher the dependency between multiple images, the lower the independence, and the lower the dependency, the higher the independence. The multiple unseparated images relating to this disclosure have a high degree of dependency on each other. Furthermore, the multiple separated images generated from the multiple unseparated images have a low degree of dependency.

[0117] Referring again to Figure 7, the first image 710 may be a multichannel image in which multiple channels correspond to one unseparated image, each contained within multiple unseparated images. The multiple unseparated images may be highly dependent on each other, as described above, and pixels at the same positions contained within each unseparated image may have the same or similar pixel values. Therefore, when determining a histogram based on the values ​​of pixels at the same positions contained within each of the multiple unseparated images, the magnitudes of the first and second elements contained within the sequence pairs of channel values ​​are alternatively the same or similar. For example, a specific correlation may be shown between the values ​​of the first and second elements, as in the first-1 histogram 730 generated based on the values ​​of pixels at the same positions contained within each of the multiple unseparated images.

[0118] In this disclosure, a histogram generated based on the values ​​of pixels at the same positions in each of multiple single-channel images is also called a “joint histogram” for that image. The probability distribution function determined from the joint histogram is also called a “joint probability distribution.” For example, the first-1 histogram 730 may be a joint histogram for multiple unseparated images contained in the first image 710.

[0119] When generating a histogram based on the values ​​of pixels at different locations within multiple unseparated images contained in the first image 710, there may be no particular trend between the magnitudes of the first and second elements in the channel value sequence pair. For example, as in the first-second histogram 750 generated based on the values ​​of pixels at different locations within each of the multiple unseparated images, there may be no particular correlation between the values ​​of the first and second elements in the channel value sequence pair.

[0120] In this disclosure, a histogram generated based on the values ​​of pixels at different positions contained in each of multiple single-channel images is also called a “marginal histogram” for that image. The probability distribution function determined from the marginal histogram is also called a “marginal probability distribution.” For example, the first-second histogram 7500 may be a marginal histogram for multiple unseparated images contained in the first image 710.

[0121] The following describes two or more histograms that are determined when multiple channels in a multichannel image are each separate images, referring to Figure 10.

[0122] Figure 10 is an illustrative diagram showing histograms determined in different ways from a multichannel image in another embodiment of the present disclosure. The second image 1010 may be, for example, a multichannel image containing multiple separated images. The second image 1010 may be a multichannel image containing multiple separated images generated as a result of the processor 110 computing multiple unseparated images using an unmixing matrix. Hereinafter, in the present disclosure, the second image 1010 will be described as a multichannel image having two channels, but this is merely an assumption for the sake of explanation and the present disclosure should not be limited thereto. For example, the first image (or the first channel of the second image) included in the second image 1010 may be an image of biomolecule "A1" contained in the sample. Or, for example, the second image (or the second channel of the second image) included in the second image 1010 may be an image of biomolecule "B1" contained in the sample.

[0123] Histograms 2-1 1030 and 2-2 1050 may be histograms determined for the second image 1010 in a manner similar to that of histograms 1-1 730 and 1-2 750, as described with reference to Figure 7. For example, histogram 2-1 1030 may be a combined histogram determined based on the values ​​of pixels at the same positions within multiple single-channel images contained in the second image 1010, and histogram 2-2 1050 may be a marginal histogram determined based on the values ​​of pixels at different positions within multiple single-channel images contained in the second image 1010. In multiple isolated images, there may not be any particular trend among the pixel values ​​of each corresponding pixel at the same position. That is, histograms determined for multiple isolated images with low interdependence may not show any particular correlation. For example, when generating a second-first histogram 1030 and a second-second histogram 1050 for a second image 1010 containing multiple separated images, there may be no particular trend between the magnitudes of the first and second elements in the channel value sequence pairs in the two generated histograms.

[0124] Figure 11 is a flowchart illustrating the operation of server 100, which updates the parameters of the unmixing matrix, in one embodiment of the present disclosure.

[0125] The processor 110 may acquire multiple unseparated images of a sample containing multiple biomolecules (S1110). The processor 110 may use a first unmixing matrix to generate multiple separated images from the multiple unseparated images, each corresponding to a different biomolecule (S1120). S1110 or S1120 may be executed by the processor 110 in the same or similar manner as S610 or S620 described in Figure 6.

[0126] Next, the processor 110 may evaluate the dependencies between multiple separated images based on the multiple separated images generated by the unmixing matrix (S1130). The dependencies between the multiple separated images may be evaluated based on a specific algorithm executed by the processor 110, or the processor 110 may evaluate them by performing predetermined operations using an artificial neural network model. The processor 110 may also modify the parameters of the unmixing matrix based on the dependency evaluation results so that the dependencies between the multiple separated images are reduced (S1140).

[0127] According to various embodiments of this disclosure, the processor 110 may evaluate the dependencies between multiple isolated images based on histograms generated for the multiple isolated images. Specifically, the processor 110 may calculate a value indicating the dependencies between the multiple isolated images (hereinafter also referred to as the "dependency evaluation value") based on the histograms generated for the multiple isolated images using a predetermined algorithm (or mathematical formula).

[0128] In a first embodiment in which the dependency between multiple isolated images is evaluated based on histograms generated for multiple isolated images, the processor 110 may evaluate the dependency by calculating mutual information between the multiple isolated images. "Mutual information" is a value derived from information theory, and the mutual information between two variables may mean the total amount of information shared by the two variables. For example, the mutual information between two random variables may be 0. The processor 110 may calculate the mutual information based on a combined histogram or a marginal histogram generated based on the multiple isolated images. The mutual information for two isolated images may be expressed, for example, as shown in the following formula (8).

[0129]

number

[0130] In a second embodiment in which the dependency between multiple separated images is evaluated based on histograms generated for multiple separated images, the processor 110 may evaluate the dependency by calculating the Kullback-Leibler divergence value between the multiple separated images. The "Kullback-Leibler divergence" is a function that can be used to calculate the difference between two different probability distributions, and the value of this function represents the difference in information entropy between the two different probability distributions. The larger the Kullback-Leibler divergence value, the greater the difference between the two different probability distributions, and the better they can be distinguished. The Kullback-Leibler divergence value may be defined as shown in the following formula (9).

[0131]

number

[0132] In a third embodiment in which the dependency between multiple separated images is evaluated based on histograms generated for multiple separated images, the processor 110 may evaluate the dependency by calculating the cross-entropy value between the multiple separated images. "Cross-entropy" means the average number of bits required to distinguish between two probability distributions, and this value represents the difference between two different probability distributions. The larger the cross-entropy value, the greater the difference between the two different probability distributions, and the better they can be distinguished. The cross-entropy value may be defined as shown in the following formula (10).

[0133]

number

[0134] In a fourth embodiment in which the dependency between multiple isolated images is evaluated based on histograms generated for multiple isolated images, the processor 110 may evaluate the dependency by calculating a Rand Index between the multiple isolated images. The "Rand Index" is a value that indicates the similarity between two datasets. The larger the Rand Index, the greater the difference between the two datasets, and the better they can be distinguished. For example, if the two datasets (X and Y) are X = {X,X 12 ,...,X i} and Y={Y,Y12 ,...,Y j Assuming that it can be expressed as}, the superposition between the two datasets may be shown as in Table 1 below. [Table 1] For example, dataset X may contain data on combined histograms or combined probability distributions between multiple isolated images, and dataset Y may contain data on marginal histograms or marginal probability distributions between multiple isolated images. The Rand index based on Table 1 in this way may be defined as shown in equation (11) below.

[0135]

number

[0136] The first to fourth embodiments relating to dependency evaluation described above are merely illustrative examples to illustrate how the processor 110 evaluates dependencies between multiple isolated images, and the disclosure should not be limited thereto. The processor 110 of the disclosure may evaluate dependencies (or similarities) based on histograms generated for the multiple isolated images in a variety of ways.

[0137] The server 100 relating to this disclosure may modify at least one parameter included in the unmixing matrix based on the calculated dependency evaluation results. The following describes specific parameter modification methods, again referring to the first to fourth embodiments of the dependency evaluation described above.

[0138] In the first embodiment, where the dependency between multiple separated images is evaluated based on mutual information, the processor 110 may modify the parameters of the unmixing matrix in a direction that reduces the calculated mutual information. In this disclosure, the expression "modifying the parameters of the unmixing matrix in a direction that reduces a specific value calculated for multiple separated images" may mean that, as a result of modifying the parameters of the unmixing matrix that forms the basis for generating multiple separated images from multiple unseparated images, the specific value calculated for the multiple separated images generated by the modified unmixing matrix is ​​smaller than the specific value calculated for the multiple separated images generated by the unmixing matrix before modification. Hereinafter, the expression "modifying the parameters of the unmixing matrix in a direction that reduces a specific value" will be used interchangeably with the expression "modifying the parameters of the unmixing matrix so that a specific value is reduced." To modify the parameters of the unmixing matrix in a direction that reduces mutual information, the processor 110 may use, for example, a loss function such as equation (12) below. Equation (12) shows the loss function in one embodiment according to equation (8) described above.

[0139]

number

[0140] In the second embodiment, where the dependency between multiple separated images is evaluated based on the Kullback-Leibler divergence value, the processor 110 may use a loss function such as equation (13) below to modify the parameters of the unmixing matrix in a direction that reduces the Kullback-Leibler divergence value. Equation (13) shows the loss function in one embodiment according to equation (9) described above.

[0141]

number

[0142] Assuming there are two separate images, X' in equation (13) may represent the joint probability distribution determined between image (X1 - α × X2) and image (X2), and Y' may represent the marginal probability distribution determined between image (X1) and image (X2 - β × X1).

[0143] In the third embodiment, where the dependency between multiple separated images is evaluated based on the cross-entropy value, the processor 110 may use a loss function such as equation (14) below to modify the parameters of the unmixing matrix in a direction that reduces the cross-entropy value. Equation (14) shows the loss function in one embodiment according to equation (10) described above.

[0144]

number

[0145] Assuming there are two separate images, X' in equation (14) may represent the joint probability distribution determined between image (X1 - α × X2) and image (X2), and Y' may represent the marginal probability distribution determined between image (X1) and image (X2 - β × X1).

[0146] In the fourth embodiment, where the dependency between multiple separated images is evaluated based on the Rand index, the processor 110 may use a loss function such as equation (15) below to modify the parameters of the unmixing matrix in a direction that reduces the Rand index. Equation (15) shows the loss function in one embodiment according to equation (11) described above.

[0147]

number

[0148] Assuming there are two separate images, X' in equation (15) may represent the joint probability distribution determined between image (X1 - α × X2) and image (X2), and Y' may represent the marginal probability distribution determined between image (X1) and image (X2 - β × X1).

[0149] The processor 110 may determine at least one parameter (e.g., α or β in equations (12) to (15)) that minimizes various loss functions such as those in equations (12) to (15) described above, based on the following equation (16).

[0150]

number

[0151] As described above, the server 100 relating to this disclosure can modify at least one parameter included in the unmixing matrix based on the calculated dependency evaluation result.

[0152] According to some additional embodiments of this disclosure, the processor 110 may evaluate the dependencies between multiple separated images based on an artificial neural network model and modify the parameters of the unmixing matrix based on the dependency evaluation results. The method by which the processor 110 modifies the parameters of the unmixing matrix based on an artificial neural network model according to this disclosure will be specifically described with reference to the following drawings.

[0153] Next, the processor 110 may determine whether predetermined critical conditions are met (S1145).

[0154] In one embodiment, the predetermined critical condition may be a condition based on the number of updates to the unmixing matrix. For example, suppose the predetermined critical condition is a condition that is satisfied when the unmixing matrix is ​​modified N times (where N is a natural number greater than or equal to 1). In this case, the processor 110 may count the number of updates each time the unmixing matrix is ​​updated, and if the counted number of updates is N, it may determine that the predetermined critical condition is satisfied.

[0155] In one embodiment in which the processor 110 calculates dependency evaluation values ​​between multiple separated images, the predetermined critical condition may be a condition based on the magnitude of the calculated dependency evaluation value. For example, it is assumed that the predetermined critical condition is satisfied when the magnitude of the dependency evaluation value calculated between multiple separated images is 0.2 or less. In this case, the processor 110 may determine that the predetermined critical condition is satisfied if, as a result of executing S1130, the magnitude of the dependency evaluation value calculated between multiple separated images is 0.2 or less.

[0156] Furthermore, in one embodiment in which the processor 110 evaluates dependencies based on an artificial neural network model, the predetermined critical condition may be a condition based on at least one of the number of training iterations of the artificial neural network model, the magnitude of the output value, or the loss value (Loss value, the error between the output value and ground truth). For example, suppose the predetermined critical condition is a condition that is satisfied when the loss value of the artificial neural network model is 0.1 or less. In this case, the processor 110 may determine that the predetermined critical condition is satisfied if the loss value of the artificial neural network model is 0.1 or less.

[0157] If the processor 110 determines whether a predetermined critical condition is met and finds that the condition is not satisfied, the processor 110 may repeat steps S1120 to S1140 described above until the predetermined critical condition is met.

[0158] If the processor 110 determines that a predetermined critical condition is met, and the condition is met, the processor 110 may terminate the update of the unmixing matrix. The updated unmixing matrix may be a matrix in which at least one parameter has been modified compared to the unmixing matrix before the update.

[0159] Figure 12 is a conceptual diagram illustrating the process of evaluating the dependencies between multiple separated images 1210 based on an artificial neural network model 1250 in one embodiment of the present disclosure. The artificial neural network model 1250 may be an artificial neural network model (hereinafter also referred to as the "classification model") that receives input data and generates output data for determining the type of input data. Hereinafter, with reference to Figure 12, for the sake of explanation, it will be assumed that the multiple separated images 1210 include three separated images, but the present disclosure is not limited thereto.

[0160] The processor 110 may generate at least one input data to be input to the artificial neural network model 1250 based on a plurality of separated images 1210. In one embodiment, the processor 110 may generate input data by sampling data from at least one probability distribution among two or more distinct probability distributions relating to the plurality of separated images 1210. In this disclosure, the term “sampling” may mean the operation of selecting or extracting a predetermined number of elements based on the probability values ​​of each element contained in a particular probability distribution. For example, if a first element contained in a particular probability distribution has a higher probability value than a second element, and one element is sampled from this particular probability distribution, the probability of the first element being selected may be higher than the probability of the second element being selected. Alternatively, for example, if all elements contained in a particular probability distribution have the same probability value, and one element is sampled from this particular probability distribution, all elements may have an equal probability of being selected. In this disclosure, the probability distribution for which the processor 110 samples data may be, for example, a probability distribution determined based on the values ​​of pixels at the same positions in each of the multiple separated images 1210 (hereinafter also referred to as the "first probability distribution"). The first probability distribution may be determined by the processor 110 performing the method described with reference to Figure 8 for the multiple separated images 1210. The processor 110 may generate input data by sampling data from the first probability distribution. Alternatively, the probability distribution for which the processor 110 samples data may be, for example, a probability distribution determined based on the values ​​of pixels at different positions in each of the multiple separated images 1210 (hereinafter also referred to as the "second probability distribution"). The second probability distribution may be determined by the processor 110 performing the method described with reference to Figure 9 for the multiple separated images 1210. The processor 110 may generate input data by sampling data from the second probability distribution.

[0161] In one embodiment, the input data 1231 sampled from the first probability distribution may be expressed as shown in the following formula (17).

[0162]

number

[0163] In one embodiment, the input data 1233 sampled from the second probability distribution may be expressed as shown in the following formula (18).

[0164]

number

[0165] In one embodiment, the processor 110 may input input data to a classification model and determine the type of input data based on the output data of the classification model. If the input data is data sampled according to one of two or more probability distributions determined to be different with respect to a plurality of isolated images, the classification model may generate output data to determine the specific probability distribution on which the input data is based. For example, the input data for the classification model may be input data 1231 sampled from a first probability distribution or input data 1233 sampled from a second probability distribution. In this case, the output data of the classification model may be data that determines the probability distribution associated with the input data. That is, if the input data is input data 1231 sampled from a first probability distribution, the classification model may output information indicating the first probability distribution as output data for that input data. Also, if the input data is input data 1233 sampled from a second probability distribution, the classification model may output information indicating the second probability distribution as output data for the input data.

[0166] To train an artificial neural network model 1150, which is a classification model according to one embodiment of this disclosure, the processor 110 may generate training data by labeling input data 1131 sampled from a first probability distribution and input data 1133 sampled from a second probability distribution differently from each other. For example, the processor 110 may label the input data 1131 sampled from the first probability distribution with "1" as the ground truth, and the input data 1133 sampled from the second probability distribution with "0" as the ground truth. When the classification model is trained based on the training data thus generated, the processor 110 may input the input data included in the training data into the classification model, obtain output data (e.g., a real number between 0 and 1) output by the classification model, and train the classification model by updating the value of at least one parameter included in the classification model using a backpropagation technique based on the difference between the ground truth labeled for this input data and the output data of the classification model.

[0167] Figure 13 is a conceptual diagram illustrating the process of evaluating the dependencies between multiple isolated images 1310 based on an artificial neural network model 1350 in another embodiment of the present disclosure. The artificial neural network model 1350 may be an artificial neural network model (hereinafter also referred to as the "prediction model") that receives multiple input data and generates output data to predict specific values ​​associated with the multiple input data. Hereinafter, with reference to Figure 13, for the sake of explanation, it will be assumed that the multiple isolated images 1310 include two isolated images, but this will not limit the present disclosure.

[0168] The processor 110 may generate a plurality of input data to be input to the artificial neural network model 1350 based on a plurality of isolated images 1310. The processor 110 may generate a plurality of input data sampled from different probability distributions determined based on the plurality of isolated images 1310. Hereinafter, it will be assumed that the artificial neural network model 1350 receives two input data, but this will not limit the disclosure, and the artificial neural network model 1350 of this disclosure may receive three or more input data. In one embodiment, the processor 110 may generate two input data by sampling data from two different probability distributions relating to the plurality of isolated images 1310. The two different probability distributions relating to the plurality of isolated images 1310 may be, for example, a probability distribution determined based on the values ​​of pixels at the same position in each of the plurality of isolated images 1310 (hereinafter also referred to as the "third probability distribution"), or a probability distribution determined based on the values ​​of pixels at different positions in each of the plurality of isolated images 1310 (hereinafter also referred to as the "fourth probability distribution"). In this case, the third probability distribution may be determined by the processor 110 performing the method described with reference to Figure 8 on the multiple separated images 1310, and the fourth probability distribution may be determined by the processor 110 performing the method described with reference to Figure 9 on the multiple separated images 1310. The processor 110 may generate multiple input data to be input to the artificial neural network model 1350 by sampling input data 1331 from the third probability distribution and input data 1333 from the fourth probability distribution.

[0169] The processor 110 may input multiple input data to the artificial neural network model 1350 and obtain specific values ​​associated with the multiple input data that the artificial neural network model 1350 predicts. The "specific values" associated with the multiple input data that the artificial neural network model 1350 predicts may mean the values ​​output by the artificial neural network model 1350 after receiving the multiple input data.

[0170] The processor 110 may calculate dependency evaluation values ​​for multiple separated images based on specific values ​​obtained. The type of dependency evaluation value calculated by the processor 110 based on the artificial neural network model may be determined by various embodiments, as described above. For the sake of explanation, below we will assume that the processor 110 calculates the "mutual information" described above as an example of a dependency evaluation value calculated based on the artificial neural network model. In one embodiment, the mutual information between multiple separated images calculated based on specific values ​​predicted by the prediction model may be expressed as shown in the following formula (19).

[0171]

number

[0172] To train the artificial neural network model 1350 as a predictive model as described above, the processor 110 may calculate mutual information based on a specific value output by the predictive model and train the predictive model to maximize the calculated mutual information. That is, when the predictive model predicts a specific value for multiple input data sampled from multiple separated images, the value of at least one parameter included in the predictive model may be updated to predict a specific value in a direction that maximizes the mutual information for the multiple separated images. For example, the processor 110 may apply gradient descent (or gradient ascent) to a predetermined mutual information calculation formula so that the value of at least one parameter included in the predictive model is updated by a chain rule.

[0173] The unmixing matrix and artificial neural network models of this disclosure may be trained adversarially. In this disclosure, the expression "trained adversarially" may mean that the values ​​of at least one parameter contained in each object are changed as the two objects are trained to solve opposing tasks.

[0174] In one embodiment of adversarial learning relating to this disclosure, if the artificial neural network model corresponds to a classification model, the value of at least one element in the unmixing matrix may be updated so that the first and second input data input to the classification model are not appropriately distinguished by the artificial neural network model, and the value of at least one parameter in the artificial neural network model may be updated so that the first and second input data input are appropriately distinguished. For example, the first input data may be data sampled from a first probability distribution, and the second input data may be data sampled from a second probability distribution. The classification model may be an artificial neural network model that receives input data sampled from each of a plurality of separated images, each of which is determined to be different from the other, and determines the probability distribution associated with each input data. The unmixing matrix may also generate a plurality of separated images that form the basis for generating each input data. That is, the unmixing matrix generates a plurality of separated images from a plurality of unseparated images, and the classification model may determine the type of input data sampled from each of the plurality of separated images. Therefore, the classification model learns to appropriately distinguish between multiple input data, while the unmixing matrix learns to be indistinguishable from multiple input classification models sampled from each of the multiple separated images. In this way, the classification model and the unmixing matrix learn to adapt to each other.

[0175] In one embodiment, if the two input classification models input to the classification model appropriately distinguish between the unseparated images, the processor 110 may determine that the classification model has not been sufficiently trained and may perform further training of the classification model. In this case, the multiple separated images generated from the multiple unseparated images using the unmixing matrix may be determined to be dependent on each other. Conversely, if the two input classification models input to the classification model appropriately distinguish between the unseparated images, the processor 110 may determine that the classification model has been sufficiently trained and may interrupt the training of the classification model. In this case, the multiple separated images generated from the multiple unseparated images using the unmixing matrix may be determined to be independent of each other.

[0176] In this disclosure, "appropriately distinguishable by two types of input classification models input to the classification model" or "not appropriately distinguishable by two types of input classification models input to the classification model" may be determined quantitatively or numerically. For example, when the classification model generates output data for determining the type of input data, the processor 110 may evaluate the accuracy (or confidence) of the classification model using a test dataset in which ground truth is labeled for a predetermined number of input data. Specifically, the processor 110 may determine that the classification model has been sufficiently trained if it outputs classification results within the ground truth and error range for a certain number or a certain proportion of the input data included in the test dataset. Alternatively, the processor 110 may determine that the classification model has been sufficiently trained if the change in the accuracy of the classification model falls below a threshold as training progresses. The above-described method for evaluating the accuracy of the classification model is merely an example for illustrative purposes and should not limit this disclosure.

[0177] In another embodiment relating to adversarial learning as disclosed herein, if the artificial neural network model is a predictive model, the value of at least one parameter included in the predictive model may be updated to predict a specific value in the direction that maximizes a dependency evaluation value (e.g., mutual information, Kullback-Leibler information value, cross-entropy value, Rand exponent, etc.) for a plurality of separated images, and the value of at least one element included in the unmixing matrix may be updated to generate a plurality of separated images in the direction that minimizes the dependency evaluation value for a plurality of separated images. The predictive model may be an artificial neural network model that takes input data of a plurality of input data sampled based on probability distributions determined differently from each other from a plurality of separated images, and predicts a specific value associated with the input data. The unmixing matrix may also generate a plurality of separated images that form the basis for generating the plurality of input data. That is, the unmixing matrix generates a plurality of separated images from a plurality of unseparated images, and the predictive model may take input data of a plurality of input data sampled from each of the plurality of separated images, and predict a specific value associated with the input data. Furthermore, a specific value predicted by the prediction model may serve as the basis for calculating dependency evaluation values ​​for multiple separated images. Thus, the prediction model and the unmixing matrix learn to adapt to each other by being trained in a direction that maximizes the dependency evaluation values ​​for multiple separated images, and the unmixing matrix learns in a direction that minimizes the dependency evaluation values ​​for multiple separated images.

[0178] In one embodiment, if the dependency evaluation value calculated based on a specific value output by the prediction model does not exceed a predetermined threshold, the processor 110 may determine that the prediction model has not been sufficiently trained and may further train the prediction model. In this case, the multiple separated images generated from multiple unseparated images using the unmixing matrix may be determined to be dependent on each other. Conversely, if the dependency evaluation value calculated based on a specific value output by the prediction model exceeds a predetermined threshold, the processor 110 may determine that the prediction model has been sufficiently trained and may interrupt the training of the prediction model. In this case, the multiple separated images generated from multiple unseparated images using the unmixing matrix may be determined to be independent of each other. On the other hand, the processor 110 may determine the degree of training of the prediction model based on the number of times the prediction model has been trained, for example, the number of training epochs.

[0179] As described above, according to various embodiments of the present disclosure, the processor 110 can evaluate the dependencies between multiple separated images and determine the parameters of the unmixing matrix based on the evaluation results. Based on the unmixing matrix thus determined, the processor 110 according to the present disclosure can generate separated images of each biomolecule contained in the sample from multiple unseparated images of the sample.

[0180] The following describes a method for acquiring multiple unseparated images in one embodiment of the contents of this disclosure.

[0181] As explained with reference to Figure 4, conventional methods for acquiring images of individual biomolecules from a sample containing multiple biomolecules are constrained by the requirement that the emission spectra of the fluorescent substances labeled to each biomolecule must not overlap as much as possible, thus limiting the simultaneous use of a maximum of four fluorescent substances. Furthermore, after acquiring images of four biomolecules contained in a sample using four fluorescent substances, in order to acquire images of other biomolecules contained in the same sample, a post-processing step to remove the existing fluorescent substances had to be performed before newly labeling the other biomolecules. Post-processing steps to remove fluorescent substances may include, for example, steps to inactivate the fluorescent substances or steps to remove the substances used to label antibodies or biomolecules labeled with fluorescent substances.

[0182] On the other hand, the image processing method according to this disclosure does not require the inactivation or removal process of fluorescent substances that conventional methods require. As a result, in one embodiment of this disclosure, multiple unseparated images can be obtained in a manner different from conventional methods.

[0183] Multiple unseparated images according to one embodiment of this disclosure may be generated sequentially by performing one cycle of staining and imaging on a sample two or more times. Alternatively, multiple unseparated images according to this disclosure may be generated sequentially by performing one cycle of staining and imaging two or more times without going through a post-processing step to remove the fluorescent substance. Thus, the image processing method according to this disclosure has the effect of generating multiple separated images quickly and effectively by generating separated images for each biomolecule without going through the conventional step of removing the fluorescent substance.

[0184] Figure 14 is a conceptual diagram showing the process of sequentially acquiring multiple images in one embodiment of the present disclosure. Assume that the sample 1400 in Figure 14 contains N biomolecules (where N is a natural number greater than or equal to 1). When the first biomolecule 1401 contained in the sample 1400 is stained by the first staining, the processor 110 may acquire a first unseparated image 1410 by imaging the sample 1400 after the first staining with the imaging unit. The first unseparated image 1410 displays the first biomolecule 1401 that was stained by the first staining, and the remaining unstained biomolecules are not displayed. Next, when the second biomolecule 1402 contained in the sample 1400 is stained by the second staining, the processor 110 may acquire a second unseparated image 1420 by imaging the sample 1400 after the second staining with the imaging unit. On the second unseparated image 1420, the first biomolecule 1401 stained by the first staining and the second biomolecule 1402 stained by the second staining are displayed, while the remaining unstained biomolecules are not displayed. By repeating this process, the processor 110 may acquire the nth unseparated image 1430 by taking a picture of the sample 1400 after the nth staining with the imaging unit when the nth biomolecule 1404 contained in the sample 1400 is stained by the Nth staining. The nth unseparated image 1430 may be represented to include the first biomolecule 1401 stained by the first staining, the second biomolecule 1402 stained by the second staining, ..., and the nth biomolecule 1404 stained by the Nth staining. In this way, when multiple unseparated images are acquired sequentially, the unseparated image acquired after the "i+1"th staining will represent one more biomolecule than the unseparated image acquired after the "i"th staining. Hereinafter, in this disclosure, when comparing the unseparated image obtained after the i-th staining with the unseparated image obtained after the i+1-th staining, one additional biomolecule represented on the unseparated image obtained after the i-th staining will be referred to as the "biomolecule labeled in the i+1-th staining process" (where i is a natural number greater than or equal to 1).For example, the second biomolecule 1402, which is additionally represented on the second unseparated image 1420 compared to the first unseparated image 1410 in Figure 14, may be referred to as a "biomolecule labeled in the second staining process," and the Nth biomolecule 1403, which is additionally represented on the Nth unseparated image 1430 compared to the (N-1)th unseparated image, may be referred to as a "biomolecule labeled in the Nth staining process."

[0185] As explained in Figure 14, the image processing method relating to this disclosure may generate a separated image for a specific biomolecule based on two or more consecutive unseparated images from a plurality of sequentially acquired unseparated images. In this disclosure, expressions such as "two or more unseparated images acquired sequentially" or "two or more unseparated images acquired by sequentially staining a plurality of biomolecules contained in a sample" may mean two or more unseparated images acquired by sequentially performing a cycle including staining and imaging for each of the plurality of biomolecules contained in a sample. For example, two consecutively acquired unseparated images may include an unseparated image taken after the i-th stain (where i is a natural number of 1 or more) and an unseparated image taken after the (i+1)th stain. Also, for example, three consecutively acquired unseparated images may include an unseparated image taken after the i-th stain, an unseparated image taken after the (i+1)th stain, and an unseparated image taken after the (i+2)th stain. Hereafter, for convenience of explanation, the expression "a plurality of unseparated images acquired sequentially" will be used interchangeably with the expression "a plurality of consecutive unseparated images."

[0186] Figure 15 is a conceptual diagram illustrating an exemplary method in one embodiment of the present disclosure for obtaining a separated image of at least one biomolecule from two consecutive unseparated images from a plurality of sequentially acquired unseparated images. The processor 110 according to the present disclosure may generate a separated image of the biomolecule labeled during the i+1th staining process by performing calculations on two consecutive unseparated images, namely, the unseparated image acquired after the i-th staining and the unseparated image acquired after the i+1-th staining.

[0187] In one embodiment, the processor 110 may acquire a first unseparated image 1510 by staining one biomolecule (e.g., protein A) contained in the sample with a first staining step and then photographing the sample. The processor 110 may also acquire a second unseparated image 1530 by staining another biomolecule (e.g., protein B) contained in the sample with a subsequent secondary staining step and then photographing the sample. The processor 110 may also acquire a third unseparated image 1550 by staining yet another biomolecule (e.g., protein C) contained in the sample with a subsequent tertiary staining step and then photographing the sample. The processor 110 may perform calculations on two consecutive unseparated images based on at least one parameter to generate a separated image for at least one biomolecule. For example, the processor 110 may generate a separated image 1520 of the biomolecule (i.e., protein B) labeled in the second staining step by performing calculations on the first unseparated image 1510 and the second unseparated image 1530. Furthermore, the processor 110 may generate a separated image 1540 for the biomolecule labeled in the third staining process (i.e., protein C) by performing calculations on the second unseparated image 1530 and the third unseparated image 1550. On the other hand, the separated image for the biomolecule labeled in the first staining process (i.e., protein A) may be obtained as the first unseparated image 1510.

[0188] Figure 16 is a conceptual diagram illustrating an exemplary method in one embodiment of the present disclosure for obtaining separation images of at least two biomolecules from three consecutive unseparated images from a plurality of sequentially acquired unseparated images. The first unseparated image 1610, the second unseparated image 1630, and the third unseparated image 1650 shown in Figure 16 may be obtained in the same manner as the first unseparated image 1510, the second unseparated image 1530, and the third unseparated image 1550 in Figure 15, respectively. The processor 110 may perform calculations on the three consecutive unseparated images based on at least one parameter to generate separation images for at least two biomolecules. For example, the processor 110 may simultaneously calculate the first unseparated image 1610, the second unseparated image 1620, and the third unseparated image 1630 to generate a separation image 1620 for a biomolecule labeled in the second staining process (i.e., protein B) and a separation image 1640 for a biomolecule labeled in the third staining process (i.e., protein C). On the other hand, the separation image for the biomolecule labeled in the first staining process (i.e., protein A) may be obtained as the first unseparated image 1610.

[0189] Figures 15 or 16 illustrate a method for obtaining a separated image of a biomolecule based on two or three consecutive unseparated images, but this should not limit the disclosure, as the disclosure may also allow for the generation of a separated image of a biomolecule based on any number of two or more consecutive unseparated images. For the sake of explanation, a method for obtaining a separated image based on two consecutive unseparated images will be described below.

[0190] Figure 17 is a flowchart illustrating the operation of a server 100 in one embodiment of the present disclosure, which generates a separated image of at least one biomolecule based on two consecutive unseparated images.

[0191] The processor 110 may acquire a first unseparated image of a sample containing a first biomolecule labeled with a first fluorescent substance and an unlabeled second biomolecule (S1710). In the embodiment described in Figure 17, the first biomolecule may be a biomolecule labeled with a specific fluorescent substance in the i-th staining step of the sample (i.e., the i-th staining step). The processor 110 may acquire an unseparated image of the sample in which the first biomolecule has been stained by an imaging unit (not shown), or it may acquire an unseparated image of the sample in which the first biomolecule has been stained by an external device or a user terminal 200.

[0192] Next, the processor 110 may acquire a second unseparated image of the sample containing a first biomolecule labeled with a first fluorescent substance and a second biomolecule labeled with a second fluorescent substance (S1720). In the embodiment described in Figure 17, the second biomolecule may be a biomolecule labeled with a specific fluorescent substance in the i+1th staining step of the sample (i.e., the i+1th staining step). That is, the first unseparated image and the second unseparated image acquired in S1710 may be two images acquired sequentially. That is, the first biomolecule may be a biomolecule labeled in the i-th staining step, and the second biomolecule may be a biomolecule labeled in the i+1th staining step. The processor 110 may acquire an unseparated image of the sample stained with the second biomolecule by an imaging unit (not shown), or it may acquire an unseparated image of the sample stained with the second biomolecule by receiving an unseparated image of the sample stained with the second biomolecule from an external device or user terminal 200.

[0193] Unless otherwise clearly indicated in the context, the first unseparated image may refer to an unseparated image taken after labeling the first biomolecule with the first fluorescent substance in the i-th staining step for a particular sample, and the second unseparated image may refer to an unseparated image taken after labeling the second biomolecule with the second fluorescent substance in the i+1-th staining step for the same particular sample. In such a first unseparated image, the first biomolecule may be displayed, but the second biomolecule may not. In the second unseparated image, both the first and second biomolecules may be displayed.

[0194] Next, the processor 110 may generate a separated image of the second biomolecule based on the first unseparated image and the second unseparated image (S1730). In one embodiment, the processor 110 may use an unmixing matrix to compute the first unseparated image and the second unseparated image and generate a separated image of the second biomolecule based on the result of the computation. In addition, the value of at least one element included in the unmixing matrix may be determined based on a trained artificial neural network model. Since the common explanation of the unmixing matrix or artificial neural network model is as described above, the following explanation will omit redundant content and will only describe the differences.

[0195] In one embodiment, a first unseparated image and a second unseparated image acquired consecutively from the same sample may each be images captured by detecting light in the same specific wavelength band from the sample. More specifically, the first unseparated image may be an image captured by detecting light in the first wavelength band from the light emitted by a sample containing a first biomolecule labeled with a first fluorescent substance, and the second unseparated image may be an image captured by detecting light in the second wavelength band from the light emitted by a sample containing a second biomolecule labeled with a second fluorescent substance. In this case, if the first wavelength band and the second wavelength band are the same, the first unseparated image and the second unseparated image may be images captured by detecting light in the same specific wavelength band. In this disclosure, the term "light in a specific wavelength band" may mean light having wavelengths in a specific range. For example, light in a specific wavelength band may mean light having wavelengths between 400 nm and 450 nm. In order to capture an unseparated image by detecting light in a specific wavelength band, the wavelength range of light detected by the imaging unit may be adjusted, the sample may be irradiated with light in a specific wavelength band, or a predetermined filter may be installed between the imaging unit and the sample.

[0196] In one embodiment, the first unseparated image and the second unseparated image acquired sequentially from the same sample may each be images acquired based on the same emission filter (hereinafter also referred to as the "sixth emission filter").

[0197] Figure 18 is an illustrative diagram showing the emission spectra of multiple fluorescent substances and the wavelength bands that pass through a specific emission filter. As described above, an unseparated image may be obtained by irradiating a fluorescent substance with light of a specific wavelength to which the fluorescent substance reacts, filtering the light emitted by the excited fluorescent substance with a corresponding emission filter, and photographing the light that has passed through the emission filter. For example, suppose the first unseparated image is an image obtained after labeling a first biomolecule contained in the sample with one of the fluorescent substances "Alexa405", "CF405S", or "ATTO390", and the second unseparated image is an image obtained after labeling a second biomolecule contained in the same sample with one of the fluorescent substances "Alexa405", "CF405S", or "ATTO390". In this case, the first and second unseparated images may be images obtained based on the same sixth emission filter, for example. If the wavelength band 1800 through which the sixth emission filter passes is, for example, as shown by the dotted line section in Figure 18, then both the first unseparated image and the second unseparated image may be acquired by detecting that the light in the wavelength band 1800 through which the sixth emission filter passes is the same.

[0198] According to one embodiment of the present disclosure, the first fluorescent substance used to acquire the first unseparated image and the second fluorescent substance used to acquire the second unseparated image may be the same fluorescent substance as the first. For example, the first and second fluorescent substances may be the same fluorescent substance as the first and second fluorescent substances, such as "Alexa405", "Alexa488", "Alexa546", "Alexa647", "CF594", "CF405S", "ATTO390", or any other variety of fluorescent substances.

[0199] According to one embodiment of the present disclosure, the first fluorescent substance used to acquire the first unseparated image and the second fluorescent substance used to acquire the second unseparated image may be fluorescent substances having similar emission spectra. In the present disclosure, two or more fluorescent substances having similar emission spectra may be expressed as a "combination of fluorescent substances having similar emission spectra." A method for determining a "combination of fluorescent substances having similar emission spectra" will be described below with reference to Figure 19. Figure 19 is an illustrative diagram showing the emission spectra of several fluorescent substances and the wavelength values ​​at which the signal intensity is maximized in each emission spectrum.

[0200] The processor 110 may determine a combination of fluorescent substances having similar emission spectra based on the intensity of the signals in the emission spectra of each fluorescent substance. For the sake of explanation, the following description will assume that a combination of fluorescent substances having similar emission spectra consists of two fluorescent substances.

[0201] The processor 110 may determine two fluorescent substances as a combination of fluorescent substances having similar emission spectra if the wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of each of the two fluorescent substances satisfies a predetermined condition (hereinafter also referred to as the "fluorescent substance combination condition"). More specifically, assuming that the first unseparated image is an image taken after labeling a first biomolecule contained in the sample with a first fluorescent substance, and the second unseparated image is an image taken after labeling a second biomolecule contained in the sample with a second fluorescent substance, the first and second fluorescent substances may be a combination of fluorescent substances having similar emission spectra if the first wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of the first fluorescent substance and the second wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of the second fluorescent substance satisfies a predetermined condition.

[0202] In one embodiment of the fluorescent material combination conditions, the processor 110 may determine that the fluorescent material combination conditions are met if the first wavelength value at which the intensity of the emission signal is maximum in the emission spectrum of the first fluorescent material and the second wavelength value at which the intensity of the emission signal is maximum in the emission spectrum of the second fluorescent material are below a predetermined threshold. For example, as shown in Figure 19, the wavelength value at which the intensity of the emission signal is maximum in the emission spectrum of "Alexa405" (hereinafter also referred to as "the maximum wavelength value of Alexa405") may be approximately 420 nm, and the wavelength value at which the intensity of the emission signal is maximum in the emission spectrum of "CF405S" (hereinafter also referred to as "the maximum wavelength value of CF405S") may be approximately 430 nm. Also, the wavelength value at which the intensity of the emission signal is maximum in the emission spectrum of "ATTO390" (hereinafter also referred to as "the maximum wavelength value of ATTO390") may be approximately 480 nm.

[0203] In one embodiment relating to Figure 19, if the processor 110 has a predetermined threshold of 20 nm, the difference between the maximum wavelength of Alexa 405 (e.g., 420 nm) and the maximum wavelength of CF405S (e.g., 430 nm) is 10 nm, which is below the predetermined threshold. Therefore, Alexa 405 and CF405S may be judged to satisfy the combination condition for fluorescent materials. On the other hand, the difference between the maximum wavelength of Alexa 405 and the maximum wavelength of ATTO 390 (e.g., 480 nm) is 60 nm, which is not below the predetermined threshold. Therefore, Alexa 405 and ATTO 390 may be judged to not satisfy the combination condition for fluorescent materials. Similarly, since the difference between the maximum wavelengths of CF405S and ATTO 390 is 50 nm, they may also be judged to not satisfy the combination condition for fluorescent materials.

[0204] In one embodiment relating to Figure 19, if the processor 110 has a predetermined threshold of 60 nm, the difference between the maximum wavelength of Alexa 405 (e.g., 420 nm) and the maximum wavelength of CF405S (e.g., 430 nm) is 10 nm, which is below the predetermined threshold. Therefore, Alexa 405 and CF405S may be judged to satisfy the combination condition for fluorescent materials. Similarly, the difference between the maximum wavelength of Alexa 405 and the maximum wavelength of ATTO 390 (e.g., 480 nm) is 60 nm, which is below a predetermined threshold. Therefore, Alexa 405 and ATTO 390 may be judged to satisfy the combination condition for fluorescent materials. Likewise, since the difference between the maximum wavelengths of CF405S and ATTO 390 is 50 nm, they may be judged to satisfy the combination condition for fluorescent materials. In another embodiment relating to the combination conditions for fluorescent materials, the processor 110 may determine that the combination conditions for fluorescent materials are met if the ratio of the smaller wavelength value to the larger wavelength value among the first wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of the first fluorescent material and the second wavelength value at which the intensity of the emission signal is maximized in the emission spectrum of the second fluorescent material is greater than or equal to a predetermined threshold. For example, if the predetermined threshold of the processor 110 is 0.95, the ratio of the smaller wavelength value to the larger wavelength value among the maximum wavelength values ​​of Alexa405 and CF405S is approximately 0.977 (=420 / 430), which is greater than or equal to the predetermined threshold, and therefore, Alexa405 and CF405S may be determined to satisfy the combination conditions for fluorescent materials. On the other hand, the ratio of the smaller wavelength value to the larger wavelength value among the maximum wavelength values ​​of Alexa405 and ATTO390 is 0.875 (=420 / 480), which is below the predetermined threshold ratio, so Alexa405 and ATTO390 may be judged as not meeting the combination conditions for fluorescent materials. Similarly, the ratio of the respective maximum wavelength values ​​of CF405S and ATTO390 is approximately 0.896 (=430 / 480), which is below the predetermined threshold ratio, so CF405S and ATTO390 may also be judged as not meeting the combination conditions for fluorescent materials.

[0205] The specific numerical values ​​of the predetermined threshold or predetermined threshold ratio are merely illustrative and should not limit this disclosure. The predetermined threshold or predetermined threshold ratio may be set to a variety of real values ​​depending on the type of fluorescent substance used in the staining process.

[0206] According to the conventional image processing method described with reference to Figure 4, in order to obtain accurate images of each of multiple biomolecules, it was necessary to select fluorescent substances such that the emission spectra of the fluorescent substances used to label each of the multiple biomolecules did not overlap as much as possible. On the other hand, according to the image processing method of this disclosure, when generating multiple separated images from multiple unseparated images, the limitations of the conventional method described above can be greatly reduced. Furthermore, in the process of sequentially staining multiple biomolecules contained in a sample and acquiring unseparated images continuously, it is also possible to use the same fluorescent substance or similar fluorescent substances for each stain. As a result, in this disclosure, multiple unseparated images acquired sequentially can be acquired based on light in the same or similar wavelength bands by methods such as those described in some embodiments above, and multiple unseparated images can be acquired more quickly and easily than in the conventional method without having to go through additional steps such as changing emission filters or removing fluorescent substances.

[0207] In additional embodiments disclosed herein, if the multiple unseparated images are images obtained by sequentially staining multiple biomolecules contained in the sample, the unmixing matrix may be a triangular matrix.

[0208] In one embodiment relating to the triangular matrix of this disclosure, if a plurality of unseparated images include two unseparated images, the unmixing matrix, which is a triangular matrix, may be expressed as shown in the following equation (20).

[0209]

number

[0210] In another embodiment relating to the triangular matrix of this disclosure, if the multiple unseparated images include three unseparated images, the unmixing matrix, which is a triangular matrix, may be expressed as shown in equation (21) below.

[0211]

number

[0212] In the detailed explanation based on the above formulas (20) and (21), the triangular matrix used as the unmixing matrix was described as a lower triangular matrix. However, this disclosure is not limited to this, and the triangular matrix in this disclosure may also be an upper triangular matrix. Thus, if the multiple unseparated images are images obtained by sequentially staining multiple biomolecules contained in the sample, the unmixing matrix in this disclosure may be a triangular matrix, thereby enabling the processor 110 to obtain a separated image of at least one biomolecule by performing matrix operations more quickly.

[0213] In the flowcharts relating to the disclosures herein, each step of the method or algorithm is described in a sequential order, but the steps may be performed in any order, not just sequentially. The descriptions of the flowcharts herein do not preclude changes or modifications to the method or algorithm, nor do they imply that any step is essential or preferred. In one embodiment, at least some steps may be performed in parallel, iteratively, or heuristically. In one embodiment, at least some steps may be omitted, or other steps may be added.

[0214] Various embodiments relating to the disclosed herein may be implemented as software on a machine-readable storage medium. The software may be software for implementing the various embodiments relating herein. The software may be inferred from the various embodiments relating herein by a programmer in the art to which the disclosed herein pertains. For example, the software may be a program that includes machine-readable instructions (e.g., code or code segment). The machine is a device capable of operating according to instructions called from the storage medium, and may be, for example, a computer. In one embodiment, the machine may be a computing device according to one of the various embodiments relating herein. In one embodiment, the processor of the machine may execute a called instruction, enabling the components of the machine to perform the function corresponding to that instruction. In one embodiment, the processor may be processors 110, 210 according to the embodiments relating herein. The storage medium may mean all types of recording medium on which machine-readable data is recorded. The recording medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data recording device, etc. In one embodiment, the recording medium may be memory 130, 230. In one embodiment, the recording medium may be implemented in a distributed form across computer systems connected to a network. The software may be stored and executed in a distributed manner across computer systems. The recording medium may be a non-transitory recording medium. A non-transitory recording medium means a tangible medium that exists regardless of whether the data is recorded semi-permanently or temporarily, and does not include signals that are transmitted transiently.

[0215] Although the technical concept relating to the disclosed content of this specification has been explained above through various embodiments, the technical concept relating to the disclosed content of this specification includes a variety of substitutions, modifications, and alterations that are possible to the extent that can be understood by a person with ordinary skill in the art to which the disclosed content of this specification pertains. Furthermore, such substitutions, modifications, and alterations should be understood to be included within the scope of the appended claims.

Claims

1. An image processing method performed on an electronic device including one or more processors and one or more memories on which instructions executed by the one or more processors are recorded, To obtain a first unseparated image of a sample containing a first biomolecule labeled with a first fluorescent substance and a second biomolecule that is not labeled. To obtain a second unseparated image of the sample containing the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with the second fluorescent substance, and This includes generating a separated image of the second biomolecule based on the first unseparated image and the second unseparated image, The first fluorescent substance and the second fluorescent substance are determined such that a first wavelength value, which maximizes the intensity of the emission signal within the emission spectrum of the first fluorescent substance, and a second wavelength value, which maximizes the intensity of the emission signal within the emission spectrum of the second fluorescent substance, satisfy predetermined conditions. The method involves obtaining the second unseparated image by first taking a first unseparated image of the sample, then labeling the second biomolecule contained in the sample with a second fluorescent substance, and then photographing the sample.

2. The method according to claim 1, wherein the first unseparated image and the second unseparated image are images captured by detecting light of the same specific wavelength band from the sample.

3. The first unseparated image and the second unseparated image are images acquired based on the same emission filter, The method according to claim 1, wherein the light-emitting filter allows light in a specific wavelength band to pass through.

4. The method according to claim 1, wherein the first fluorescent substance and the second fluorescent substance are the same fluorescent substance.

5. The aforementioned predetermined conditions are: The method according to claim 1, wherein the condition is satisfied when the difference between the first wavelength value and the second wavelength value is less than or equal to a predetermined threshold.

6. The aforementioned predetermined conditions are: The method according to claim 1, wherein the condition is satisfied when the ratio of the smaller wavelength value to the larger wavelength value among the first wavelength value and the second wavelength value is greater than or equal to a predetermined threshold.

7. The generation of the aforementioned separated image is The method according to claim 1, comprising calculating the first unseparated image and the second unseparated image using an unmixing matrix.

8. The method according to claim 7, wherein the value of at least one element included in the unmixing matrix is ​​determined based on a trained artificial neural network model.

9. The method further includes obtaining a third unseparated image of the sample comprising the first biomolecule labeled with the first fluorescent substance, the second biomolecule labeled with the second fluorescent substance, and the third biomolecule labeled with the third fluorescent substance. The above generation is, The additional step includes generating a separated image of the third biomolecule based on the third unseparated image, The first unseparated image is an image obtained by photographing a sample containing the unlabeled second biomolecule and the unlabeled third biomolecule. The method according to claim 1, wherein the second unseparated image is an image obtained by photographing a sample containing the unlabeled third biomolecule.

10. The second unseparated image is, The first unseparated image of the sample is taken, and then the second biomolecule contained in the sample is labeled with the second fluorescent substance, and the sample is photographed again to obtain the first unseparated image of the sample. The XL3 unseparated image is The method according to claim 9, wherein the second unseparated image of the sample is taken, the third biomolecule contained in the sample is labeled with the third fluorescent substance, and then the sample is photographed.

11. An electronic device, One or more processors, and One or more memories containing instructions executed by the one or more processors Includes, The one or more processors described above are: A first unseparated image was obtained of a sample containing a first biomolecule labeled with a first fluorescent substance and a second biomolecule that was not labeled. A second unseparated image of the sample containing the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with the second fluorescent substance is obtained. Based on the first unseparated image and the second unseparated image, a separated image of the second biomolecule is generated. The first fluorescent substance and the second fluorescent substance are determined such that a first wavelength value, which maximizes the intensity of the emission signal within the emission spectrum of the first fluorescent substance, and a second wavelength value, which maximizes the intensity of the emission signal within the emission spectrum of the second fluorescent substance, satisfy predetermined conditions. The electronic device obtains the second unseparated image by first capturing a first unseparated image of the sample, then labeling the second biomolecule contained in the sample with a second fluorescent substance, and then capturing the sample.

12. The electronic apparatus according to claim 11, wherein the first unseparated image and the second unseparated image are images captured by detecting light of the same specific wavelength band relative to the sample.

13. The first unseparated image and the second unseparated image are images acquired based on the same emission filter, The electronic device according to claim 11, wherein the light-emitting filter allows light in a specific wavelength band to pass through.

14. The electronic device according to claim 11, wherein the first fluorescent substance and the second fluorescent substance are the same fluorescent substance.

15. The aforementioned electronic device is Including the photography department, The one or more processors described above are: A first unseparated image is obtained by photographing the sample, which includes the first biomolecule labeled with the first fluorescent substance and the second biomolecule that is not labeled, using the imaging unit. The electronic apparatus according to claim 11, wherein a second unseparated image is obtained by photographing the sample containing the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with the second fluorescent substance using the imaging unit.

16. A non-temporary computer-readable recording medium that records instructions for causing one or more processors to perform an operation when executed by one or more processors, The instruction is performed by one or more processors. A first unseparated image was obtained of a sample containing a first biomolecule labeled with a first fluorescent substance and a second biomolecule that was not labeled. A second unseparated image of the sample containing the first biomolecule labeled with the first fluorescent substance and the second biomolecule labeled with the second fluorescent substance is obtained. Based on the first unseparated image and the second unseparated image, a separated image of the second biomolecule is generated. The first fluorescent substance and the second fluorescent substance are determined such that a first wavelength value, which maximizes the intensity of the emission signal within the emission spectrum of the first fluorescent substance, and a second wavelength value, which maximizes the intensity of the emission signal within the emission spectrum of the second fluorescent substance, satisfy predetermined conditions. The second unseparated image is obtained by taking a first unseparated image of the sample, then labeling the second biomolecule contained in the sample with a second fluorescent substance, and then taking an image of the sample, and is recorded on a computer-readable recording medium.

Images