Bronchial image generation method, electronic device, and computer-readable storage medium

Abstract

The application relates to the field of medical devices, and provides a bronchus image generation method, an electronic device and a computer readable storage medium, which can efficiently and inexpensively obtain a bronchus image. The method comprises the following steps: acquiring a basic database, wherein the basic database comprises a plurality of real lung pictures and lung body membrane pictures; selecting a picture from the basic database as an original picture; inputting the original picture into a pre-trained target generative adversarial network to obtain a target picture corresponding to the original picture output by the pre-trained target generative adversarial network. Compared with the prior art, the technical scheme provided by the application can efficiently and inexpensively obtain a bronchus image.

Description

Technical Field

[0001] This invention relates to the field of medical devices, and more particularly to a method for generating bronchial images, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of artificial intelligence (AI) technology, it has achieved significant breakthroughs in the medical field, such as medical image registration and endoscopic navigation. The performance of AI technology relies on large-scale, high-quality medical data as a training set to train the model. Taking bronchoscopic navigation in pulmonary intervention as an example, the corresponding algorithm usually requires bronchoscopy on a lung phantom to collect lung phantom images for simulation testing, which can cost tens of thousands of dollars. On the other hand, when lung images are needed as a training set for testing, data must also be collected from living lungs.

[0003] As can be seen from the above description, the cost of acquiring both lung phantom images and images of the lungs is extremely high, hindering the promotion and application of AI technology in the medical field. Summary of the Invention

[0004] The bronchial image generation method, electronic device, and computer-readable storage medium provided in this application embodiment can efficiently and cost-effectively acquire bronchial images.

[0005] One embodiment of this application provides a method for generating bronchial images, including:

[0006] Obtain a basic database containing several real images of the lungs and lung membranes;

[0007] The images selected from the aforementioned basic database will be used as the original images;

[0008] The original image is input into a pre-trained target generative adversarial network to obtain a target image corresponding to the original image, output by the pre-trained target generative adversarial network.

[0009] One aspect of this application also provides a bronchial image generation device, including:

[0010] The acquisition module is used to acquire a basic database, which contains several real images of the lungs and lung membranes.

[0011] The selection module is used to select images from the basic database as the original images;

[0012] The generation module is used to input the original image into a pre-trained target generative adversarial network to obtain a target image corresponding to the original image output by the pre-trained target generative adversarial network.

[0013] One aspect of this application also provides an electronic device, including: a memory and a processor;

[0014] The memory stores executable program code;

[0015] The processor coupled to the memory calls the executable program code stored in the memory to execute the bronchial image generation method provided in the above embodiments.

[0016] One aspect of this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when run by a processor, implements the bronchial image generation method provided in the above embodiments.

[0017] As can be seen from the technical solution provided in this application, on the one hand, since the basic database contains a wide range of real lung images and lung membrane images, and the acquisition method is simple, there is no acquisition cost problem. By inputting the original image into a pre-trained target generative adversarial network, a target image corresponding to the original image is obtained from the output of the pre-trained target generative adversarial network. Therefore, bronchial images can be acquired efficiently and cheaply, reducing the cost of various algorithms such as medical image registration and / or endoscopic navigation that require simulation testing using lung phantoms or live data collection, which is conducive to the application and promotion of AI technology in the medical field. On the other hand, compared with lung images acquired through bronchoscopy and virtual bronchial tree slice images obtained through CT scans, the modal difference between the simulated lung images and virtual bronchial tree slice images generated by the pre-trained target generative adversarial network is smaller, which can reduce the modal difference between the two, thereby facilitating matching during endoscopic navigation and improving navigation performance. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of a bronchial image generation method provided in an embodiment of this application;

[0020] Figure 2This is a schematic diagram of the structure of a target generative adversarial network provided in an embodiment of this application;

[0021] Figure 3 A schematic diagram of the bronchial image generation device provided in the embodiments of this application;

[0022] Figure 4 This application provides a schematic diagram of the target adversarial network as a first generative adversarial network in an embodiment of the present application.

[0023] Figure 5 This application provides a schematic diagram of the target adversarial network as a second generative adversarial network in an embodiment of the present application.

[0024] Figure 6a This application provides a schematic diagram of the target adversarial network as a third generative adversarial network in an embodiment of the present application.

[0025] Figure 6b A schematic diagram of the target adversarial network as a third generative adversarial network is provided for another embodiment of this application;

[0026] Figure 7 A schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] See Figure 1 The following is a flowchart illustrating the implementation of a bronchial image generation method according to an embodiment of this application. (See attached...) Figure 1 As shown, the method mainly includes:

[0029] Step S101: Obtain the basic database, which contains several real images of the lungs and lung membranes.

[0030] In this embodiment, real lung images and lung phantom images can exist in a basic database. "Existing" means that they are widely available and easy to obtain. For example, real lung images may be obtained from data collection on the lungs of patients who must undergo endoscopic interventions, or from historical images obtained from data collection on the lungs of a large number of patients who underwent endoscopic interventions before AI technology was applied to the medical field. They may also be generated from lung images generated by other neural networks or from lung images generated by the pre-trained target generative adversarial network mentioned in this embodiment, etc. In other words, the acquisition of these real lung images is either not specifically for the purpose of implementing the technical solution of this embodiment, and the acquisition of these real lung images in this embodiment can be regarded as a "convenient" behavior, or it completely avoids the live animal acquisition process (e.g., generated by a pre-trained neural network). Similarly, the acquisition of existing lung phantom images in this embodiment is also purely a "convenient" behavior. On the one hand, because existing real lung images and lung phantom images are widely available and easy to obtain, the cost of implementing the technical solution of this application or obtaining these existing real lung images and lung phantom images is almost negligible. On the other hand, there is no need for a one-to-one correspondence between the aforementioned real lung images and lung phantom images, which simplifies the process of obtaining real lung images and lung phantom images and objectively reduces the cost of obtaining these real lung images and lung phantom images.

[0031] Step S102: Select the images from the basic database as the original images.

[0032] The original image means that it can be used as input data for neural networks (e.g., generative adversarial networks) or for training neural networks.

[0033] Step S103: Input the original image into the pre-trained target generative adversarial network to obtain the target image corresponding to the original image output by the pre-trained target generative adversarial network.

[0034] In the embodiments of this application, the pre-trained target generative adversarial network is obtained by training a target generative adversarial network (GAN). This means that the technical solution of this application also includes the process of training a GAN. Specifically, the technical solution of this application also includes: obtaining a training set of the target generative adversarial network, wherein the training set includes simulated lung phantom images, simulated lung interior images, real lung interior images, lung phantom images and / or virtual bronchial tree slice images; and training the target generative adversarial network using the training set to obtain the trained target generative adversarial network.

[0035] like Figure 2 As shown in the above embodiment, the target generative adversarial network mainly includes a forward generator 201, a backward generator 202, a forward discriminator 203 corresponding to the forward generator 201, and a backward discriminator 204 corresponding to the backward generator 202. The forward generator 201 can be any model that can output images, such as the simplest fully connected neural network or deconvolutional network. This means that when an n-dimensional vector is input, both the forward generator 201 and the backward generator 202 can be models that can output images. The input of the backward generator 202 is set to be the output of the forward generator 201 or the input is of the same type as the output of the forward generator 201, and its output is of the same type as the input of the forward generator 201 or the input of the forward generator 201. For the forward discriminator 203 or the reverse discriminator 204, it can be any discriminator model. In this embodiment, the forward discriminator 203 or the reverse discriminator 204 can be a binary classification discriminator, whose input is an image and whose output is the true or false label of the image. As an embodiment of this application, training the target generative adversarial network using a training set to obtain the trained target generative adversarial network can be achieved by: constructing a target loss function; inputting the training samples contained in the training set into the generative adversarial network, training the generative adversarial network based on the target loss function until the value of the target loss function is minimized, thereby obtaining the trained target loss function. The aforementioned target loss function includes the adversarial loss function (denoted as...). Consistent loss function (denoted as) ) and continuous loss function (denoted as ), etc. In the embodiments of this application, the expression for the adversarial loss function is: = arg arg log D(Y)+(1-logD(G(X))), where Y represents image data extracted from images generated by the target generative adversarial network, X represents image data extracted from simulated lung phantom images, simulated lung interior images, real lung interior images, lung phantom images, and / or virtual bronchial tree slice images, G(X) represents the generator with input X, and D(Y) represents the discriminator with input Y. The goal of adjusting the adversarial loss function is to balance the generator G and the discriminator D, so that the data generated by the generator G is judged as real; the expression for the consistency loss function is: =arg Its goal is to make the generated data pass through a reverse generator. The generated data has the smallest error compared to the input data X; the expression for the continuous loss function is: = The goal is to ensure that the generated data and the input data have the same optical flow variation, guaranteeing that the generated data also conforms to the same temporal distribution. In the expression of the continuous loss function, Indicates the solution , express , Let X represent the data at time t. In the above embodiment, the training samples contained in the training set are input into the generative adversarial network (GAN), and the GAN is trained based on the target loss function until the value of the target loss function is minimized. Specifically, the target loss function can be obtained by: using the training set as the input of the forward generator, adjusting the parameters of the forward discriminator or the forward discriminator based on the output of the forward generator until the value of the target loss function is minimized; or using the training set as the input of the reverse generator, adjusting the parameters of the reverse discriminator or the reverse generator based on the output of the reverse discriminator until the value of the target loss function is minimized. For example, during training, the forward discriminator receives real data and fake data generated by the forward generator. The parameters of both the forward generator and the forward discriminator can be tuned simultaneously based on the final output: if the forward discriminator makes a correct judgment, the parameters of the forward generator are adjusted to make the generated fake data more realistic; if the forward discriminator makes an incorrect judgment, its parameters are adjusted to avoid future errors. Training continues until both reach a balanced and harmonious state, where the target loss function, consistent loss function, and continuous loss function are minimized. Tuning the parameters of the inverse generator and the inverse discriminator can be done similarly, and will not be elaborated further.

[0036] Figure 2 The example target generative adversarial network is trained until its target loss function reaches its minimum value, thus obtaining the pre-trained target generative adversarial network. Subsequently, the pre-trained target generative adversarial network can generate target images corresponding to the original images according to actual needs.

[0037] Since the target image may be an image of the lung membrane, a simulated lung image, or a virtual bronchial tree slice image, meaning there is a need for images of the lung membrane, a need for simulated lung images, or a need for virtual bronchial tree slice images, the following explains each case separately:

[0038] When there is a need to obtain lung phantom images, the original images in the above embodiments are real lung images in the base database, and the target adversarial network is a first generative adversarial network. In this case, as an embodiment of this application, inputting the original image into a pre-trained target generative adversarial network to obtain a target image corresponding to the original image output by the pre-trained target generative adversarial network can be achieved by: inputting the selected real lung image into the pre-trained first generative adversarial network to obtain a simulated lung phantom image output by the first generative adversarial network corresponding to the selected real lung image. For example... Figure 4 The diagram shown is a schematic of the structure where the target adversarial network is the first generative adversarial network.

[0039] When there is a need to obtain simulated intrapulmonary images, the original image is a lung membrane image from the base database, and the target adversarial network is a pre-trained second generative adversarial network. In another embodiment of this application, inputting the original image into the pre-trained target generative adversarial network to obtain the target image corresponding to the original image output by the pre-trained target generative adversarial network can be achieved by: inputting a selected lung membrane image into the pre-trained second generative adversarial network to obtain a simulated intrapulmonary image corresponding to the selected lung membrane image output by the second generative adversarial network. For example... Figure 5 The diagram shown is a schematic of the target adversarial network being a second generative adversarial network.

[0040] When there is a need to obtain virtual bronchial tree slice images, the original image is a real lung image or pleural membrane image from the base database, and the target adversarial network is a pre-trained third generative adversarial network. In another embodiment of this application, inputting the original image into the pre-trained target generative adversarial network to obtain the target image corresponding to the original image output by the pre-trained target generative adversarial network can be achieved by: inputting the selected real lung image or pleural membrane image into the pre-trained third generative adversarial network to obtain a virtual bronchial tree slice image output by the third generative adversarial network corresponding to the selected real lung image or pleural membrane image. Figure 6a The diagram shown is a schematic diagram of the target adversarial network of an embodiment of this application, which is a third generative adversarial network. The third generative adversarial network uses real lung images as the original images. Figure 6bThe diagram shown illustrates the structure of a third generative adversarial network (GAN) as the target adversarial network in another embodiment of this application. This third GAN uses lung membrane images as the original images. Compared to real lung images acquired through bronchoscopy and virtual bronchial tree slice images obtained through CT scans, the modal difference between the simulated lung images obtained through a pre-trained second GAN and the virtual bronchial tree slice images obtained through a pre-trained third GAN is smaller. This reduces the modal difference between the two, which is beneficial for matching during navigation and improves navigation performance.

[0041] After obtaining the target images corresponding to the original images output by the pre-trained target generative adversarial network (PGA) in step S103, these generated target images can be added to the basic database so that they can be used as original images input to the GPA for processing. For example, inputting a target image, i.e., a lung membrane image, into the GPA will generate a corresponding simulated lung image or a virtual bronchial tree slice image; or, inputting a target image, i.e., a simulated lung image, into the GPA will generate a corresponding lung membrane image or a virtual bronchial tree slice image, and so on.

[0042] As can be seen from the technical solution provided in this application, on the one hand, since the basic database contains a wide range of real lung images and lung membrane images, and the acquisition method is simple, there is no acquisition cost problem. By inputting the original image into a pre-trained target generative adversarial network, a target image corresponding to the original image is obtained from the output of the pre-trained target generative adversarial network. Therefore, bronchial images can be acquired efficiently and cheaply, reducing the cost of various algorithms such as medical image registration and / or endoscopic navigation that require simulation testing using lung phantoms or live data collection, which is conducive to the application and promotion of AI technology in the medical field. On the other hand, compared with lung images acquired through bronchoscopy and virtual bronchial tree slice images obtained through CT scans, the modal difference between the simulated lung images and virtual bronchial tree slice images generated by the pre-trained target generative adversarial network is smaller, which can reduce the modal difference between the two, thereby facilitating matching during endoscopic navigation and improving navigation performance.

[0043] See Figure 3 This is a schematic diagram of a bronchial image generation device according to an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. The device may be a computer terminal, or a software module configured on that computer terminal. Figure 3 As shown, the device includes: an acquisition module 301, a selection module 302, and a generation module 303, which are described in detail below:

[0044] The acquisition module 301 is used to acquire a basic database, which contains several real lung images and lung membrane images;

[0045] Module 302 is used to select images from the base database as the original images;

[0046] The generation module 303 is used to input the original image into a pre-trained target generative adversarial network to obtain a target image corresponding to the original image output by the pre-trained target generative adversarial network.

[0047] Furthermore, when there is a need to obtain lung membrane images, the original images in the above embodiments are real lung images in the base database, the target adversarial network is a first generative adversarial network, and the generation module 303 is specifically used to obtain a simulated lung membrane image output by the first generative adversarial network corresponding to the selected real lung image by inputting the selected real lung image into the pre-trained first generative adversarial network.

[0048] Furthermore, when there is a need to obtain simulated lung images, the original images in the above embodiments are lung membrane images in the basic database, the target adversarial network is a pre-trained second generative adversarial network, and the generation module 303 is specifically used to obtain simulated lung images corresponding to the selected lung membrane images by inputting the selected lung membrane images into the pre-trained second generative adversarial network.

[0049] Furthermore, when there is a need to obtain virtual bronchial tree slice images, the original image is a real lung image or lung membrane image in the base database, the target adversarial network is a pre-trained third generative adversarial network, and the generation module 303 is specifically used to obtain a virtual bronchial tree slice image corresponding to the selected real lung image or lung membrane image by inputting the selected real lung image or lung membrane image into the pre-trained third generative adversarial network.

[0050] Furthermore, Figure 3 The example apparatus may also include an adding module for adding the generated target image to the base database so that the target image is used as the raw image input to the target generative adversarial network for processing.

[0051] Furthermore, Figure 3 The example device may also include a training set acquisition module and a training module, wherein:

[0052] The training set acquisition module is used to acquire the training set of the target generative adversarial network. The training set includes simulated lung phantom images, simulated lung interior images, real lung interior images, lung phantom images, and virtual bronchial tree slice images.

[0053] The training module is used to train the target generative adversarial network using the training set to obtain the trained target generative adversarial network.

[0054] Furthermore, the target generative adversarial network in the above embodiments includes a forward generator, a backward generator, a forward discriminator corresponding to the forward generator, and a backward discriminator corresponding to the backward generator. The training module may further include a construction unit and an optimization unit, wherein:

[0055] A construction unit is used to construct a target loss function, which includes an adversarial loss function, a consistent loss function, and a continuous loss function.

[0056] The tuning unit is used to input the training samples contained in the training set into the target generative adversarial network, and train the target generative adversarial network based on the target loss function until the value of the target loss function is minimized, so as to obtain the trained target loss function.

[0057] Furthermore, the aforementioned adversarial loss function is: Where Y represents the image data extracted from the image generated by the target generative adversarial network, X represents the image data extracted from simulated lung phantom images, simulated lung images, real lung images, lung phantom images, and / or virtual bronchial tree slice images, G(X) represents the generator with input X, and D(Y) represents the discriminator with input Y; the consistency loss function is... , It is a reverse generator; the continuous loss function is , Indicates the solution , express , Let X represent the data at time t.

[0058] Furthermore, the aforementioned tuning unit may include a first parameter adjustment unit or a second parameter adjustment unit, wherein:

[0059] The first parameter adjustment unit is used to adjust the parameters of the forward discriminator based on the output of the forward generator, or to adjust the parameters of the forward generator based on the output of the forward discriminator, until the value of the target loss function is minimized.

[0060] The second parameter adjustment unit is used to adjust the parameters of the inverse discriminator based on the output of the inverse generator, or to adjust the parameters of the inverse generator based on the output of the inverse discriminator, until the value of the target loss function is minimized.

[0061] For details on how each module implements its function, please refer to [link / reference]. Figure 1The relevant details in the illustrated embodiments will not be repeated here.

[0062] As can be seen from the technical solution provided in this application, on the one hand, since the basic database contains a wide range of real lung images and lung membrane images, and the acquisition method is simple, there is no acquisition cost problem. By inputting the original image into a pre-trained target generative adversarial network, a target image corresponding to the original image is obtained from the output of the pre-trained target generative adversarial network. Therefore, bronchial images can be acquired efficiently and cheaply, reducing the cost of various algorithms such as medical image registration and / or endoscopic navigation that require simulation testing using lung phantoms or live data collection, which is conducive to the application and promotion of AI technology in the medical field. On the other hand, compared with lung images acquired through bronchoscopy and virtual bronchial tree slice images obtained through CT scans, the modal difference between the simulated lung images and virtual bronchial tree slice images generated by the pre-trained target generative adversarial network is smaller, which can reduce the modal difference between the two, thereby facilitating matching during endoscopic navigation and improving navigation performance.

[0063] See Figure 7 The present application provides a schematic diagram of the hardware structure of an electronic device according to an embodiment.

[0064] For example, the electronic device can be any of various types of computer system devices that are non-movable or movable or portable and perform wireless or wired communication. Specifically, the electronic device can be a desktop computer, server, mobile phone or smartphone (e.g., iPhone™-based, Android™-based phone), portable gaming device (e.g., Nintendo DS™, PlayStation Portable™, Gameboy Advance™, iPhone™), laptop computer, PDA, portable internet device, portable medical device, smart camera, music player and data storage device, other handheld devices and such as watches, headphones, pendants, etc. The electronic device can also be other wearable devices (e.g., such as electronic glasses, electronic clothing, electronic bracelets, electronic necklaces and other head-mounted devices (HMDs)).

[0065] like Figure 7As shown, the electronic device 100 may include a control circuit, which may include a storage and processing circuit 300. The storage and processing circuit 300 may include a memory, such as a hard disk drive, a non-volatile memory (e.g., flash memory or other electronically programmable erasure-limited memory used to form a solid-state drive), a volatile memory (e.g., static or dynamic random access memory), etc., which are not limited in this embodiment. The processing circuit in the storage and processing circuit 300 can be used to control the operation of the electronic device 100. This processing circuit may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits (ASICs), display driver integrated circuits, etc.

[0066] The storage and processing circuitry 300 can be used to run software in the electronic device 100, such as internet browsing applications, Voice over Internet Protocol (VoIP) telephone calling applications, email applications, media playback applications, operating system functions, etc. This software can be used to perform various control operations, such as image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display functions based on status indicators such as LED status lights, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the electronic device 100, etc., which are not limited in the embodiments of this application.

[0067] Furthermore, the memory stores executable program code, and the processor coupled to the memory calls the executable program code stored in the memory to execute the radio frequency operation prompting method as described in the foregoing embodiments.

[0068] The executable program code includes the following as described above. Figure 3 The various modules in the bronchial image generation device described in the illustrated embodiment include, for example, the acquisition module 301, the selection module 302, and the generation module 303.

[0069] The electronic device 100 may further include input / output circuitry 420. Input / output circuitry 420 enables the electronic device 100 to input and output data, allowing the electronic device 100 to receive data from external devices and also allowing the electronic device 100 to output data from the electronic device 100 to external devices. Input / output circuitry 420 may further include sensors 320. Sensors 320 may include ambient light sensors, light- and capacitance-based proximity sensors, touch sensors (e.g., light-based touch sensors and / or capacitive touch sensors, wherein the touch sensor may be part of a touch display screen or used independently as a touch sensor structure), accelerometers, and other sensors, etc.

[0070] The input / output circuit 420 may also include one or more displays, such as display 140. Display 140 may include one or more of the following: liquid crystal display, organic light-emitting diode display, electronic ink display, plasma display, and displays using other display technologies. Display 140 may include a touch sensor array (i.e., display 140 may be a touch screen). The touch sensor may be a capacitive touch sensor formed by an array of transparent touch sensor electrodes (e.g., indium tin oxide (ITO) electrodes), or it may be a touch sensor formed using other touch technologies, such as acoustic touch, pressure-sensitive touch, resistive touch, optical touch, etc., which are not limited in the embodiments of this application.

[0071] The electronic device 100 may also include an audio component 360. The audio component 360 can be used to provide audio input and output functions for the electronic device 100. The audio component 360 in the electronic device 100 may include a speaker, microphone, buzzer, tone generator, and other components for generating and detecting sound.

[0072] Communication circuitry 380 can be used to provide electronic device 100 with the ability to communicate with external devices. Communication circuitry 380 may include analog and digital input / output interface circuitry, and wireless communication circuitry based on radio frequency signals and / or optical signals. The wireless communication circuitry in communication circuitry 380 may include radio frequency transceiver circuitry, power amplifier circuitry, low-noise amplifier, switches, filters, and antennas. For example, the wireless communication circuitry in communication circuitry 380 may include circuitry for supporting Near Field Communication (NFC) by transmitting and receiving near-field coupled electromagnetic signals. For example, communication circuitry 380 may include a near-field communication antenna and a near-field communication transceiver. Communication circuitry 380 may also include cellular telephone transceivers and antennas, wireless local area network transceiver circuitry and antennas, etc.

[0073] The electronic device 100 may further include a battery, power management circuitry, and other input / output units 400. Input / output units 400 may include buttons, joysticks, click wheels, scroll wheels, touchpads, keypads, keyboards, cameras, LEDs, and other status indicators.

[0074] Users can input commands through the input / output circuit 420 to control the operation of the electronic device 100, and can use the output data of the input / output circuit 420 to receive status information and other outputs from the electronic device 100.

[0075] Furthermore, this application embodiment also provides a non-transitory computer-readable storage medium, which can be configured in the server in the above embodiments. The non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the bronchial image generation method described in the foregoing embodiments.

[0076] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0077] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0078] In the embodiments provided in this application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0082] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A bronchogram generation method characterized by, The method comprises: obtaining a basic database containing a plurality of real lung pictures and lung phantom pictures; selecting a picture from the basic database as an original picture; inputting the original picture into a pre-trained target generative adversarial network to obtain a target picture corresponding to the original picture output by the pre-trained target generative adversarial network; the target picture comprises a lung phantom picture, a simulated lung picture or a virtual bronchial tree slice picture, wherein when the target picture is a lung phantom picture, the original picture is a real lung picture selected from the basic database, when the target picture is a simulated lung picture, the original picture is a lung phantom picture selected from the basic database, and when the target picture is a virtual bronchial tree slice picture, the original picture is a real lung picture or a lung phantom picture selected from the basic database.

2. The bronchial image generation method of claim 1, wherein, When there is a demand to obtain a lung phantom picture, the original picture is a real lung picture in the basic database, and the target generative adversarial network is a first generative adversarial network; inputting the original picture into a pre-trained target generative adversarial network to obtain a target picture corresponding to the original picture output by the pre-trained target generative adversarial network comprises: inputting the selected real lung picture into a pre-trained first generative adversarial network to obtain a simulated lung phantom picture corresponding to the selected real lung picture output by the pre-trained first generative adversarial network.

3. The bronchial image generation method of claim 1, wherein, When there is a demand to obtain a simulated lung picture, the original picture is a lung phantom picture in the basic database, and the target generative adversarial network is a second generative adversarial network; inputting the original picture into a pre-trained target generative adversarial network to obtain a target picture corresponding to the original picture output by the pre-trained target generative adversarial network comprises: inputting the selected lung phantom picture into a pre-trained second generative adversarial network to obtain a simulated lung picture corresponding to the selected lung phantom picture output by the pre-trained second generative adversarial network.

4. The bronchial image generation method of claim 1, wherein, When there is a demand to obtain a virtual bronchial tree slice picture, the original picture is a real lung picture or a lung phantom picture in the basic database, and the target generative adversarial network is a third generative adversarial network; inputting the original picture into a pre-trained target generative adversarial network to obtain a target picture corresponding to the original picture output by the pre-trained target generative adversarial network comprises: inputting the selected real lung picture or lung phantom picture into a pre-trained third generative adversarial network to obtain a virtual bronchial tree slice picture corresponding to the selected real lung picture or lung phantom picture output by the pre-trained third generative adversarial network.

5. The bronchial image generation method of claim 1, wherein, The method further comprises: adding the target picture to the basic database so that the target picture is input into the pre-trained target generative adversarial network as the original picture for processing.

6. The bronchial image generation method of claim 1, wherein, The method further comprises: The training set of the target generative adversarial network is obtained, and the training set includes simulated lung phantom pictures, simulated lung pictures, real lung pictures, lung phantom pictures, and virtual bronchial tree slice pictures. The target generative adversarial network is trained by using the training set, and the pre-trained target generative adversarial network is obtained.

7. The bronchial image generation method of claim 6, wherein, The target generative adversarial network includes a forward generator, a reverse generator, a forward discriminator corresponding to the forward generator, and a reverse discriminator corresponding to the reverse generator. The target generative adversarial network is trained by using the training set, and the pre-trained target generative adversarial network is obtained. The target loss function is constructed, and the target loss function includes an adversarial loss function, a consistency loss function, and a continuity loss function. The training samples in the training set are input into the target generative adversarial network, and the target generative adversarial network is trained based on the target loss function until the value of the target loss function is minimized to obtain a trained target loss function.

8. The bronchial image generation method of claim 7, wherein, The adversarial loss function is , Y represents image data extracted from an image generated by the target generative adversarial network, X represents image data extracted from the simulated lung phantom picture, simulated lung picture, real lung picture, lung phantom picture, and / or virtual bronchial tree slice picture, G(X) represents a generator with input X, and D(Y) represents a discriminator with input Y. The consistent loss function is , the is a reverse generator; The continuous loss function is , the represents solving , represents , the represents the data X at time t.

9. The bronchial image generation method of claim 7, wherein, The training samples in the training set are input into the target generative adversarial network, and the target generative adversarial network is trained based on the target loss function until the value of the target loss function is minimized to obtain a trained target loss function. The training set is used as the input of the forward generator, and the parameters of the forward discriminator are adjusted according to the output result of the forward generator or the parameters of the forward generator are adjusted according to the output result of the forward discriminator until the value of the target loss function is minimized; or The training set is used as the input of the reverse generator, and the parameters of the reverse discriminator are adjusted according to the output result of the reverse generator or the parameters of the reverse generator are adjusted according to the output result of the reverse discriminator until the value of the target loss function is minimized.

10. A bronchogram generation apparatus characterized by comprising: The device includes: The acquisition module is configured to acquire a basic database, and the basic database includes a plurality of real lung pictures and lung phantom pictures; The selection module is configured to select pictures from the basic database as original pictures; The generation module is configured to input the original pictures into a pre-trained target generative adversarial network to obtain target pictures corresponding to the original pictures output by the pre-trained target generative adversarial network; The target pictures include lung phantom pictures, simulated lung pictures, or virtual bronchial tree slice pictures, wherein when the target pictures are lung phantom pictures, the original pictures are real lung pictures selected from the basic database, when the target pictures are simulated lung pictures, the original pictures are lung phantom pictures selected from the basic database, and when the target pictures are virtual bronchial tree slice pictures, the original pictures are real lung pictures or lung phantom pictures selected from the basic database.

11. An electronic device, comprising: The electronic device includes a memory and a processor; The memory stores executable program codes; The processor coupled to the memory invokes the executable program codes stored in the memory to execute the bronchial image generation method according to any one of claims 1 to 9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the bronchogram generation method according to any one of claims 1 to 9.

Images