Model training method, image reconstruction method, and related device

Abstract

The present disclosure relates to a model training method, an image reconstruction method, and a related device. A computer device acquires a pre-trained target neural network model, the target neural network model being obtained by training a plurality of groups of training samples by means of a machine learning algorithm, and each group of training samples comprising a CTP image and a CTA image which are paired with each other, wherein the CTP image and the CTA image which are paired with each other correspond to the same selected level of target tissue; a target CTP image is input into the target neural network model, such that the target neural network model reconstructs a target CTA image on the basis of the target CTP image. The blood vessel imaging effect of the target CTA image is close to that of a real CTA image or is even better that that of the real CTA image, thereby providing a real and accurate reference basis for examination of each tissue and organ, and a patient can undergo one less CTA examination, thereby increasing the speed of one-stop CT examination for stroke, avoiding exposure to radiation of one more CTA examination, and reducing one injection of a contrast agent, lowering the risk of side effects generated by the contrast agent.

Description

Model training method, image reconstruction method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on October 9, 2023, with application number 202311298638.3 and application name “Model training method, image reconstruction method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to a model training method, an image reconstruction method and related equipment. Background Art

[0003] Computed Tomography Angiography (CTA) is a type of computed tomography (CT) that can be used to image blood vessels throughout the body, effectively visualizing arteries and veins. By injecting a contrast agent into the blood vessels, images are created to identify aneurysms (blood vessels that have enlarged and may rupture), narrowed blood vessels due to atherosclerosis (fatty deposits that form plaques on artery walls), abnormal blood vessel structures within the brain, blood vessels damaged by injury, blood clots that may form in leg veins and travel to the lungs, and tumors supplied by blood vessels. CTA can be used to display blood vessels in the heart, aorta, and other large vessels, lungs, kidneys, head, neck, arms, and legs. It can also be used to locate bleeding in the gastrointestinal system from arteries or veins. Information from CTA can help prevent strokes or heart attacks.

[0004] Computed Tomography Perfusion (CTP) is a new imaging modality that can assess the brain's microcirculation. Unlike traditional CT imaging, which assesses brain structure and morphology, CTP collects physiological information about cerebral perfusion status. CTP imaging can significantly impact the initial assessment of patients with acute cerebral infarction by helping to identify the ischemic penumbra. On the other hand, diffusion magnetic resonance imaging can identify dead brain tissue, while CTP imaging can help identify dying brain tissue, which, once identified, can be recovered with timely and appropriate treatment. Several studies have demonstrated the potential role and development of CT perfusion imaging in the assessment and tracking of intracranial and extracranial stenotic and occlusive diseases, the assessment of vasospasm after subarachnoid hemorrhage, the differentiation of tumors from infection, and the confirmation of brain death.

[0005] Brain CTA images can assist doctors in diagnosing brain diseases, while brain CTP images can reveal blood flow velocity and blood supply to brain tissue. Therefore, brain CTA and CTP are currently important diagnostic tools for stroke-related diseases. For patients with acute ischemic cerebrovascular disease caused by anterior circulation large artery occlusion, if the time window for onset is beyond 6 to 24 hours or the onset time is unknown, CTP imaging can assess the core infarct area and ischemic penumbra, helping to select patients suitable for intra-arterial treatment. Therefore, in a one-stop CT examination for stroke, CTP should be performed in addition to the plain scan and CTA: plain scan first, then CTA, and finally CTP. However, if CTA is performed first to identify abnormal vessels, the patient must wait for the contrast agent to be expelled before a new contrast agent is injected for CTP. This prolongs the examination time for stroke treatment and exposes the patient to multiple radiation doses and contrast agent injections.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a model training method, an image reconstruction method, and related equipment for reconstructing a CTA image with high-quality vascular imaging effect from a CTP image using a pre-trained target neural network model.

[0008] In a first aspect, an embodiment of the present application provides an image reconstruction method, the method comprising:

[0009] Acquire the target CTP image to be processed;

[0010] Obtaining a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple sets of training samples using a machine learning algorithm, and each set of training samples includes paired CTP images and CTA images, wherein the paired CTP images and CTA images correspond to the same selected layer of the target tissue;

[0011] The target CTP image is input into the target neural network model so that the target neural network model reconstructs the target CTA image corresponding to the target CTP image based on the target CTP image, and obtains the target CTA image output by the target neural network model.

[0012] In a second aspect, an embodiment of the present application provides a model training method, the method comprising:

[0013] Acquiring multiple sets of training samples, each set of training samples including paired CTP images and CTA images, wherein the paired CTP images and CTA images correspond to the same selected layer of target tissue;

[0014] Get the initial neural network model;

[0015] Multiple groups of training samples are input into the initial neural network model so that the initial neural network model generates predicted CTA images based on the CTP images in the training samples. When the relationship between the predicted CTA images and the CTA images in the training samples meets the convergence condition, the training is stopped to obtain the target neural network model.

[0016] In a third aspect, an embodiment of the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method of the first aspect.

[0018] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0019] The target CTA image corresponding to the target CTP image is reconstructed using a pre-trained target neural network model. The vascular imaging effect of the target CTA image can be close to or even better than the real CTA image, providing a true and accurate reference basis for the examination of various tissues and organs. Therefore, for brain examinations, patients can undergo one less CTA examination, speeding up the one-stop CT examination for stroke, avoiding the radiation exposure of an additional CTA examination, and reducing the injection of contrast agent, thereby reducing the risk of side effects of contrast agent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] FIG1a is a flow chart of a model training method according to an embodiment of the present application;

[0022] FIG1b shows a schematic diagram of the architecture of an initial neural network provided according to at least one embodiment of the present disclosure;

[0023] FIG1c shows a schematic structural diagram of a time series information fusion module according to at least one embodiment of the present disclosure;

[0024] FIG1d shows a schematic diagram of training an initial neural network according to at least one embodiment of the present disclosure;

[0025] FIG2a is a schematic diagram of a flow chart of an image reconstruction method in an embodiment of the present application;

[0026] FIG2 b shows a comparison diagram of a CTA image predicted by the image reconstruction method according to an embodiment of the present disclosure and a real CTA image;

[0027] FIG3 is a schematic diagram of the structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0033] The embodiments of the present application provide a model training method, an image reconstruction method, and related equipment for reconstructing a CTA image with high-quality vascular imaging effect from a CTP image using a pre-trained target neural network model.

[0034] The following describes the model training method in the embodiment of the present application:

[0035] Referring to FIG. 1a , an embodiment of a model training method in an embodiment of the present application includes:

[0036] 101. Acquire multiple groups of training samples, each group of training samples including paired CTP images and CTA images, wherein the paired CTP images and CTA images correspond to the same selected layer of target tissue.

[0037] In this embodiment, the target tissue can be any organ or tissue in the human or animal body, such as the brain, heart, aorta and other large blood vessels, lungs, kidneys, and other organs and tissues. Taking the brain as an example, a brain CTP and brain CTA medical examination can be performed on the same patient, and CTP images and CTA images of the same brain structure can be acquired. That is, a CTP examination is performed on n selected brain slices to obtain CTP images of each selected slice; and a CTA examination is performed on m selected brain slices to obtain CTA images of each selected slice, where n and m are both positive integers greater than 1, and m>n.

[0038] When acquiring CTP images, n × t images are obtained by scanning n slices of the target tissue t times, where n and t are both positive integers greater than 1. For each selected slice, the t images obtained from the t scans are superimposed in the chronological order of the t scans, resulting in a CTP image with t time-series channels corresponding to that slice. CTA images are acquired for m slices of the target tissue, where the m slices include the n slices. For each slice in the n slices, a CTA image is identified from the CTA images of the m slices that is paired with the CTP image of that slice. These paired CTP and CTA images at that slice constitute a set of training samples. Due to the different scanning methods of CTA and CTP images, for example, for a CTP examination, the selected slice is scanned t times, resulting in t CTP images. For a CTA examination, the selected slice is scanned only once, resulting in only one CTA image. Therefore, the paired CTP and CTA images include t CTP images and one CTA image.

[0039] Furthermore, the CTP and CTA images included in the training samples are derived from CTP and CTA medical examinations, respectively. Since it is unlikely that the slices selected for scanning in CTP and CTA examinations are exactly the same, the slice thicknesses and inter-slice structures between adjacent slices in CTP and CTA images may differ. For example, a CTP scan may include slices A, B, and C, while a CTA scan may include slices A, B, C, D, and E. While both slices have slices A, B, and C, this does not necessarily mean that the CTP image for slice A will necessarily correspond to the CTA image for slice A. For example, if the morphological structure of the CTP image for slice A is more similar to the morphological results of the CTA image for slice B, then the CTP image for slice A will be considered a match with the CTA image for slice B.

[0040] The CTA image that is paired with the CTP image at the m layers is determined from the CTA images at the m layers. The pairing of the CTP image and the CTA image can be determined manually. A person can select a CTA image that is morphologically similar or identical to the layer corresponding to the CTP image from the CTA images at the m layers, and pair it with the CTP image.

[0041] For example, assuming the n slices are slices A, B, and C of the target tissue, CTP images can be obtained for each of these slices; and assuming the m slices are slices A, B, C, D, and E of the target tissue, CTA images can be obtained for each of these slices. For slice A, a CTA image with a morphological structure similar to or identical to slice A can be selected from the CTA images of slices A, B, C, D, and E, and this CTA image can then be paired with the CTP image of slice A. Similarly, for slice B, a CTA image with a morphological structure similar to or identical to slice B can be selected from the CTA images of slices A, B, C, D, and E, and this CTA image can then be paired with the CTP image of slice B. Similarly, for slice C, a CTA image paired with the CTP image of slice C can be obtained.

[0042] Optionally, an image matching algorithm can be used to pair the CTP image with the CTA image. For example, taking the morphological structure as the pairing criterion, for the CTP images of n layers, a CTA image that is paired with the CTP image of each layer in the n layers is determined from the CTA images of m layers. When the difference in morphological structure between a CTA image of a layer and a CTP image of a layer is less than a preset threshold, the CTA image is determined to be paired with the CTP image. In this implementation, the image matching algorithm includes rigid registration, non-rigid registration, a method based on grayscale information, and a method based on a transform domain. Among them, the method based on grayscale information includes methods such as cross-correlation method, correlation coefficient measurement, sequential similarity detection algorithm, information theory exchange information similarity criterion, and the like. The method based on the transform domain includes methods such as phase correlation method, Walsh Transform transformation, and the like.

[0043] Alternatively, for example, for a patient, t consecutive dynamic scans are performed on n selected brain slices, resulting in t scan time points. Each scan time point then produces n images. For each slice, images at t scan time points are selected for that slice and then superimposed in the chronological order of the t scans. Ultimately, n t-channel CTP images are obtained, each corresponding to a selected slice of the brain.

[0044] In each training set, the paired CTP and CTA images correspond to the same selected slice of the same brain structure. In this embodiment, the paired CTA images are determined based on the CTP image, so the slice corresponding to the paired CTA image is the selected slice corresponding to the CTP image. For example, if the selected slice corresponding to the CTP image is slice A, and from the CTA images of slices A, B, C, D, and E, the CTP image of slice A is paired with the CTA image of slice E, then both the paired CTP and CTA images correspond to selected slice A.

[0045] In this embodiment, the CTP images in the training samples can be concatenated CTP images or unconcatenated CTP images. For example, unconcatenated CTP images include four dimensions: batch size (bs), number of image channels, image length (h), and image width (w), while concatenated CTP images include five dimensions: batch size (bs), number of timing channels (t), number of image channels, image length (h), and image width (w). Optionally, the CTP images in this application can include multiple dimensions, such as four, five, or more dimensions.

[0046] For example, the five dimensions of the CTP image in the training sample include batch size (bs), the number of timing channels (t), the number of image channels, the image length (h), and the image width (w). For conventional color images, the number of image channels is 3, namely, RGB (red, green, and blue) channels. For CTP images, they are usually single-channel grayscale images, so the number of image channels is 1. In this embodiment, the number of image channels can be set to a preset value. The number of image channels is related to the color of the image, and the value of the number of image channels is, for example, 1, 3, etc. The other dimensions of the above five dimensions are input values. Optionally, the number of image channels can also be set to an input value, for example, "c" is used to represent the number of image channels.

[0047] 102. Obtain an initial neural network model.

[0048] Figure 1b illustrates a schematic diagram of the architecture of an initial neural network provided according to at least one embodiment of the present disclosure. As shown in Figure 1b , initial neural network 100 includes a backbone neural network 110 and a temporal information fusion module 120. The output of backbone neural network 110 serves as the input to temporal information fusion module 120. The input to backbone neural network 110 is a training sample, such as the training sample obtained by executing step 101. The output of temporal information fusion module 120 is a predicted CTA image.

[0049] In this embodiment, the initial neural network model includes a backbone neural network and a temporal information fusion module. The backbone neural network can be any neural network, such as a convolutional neural network (CNN), a feedforward neural network, a recurrent neural network (LSTM / GRU / attention), a generative adversarial network (GAN), or the like. For example, the backbone neural network 110 is a convolutional neural network, or the backbone neural network 110 includes a convolutional neural network and a concatenation module, where the concatenation module can concatenate input images.

[0050] The temporal information fusion module includes any functional module capable of performing image dimensionality reduction. For example, dimensionality reduction can include eliminating the dimension of the number of temporal channels. Because a single-channel CTA image is available for a selected slice, while t channels are available for CTP images, eliminating the temporal channel-dependent dimension is necessary when deriving a CTA image from a CTP image.

[0051] In this embodiment, the backbone neural network is connected to the temporal information fusion module, which includes the above-mentioned functional module capable of performing image dimensionality reduction operations and the convolution layer, thereby forming the initial neural network model.

[0052] Figure 1c shows a schematic diagram of the structure of a temporal information fusion module provided according to at least one embodiment of the present disclosure. As shown in Figure 1c, the temporal information fusion module 120' includes an image dimensionality reduction module 121 and a convolution module 122. The convolution layer module 122 may include at least one convolution layer, for example, the convolution layer module 122 includes two convolution layers. The image dimensionality reduction module 121 has multiple computing units, for example, the number of computing units corresponds to the number of temporal channels t+1, for example, when t equals 6, there are 7 computing units. In this embodiment, the input of the image dimensionality reduction module 121 is the output of the backbone neural network, for example, the output of the backbone neural network is a 5-dimensional feature map. The output of the image dimensionality reduction module 121 is a 4-dimensional feature map, which is also the input of the convolution layer module 122. The output of the convolution layer module 122 is a single 4-dimensional feature map of the predicted CTA image. Optionally, the single feature map output by the convolution layer module 122 can be used as the predicted CTA image.

[0053] 103. Input multiple groups of the training samples into the initial neural network model so that the initial neural network model generates predicted CTA images based on the CTP images in the training samples, and stop training when the relationship between the predicted CTA images and the CTA images in the training samples meets the convergence condition to obtain a target neural network model.

[0054] In this embodiment, the dimensions of the CTP image in the training sample include the number of time-series channels, t, where t is a positive integer greater than 1. The time-series channels correspond to the scanning time points of the CTP image. That is, if there are t scanning time points, the obtained CTP image has t time-series channels. For example, a CTP image may have five dimensions, including the number of time-series channels, t.

[0055] In this embodiment, the CTP images in the acquired training samples include superimposed CTP images and non-superimposed CTP images. When the CTP image is a non-superimposed CTP image, an initial neural network model is required to perform superimposition processing on the CTP image. When the CTP image is a superimposed CTP image, the initial neural network model does not need to perform superimposition processing on the CTP image. For example, the backbone neural network of the initial neural network model only includes a convolution layer (also referred to as a convolutional neural network), then the CTP image of the corresponding training sample is a superimposed CTP image. For another example, the backbone neural network of the initial neural network model includes a superimposition module and a convolution layer, then the CTP image in the corresponding training sample may be a non-superimposed CTP image, and the superimposition module performs superimposition processing on the CTP image.

[0056] Optionally, the initial neural network model includes a backbone neural network and a time series information fusion module. When the initial neural network model generates a predicted CTA image based on a CTP image in a training sample, the backbone neural network processes multiple CTP images included in the CTP image in the training sample to obtain multiple initial feature maps, and processes the multiple initial feature maps according to the order of the multiple time series channels to obtain multiple feature maps, wherein the multiple CTP images are associated with the multiple time series channels, and the dimensions of the multiple feature maps are higher than the dimensions of the multiple initial feature maps.

[0057] For example, the backbone neural network can perform stacking processing on multiple CTP images. For example, the dimension of the CTP image input to the backbone neural network is [bs, 1, h, w], where 1 is the number of image channels or the number of feature maps. The backbone neural network stacks multiple 4-dimensional CTP images. The number of multiple CTP images in each group of training samples depends on the number of time points, that is, the number of time-series channels t. The dimension of the stacked CTP image is [bs, t, 1, h, w]. The stacking changes the CTP image from 4 dimensions to 5 dimensions. In this implementation, multiple initial feature maps correspond to 4-dimensional CTP images, and multiple feature maps correspond to the stacked CTP images. Therefore, the dimensions of the multiple feature maps are higher than the dimensions of the multiple initial feature maps. Optionally, after obtaining multiple feature maps, the backbone neural network performs convolution processing on the multiple feature maps. For example, the dimensions of the multiple feature maps are [bs, t, 1, h, w]. After the convolution layer in the backbone neural network performs convolution processing on the multiple feature maps, the dimensions of the multiple feature maps obtained are [bs, t, C, h, w], where C is the number of convolution kernels in the convolution layer. The number of convolution kernels can be adjusted according to the actual design. For example, if the value of C is 32, the dimensions of the multiple feature maps obtained are [bs, t, 32, h, w]. In this case, 32 can indicate that the number of feature maps in each of the multiple feature maps has been expanded from 1 to 32.

[0058] Concatenation refers to connecting multiple images at the channel level. For example, if there are three single-channel images with a height and width of 512, that is, the size of each image is (1, 512, 512), after concatenating these three single-channel images, a three-channel image with a size of (3, 512, 512) can be obtained.

[0059] For example, five selected slices of the brain (assuming they are numbered slices 1 to 5) are scanned three times respectively, that is, there are three scanning time points (point A, point B, point C), each scanning time point generates five images, and three images are obtained for each slice, that is, a total of 15 images are obtained. If they are numbered A1 to A5, B1 to B5, and C1 to C5, the three images A1B1C1 of slice 1, the three images A2B2C2 of slice 2, the three images A3B3C3 of slice 3, the three images A4B4C4 of slice 4, and the three images A5B5C5 of slice 5 can be superimposed, and finally a three-channel image with three timing channels corresponding to each slice is obtained.

[0060] As another example, the input to the backbone neural network can be the multiple feature maps described above, namely, the concatenated CTP image. For example, if the dimensions of the CTP image input to the backbone neural network are [bs, t, 1, h, w], the backbone neural network can directly perform convolution processing on the multiple feature maps. The convolution processing method can be found in the description above and will not be repeated here.

[0061] Further optionally, when the initial neural network model generates a predicted CTA image based on the CTP image in the training sample, the temporal information fusion module performs a weighted sum operation on multiple feature maps to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction, and performs a convolution operation on the multiple feature maps after dimensionality reduction to obtain a single feature map, and the single feature map serves as the predicted CTA image, or the single feature map corresponds to the predicted CTA image.

[0062] In this embodiment, the feature map is reduced in dimension by performing a weighted sum operation, and the feature map after dimensionality reduction is further subjected to a convolution operation to obtain a predicted feature map, that is, a predicted CTA image.

[0063] Further optionally, the temporal information fusion module performs a weighted summation operation on the multiple feature maps to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction, including: the temporal information fusion module performs a scalar multiplication operation on each feature map in the multiple feature maps and the corresponding weight parameter to obtain multiple weight products; the temporal information fusion module performs a matrix addition operation on the multiple weight products to obtain multiple feature maps after dimensionality reduction to eliminate the dimensions related to the temporal channel, wherein the multiple feature maps in the target CTP image are related to the temporal channel, and the multiple feature maps after dimensionality reduction are not related to the temporal channel.

[0064] In one optional implementation, for example, when the initial neural network model generates a predicted CTA image based on a CTP image in a training sample, the backbone neural network is configured to process the four-dimensional data on each temporal channel of the CTP image in the training sample to obtain t first four-dimensional feature maps, and then stack the t first four-dimensional feature maps based on the order of their temporal channels to obtain a five-dimensional feature map. The temporal information fusion module is configured to perform dimensionality reduction on the image channels of the five-dimensional feature map to obtain a second four-dimensional feature map. The convolutional layer in the temporal information fusion module performs a convolution operation on the second four-dimensional feature map, which is equivalent to compressing the dimension of the number of temporal channels in the second four-dimensional feature map. For example, the dimensions of the five-dimensional feature map are [bs, t, c, h, w], and the dimensions of the second four-dimensional feature map are [bs, t, h, w], where bs represents the batch size, t represents the number of temporal channels, c represents the dimension of the image channels, h represents the image length, and w represents the image width. After the convolution operation, the dimension of the number of temporal channels is compressed to 1, resulting in a predicted CTA image with dimensions [bs, 1, h, w]. Furthermore, a loss function can be calculated based on the predicted CTA image and the CTA images in the training sample. Training of the initial neural network model is terminated when the loss function meets the convergence condition. In this embodiment, concatenating the t first four-dimensional feature maps results in the four-dimensional feature map being upscaled to a five-dimensional feature map. In this embodiment, the dimension of the image channel is first reduced, and then the dimension of the number of time series channels is compressed, and finally the dimension of the predicted CTA image is obtained as [bs, 1, h, w].

[0065] In some embodiments, the initial neural network model performs dimensionality reduction processing on the dimension of the number of time series channels. In other embodiments, the initial neural network model can also perform dimensionality reduction processing on other dimensions. For example, a CTP image can have dimensions related to image channels, such as the number of image channels. The initial neural network model can perform dimensionality reduction processing on the dimension of the number of image channels, where the number of image channels is related to the color of the image. For example, when the image is a grayscale image, the number of image channels is 1, and when the image is an RGB image, the number of image channels is 3. The dimensionality reduction method for the number of image channels can be referred to the dimensionality reduction method in the related art and is not limited here.

[0066] Optionally, after obtaining the predicted CTA image or the final feature map, a loss function is calculated based on the predicted CTA image and the CTA images in the training sample, and then a determination is made as to whether the loss function result meets a convergence condition. The convergence condition may be that the fluctuation amplitude of the loss function value is less than a preset numerical range, or that the loss function value is less than a preset threshold, which are not limited here. If the loss function result meets the convergence condition, the current training is determined to be complete. If the loss function result does not meet the convergence condition, the current training is determined to be incomplete, and the above-described training method is repeated until the training is complete.

[0067] FIG1d shows a schematic diagram of training an initial neural network according to at least one embodiment of the present disclosure.

[0068] As shown in FIG1d , the initial neural network 100 ′ includes a backbone neural network 110 ′ and a temporal information fusion module 120 ″. The backbone neural network 110 ′ may be the backbone neural network 110 , and the temporal information fusion module 120 ″ may be the temporal information fusion module 120 ′ or the temporal information fusion module 120 .

[0069] During the training process, the backbone neural network 110' inputs pre-paired CTP images and CTA images. The dimension of the input image of the backbone neural network 110' is [bs, t, 1, h, w], where bs is the batch size, t is the number of temporal channels, 1 is the number of image channels (here the default image is a grayscale image, so the number of image channels is 1), h is the height of the image, and w is the width of the image. For example, both h and w can be set to 512. The dimension of the feature map output by the backbone neural network is [bs, t, 32, h, w], where 32 is the number of feature maps. The feature map output no longer has the dimension of the number of image channels, and the dimension of the number of feature maps is added. For ease of illustration, the dimension bs is not shown in Figure 1d.

[0070] The output of the backbone neural network 110' is still a 5-dimensional feature map. The image dimensionality reduction module 120" refers to a learnable parameter W for the feature map output by the backbone neural network 110'. i , the parameter W i The similarity between the feature map at each time point and the target CTA image and the weight of the current feature map in the final output CTA image, where i ranges from 1 to t, is represented. This embodiment uses this weight to achieve image dimensionality reduction.

[0071] The CTP image depicts the entire process from the contrast agent flowing into the tissue to the outflow of the tissue, which includes the early arterial phase (the initial stage of contrast agent entering the artery, for example, 10-20 seconds), the arterial phase (for example, 20-35 seconds, when the arterial contrast agent concentration is high and stable), the early venous phase (for example, 35-50 seconds, when the contrast agent flows back from the artery to the vein and the vein begins to enhance), the venous phase (for example, 50-70 seconds, when the venous contrast agent concentration is high), and the delayed phase (for example, after 70 seconds, when the contrast agent is evenly distributed in the tissue space). Since there is sufficient arterial enhancement and limited venous enhancement at the early time point of the CTP image, the blood vessels in the CTA image predicted from the arterial phase CTP image are more similar to the blood vessels in the real CTA image. Based on this, the above-mentioned parameter W is introduced in this embodiment. i By using similarity and weight to reduce the dimension of feature maps at multiple time points, features that are the same or similar to those of the real CTA image can be retained, making the predicted CTA image similar to the real CTA image or the features more significant.

[0072] Returning to Figure 1d, the input of the image dimensionality reduction module 120" is a feature map of dimension [bs, t, 32, h, w]. The image dimensionality reduction module 120" includes t+1 computing units, where t computing units input the feature map of 1 time point, that is, each computing unit processes a feature map of dimension [bs, 32, h, w]. The t computing units perform scalar multiplication operations on their respective feature maps.

[0073] Assume that X represents the feature map output by the backbone neural network 110', X=[x1,x2,x3,……,x t ], where x1~x t are the 4-dimensional feature maps at t time points. i The sum of is equal to 1, that is The scalar multiplication operation performed by each of the t computational units is W i ·x i , that is, each computing unit performs a multiplication operation on the feature map and the weight parameter. The obtained t multiplication results are further used for matrix addition operation. For example, there is 1 computing unit used for matrix addition operation.

[0074] The matrix addition operation can be expressed as Where W i ·x iis the result of the previous multiplication operation, and Y represents the feature map after weighted summation, whose dimension is [bs, 32, h, w]. Y is used as the input of the convolution layer module 122". After the convolution processing of the convolution layer module 122", the number of feature maps can be reduced from 32 to 1, and the feature map of the predicted CTA image is obtained. The dimension of the feature map of the predicted CTA image is [bs, 1, h, w]. In this example, the initial neural network model does not stack the CTP image, and its input is a 5-dimensional feature map. The image dimensionality reduction module 120" first reduces the dimension of the number of temporal channels, and then compresses the dimension of the number of image channels. Finally, the dimension of the feature map of the predicted CTA image is [bs, 1, h, w].

[0075] Next, the loss function is used to calculate the predicted 4-dimensional feature map and the 4-dimensional feature map of the real CTA image in the training sample. When the result of the loss function meets the convergence condition, the training of the initial neural network model 100' is completed, otherwise the training continues.

[0076] By training the neural network model through the above-mentioned model training steps, the model can acquire the ability to reconstruct (also called predict) CTA images from CTP images. Moreover, since both the CTP image and the reconstructed CTA image are targeted at the same layer of the target tissue, the reconstructed CTA image can be close to the real CTA image in terms of vascular imaging, or even better than the real CTA image, providing a true and accurate reference for the examination of various tissues and organs.

[0077] The following will further describe the embodiment of the present application in detail based on the embodiment shown in Figures 1a-1d above. Referring to Figure 2a, an embodiment of the image reconstruction method in the embodiment of the present application includes:

[0078] 201. Obtain a target CTP image to be processed.

[0079] The target CTP image may be a real CTP image obtained by performing a CTP scan on the target tissue of the subject, that is, a CTP scan is performed on a selected layer of the target tissue, and a CTP image is generated based on the scanning result.

[0080] 202. Obtain a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple groups of training samples using a machine learning algorithm, and each group of training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue.

[0081] In this embodiment, the target neural network model may be a target neural network model trained using the model training method of the embodiment shown in FIG1a . Where the target tissue includes the brain, the target CTP image may be a brain CTP image obtained by performing a CTP examination of the brain. The machine learning algorithm may be a deep learning algorithm.

[0082] 203. Input the target CTP image into the target neural network model, so that the target neural network model reconstructs the target CTA image corresponding to the target CTP image based on the target CTP image, and obtains the target CTA image output by the target neural network model.

[0083] When a target CTP image is obtained, the target CTP image can be input into the target neural network model. Based on the ability to reconstruct CTA images from CTP images acquired during model training, the target neural network model processes the target CTP image and outputs a target CTA image corresponding to the reconstructed target CTP image. Because the target CTA image is reconstructed based on the target CTP image, the target CTA image and the target CTP image correspond to the same layer of the target tissue. Consequently, the vascular imaging effect of the target CTA image can be close to or even better than that of a real CTA image, providing a true and accurate reference for the examination of various tissues and organs. It is understood that the target neural network model is obtained by training the initial neural network described above. The target neural network model has the same architecture as the initial neural network model, as shown in Figures 1b-1c, which will not be further described here. Similarly, the process of predicting CTA images using the target neural network model is similar or identical to the process of predicting CTA images during training, with the only difference being that no loss function is required for calculation, as shown in Figure 1d, which will not be further described here.

[0084] In this embodiment, a pre-trained target neural network model is used to reconstruct the target CTP image into its corresponding target CTA image. The vascular imaging effect of the target CTA image can be close to or even better than the real CTA image, providing a true and accurate reference basis for the examination of various tissues and organs. Therefore, for brain examinations, patients can undergo one less CTA examination, speeding up the one-stop CT examination for stroke, avoiding the radiation exposure of an additional CTA examination, and reducing the injection of contrast agent, thereby reducing the risk of side effects of the contrast agent.

[0085] FIG2 b shows a comparison diagram of a CTA image predicted by the image reconstruction method according to an embodiment of the present disclosure and a real CTA image.

[0086] Figure 2b shows the target CTP image, actual CTA image, and predicted CTA image for a brain window, as well as the target CTP image, actual CTA image, and predicted CTA image for a vascular window. It should be understood that only one target CTP image is shown in Figure 2b, which does not represent the total number of target CTP images input into the trained neural network model.

[0087] As shown in Figure 2b, for brain windows, the predicted CTA image has lower noise and appears clearer than the real CTA image. For vascular windows, the features in the predicted CTA image are clearer than those in the real CTA image.

[0088] The following describes a computer device in an embodiment of the present application. Referring to FIG3 , an embodiment of the computer device in an embodiment of the present application includes:

[0089] The computer device 300 may include one or more central processing units (CPUs) 301 and a memory 305 . The memory 305 stores one or more application programs or data.

[0090] Memory 305 can be volatile or persistent storage. The program stored in memory 305 can include one or more modules, each of which can include a series of instruction operations on the computer device. Furthermore, the central processing unit 301 can be configured to communicate with memory 305 and execute the series of instruction operations in memory 305 on the computer device 300.

[0091] The computer device 300 may also include one or more power supplies 302, one or more wired or wireless network interfaces 303, one or more input and output interfaces 304, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0092] The central processing unit 301 can execute the operations executed by the computer device in the embodiments shown in Figures 1a to 2b above, and the details will not be repeated here.

[0093] An embodiment of the present application also provides a computer storage medium, one embodiment of which includes: the computer storage medium stores instructions, and when the instructions are executed on a computer, the computer performs the operations performed by the computer device in the embodiments shown in Figures 1a and 2b above.

[0094] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0096] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0097] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0099] An embodiment of the present application provides an image reconstruction method, the method comprising:

[0100] Acquire the target CTP image to be processed;

[0101] Obtaining a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple sets of training samples using a machine learning algorithm, and each set of training samples includes paired CTP images and CTA images, wherein the paired CTP images and CTA images correspond to the same selected layer of the target tissue;

[0102] The target CTP image is input into the target neural network model so that the target neural network model reconstructs the target CTA image corresponding to the target CTP image based on the target CTP image, and obtains the target CTA image output by the target neural network model.

[0103] Optionally, the target neural network model reconstructs a target CTA image corresponding to the target CTP image according to the target CTP image, including:

[0104] The target neural network model performs a weighted sum operation on multiple feature maps in the target CTP image to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction;

[0105] The target neural network model performs a convolution operation on the multiple feature maps after dimensionality reduction to obtain a single feature map, and the single feature map serves as the target CTA image or the single feature map corresponds to the target CTA image.

[0106] Optionally, the target neural network model performs a weighted sum operation on multiple feature maps in the target CTP image, including:

[0107] The target neural network model performs a scalar multiplication operation on each feature map in the multiple feature maps and a corresponding weight parameter to obtain a plurality of weight products;

[0108] The target neural network model performs a matrix addition operation on the multiple weight products to obtain the multiple feature maps after dimensionality reduction to eliminate dimensions related to the timing channel, wherein the multiple feature maps in the target CTP image are related to the timing channel, and the multiple feature maps after dimensionality reduction are not related to the timing channel.

[0109] Optionally, before the target neural network model performs a weighted sum operation on the multiple feature maps in the target CTP image, the method further includes:

[0110] The target neural network model processes a plurality of CTP images included in the target CTP image to obtain a plurality of initial feature maps, wherein the plurality of CTP images are associated with a plurality of time series channels;

[0111] The target neural network model processes the multiple initial feature maps according to the order of the multiple time series channels to obtain the multiple feature maps, wherein the dimensions of the multiple feature maps are higher than the dimensions of the multiple initial feature maps.

[0112] Optionally, the target tissue includes a brain, and the target CTP image is a brain CTP image obtained by performing a CTP examination on the brain.

[0113] Optionally, the target CTP image includes a plurality of CTP images, wherein the number of the plurality of CTP images is equal to the number of time series channels.

[0114] The present invention provides a model training method, which includes:

[0115] Acquiring multiple sets of training samples, each set of training samples including paired CTP images and CTA images, wherein the paired CTP images and CTA images correspond to the same selected layer of target tissue;

[0116] Get the initial neural network model;

[0117] Multiple groups of training samples are input into the initial neural network model so that the initial neural network model generates predicted CTA images based on the CTP images in the training samples. When the relationship between the predicted CTA images and the CTA images in the training samples meets the convergence condition, the training is stopped to obtain the target neural network model.

[0118] Optionally, the initial neural network model includes a backbone neural network and a time series information fusion module;

[0119] When the initial neural network model generates the predicted CTA image according to the CTP image in the training sample,

[0120] The backbone neural network processes multiple CTP images included in the CTP images in the training sample to obtain multiple initial feature maps, and processes the multiple initial feature maps according to the order of the multiple timing channels to obtain multiple feature maps, wherein the multiple CTP images are associated with multiple timing channels, and the dimensions of the multiple feature maps are higher than the dimensions of the multiple initial feature maps.

[0121] Optionally, the model training method further includes:

[0122] When the initial neural network model generates the predicted CTA image according to the CTP image in the training sample,

[0123] The temporal information fusion module performs a weighted summation operation on the multiple feature maps to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction, and performs a convolution operation on the multiple feature maps after dimensionality reduction to obtain a single feature map, and the single feature map serves as the predicted CTA image, or the single feature map corresponds to the predicted CTA image.

[0124] Optionally, the time series information fusion module performs a weighted sum operation on the multiple feature maps to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction, including:

[0125] The temporal information fusion module performs a scalar multiplication operation on each feature map and a corresponding weight parameter in the plurality of feature maps to obtain a plurality of weight products;

[0126] The temporal information fusion module performs a matrix addition operation on the multiple weight products to obtain the multiple feature maps after dimensionality reduction to eliminate dimensions related to the temporal channel, wherein the multiple feature maps in the target CTP image are related to the temporal channel, and the multiple feature maps after dimensionality reduction are not related to the temporal channel.

[0127] Optionally, the dimension of the CTP image in the training sample includes the number of time series channels t, where t is a positive integer greater than 1;

[0128] When the initial neural network model generates the predicted CTA image based on the CTP image in the training sample, the backbone neural network processes the N-dimensional data on each time series channel of the CTP image in the training sample to obtain t first N-dimensional feature maps, and the t first N-dimensional feature maps are superimposed based on the order of the time series channels of the t first N-dimensional feature maps to obtain an N+1-dimensional feature map, where N is a positive integer;

[0129] The temporal information fusion module is used to perform a dimensionality reduction operation on the N+1-dimensional feature map to obtain a second N-dimensional feature map, and the convolution layer in the temporal information fusion module is used to perform a convolution process on the second N-dimensional feature map to obtain the predicted CTA image.

[0130] Optionally, stopping the training when the relationship between the predicted CTA image and the CTA image in the training sample satisfies a convergence condition includes:

[0131] A loss function is calculated based on the predicted CTA image and the CTA image in the training sample, and the training of the initial neural network model is stopped when a result of the loss function meets a convergence condition.

[0132] Optionally, the dimensions of the CTP images in the training samples include batch size, the number of temporal channels, the number of image channels, image length, and image width.

[0133] Optionally, obtaining multiple sets of training samples includes:

[0134] Obtain n×t images by performing t scans on n selected slices of the target tissue, where n and t are both positive integers greater than 1;

[0135] For each of the n slices, superimposing the t images according to the time sequence of the t scans performed on the slice to obtain a CTP image having t sequential channels corresponding to the slice;

[0136] Acquiring CTA images of m layers of the target tissue, where the m layers include the n layer, and m is a positive integer greater than n;

[0137] For each of the n layers, a CTA image paired with the CTP image of the layer is determined from the CTA images of the m layers, and the paired CTP images and CTA images on the layer constitute a set of training samples.

[0138] Optionally, the paired CTP image and CTA image correspond to the same selected layer of the same brain structure.

[0139] An embodiment of the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the aforementioned graphics reconstruction method or model training method when executing the computer program.

[0140] An embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the aforementioned graphics reconstruction method or model training method.

Claims

1. An image reconstruction method, comprising: Acquire the target CTP image to be processed; Acquire a pre-trained target neural network model, wherein the target neural network model is obtained by training multiple groups of training samples using a machine learning algorithm, and each group of training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue; The target CTP image is input into the target neural network model so that the target neural network model reconstructs the target CTA image corresponding to the target CTP image according to the target CTP image, and obtains the target CTA image output by the target neural network model.

2. The method according to claim 1, wherein: The target neural network model reconstructs a target CTA image corresponding to the target CTP image according to the target CTP image, including: The target neural network model performs a weighted sum operation on multiple feature maps in the target CTP image to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction; The target neural network model performs a convolution operation on the multiple feature maps after dimensionality reduction to obtain a single feature map, and the single feature map serves as the target CTA image or the single feature map corresponds to the target CTA image.

3. The method according to claim 2, wherein: The target neural network model performs a weighted sum operation on a plurality of feature maps in the target CTP image, including: The target neural network model performs a scalar multiplication operation on each feature map in the multiple feature maps and a corresponding weight parameter to obtain multiple weight products; The target neural network model performs a matrix addition operation on the multiple weight products to obtain the multiple feature maps after dimensionality reduction to eliminate the dimensions related to the timing channel, wherein the multiple feature maps in the target CTP image are related to the timing channel, and the multiple feature maps after dimensionality reduction are related to the timing channel. The Tao has nothing to do with it.

4. The method according to claim 2 or 3, wherein: Before the target neural network model performs a weighted sum operation on a plurality of feature maps in the target CTP image, the method further includes: The target neural network model processes a plurality of CTP images included in the target CTP image to obtain a plurality of initial feature maps, wherein the plurality of CTP images are associated with a plurality of time series channels; The target neural network model processes the multiple initial feature maps according to the order of the multiple time series channels to obtain the multiple feature maps, wherein the dimensions of the multiple feature maps are higher than the dimensions of the multiple initial feature maps.

5. The method according to any one of claims 1 to 4, wherein: The target tissue includes the brain, and the target CTP image is a brain CTP image obtained by performing a CTP examination on the brain.

6. The method according to any one of claims 1 to 5, wherein: The target CTP image includes a plurality of CTP images, wherein the number of the plurality of CTP images is equal to the number of timing channels.

7. A model training method, comprising: Acquire multiple groups of training samples, each group of the training samples includes paired CTP images and CTA images, and the paired CTP images and CTA images correspond to the same selected layer of the target tissue; Get the initial neural network model; Multiple groups of the training samples are input into the initial neural network model so that the initial neural network model generates a predicted CTA image based on the CTP image in the training samples. When the relationship between the predicted CTA image and the CTA image in the training samples meets the convergence condition, the training is stopped to obtain a target neural network model.

8. The method according to claim 7, wherein: The initial neural network model includes a backbone neural network and a time series information fusion module; When the initial neural network model generates the predicted CTA image according to the CTP image in the training sample, The backbone neural network processes multiple CTP images included in the CTP images in the training samples to obtain multiple initial feature maps, and processes the multiple initial feature maps according to the order of the multiple timing channels to obtain multiple feature maps, wherein the multiple CTP images are associated with multiple timing channels, and the dimensions of the multiple feature maps are higher than the dimensions of the multiple initial feature maps.

9. The method according to claim 8, wherein: Also includes: When the initial neural network model generates the predicted CTA image according to the CTP image in the training sample, The temporal information fusion module performs a weighted sum operation on the multiple feature maps to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction, and performs a convolution operation on the multiple feature maps after dimensionality reduction to obtain a single feature map, and the single feature map serves as the predicted CTA image, or the single feature map corresponds to the predicted CTA image.

10. The method according to claim 9, wherein: The time series information fusion module performs a weighted sum operation on the multiple feature maps to reduce the dimensions of the multiple feature maps to obtain multiple feature maps after dimensionality reduction, including: The temporal information fusion module performs a scalar multiplication operation on each feature map and a corresponding weight parameter in the multiple feature maps to obtain multiple weight products; The timing information fusion module performs a matrix addition operation on the multiple weight products to obtain the multiple feature maps after dimensionality reduction to eliminate the dimensions related to the timing channel, wherein the multiple feature maps in the target CTP image are related to the timing channel, and the multiple feature maps after dimensionality reduction are independent of the timing channel.

11. The method according to any one of claims 7 to 10, wherein: The dimension of the CTP image in the training sample includes the number of time series channels t, where t is a positive integer greater than 1; When the initial neural network model generates the predicted CTA image according to the CTP image in the training sample, the backbone neural network processes the N-dimensional data on each time series channel of the CTP image in the training sample to obtain t first N-dimensional feature maps, and the t first N-dimensional feature maps are superimposed based on the sequence of the time series channels of the t first N-dimensional feature maps to obtain an N+1-dimensional feature map, where N is a positive integer; The temporal information fusion module is used to perform a dimensionality reduction operation on the N+1 dimensional feature map to obtain a second N dimensional feature map, and the convolution layer in the temporal information fusion module is used to perform a convolution process on the second N dimensional feature map to obtain the predicted CTA image.

12. The method according to any one of claims 7 to 11, wherein: The step of stopping the training when the relationship between the predicted CTA image and the CTA image in the training sample satisfies a convergence condition comprises: A loss function is calculated according to the predicted CTA image and the CTA image in the training sample, and the training of the initial neural network model is stopped when a result of the loss function meets a convergence condition.

13. The method according to any one of claims 7 to 12, wherein: The dimensions of the CTP images in the training samples include batch size, the number of time series channels, the number of image channels, image length, and image width.

14. The method according to any one of claims 7 to 13, wherein: The obtaining of multiple sets of training samples comprises: Obtain n×t images by scanning n selected layers of the target tissue t times, where n and t are both positive integers greater than 1; For each of the n layers, the t images are superimposed in the time sequence of the t scans performed on the layer to obtain a CTP image with t sequential channels corresponding to the layer; Acquire CTA images of m layers of the target tissue, where the m layers include the n layer, and m is a positive integer greater than n; For each of the n layers, a CTA image paired with the CTP image of the layer is determined from the CTA images of the m layers, and the CTP images and CTA images paired with each other on the layer constitute a group of the training samples.

15. The method according to any one of claims 7 to 14, wherein: The paired CTP image and CTA image correspond to the same selected layer of the same brain structure.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 15 is implemented.

17. A computer storage medium, wherein: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 15.

Images