Training method of image processing model, image processing method, device and medium

Abstract

The invention provides an image processing model training method, an image processing method, an image processing device and a medium. The method comprises the steps of obtaining a sample image and annotation data for the sample image, wherein the annotation data comprises an annotated abnormal region and at least a first non-abnormal region of the sample image; obtaining at least two sub-networks, each of the at least two sub-networks being capable of being used to output a prediction result for segmenting the anomaly based on the sample image, the at least two sub-networks including a first sub-network and a second sub-network, the second sub-network being obtained based on at least one policy, the at least one strategy comprises a first strategy, and the first strategy is used for reducing the similarity of a first prediction result of the first sub-network for the sample image and a second prediction result of the second sub-network for the sample image in a first non-abnormal region of the sample image; and constructing an image processing model based on the at least two sub-networks.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a training method for an image processing model, an image processing method, an apparatus, a computing device, and a storage medium. Background Art

[0002] Currently, doctors often use medical image sequences scanned by medical scanning equipment (such as nuclear magnetic resonance imaging scanners, magnetic resonance imaging scanners, computer tomography scanners, etc.) to make medical diagnoses. In this process, preprocessing the image, such as segmenting the image area, may be very convenient for assisting doctors in diagnosis. Summary of the invention

[0003] According to one aspect of the present disclosure, there is provided a training method for an image processing model, comprising: obtaining a sample image and annotated data for the sample image, the annotated data comprising an annotated abnormal region and at least a first non-abnormal region of the sample image; obtaining at least two subnetworks, each of the at least two subnetworks being capable of being used to output a prediction result for segmenting anomalies based on the sample image, the at least two subnetworks comprising a first subnetwork and a second subnetwork, wherein the second subnetwork is obtained based on at least one strategy, the at least one strategy comprising a first strategy, the first strategy being used to reduce the similarity between a first prediction result of the first subnetwork for the sample image and a second prediction result of the second subnetwork for the sample image in the first non-abnormal region of the sample image; and constructing the image processing model based on the at least two subnetworks.

[0004] According to another aspect of the present disclosure, there is provided an image processing method, comprising: obtaining an image processing model by a method according to an embodiment of the present disclosure; and obtaining a segmentation result by the image processing model based on an input image.

[0005] According to yet another aspect of the present disclosure, a training device for an image processing model is provided, comprising: a sample image obtaining unit, for obtaining a sample image and annotation data for the sample image, the annotation data comprising an annotated abnormal region and at least a first non-abnormal region of the sample image; a network obtaining unit, for obtaining at least two sub-networks, each of the at least two sub-networks being capable of being used to output a prediction result for segmenting anomalies based on the sample image, the at least two sub-networks comprising a first sub-network and a second sub-network, wherein the second sub-network is obtained based on at least one strategy, the at least one strategy comprising a first strategy, the first strategy being used to reduce the similarity between a first prediction result of the first sub-network for the sample image and a second prediction result of the second sub-network for the sample image in the first non-abnormal region of the sample image; and a construction unit, for constructing the image processing model based on the at least two sub-networks.

[0006] According to yet another aspect of the present disclosure, an image processing device is provided, comprising: a model acquisition unit, for obtaining an image processing model through a method or device according to an embodiment of the present disclosure; and a segmentation unit, for obtaining a segmentation result through the image processing model based on an input image.

[0007] According to another aspect of the present disclosure, a computing device is provided, comprising: a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement a training method for an image processing model according to one or more embodiments of the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, a training method for an image processing model according to one or more embodiments of the present disclosure is implemented.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements a training method for an image processing model according to one or more embodiments of the present disclosure.

[0010] These and other aspects of the disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is a schematic diagram illustrating an example system in which the various methods described herein may be implemented according to an exemplary embodiment;

[0013] Figure 2 is a flow chart illustrating a training method for an image processing model according to an exemplary embodiment;

[0014] Figure 3 is a schematic diagram illustrating a sample image according to an exemplary embodiment;

[0015] Figure 4 is a flowchart illustrating an image processing method according to an exemplary embodiment;

[0016] Figure 5 is a schematic block diagram illustrating a training apparatus for an image processing model according to an exemplary embodiment;

[0017] Figure 6 is a schematic block diagram illustrating an image processing apparatus according to an exemplary embodiment;

[0018] Figure 7 is a block diagram illustrating an exemplary computer device that can be applied to the exemplary embodiments. DETAILED DESCRIPTION

[0019] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0020] The terms used in the description of various examples described in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. As used herein, the term "plurality" means two or more, and the term "based on" should be interpreted as "based at least in part on". In addition, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations.

[0021] Exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0022] Figure 1 is a schematic diagram illustrating an example system 100 in which the various methods described herein may be implemented, according to an example embodiment.

[0023] refer to Figure 1 The system 100 includes a client device 110 , a server 120 , and a network 130 that communicatively couples the client device 110 and the server 120 .

[0024] The client device 110 includes a display 114 and a client application (APP) 112 that can be displayed via the display 114. The client application 112 can be an application that needs to be downloaded and installed before running or a small program (liteapp) as a lightweight application. In the case where the client application 112 is an application that needs to be downloaded and installed before running, the client application 112 can be pre-installed on the client device 110 and activated. In the case where the client application 112 is a small program, the user 102 can directly run the client application 112 on the client device 110 by searching for the client application 112 in the host application (for example, by the name of the client application 112, etc.) or scanning the graphic code of the client application 112 (for example, a bar code, a QR code, etc.), without installing the client application 112. In some embodiments, the client device 110 can be any type of mobile computer device, including a mobile computer, a mobile phone, a wearable computer device (for example, a smart watch, a head-mounted device, including smart glasses, etc.) or other types of mobile devices. In some embodiments, the client device 110 may alternatively be a stationary computer device, such as a desktop computer, a server computer or other types of stationary computer devices. In some optional embodiments, the client device 110 may also be or include a medical image printing device.

[0025] The server 120 is typically a server deployed by an Internet Service Provider (ISP) or an Internet Content Provider (ICP). The server 120 may represent a single server, a cluster of multiple servers, a distributed system, or a cloud server that provides basic cloud services (such as cloud databases, cloud computing, cloud storage, and cloud communications). It will be understood that although Figure 1 The server 120 is shown in FIG. 1 in communicating with only one client device 110 , but the server 120 may provide background services for multiple client devices simultaneously.

[0026] Examples of network 130 include a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and / or a combination of communication networks such as the Internet. Network 130 can be a wired or wireless network. In some embodiments, the data exchanged through network 130 is processed using technologies and / or formats including hypertext markup language (HTML), extensible markup language (XML), etc. In addition, encryption technologies such as secure socket layer (SSL), transport layer security (TLS), virtual private network (VPN), Internet protocol security (IPsec) can also be used to encrypt all or some links. In some embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0027] The system 100 may also include an image acquisition device 140. In some embodiments, Figure 1 The image acquisition device 140 shown can be a medical scanning device, including but not limited to scanning or imaging devices used in positron emission tomography (PET), positron emission tomography with computerized tomography (PET / CT), single photon emission computed tomography with computerized tomography (SPECT / CT), computerized tomography (CT), medical ultrasonography, nuclear magnetic resonance imaging (NMRI), magnetic resonance imaging (MRI), cardiovascular angiography (CA), digital radiography (DR), etc. For example, the image acquisition device 140 may include a digital subtraction angiography scanner, a magnetic resonance angiography scanner, a tomography angiography scanner, a positron emission tomography scanner, a positron emission computed tomography scanner, a single photon emission computed tomography scanner, a computed tomography scanner, a medical ultrasound examination device, a nuclear magnetic resonance imaging scanner, a magnetic resonance imaging scanner, a digital radiography scanner, etc. The image acquisition device 140 may communicate with a server (e.g., Figure 1The image processing system is connected to the server 120 in the figure or a separate server of the imaging system (not shown in the figure) to realize image data processing, including but not limited to converting the scan data (for example, converting it into a medical image sequence), compressing it, correcting its pixels, and reconstructing it in three dimensions.

[0028] The image acquisition device 140 may be connected to the client device 110 , for example, via the network 130 , or directly connected to the client device in other ways to communicate with the client device.

[0029] Optionally, the system may further include an intelligent computing device or computing card 150. The image acquisition device 140 may include or be connected (e.g., removably connected) to such a computing card 150, etc. As an example, the computing card 150 may implement image data processing, including but not limited to conversion, compression, pixel correction, reconstruction, etc. As another example, the computing card 150 may implement a training method for an image processing model according to an embodiment of the present disclosure.

[0030] The system may also include other parts not shown, such as a data storage unit. The data storage unit may be a database, a data repository, or one or more devices for data storage in other forms, and may be a conventional database, or may include a cloud database, a distributed database, etc. For example, direct image data formed by the image acquisition device 140 or a medical image sequence or three-dimensional image data obtained through image processing may be stored in the data storage unit for subsequent retrieval by the server 120 and the client device 110 from the data storage unit. In addition, the above-mentioned image acquisition device 140 may also directly provide the image data or the medical image sequence or three-dimensional image data obtained through image processing to the server 120 or the client device 110, etc.

[0031] The user can use the client device 110 to view the collected images or videos, including preliminary image data or images after analysis and processing, view analysis results, interact with the collected images or analysis results, input collection instructions, configuration data, etc. The client device 110 can send configuration data, instructions or other information to the image acquisition device 140 to control the collection and data processing of the image acquisition device.

[0032] For the purpose of the embodiments of this disclosure, Figure 1In the example of , the client application 112 may be an image sequence management application, which may provide various functions, such as storage management, indexing, sorting, and classification of the acquired image sequences. Correspondingly, the server 120 may be a server used together with the image sequence management application. The server 120 may provide image sequence management services to the client application 112 running in the client device 110 based on user requests or instructions generated according to the embodiments of the present disclosure, such as managing the image sequence storage in the cloud, storing and classifying the image sequence according to a specified index (including, for example, but not limited to, sequence type, patient identification, body part, acquisition target, acquisition stage, acquisition machine, whether lesions are detected, severity, etc.), and retrieving and providing the image sequence to the client device according to the specified index, etc. Alternatively, the server 120 may also provide or allocate such service capabilities or storage space to the client device 110, and the client application 112 running in the client device 110 may provide corresponding image sequence management services according to user requests or instructions generated according to the embodiments of the present disclosure, etc. It is to be understood that the above is only an example, and the present disclosure is not limited thereto.

[0033] Figure 2 2 is a flowchart illustrating a method 200 for training an image processing model according to an exemplary embodiment. The method 200 may be performed on a client device (e.g., Figure 1 The execution of each step of the method 200 may be performed at the client device 110 shown in FIG. Figure 1 In some embodiments, the method 200 may be performed on a server (e.g., Figure 1 In some embodiments, method 200 may be performed by a client device (eg, client device 110) and a server (eg, server 120) in combination.

[0034] In the following, each step of the method 200 is described in detail.

[0035] refer to Figure 2 In step 210, a sample image and annotation data for the sample image are obtained, wherein the annotation data includes an annotated abnormal region and at least a first non-abnormal region of the sample image.

[0036] In step 220, at least two subnetworks are obtained, each of the at least two subnetworks can be used to output a prediction result for segmenting anomalies based on the sample image, and the at least two subnetworks include a first subnetwork and a second subnetwork, wherein the second subnetwork is obtained based on at least one strategy, and the at least one strategy includes a first strategy, and the first strategy is used to reduce the similarity between a first prediction result of the first subnetwork for the sample image and a second prediction result of the second subnetwork for the sample image in the first non-abnormal area of ​​the sample image.

[0037] In step 230, the image processing model is constructed based on the at least two sub-networks.

[0038] Through the above method, the similarity of different models in the first non-abnormal area can be reduced. Since the abnormal and non-abnormal often lack clear boundaries in pixels, and some specific non-abnormal areas such as transition parts, reflective parts, etc. sometimes have similar morphologies to abnormal areas, the recognition of models in these areas is often prone to errors. For such common phenomena, the effect of improving accuracy through large amounts of data training is often poor, or it is easy to have overfitting and other phenomena. For example, the model obtains samples that are accurately labeled (for example, accurately labeled by experienced people), but because the pixel values ​​near the accurately labeled boundaries in the image may be very similar, it seems that there is no clear boundary, so the efficiency of machine learning is very low, and it is difficult to obtain a true judgment method. On the contrary, according to the embodiments of the present disclosure, such a way of thinking is provided, that is, a single model can be allowed to make mistakes in "error-prone areas" and "fuzzy areas", but as long as the mistakes of multiple models are different. Through the strategy of reducing the similarity of different models in such error-prone areas, although each model has the possibility of making mistakes in such areas, it often does not have the same mistakes, so the integrated model will have a better performance in the first non-abnormal area.

[0039] It is understandable that the first non-abnormal region here can be a model error-prone region marked by a doctor or other practitioner. It is also understandable that the wording of "first non-abnormal region" does not require that the first non-abnormal region is a single or continuous region. For example, multiple fuzzy boundaries, reflections, occlusions or other error-prone regions in an image can be collectively referred to as first non-abnormal regions, and the present disclosure is not limited thereto.

[0040] In addition, it is understood that, in this article, although each network including the first subnetwork and the second subnetwork is described as being used to output a prediction result for segmenting anomalies based on the sample image, it is not required that each subnetwork itself is a complete "segmentation" network. In some examples, subnetworks such as the first subnetwork and the second subnetwork can be segmentation networks. In other examples, subnetworks such as the first subnetwork and the second subnetwork can be feature extraction networks, for example, the extracted features can be used for segmentation networks. It is understood that subnetworks such as the first subnetwork and the second subnetwork can be composed of at least several layers of convolution, and can be used to obtain segmentation results or results of segmentation-related features. For example, in the case where the at least two subnetworks are feature extraction networks, constructing the image processing model based on the at least two subnetworks can also include constructing an image processing model based on the at least two subnetworks and one or more additional segmentation parts, so that the constructed image processing model can output prediction results, prediction values ​​and / or abnormal segmentation maps for segmenting anomalies based on sample images. It is understood that the present disclosure is not limited to this.

[0041] In addition, it is understandable that although the naming methods such as "first sub-network" and "second sub-network" are used, such expressions do not require that each sub-network must exist in conjunction with other networks, nor do they require that each sub-network must be part of a more complete large network. The naming methods such as first sub-network and second sub-network are only used to describe that the sub-network can be used with other networks, but do not exclude that the sub-network itself can be regarded as a complete network, trained separately, used separately, etc.

[0042] According to some embodiments, the at least one strategy may further include a second strategy, wherein the second strategy is used to reduce the difference between the abnormal segmentation indicated by the second prediction result and the marked abnormal region.

[0043] Exemplarily, each of the first prediction result and the second prediction result may include a prediction value of whether the corresponding pixel of the sample image corresponds to an abnormality or a lesion, for example, a predicted probability. It is understood that, in the case where the prediction result is a probability value of whether each pixel corresponds to an abnormality, the abnormal segmentation indicated by the prediction result may refer to whether the probability value appears as an abnormal or non-abnormal segmentation result under a specific threshold or activation function. In other examples, the second prediction result may also be a segmentation of abnormal and non-abnormal areas.

[0044] As a non-limiting example, the method may include adjusting parameters of the second sub-network based on at least the first strategy. For example, the method may include: obtaining a first sub-network and a second sub-network; obtaining the first prediction result and the second prediction result for the sample image based on the first sub-network and the second sub-network; and adjusting parameters of the first sub-network and the second sub-network based on at least the overlap between the predicted values ​​of the first prediction result and the second prediction result in the first non-abnormal area of ​​the sample image.

[0045] According to some embodiments, the second sub-network may be trained based on the at least one strategy by: determining an anomaly prediction loss of the second sub-network based on the difference between the anomaly segmentation indicated by the second prediction result and the labeled anomaly region; determining an inter-network similarity loss of the second sub-network based on the similarity between the second prediction result and the first prediction result in the first non-abnormal region of the sample image; and adjusting parameters of the second sub-network based on the anomaly prediction loss and the inter-network similarity loss.

[0046] Exemplarily, the method may include obtaining at least two sub-networks, wherein the at least two sub-networks include a first sub-network and a second sub-network, and each of the at least two sub-networks is used to output a lesion region segmentation map based on a sample image.

[0047] According to some embodiments, the method 200 may further include adjusting parameters of the first sub-network to reduce the difference between the abnormal segmentation indicated by the first prediction result and the annotated abnormal region.

[0048] In such an embodiment, the first network and the second network may be trained separately (eg, simultaneously) so that the output of the first network is as close as possible to the abnormal region.

[0049] According to some embodiments, method 200 may further include adjusting parameters of the first sub-network to reduce the similarity between the first prediction result and the second prediction result in the first non-abnormal region.

[0050] In such an embodiment, the first network and the second network may be trained separately (e.g., simultaneously), and the training process may include, in addition to adjusting the parameters of the second network, adjusting the parameters of the first network so that the first network does not overlap with the second network in the first non-abnormal region.

[0051] According to some embodiments, obtaining at least two subnetworks may include: obtaining the initialized first subnetwork and the initialized second subnetwork; and training the first subnetwork and the second subnetwork based on the sample image until a training end condition is met. Exemplarily, the training end condition may include at least one of the following: the difference between the abnormal segmentation indicated by the first prediction result of the first subnetwork for the sample image and the annotated abnormal area is less than a first threshold; the difference between the abnormal segmentation indicated by the second prediction result of the second subnetwork for the sample image and the annotated abnormal area is less than a second threshold; and the similarity between the first prediction result of the first subnetwork for the sample image and the second prediction result of the second subnetwork for the sample image in the first non-abnormal area of ​​the sample image is less than a third threshold. Again exemplarily, the training end condition may include satisfying two of the above, or all of them.

[0052] As a non-limiting example, the method according to an embodiment of the present disclosure may include: determining the lesion area loss of the first sub-network based on the difference between the lesion area predicted by the first prediction result and the marked abnormal area, and determining the overlap loss of the first sub-network, wherein the overlap loss is based on the overlap between the prediction results of the first sub-network and the first sub-network in the second non-abnormal area of ​​the sample image.

[0053] According to some embodiments, the first subnetwork and the second subnetwork may be obtained by the following operations: using the sample image to respectively train a plurality of subnetworks including the first subnetwork and the second subnetwork, until the difference between the abnormal segmentation indicated by the corresponding prediction result of each subnetwork in the plurality of subnetworks and the annotated abnormal region is less than a fourth threshold value; and selecting two networks satisfying the dissimilar condition from the trained plurality of subnetworks as the first subnetwork and the second subnetwork. As a specific non-limiting embodiment, the difference between the abnormal segmentation indicated by the corresponding prediction result and the annotated abnormal region may include the difference between the predicted lesion region corresponding to the prediction result and the true lesion region, such as but not limited to the absolute area difference or relative area difference between the two regions, or the difference in the predicted probability values ​​(e.g., the sum of the squares of the difference in the probability values ​​of each pixel), etc.

[0054] In such an embodiment, the two networks satisfying the dissimilarity condition may be two networks whose prediction results for the sample image in the first non-abnormal area have a similarity less than a fifth threshold, or the two networks satisfying the dissimilarity condition may be two networks among the multiple sub-networks whose corresponding prediction results for the sample image in the first non-abnormal area have the smallest similarity.

[0055] According to such an embodiment, multiple networks can be trained separately, two (N) networks with the lowest overlap can be selected, and such dissimilar networks can be integrated to obtain a model with overall good performance in the first non-abnormal area.

[0056] According to some embodiments, the first subnetwork and the second subnetwork may be obtained by the following operations: training the first subnetwork based on the sample image until the difference between the abnormal segmentation indicated by the first prediction result of the first subnetwork for the sample image and the marked abnormal area is less than a sixth threshold; constructing the second subnetwork based on the first subnetwork so that the initial weights of the second subnetwork correspond to the weights of the trained first subnetwork; and training the second subnetwork based on the sample image until the similarity between the second prediction result of the second subnetwork for the sample image and the first prediction result in the first non-abnormal area is less than a seventh threshold.

[0057] In such an embodiment, a network may be trained first, and after the training is completed, it is used as the initial state of the new network, and a new penalty is added to make it dissimilar to the previous network.

[0058] Exemplarily, obtaining the first subnetwork and the second subnetwork may also include training the first subnetwork based on the sample image until the difference between the abnormal segmentation indicated by the second prediction result of the second subnetwork for the sample image and the marked abnormal area is less than a predetermined difference threshold.

[0059] Exemplarily, obtaining the first subnetwork and the second subnetwork may also include training the first subnetwork based on the sample image until one or more of the following: the difference between the non-abnormal segmentation indicated by the first prediction result of the first subnetwork for the sample image and the annotated second non-abnormal region is less than a predetermined difference threshold, and / or the difference between the non-abnormal segmentation indicated by the second prediction result of the second subnetwork for the sample image and the annotated second non-abnormal region is less than a predetermined difference threshold. The second non-abnormal region may be a relatively clear region where the model is not prone to errors. For example, the annotated data may include an annotated second non-abnormal region of the sample image. For example, in addition to the error-prone first non-abnormal region, the remaining non-abnormal regions may be regarded as second non-abnormal regions. Thus, the model may have better recognition capabilities in the remaining non-abnormal regions except the first non-abnormal region.

[0060] According to some embodiments, obtaining at least two sub-networks may include: obtaining the at least two initialized sub-networks; determining a loss value and a penalty value, wherein the loss value represents the difference between the abnormal segmentation indicated by the at least two sub-networks for each of the at least two prediction results of the sample image and the marked abnormal area, and wherein the penalty value represents the similarity between the at least two prediction results in the first non-abnormal area of ​​the sample image; and adjusting the parameters of at least one network of the at least two sub-networks based on the loss value and the penalty value.

[0061] In such an embodiment, multiple networks may also be trained simultaneously to determine loss values ​​(abnormal prediction loss) and penalty values ​​(coincidence). As a more specific example, the parameters of each network in the at least two sub-networks may be adjusted based on the loss value and penalty value.

[0062] According to some embodiments, the penalty value may be the dot product of the corresponding at least two penalty vectors of the at least two sub-networks, wherein the penalty vector of each sub-network corresponds to the portion of the prediction result of the sub-network for the sample image in the first non-abnormal area. In such an embodiment, the dot product of all results may be used as the penalty.

[0063] According to some embodiments, the penalty value may be the sum of the pairwise dot product of the at least two penalty vectors of the at least two sub-networks, wherein the penalty vector of each sub-network corresponds to the portion of the prediction result of the sub-network for the sample image in the first non-abnormal region. For example, the at least two sub-networks may be three or more sub-networks, and in such an embodiment, the sum of the pairwise dot product may be used as the penalty.

[0064] According to some embodiments, constructing the image processing model based on the at least two sub-networks may include: constructing the image processing model based on the at least two sub-networks and an integration network, the integration network being used to integrate at least two prediction results output by the at least two sub-networks as the output of the image processing model.

[0065] For example, the integration network may receive the segmentation results of each of the at least two sub-networks as input, and integrate the segmentation results as the final output of the model.

[0066] According to some embodiments, integrating the at least two prediction results output by the at least two sub-networks may include averaging the at least two prediction results.

[0067] As a specific non-limiting example, each prediction result may be a probability value that a pixel or each sub-region in the image corresponds to an abnormality, and averaging the prediction results may be averaging the probability values ​​pixel by pixel or sub-region by sub-region.

[0068] According to some embodiments, integrating the at least two prediction results output by the at least two sub-networks may include performing a weighted average on the at least two prediction results, and wherein the weight corresponding to each sub-network in the weighted average operation is based on the similarity between the prediction results of the corresponding sub-network and other networks in the at least two sub-networks in the first non-abnormal area.

[0069] For example, the weight may be inversely proportional to the similarity. Thus, the output of the network that is more similar to other networks may be reduced, so that the performance of the integrated model in the first non-abnormal area is more randomized, and thus a more accurate output is obtained.

[0070] According to some embodiments, the sample image may be an image collected from a human body, the marked abnormal region may correspond to a lesion, and the first non-abnormal region may correspond to a reflective region around the lesion.

[0071] As an example, there may be reflections or other artifacts around lesions or abnormalities such as calcifications. In such cases, the boundaries may be very blurred. The size of the calcification seen may also be different under different window widths and beds. By marking the reflection as the first non-abnormal area, each model no longer needs to strictly learn the boundary between the reflection area and the true value area. Instead, multiple models are not similar in the reflection area, so that the fusion results of multiple models can distinguish between the true value and the reflection area.

[0072] According to some embodiments, the annotated data may further include a second non-abnormal region of the sample image, and wherein the at least one strategy further includes a strategy for reducing the difference between the non-abnormal segmentation indicated by the second prediction result and the second non-abnormal region.

[0073] As a specific non-limiting example, the annotated data may include a truth map and a segmentation map. For example, the truth map has a first value in the annotated abnormal area and a second value in the first non-abnormal area and the second non-abnormal area; the segmentation map has different values ​​in the annotated abnormal area, the first non-abnormal area, and the second non-abnormal area. As another example, the annotated data may include only a segmentation map. It is understood that the above is only an example and the present disclosure is not limited thereto.

[0074] Combination Figure 3 FIG. 3 shows an exemplary sample image 300 and its annotation data according to an exemplary embodiment of the present disclosure. Figure 3 As shown, the sample image 300 may include a true value region (abnormal region) 301, an error-prone region (first non-abnormal region) 302, and other non-true value regions (second non-abnormal regions) 303. It is understood that the number, shape, and color of regions in the figure are only examples, and the present disclosure is not limited thereto.

[0075] In some embodiments, the method may include obtaining a sample image set and a labeled data set for the sample image set, each labeled data in the labeled data set including an abnormal area, an error-prone non-abnormal area, and a clear non-abnormal area for a corresponding sample image in the sample image set, and the present disclosure is not limited to this.

[0076] According to some embodiments, the method may further include adjusting parameters of the first sub-network to reduce the difference between the non-abnormal segmentation indicated by the first prediction result and the second non-abnormal region.

[0077] It can be understood that, although many thresholds are described above in the form of first threshold, second threshold, third threshold, etc., such expressions do not mean that such thresholds have the same or different values.

[0078] Although various operations are depicted in the drawings as being in a particular order, this should not be understood as requiring that the operations must be performed in the particular order shown or in sequential order, nor should it be understood as requiring that all illustrated operations must be performed to achieve desirable results.

[0079] It is to be understood that throughout the present disclosure, an image or sample image may be a single image or a sequence of images, may be or may include two-dimensional image data, or may be or include three-dimensional image data. An image or sample image may be image data that is directly acquired and stored or otherwise sent to a terminal device for use by a user. An image or sample image may also be processed image data after various image processing. An image or sample image may also undergo other analysis processes (e.g., an analysis process for whether there are pathological features or lesions) and include analysis results (e.g., the encircling of an area of ​​interest, the segmentation results of a tissue, etc.). It is to be understood that the present disclosure is not limited thereto.

[0080] Figure 4 is a flow chart illustrating an image processing method 400 according to an exemplary embodiment.

[0081] At step 410 , an image processing model is obtained by a method according to an embodiment of the present disclosure.

[0082] At step 420 , a segmentation result is obtained based on the input image through an image processing model.

[0083] Exemplarily, through the image processing model constructed according to the embodiments of the present disclosure, obtaining a segmentation result based on the input image may include inputting the input image into the first subnetwork and the second subnetwork described in the embodiments of the present disclosure, respectively, obtaining a first prediction result and a second prediction result, respectively, and obtaining a segmentation result based on the first prediction result and the second prediction result.

[0084] According to some exemplary embodiments, an image processing method is also disclosed, including: obtaining a target image; obtaining at least two prediction results for the target image based on at least two sub-networks, each of the at least two sub-networks being capable of being used to output a prediction result for segmenting anomalies in the input image based on an input image, the at least two sub-networks including a first sub-network and a second sub-network, wherein the second sub-network is obtained based on at least one strategy, the at least one strategy including a first strategy for: when targeting the same input image, reducing the similarity between the prediction result of the first sub-network and the prediction result of the second sub-network in at least one error-prone area of ​​the input image; and obtaining a final prediction result for anomaly segmentation of the target image based on the at least two prediction results.

[0085] In such an embodiment, differentiated prediction results can be obtained during image processing. Such differentiated prediction results have differences in error-prone areas, so integrating them can obtain more accurate results.

[0086] According to some embodiments, obtaining a final prediction result for anomaly segmentation of the target image based on the at least two prediction results comprises performing a weighted average on the at least two prediction results to obtain the final prediction result.

[0087] According to some embodiments, in the weighted average, the weight of each of the at least two subnetworks may be based on the similarity between the subnetwork and at least one other subnetwork of the at least two subnetworks in the at least one error-prone region. In such an embodiment, a relatively similar subnetwork may be assigned a lower weight, and a relatively dissimilar subnetwork may be assigned a higher weight, thereby ensuring a lower overall average level and a more accurate recognition effect in the error-prone non-abnormal region.

[0088] It will be understood that, unless otherwise stated, the various embodiments or features thereof described throughout this document may be combined with each other, and the present disclosure is not limited thereto.

[0089] Figure 5is a schematic block diagram illustrating a training apparatus 500 for an image processing model according to an exemplary embodiment. The training apparatus 500 for an image processing model may include: a sample image obtaining unit 510, a network obtaining unit 520, and a construction unit 530.

[0090] The sample image obtaining unit 510 can be used to obtain a sample image and annotation data for the sample image, wherein the annotation data includes an annotated abnormal region and at least a first non-abnormal region of the sample image. The network obtaining unit 520 can be used to obtain at least two subnetworks, each of which can be used to output a prediction result for segmenting anomalies based on the sample image, wherein the at least two subnetworks include a first subnetwork and a second subnetwork, wherein the second subnetwork is obtained based on at least one strategy, wherein the at least one strategy includes a first strategy, wherein the first strategy is used to reduce the similarity between the first prediction result of the first subnetwork for the sample image and the second prediction result of the second subnetwork for the sample image in the first non-abnormal region of the sample image. The construction unit 530 can be used to construct the image processing model based on the at least two subnetworks.

[0091] It should be understood that Figure 5 The various modules of the apparatus 500 shown in FIG. Figure 2 The steps in the method 200 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 200 are also applicable to the apparatus 500 and the modules included therein. For the sake of brevity, some operations, features and advantages are not described in detail here.

[0092] Figure 6 6 is a schematic block diagram illustrating an image processing apparatus 600 according to an exemplary embodiment. The training apparatus 600 for an image processing model may include a model acquisition unit 610 and a segmentation unit 620. The model acquisition unit 610 may be used to obtain an image processing model by a method or apparatus according to an embodiment of the present disclosure. The segmentation unit 620 may be used to obtain a segmentation result by the image processing model based on an input image.

[0093] It should be understood that Figure 6 The modules of the apparatus 600 shown in FIG. 6 can be used in conjunction with the reference Figure 4 The steps in the method 400 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 400 are also applicable to the apparatus 600 and the modules included therein. For the sake of brevity, some operations, features and advantages are not described in detail here.

[0094] According to an embodiment of the present disclosure, a computing device is also disclosed, including a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the steps of a training method for an image processing model or an image processing method and its variants according to an embodiment of the present disclosure.

[0095] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium is also disclosed, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the training method for an image processing model and its variant examples according to an embodiment of the present disclosure are implemented.

[0096] According to an embodiment of the present disclosure, a computer program product is also disclosed, including a computer program, wherein when the computer program is executed by a processor, the steps of the training method for an image processing model and its variant examples according to the embodiment of the present disclosure are implemented.

[0097] Although specific functions are discussed above with reference to specific modules, it should be noted that the functions of the various modules discussed herein may be divided into multiple modules, and / or at least some functions of multiple modules may be combined into a single module. The specific module discussed herein performs an action including the specific module itself performing the action, or alternatively the specific module calls or otherwise accesses another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, the specific module that performs an action may include the specific module itself that performs the action and / or another module that the specific module calls or otherwise accesses to perform the action. For example, the multiple modules or units described in the various embodiments of the present disclosure may be combined into a single module or unit in some embodiments. As used herein, the phrase "entity A initiates action B" may refer to entity A issuing an instruction to perform action B, but entity A itself does not necessarily perform the action B.

[0098] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Figure 5The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program codes / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuits. For example, in some embodiments, one or more modules or units described in the embodiments of the present disclosure can be implemented together in a system on chip (System on Chip, SoC). SoC may include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and may optionally execute the received program code and / or include embedded firmware to perform functions.

[0099] According to one aspect of the present disclosure, a computing device is provided, which includes a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any method embodiment described above.

[0100] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0101] According to one aspect of the present disclosure, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the steps of any method embodiment described above are implemented.

[0102] In the following, combined Figure 7 Illustrative examples of such a computer device, non-transitory computer-readable storage medium, and computer program product are described.

[0103] Figure 7 An example configuration of a computer device 700 that can be used to implement the methods described herein is shown. For example, Figure 1 The server 120 and / or the client device 110 shown in FIG. 7 may include an architecture similar to the computer device 700. The above-mentioned training device / apparatus for the image processing model may also be implemented in whole or at least in part by the computer device 700 or a similar device or system.

[0104] Computer device 700 can be a variety of different types of devices, such as a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computer device or computing system. Examples of computer device 700 include, but are not limited to, a desktop computer, a server computer, a laptop or netbook computer, a mobile device (e.g., a tablet computer, a cellular or other wireless phone (e.g., a smart phone), a notepad computer, a mobile station), a wearable device (e.g., glasses, a watch), an entertainment device (e.g., an entertainment appliance, a set-top box communicatively coupled to a display device, a game console), a television or other display device, a car computer, and the like. Therefore, computer device 700 can range from a full-resource device with a large amount of memory and processor resources (e.g., a personal computer, a game console) to a low-resource device with limited memory and / or processing resources (e.g., a traditional set-top box, a handheld game console).

[0105] The computer device 700 may include at least one processor 702, memory 704, communication interface(s) 706, a display device 708, other input / output (I / O) devices 710, and one or more mass storage devices 712 that can communicate with each other, such as via a system bus 714 or other appropriate connection.

[0106] The processor 702 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores. The processor 702 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, the processor 702 may be configured to obtain and execute computer-readable instructions stored in the memory 704, mass storage device 712, or other computer-readable media, such as program codes of an operating system 716, program codes of an application program 718, program codes of other programs 720, and the like.

[0107] The memory 704 and the mass storage device 712 are examples of computer-readable storage media for storing instructions that are executed by the processor 702 to implement the various functions described above. For example, the memory 704 may generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, the mass storage device 712 may generally include a hard drive, a solid-state drive, a removable medium, including external and removable drives, a memory card, a flash memory, a floppy disk, an optical disk (e.g., a CD, a DVD), a storage array, a network attached storage, a storage area network, etc. The memory 704 and the mass storage device 712 may all be collectively referred to herein as memory or computer-readable storage media, and may be a non-transitory medium capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by the processor 702 as a specific machine configured to implement the operations and functions described in the examples herein.

[0108] A plurality of program modules may be stored on the mass storage device 712. These programs include an operating system 716, one or more application programs 718, other programs 720, and program data 722, and they may be loaded into the memory 704 for execution. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components / functions: method 200 and / or method 400 (including any suitable steps of method 200, 400), and / or additional embodiments described herein.

[0109] Although in Figure 7 700, but modules 716, 718, 720, and 722, or portions thereof, may be implemented using any form of computer-readable media accessible by computer device 700. As used herein, "computer-readable media" includes at least two types of computer-readable media, namely, computer storage media and communication media.

[0110] Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented by any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data. Computer storage media includes but is not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computer device.

[0111] In contrast, communication media may embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism. Computer storage media as defined herein does not include communication media.

[0112] The computer device 700 may also include one or more communication interfaces 706 for exchanging data with other devices, such as through a network, direct connection, etc., as discussed above. Such communication interfaces may be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless (such as IEEE802.11 wireless LAN (WLAN)) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a wireless network interface. TM The communication interface 706 may facilitate communication within a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. The communication interface 706 may also provide for communication with external storage devices (not shown) such as in a storage array, network attached storage, storage area network, etc.

[0113] In some examples, a display device 708 such as a monitor may be included for displaying information and images to the user. Other I / O devices 710 may be devices that receive various inputs from the user and provide various outputs to the user, and may include a touch input device, a gesture input device, a camera, a keyboard, a remote control, a mouse, a printer, an audio input / output device, and the like.

[0114] Although the present disclosure has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative and exemplary rather than restrictive; the present disclosure is not limited to the disclosed embodiments. Variations to the disclosed embodiments will be understood and implemented by those skilled in the art in practicing the claimed subject matter by studying the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps that are not listed, and the word "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A training method for an image processing model, comprising: Obtaining a sample image and annotation data for the sample image, wherein the annotation data includes an annotated abnormal region and at least a first non-abnormal region of the sample image; Obtain at least two subnetworks, each of the at least two subnetworks can be used to output a prediction result for segmenting anomalies based on the sample image, the at least two subnetworks include a first subnetwork and a second subnetwork, wherein the second subnetwork is obtained based on at least one strategy, the at least one strategy includes a first strategy, and the first strategy is used to reduce the similarity between a first prediction result of the first subnetwork for the sample image and a second prediction result of the second subnetwork for the sample image in the first non-abnormal region of the sample image; as well as The image processing model is constructed based on the at least two sub-networks.

2. The method according to claim 1, wherein: The at least one strategy also includes a second strategy, and the second strategy is used to reduce the difference between the abnormal segmentation indicated by the second prediction result and the marked abnormal area.

3. The method according to claim 2, wherein: The second sub-network is trained based on the at least one strategy by: Determining an anomaly prediction loss of the second subnetwork based on a difference between the anomaly segmentation indicated by the second prediction result and the anomaly region marked; Determine an inter-network similarity loss of the second sub-network based on a similarity between the second prediction result and the first prediction result in the first non-abnormal region of the sample image; as well as The parameters of the second sub-network are adjusted based on the anomaly prediction loss and the inter-network similarity loss.

4. The method according to any one of claims 1-3, further comprising adjusting parameters of the first sub-network to reduce the difference between the abnormal segmentation indicated by the first prediction result and the annotated abnormal region.

5. The method according to any one of claims 1-4, further comprising adjusting parameters of the first sub-network to reduce the similarity between the first prediction result and the second prediction result in the first non-abnormal area.

6. The method according to claim 4 or 5, wherein: Obtaining at least two subnetworks includes: obtaining the initialized first sub-network and the initialized second sub-network; and The first sub-network and the second sub-network are trained based on the sample image until a training end condition is met, wherein the training end condition includes at least one of the following: The difference between the abnormal segmentation indicated by the first prediction result of the first sub-network for the sample image and the marked abnormal region is less than a first threshold; The difference between the abnormal segmentation indicated by the second prediction result of the second sub-network for the sample image and the marked abnormal region is less than a second threshold; and A similarity between a first prediction result of the first sub-network for the sample image and a second prediction result of the second sub-network for the sample image in the first non-abnormal region of the sample image is less than a third threshold.

7. The method according to any one of claims 1 to 5, wherein: The first sub-network and the second sub-network are obtained by the following operations: Using the sample image to train a plurality of sub-networks including the first sub-network and the second sub-network respectively until the difference between the abnormal segmentation indicated by the corresponding prediction result of each sub-network in the plurality of sub-networks and the marked abnormal region is less than a fourth threshold; as well as Two networks satisfying the dissimilar condition are selected from the multiple trained sub-networks as the first sub-network and the second sub-network, wherein: The similarity of the prediction results of the two networks satisfying the dissimilarity condition for the sample image in the first non-abnormal area is less than a fifth threshold, or The two networks satisfying the dissimilarity condition are the two networks whose corresponding prediction results for the sample image in the first non-abnormal region, among the multiple sub-networks, have the smallest similarity.

8. The method according to any one of claims 1 to 5, wherein: The first sub-network and the second sub-network are obtained by the following operations: Training the first sub-network based on the sample image until a difference between an abnormal segmentation indicated by the first prediction result of the first sub-network for the sample image and the marked abnormal region is less than a sixth threshold; constructing the second sub-network based on the first sub-network so that the initial weights of the second sub-network correspond to the weights of the trained first sub-network; as well as The second sub-network is trained based on the sample image until the similarity between the second prediction result of the second sub-network for the sample image and the first prediction result in the first non-abnormal area is less than a seventh threshold.

9. The method according to any one of claims 1 to 5, wherein: Obtaining at least two subnetworks includes: Obtaining the at least two initialized sub-networks; determining a loss value and a penalty value, wherein the loss value represents a difference between an abnormal segmentation indicated by each of at least two prediction results of the at least two sub-networks for the sample image and the annotated abnormal region, and wherein the penalty value represents a similarity between the at least two prediction results in the first non-abnormal region of the sample image; and A parameter of at least one network of the at least two sub-networks is adjusted based on the loss value and the penalty value.

10. The method according to claim 9, wherein: The penalty value is the dot product of at least two corresponding penalty vectors of the at least two sub-networks, wherein the penalty vector of each sub-network corresponds to the portion of the prediction result of the sub-network for the sample image in the first non-abnormal area.

11. The method according to claim 9, wherein: The penalty value is the sum of the dot products of the corresponding at least two penalty vectors of the at least two sub-networks, wherein the penalty vector of each sub-network corresponds to the part of the prediction result of the sub-network for the sample image in the first non-abnormal area.

12. The method according to any one of claims 1 to 11, wherein: Constructing the image processing model based on the at least two sub-networks includes: The image processing model is constructed based on the at least two sub-networks and the integration network, and the integration network is used to integrate at least two prediction results output by the at least two sub-networks as the output of the image processing model.

13. The method according to claim 11, wherein: Integrating the at least two prediction results output by the at least two sub-networks includes averaging the at least two prediction results.

14. The method according to claim 13, wherein: Integrating the at least two prediction results output by the at least two sub-networks includes performing weighted averaging on the at least two prediction results, and wherein the weight corresponding to each sub-network in the weighted averaging operation is based on the similarity between the prediction results of the corresponding sub-network and other networks in the at least two sub-networks in the first non-abnormal area.

15. The method according to any one of claims 1 to 14, wherein: The sample image is an image collected from a human body, the marked abnormal area corresponds to a lesion, and the first non-abnormal area corresponds to a reflective area around the lesion.

16. The method according to any one of claims 1 to 15, wherein: The annotated data further includes a second non-abnormal region of the sample image, and wherein the at least one strategy further includes a strategy for reducing a difference between a non-abnormal segmentation indicated by the second prediction result and the second non-abnormal region.

17. The method of claim 16, further comprising adjusting parameters of the first subnetwork to reduce the difference between the non-abnormal segmentation indicated by the first prediction result and the second non-abnormal region.

18. An image processing method, comprising: Obtaining an image processing model by the method according to any one of claims 1 to 17; as well as Based on the input image, a segmentation result is obtained through the image processing model.

19. An image processing method, comprising: Obtaining a target image; At least two prediction results are obtained for the target image based on at least two sub-networks, each of the at least two sub-networks can be used to output a prediction result for segmenting anomalies in the input image based on the input image, the at least two sub-networks include a first sub-network and a second sub-network, wherein the second sub-network is obtained based on at least one strategy, and the at least one strategy includes a first strategy for: when targeting the same input image, reducing the similarity between the prediction result of the first sub-network and the prediction result of the second sub-network in at least one error-prone area of ​​the input image; as well as A final prediction result of anomaly segmentation for the target image is obtained based on the at least two prediction results.

20. The image processing method according to claim 19, wherein: Obtaining a final prediction result for anomaly segmentation of the target image based on the at least two prediction results includes performing a weighted average on the at least two prediction results to obtain the final prediction result.

21. The image processing method according to claim 20, wherein: In the weighted averaging, a weight of each of the at least two sub-networks is based on a similarity between the sub-network and at least one other sub-network of the at least two sub-networks in the at least one error-prone region.

22. A training device for an image processing model, comprising: A sample image obtaining unit, configured to obtain a sample image and annotation data for the sample image, wherein the annotation data includes an annotated abnormal region and at least a first non-abnormal region of the sample image; A network obtaining unit, configured to obtain at least two sub-networks, each of the at least two sub-networks being capable of being used to output a prediction result for segmenting anomalies based on the sample image, the at least two sub-networks comprising a first sub-network and a second sub-network, wherein the second sub-network is obtained based on at least one strategy, the at least one strategy comprising a first strategy, the first strategy being configured to reduce a similarity between a first prediction result of the first sub-network for the sample image and a second prediction result of the second sub-network for the sample image in the first non-abnormal region of the sample image; as well as A construction unit is used to construct the image processing model based on the at least two sub-networks.

23. An image processing device, comprising: A model obtaining unit, configured to obtain an image processing model by means of the apparatus according to claim 22; as well as The segmentation unit is used to obtain a segmentation result based on the input image through the image processing model.

24. A computing device comprising: a memory, a processor and a computer program stored on the memory, The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 21.

25. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

26. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 21 are implemented.

Images