Semantic segmentation methods and related products

Abstract

The present invention discloses a semantic segmentation method and related products. This semantic segmentation method inserts an attention map fusion layer into the self-attention mechanism layer of the last Transformer block. The attention map fusion layer is used to determine the Transformer block number (denoted as g) where the global block first appears in the attention map calculated by the first f Transformer blocks. The attention map fusion layer then fuses information from at least two attention maps calculated from the gth Transformer block to the last Transformer block to obtain a fused attention map. The information fusion operation must use the attention map calculated by the last Transformer block. This method can improve the accuracy of semantic segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a semantic segmentation method and related products. Background Art

[0002] Semantic segmentation is the task of pixel-level classification of image data. Given a set of semantic labels, each pixel in an image is assigned a semantic label. Traditional semantic segmentation models, after training, are only capable of identifying and segmenting a limited number of known categories whose labels were provided during training. However, in real-world applications, many unknown categories exist that were not provided during training. Obtaining pixel-level labels for these categories can be difficult and expensive, significantly limiting the practical application of traditional semantic segmentation models.

[0003] Chinese invention patent CN106295706B proposes a method for automatic image segmentation and semantic annotation based on a shape vision knowledge base, which automatically segments and semantically annotates unknown shapes through the shape vision knowledge base.

[0004] Chinese invention patent application CN117710784A proposes a method and system for detecting unknown category targets. The method mainly includes generating pseudo-label information for unknown category objects in each image sample in the training sample set based on a constructed semantic image segmentation model; training the category-related target detection head and the category-independent target detection head respectively; then obtaining the target image to be detected, and extracting and fusing all category-related features and all category-independent features in the target image to be detected respectively; inputting the fused category-related features into the trained category-related target detection head to obtain the detection results of known category objects, and inputting the fused category-independent features into the trained category-independent target detection head to obtain the foreground object detection results; based on the foreground object detection results and the known category object detection results, obtaining the detection results of unknown category objects in the target image. This method can effectively improve the detection accuracy of unknown category targets.

[0005] The present invention provides a new semantic segmentation method and related products. Summary of the Invention

[0006] The present invention provides a semantic segmentation method and related products to improve the accuracy of semantic segmentation.

[0007] The present invention achieves the above-mentioned purpose through the following schemes.

[0008] The present invention provides a semantic segmentation method capable of processing unknown categories, comprising:

[0009] A semantic encoder is used to convert multiple labels into multiple text representations with one-to-one correspondence, where each label represents the target category name of the pixel area obtained by semantic segmentation;

[0010] An image encoder is used to convert the target image into a corresponding image representation;

[0011] Determine a label for each pixel in the target image using the image representation and the multiple text representations;

[0012] Wherein, the image encoder includes sequentially connected f +1 Transformer block, each of which contains a self-attention mechanism layer, f ≥2, an attention map fusion layer is inserted into the self-attention mechanism layer of the last Transformer block;

[0013] The attention graph fusion layer is used to:

[0014] Before judgment f The Transformer block number where the global block first appears in the attention map calculated by the Transformer blocks is recorded as g ;

[0015] Fuse the information of at least two attention maps calculated from the gth Transformer block to the last Transformer block to obtain a fused attention map, where the information fusion operation must use the attention map calculated from the last Transformer block;

[0016] Among them, the self-attention mechanism layer of the last Transformer block is used to calculate the image representation output by the image encoder based on its V matrix and the fused attention map.

[0017] In some embodiments, the f The attention map calculated by the Transformer block is the QK attention map;

[0018] The attention map fusion layer uses the following function to determine whether a global block appears in the attention map:

[0019] ;

[0020] in, =1 means the number is i A global block appears in the attention map calculated by the Transformer block. =0 means the number is i There is no global block in the attention map calculated by the Transformer block. Indicates the number iThe attention map obtained by the Transformer block is j attention vectors, It is a preset constant used to prevent the product of attention vectors from being too small and exceeding the computer's storage range.

[0021] In some embodiments, in the step of converting the target image into a corresponding image representation using an image encoder, an image representation of each of the plurality of pixel blocks segmented from the target image is obtained;

[0022] Determining a label for each pixel in the target image according to the image representation and the multiple text representations includes:

[0023] Calculate the approximation between the image representation of each pixel block and each text representation;

[0024] The bilinear interpolation method is used to obtain the approximation between each pixel point in each pixel block and each text representation, and then determine the label corresponding to each pixel point.

[0025] In some embodiments, the image representation output by the self-attention mechanism layer of the last Transformer block is denoted as Z,

[0026] ;

[0027] Proj(·) is the projection function, is the fused attention map, is the Value matrix of the self-attention mechanism layer of the last Transformer block.

[0028] In some embodiments, the attention map calculated by the last Transformer block is a QQ attention map.

[0029] In some embodiments, the f Each Transformer block includes: a normalization layer, a self-attention mechanism layer, a residual layer, a normalization layer, a feedforward neural network layer and a residual layer arranged in that order.

[0030] In some embodiments, the fused attention map is the average of the attention maps calculated from the g-th Transformer block to the last Transformer block; or the fused attention map is the average of the attention maps calculated from a set number of consecutive Transformer blocks starting from the g-th Transformer block and the last Transformer block.

[0031] The present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a program, and the processor runs the program to execute the above method.

[0032] The present invention provides a computer program product, which executes the above method when running on a processor.

[0033] The present invention provides a storage medium, on which instructions are stored, and the instructions execute the above method when running.

[0034] In the attention mechanism of the last Transformer block in the visual encoder, each block in the attention map interacts with itself and neighboring blocks, improving the distinguishability of block-level features while integrating image-level global features from global blocks. Global contextual information provides a global understanding of the image level in semantic segmentation tasks. In the pixel-level classification task of semantic segmentation, it helps distinguish certain easily confused categories, thereby achieving better semantic segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is an information flow diagram of the semantic segmentation method of the present invention that can process unknown categories.

[0036] Figure 2 It is a structural block diagram of the electronic device of the present invention.

[0037] Figure 3 This is the network structure of the first visual encoder in the test example of the present invention.

[0038] Figure 4 This is the network structure of the last visual encoder in the test example of the present invention.

[0039] Figure 5 This is the semantic segmentation result of the test case of the present invention before model modification.

[0040] Figure 6 This is the semantic segmentation result of the test case of the present invention after the model is modified. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to specific embodiments, but the protection scope of the present invention is not limited thereto.

[0042] Combine Figure 1 , the present invention provides a semantic segmentation method, comprising:

[0043] A semantic encoder is used to convert multiple labels into multiple text representations with one-to-one correspondence, where each label represents the target category name of the pixel area obtained by semantic segmentation;

[0044] The target image is divided into blocks and input into the image encoder to obtain the image representation corresponding to each image block of the target image;

[0045] Determine a label for each pixel in the target image using the image representation and the multiple text representations;

[0046] Wherein, the image encoder includes sequentially connected f +1 Transformer block, each of which contains a self-attention mechanism layer, f ≥2, an attention map fusion layer is inserted into the self-attention mechanism layer of the last Transformer block;

[0047] The attention graph fusion layer is used to:

[0048] Before judgment f The Transformer block number where the global block first appears in the attention map calculated by the Transformer blocks is recorded as g ;

[0049] Fuse the information of at least two attention maps calculated from the gth Transformer block to the last Transformer block to obtain a fused attention map, where the information fusion operation must use the attention map calculated from the last Transformer block;

[0050] Among them, the self-attention mechanism layer of the last Transformer block is used to calculate the image representation output by the image encoder based on its V matrix and the fused attention map.

[0051] The semantic encoder and image encoder are pre-trained and aligned in the representation space.

[0052] In the attention map of a deep Transformer block, a special set of pixel patches appears. All pixel patches in the attention map have high attention scores for this special set of pixel patches, indicating that all pixel patches have significantly integrated the information of this special set of pixel patches. This special set of pixel patches is called a global patch, and it is believed that they better integrate global context information at the image level.

[0053] In the attention mechanism of the last Transformer block in the visual encoder, each pixel block in the attention map can interact with itself and neighboring pixel blocks, thereby improving the distinguishability of pixel block features and integrating image-level global features from the global block. Global context information can provide a global understanding of the image level in semantic segmentation tasks. In the pixel-level classification task of semantic segmentation, it can help distinguish certain easily confused categories, thereby achieving better semantic segmentation performance.

[0054] In some embodiments, the f The attention map calculated by the Transformer block is the QK attention map;

[0055] The attention map fusion layer uses the following function to determine whether a global block appears in the attention map:

[0056] ;

[0057] in, =1 means the number is i A global block appears in the attention map calculated by the Transformer block. =0 means the number is i There is no global block in the attention map calculated by the Transformer block. Indicates the number i The attention map obtained by the Transformer block is j attention vectors, This is a preset constant used to prevent the product of attention vectors from being too small to fit in the computer's memory. Here, vector multiplication is a dot product.

[0058] The QK attention map is the matrix obtained by multiplying the transpose of the Query matrix and the Key matrix and then processing it through the Softmax function. The specific formula is as follows: ,in, is the QK attention map, is the Query matrix, is the Key matrix, is the dimension of the Query matrix.

[0059] In some embodiments, The value ranges from 50 to 200. A typical value is 100.

[0060] In some embodiments, the image representation output by the self-attention mechanism layer of the last Transformer block is denoted as Z,

[0061] ;

[0062] Proj(·) is the projection function, is the fused attention map, is the Value matrix of the self-attention mechanism layer of the last Transformer block.

[0063] In some embodiments, the attention map calculated by the last Transformer block is a QQ attention map.

[0064] The QQ attention map is a matrix obtained by multiplying the query matrix by the transpose of the query matrix and then processing it with the Softmax function. This makes each pixel block focus on itself in the attention mechanism.

[0065] Right now , The attention map calculated for the last Transformer block, is the dimension of the Query matrix, and Q is the Query matrix.

[0066] In some embodiments, the fused attention map is the average of the attention maps calculated from the gth Transformer block to the last Transformer block.

[0067] In other embodiments, the fused attention map is a weighted sum of the attention maps calculated from the gth Transformer block to the last Transformer block, with the weight coefficient increasing with the order in which the Transformer blocks are connected.

[0068] In another embodiment, the fused attention map is the average of the attention maps calculated from a set number of consecutive Transformer blocks starting from the g-th Transformer block and the last Transformer block.

[0069] For example, the fused attention map is the mean of the attention maps computed from the g-th Transformer block and the last Transformer block.

[0070] For example, the fused attention map is the average of the attention maps calculated from the gth Transformer block, the g+1th Transformer block, and the last Transformer block.

[0071] The present invention does not limit the operation mechanism of the semantic encoder. In some embodiments, the semantic encoder operates as follows.

[0072] Provide multiple labels, each label represents the target category name of the pixel area obtained by semantic segmentation.

[0073] The target category name is, for example, a person, a skateboard, grass, a sofa, a chair, or any other target category name that needs to be distinguished.

[0074] The multiple tags are respectively constructed into corresponding target texts according to a preset template.

[0075] For example, a preset template is "a picture of *," where * represents a tag. The target text for the tag "person" is "a picture of a person," the target text for the tag "skateboard" is "a picture of a skateboard," and the target text for the tag "grass" is "a picture of grass."

[0076] Each target text is segmented separately to obtain multiple consecutive words corresponding to each target text.

[0077] For example, the target text "a picture of a person" is segmented into four words: "a", "person", "of", and "picture".

[0078] The text segmentation process can adopt existing mature segmentation methods, and the present invention is not limited to this.

[0079] Input multiple consecutive words corresponding to each target text into the text encoder to obtain the encoded text representation.

[0080] The text encoder can adopt an existing mature text encoder. The present invention does not limit the type of the text encoder, and does not limit the training method of the text encoder.

[0081] After text encoding, multiple consecutive words are converted into a text representation, which is expressed as a vector.

[0082] Encoding the target text (i.e., a complete sentence) is conducive to enriching the semantic expression.

[0083] In some embodiments, the visual encoder first preprocesses the target image so that the target image conforms to a set format representation, and then inputs the preprocessed target image into the embedding layer, and then inputs the image block output by the embedding layer into the first Transformer block.

[0084] During preprocessing, the target image is first scaled proportionally, maintaining its aspect ratio. This scales the target image's long or short sides to the specified dimensions, and then fills any missing pixels with appropriate values (such as black pixels). Next, the target image's pixel values are normalized. In the original RGB image data, the pixel values of each color component range from 0 to 255. Normalization transforms the pixel values of each color into a normal distribution with a mean of 0 and a standard deviation of 1. Specifically, the mean and standard deviation of the pixel values for each color in the target image are calculated. The mean and standard deviation are then subtracted from each pixel value and divided by the standard deviation. This approach is intended to make the model training process more stable and efficient. Because the pixel value range and distribution may vary across images, normalization can bring the pixel value distribution of all images into a similar range, making it easier for the model to learn image features and preventing certain features from being overemphasized or undervalued due to differences in pixel value ranges.

[0085] The embedding layer divides the input target image into fixed-size pixel blocks. For example, for a pixel of size (in It is high. is the width, the number of channels of the RGB image The image with size 3) will be divided into N pieces of size The pixel blocks are of size P, and the side length of the pixel blocks is P. These pixel blocks are mapped into N D-dimensional serialized vectors after linear embedding (a fully connected layer). Each serialized vector is vector-added with the corresponding position code to form ,in, is the serialized vector containing position information corresponding to the i-th pixel block, .

[0086] forward f The structure of each Transformer block can be designed according to the existing technology, and the present invention does not limit this. f Each Transformer block contains only one self-attention mechanism layer.

[0087] In some embodiments, reference Figure 3 ,forward f Each Transformer block includes: a normalization layer, a self-attention mechanism layer, a residual layer, a normalization layer, a feedforward neural network layer, and a residual layer arranged in that order.

[0088] Both normalization layers are layer normalization.

[0089] The first residual layer sums the input of the first normalization layer with the output of the self-attention layer.

[0090] The second residual layer sums the input of the second normalization layer with the output of the feedforward neural network layer.

[0091] The feedforward neural network layer is the first fully connected layer, the activation layer, and the second fully connected layer connected in sequence.

[0092] The following takes the first Transformer block as an example.

[0093] Will serialize the vector Input the first normalization layer to obtain the normalized serialized vector.

[0094] For the input vector , and calculate its mean and standard deviation , and then the input vector Converts to: ,This step helps stabilize the training process, reduce the problem of gradient disappearance or explosion, and make the distribution of input data more stable, which is beneficial for subsequent operations.

[0095] The serialized vector output by the first normalization layer is input into the self-attention mechanism layer to obtain the serialized vector output by the self-attention mechanism layer.

[0096] The operation of the self-attention mechanism layer is as follows: for each input normalized serialized vector, it passes through three different linear layers to generate the Query (Q) matrix, Key (K) matrix and Value (V) matrix. The attention map A is obtained by Query (Q) and Key (K). ;

[0097] in Denotes the dimension of the Key matrix, Q denotes the Query matrix, and K denotes the Key matrix. Using the attention map A as the weight, the Value (V) matrix is weighted to obtain the serialized vector output from the attention mechanism layer.

[0098] The serialized vector input to the first normalization layer and the serialized vector output by the self-attention mechanism layer are residually connected, that is, vector addition operation is performed to obtain the residual connected serialized vector.

[0099] Residual connections help prevent gradient vanishing, allowing the network to be deeper and learn more complex features because they allow information to be passed directly from the input layer to subsequent layers, rather than relying solely on transformations in the current layer (such as the self-attention mechanism layer here).

[0100] The residual connection serialization vector is input into the second normalization layer to obtain the normalized residual connection serialization vector. The second normalization layer operates in the same way as the first normalization layer.

[0101] The normalized residual connection serialization vector is input into the feedforward neural network layer to obtain the serialization vector output by the feedforward neural network layer.

[0102] The feedforward neural network layer has two fully connected layers and one activation layer (the activation layer is located between the two fully connected layers).

[0103] The function of the first fully connected layer of the feedforward neural network layer is to map the input features (i.e., the normalized residual connection sequence vector) to a higher-dimensional space, providing richer representation capabilities for subsequent nonlinear transformations. This helps the network learn more complex feature combinations and relationships, because higher dimensions can store more information, enabling the network to capture more subtle relationships between input features.

[0104] The high-dimensional vector output by the first fully connected layer is fed into the activation layer. The activation layer performs a nonlinear mapping on the linearly transformed high-dimensional vector, allowing the network to learn nonlinear feature relationships. Through nonlinear activation, the network can better fit various complex functions.

[0105] The function of the second fully connected layer of the feedforward neural network layer is to map the high-dimensional features after dimensional expansion and nonlinear transformation back to the original dimension, or to a dimension suitable for the next processing stage. This ensures that the dimension of the output feature is compatible with the subsequent operations of the network.

[0106] Perform vector addition on the residual connection serialization vector and the output of the feedforward neural network layer to obtain the output of the current Transformer block.

[0107] In some embodiments, reference Figure 4 , the last Transformer block only includes the normalization layer and the self-attention mechanism layer connected in sequence, and the attention map fusion layer is inserted in the self-attention mechanism layer.

[0108] The following example illustrates the complete operation process of the image encoder.

[0109] The preprocessed target image is divided into 9 pixel blocks, each consisting of 3 rows and 3 columns. After processing in the embedding layer, 9 serialized vectors are obtained. These 9 serialized vectors are input to the first normalization layer, again resulting in 9 normalized serialized vectors. These 9 normalized serialized vectors are then input to the self-attention layer, assuming single-head attention, again resulting in 9 serialized vectors. These 9 serialized vectors are then processed by the second normalization layer and the feedforward neural network layer, yielding 9 serialized vectors. These 9 serialized vectors are then summed with the 9 serialized vectors received by the second normalization layer, yielding 9 serialized vectors.

[0110] The self-attention layer of the last Transformer block outputs nine serialized vectors. These nine serialized vectors correspond to the nine pixel blocks of the target image and represent the D-dimensional image features extracted for each pixel block. The similarity between each pixel block and the text representation is calculated. This is essentially a matrix multiplication of the 9×D image features with the C×D text representations of the C text items.

[0111] In some embodiments, in the step of converting the target image into a corresponding image representation using an image encoder, an image representation of each of the plurality of pixel blocks segmented from the target image is obtained;

[0112] Determining a label for each pixel in the target image according to the image representation and the multiple text representations includes:

[0113] Calculate the approximation between the image representation of each pixel block and each text representation;

[0114] The bilinear interpolation method is used to obtain the approximation between each pixel point in each pixel block and each text representation, and then determine the label corresponding to each pixel point.

[0115] The image features of each image block are D-dimensional, that is, a 1×D vector. The 10 labels correspond to a 10×D text representation matrix. The matrix multiplication of the transpose of the image features and the text representation matrix results in a 1×10 vector, which represents the similarity between the current image block and the 10 labels respectively. Each image block will have a similar similarity matrix.

[0116] The current similarity is at the pixel block level, that is, the similarity between each pixel block and the 10 labels. To obtain the pixel-level similarity matrix, a bilinear interpolation operation is performed to obtain the pixel-level similarity matrix. For each pixel position, the argmax function is used to take the label corresponding to the maximum similarity value, which is used as the segmentation result for the current pixel.

[0117] Based on the same inventive concept, Figure 2 The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a program, and the processor runs the program to execute the above method.

[0118] Based on the same inventive concept, the present invention also provides a computer program product, which executes the above method when running on a processor.

[0119] Based on the same inventive concept, the present invention further provides a storage medium storing instructions, which execute the above method when running.

[0120] The processor is, for example, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or any other known processor type, or a combination thereof.

[0121] The memory and storage medium are, for example, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0122] A test example is provided below.

[0123] A multimodal pre-training model is provided for Lan M, Chen C, Ke Y, et al. Clearclip: Decomposing clip representations for dense vision-language inference[C] / / European Conference on Computer Vision. Springer, Cham, 2025: 143-160. The last Transformer block in the visual encoder multiplies the attention map calculated by itself with the Value matrix to obtain the image representation.

[0124] In this test case f The structure of each Transformer block is Figure 3 In the structure shown, the attention map calculated from the attention mechanism layer is the QK attention map.

[0125] The attention map calculated by the self-attention mechanism layer of the last Transformer block in this test case is the QQ attention map.

[0126] The structure of the last Transformer block in this test case before modification is the same as Figure 4 The only difference from the shown structure is that the attention map fusion layer is not inserted in the self-attention mechanism layer.

[0127] Figure 5 This is the test result of the multimodal pre-trained model. Part of the chair area (red) is mistaken for a sofa (green area).

[0128] An attention map fusion layer is added to the last Transformer block of the multimodal pre-trained model. The fused attention map is the average of the attention maps computed in the gth Transformer block, the g+1th Transformer block, and the final Transformer block. The model parameters of the remaining modules of the multimodal pre-trained model remain unchanged. The image representation is obtained by multiplying the fused attention map with the Value matrix.

[0129] Figure 6 This is the test result after the modification. The modification of the multimodal pre-training model avoids the error of confusing chairs with sofas.

[0130] The scope of protection of the present invention is not limited to the above-described embodiments. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the scope and spirit of the present invention. If such modifications and variations fall within the scope of the claims of the present invention and their equivalents, the present invention is intended to include such modifications and variations.

Claims

1. A semantic segmentation method, characterized in that: include: A semantic encoder is used to convert multiple labels into multiple text representations with one-to-one correspondence, where each label represents the target category name of the pixel area obtained by semantic segmentation; An image encoder is used to convert the target image into a corresponding image representation; Determining a label for each pixel in the target image based on the image representation and the multiple text representations; Wherein, the image encoder includes sequentially connected f +1 Transformer block, each of which contains a self-attention mechanism layer, f ≥2, an attention map fusion layer is inserted into the self-attention mechanism layer of the last Transformer block; The attention graph fusion layer is used to: Before judgment f The Transformer block number where the global block first appears in the attention map calculated by the Transformer blocks is recorded as g ; The first g The information fusion operation must use the attention map calculated by the last Transformer block; The self-attention mechanism layer of the last Transformer block is used to calculate the image representation output by the image encoder based on its V matrix and the fused attention map; forward f The attention map calculated by the Transformer block is the QK attention map; The attention map fusion layer uses the following function to determine whether a global block appears in the attention map: ; in, =1 means the number is i A global block appears in the attention map calculated by the Transformer block. =0 means the number is i There is no global block in the attention map calculated by the Transformer block. Indicates the number i The attention map obtained by the Transformer block is j attention vectors, It is a preset constant used to prevent the product of attention vectors from being too small and exceeding the computer's storage range.

2. The semantic segmentation method according to claim 1, characterized in that In the step of converting the target image into a corresponding image representation using an image encoder, an image representation of each of the plurality of pixel blocks segmented from the target image is obtained; Determining a label for each pixel in the target image according to the image representation and the multiple text representations includes: Calculate the approximation between the image representation of each pixel block and each text representation; The bilinear interpolation method is used to obtain the approximation between each pixel point in each pixel block and each text representation, and then determine the label corresponding to each pixel point.

3. The semantic segmentation method according to claim 1, characterized in that The image representation output by the self-attention mechanism layer of the last Transformer block is denoted as Z. ; Proj(·) is the projection function, is the fused attention map, is the Value matrix of the self-attention mechanism layer of the last Transformer block.

4. The semantic segmentation method according to claim 1, characterized in that The attention map calculated by the last Transformer block is the QQ attention map.

5. The semantic segmentation method according to claim 1, wherein: forward f Each Transformer block includes: a normalization layer, a self-attention mechanism layer, a residual layer, a normalization layer, a feedforward neural network layer, and a residual layer arranged in that order.

6. The semantic segmentation method according to claim 1, characterized in that The fused attention map is the average of the attention maps calculated from the g-th Transformer block to the last Transformer block; or the fused attention map is the average of the attention maps calculated from a set number of consecutive Transformer blocks starting from the g-th Transformer block and the last Transformer block.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a program, and the processor runs the program to perform the method according to any one of claims 1 to 6.

8. A computer program product, characterized in that When running on a processor, the method according to any one of claims 1 to 6 is executed.

9. A storage medium, characterized in that: The storage medium stores instructions, which, when executed, execute the method according to any one of claims 1 to 6.

Images