A visual token pruning method based on semantic complementarity and a multi-modal large language model reasoning method and device

Abstract

The application belongs to the technical field of artificial intelligence, and relates to a visual token pruning method based on semantic complementarity and a multi-modal large language model reasoning method and device. The method comprises: encoding an image by using a visual encoder to obtain an initial visual token sequence; selecting a key visual token from the initial visual token sequence; selecting a complementary visual token of the key visual token from the initial visual token sequence; performing pooling calculation on the initial visual token sequence to obtain a fusion visual token; splicing the key visual token and the complementary visual token and adding the fusion visual token to obtain a final visual token sequence; and a multi-modal large language model fusing visual information in the visual token sequence and text information in a text token sequence to obtain a reasoning result. The application can realize efficient compression of visual information, thereby reducing the computing overhead while improving the reasoning efficiency and multi-modal understanding capability of the model.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and model compression technology, and more specifically, relates to a visual token pruning method based on semantic complementarity and a multimodal large language model reasoning method and apparatus. Background Technology

[0002] Existing multimodal large language models can understand and fuse different types of data, enabling cross-modal reasoning, question answering, generation, and decision support, thereby significantly improving the perception, understanding, and interaction capabilities of artificial intelligence systems. However, this powerful multimodal understanding capability typically relies on substantial computational and storage resources, severely limiting its application in practical scenarios such as smart homes, healthcare, and education. Therefore, researching lightweight multimodal large language models is of great significance, as it can reduce computational and storage overhead while ensuring the model's performance and response efficiency in complex tasks.

[0003] In recent years, the practical application of multimodal large language models has attracted increasing attention. However, existing patents mainly focus on model architecture optimization, training methods, and the parsing and utilization of large-scale data. For example, Soochow University's patent application "Training Method and System for Multimodal Large Language Model Based on Multimodal Encoder" (patent application number CN202311477951.3, patent publication number CN117218498A) and Shanghai Jiao Tong University's patent "Training Method and System for End-to-End Multimodal Large Model for Spoken Question Answering" (patent application number CN202411114871.6, patent publication number CN118782040A) mainly focus on the optimization of multimodal large language model training methods. In addition, Harbin Institute of Technology's patent application "A Multimodal Data Parsing Method Based on Hybrid Expert Structure Large Model Training" (patent application number CN202410597545.9, patent publication number CN118551220A) focuses on model architecture design and efficient parsing of multimodal data. However, the aforementioned patents neglect how to reduce the computational cost and improve inference efficiency of large multimodal models in practical applications, making it difficult to deploy the models in resource-constrained scenarios or scenarios requiring real-time response.

[0004] In summary, token pruning for multimodal large language models is an important approach to address the problems of high computational overhead and slow inference speed in practical application scenarios, but current technologies have not yet conducted research on this task. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a visual token pruning method based on semantic complementarity and a multimodal large language model inference method and apparatus. The purpose is to achieve efficient compression of visual information by effectively filtering key visual tokens and supplementing semantic diversity information, thereby reducing computational overhead while improving the model's inference efficiency and multimodal understanding capabilities.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A visual token pruning method based on semantic complementarity includes the following steps: The image is encoded using a visual encoder to obtain an initial sequence of visual tokens; Based on the multi-head attention score of the visual encoder, key visual tokens are selected from the initial sequence of visual tokens; Based on feature similarity, complementary visual tokens of key visual tokens are selected from the initial visual token sequence; The initial visual token sequence is pooled to obtain the fused visual token; The key visual tokens and complementary visual tokens are concatenated and a fusion visual token is added to obtain the final visual token sequence.

[0007] Furthermore, the selection of key visual tokens from the initial visual token sequence based on the multi-head attention score of the visual encoder includes: The attention score for each visual token is calculated in each attention head of the visual encoder, and the scores of the visual tokens in all attention heads are averaged to obtain the multi-head attention score for each visual token. The top k visual tokens, ranked from highest to lowest based on multi-head attention scores, are selected as key visual tokens.

[0008] Furthermore, the step of selecting complementary visual tokens for key visual tokens from the initial visual token sequence based on feature similarity includes: The similarity between the feature vectors of the remaining visual tokens in the initial visual token sequence, excluding the key visual token, is calculated to obtain the first similarity. The second similarity score is obtained by calculating the similarity between the key visual token and the other visual tokens. By combining the first similarity score and the second similarity score, the complementary token score is obtained; The top k visual tokens are selected from the remaining visual tokens based on their complementary token scores, and these k tokens are used as complementary visual tokens.

[0009] Furthermore, the pooling calculation is global average pooling.

[0010] A multimodal large language model inference method includes the following steps: The visual token sequence and text token sequence obtained by the above method are input into the multimodal large language model for inference; The multimodal large language model integrates visual information from visual token sequences with textual information from text token sequences to obtain inference results.

[0011] Furthermore, the multimodal large language model is constructed using the following steps: Build a visual encoder to receive visual input, including images or videos; Build a large language model to receive text input; By combining a visual encoder with a large language model, a multimodal large language model is formed.

[0012] A visual token clipping device based on semantic complementarity, comprising: The image encoding module is used to encode the image using a visual encoder to obtain an initial visual token sequence; The key visual token selection module is used to select key visual tokens from an initial sequence of visual tokens based on the multi-head attention score of the visual encoder. A complementary visual token selection module is used to select complementary visual tokens of key visual tokens from an initial sequence of visual tokens based on feature similarity. The token fusion module is used to perform pooling calculations on the initial visual token sequence to obtain a fused visual token; The token concatenation module is used to concatenate key visual tokens and complementary visual tokens and add a fused visual token to obtain the final visual token sequence.

[0013] A multimodal large language model inference device, comprising: The token acquisition module is used to acquire the visual token sequence and text token sequence obtained by the above method, and input them into the multimodal large language model for inference; The multimodal large language model module is used to fuse visual information from the visual token sequence and textual information from the text token sequence to obtain inference results.

[0014] The present invention also provides a computer device including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.

[0015] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the above-described method.

[0016] The core innovations and beneficial effects of this invention are as follows: 1) A Robust Importance Measurement Method Based on Multi-Head Attention Score Aggregation. In terms of importance measurement, this invention quantifies token importance by calculating the average score of all attention heads in the visual encoder, evaluating importance from a multi-head, multi-viewpoint perspective. This invention abandons the traditional approach that relies solely on CLS tokens or text guidance, proposing instead the average score of multi-head attention as the benchmark for importance evaluation. Different attention heads often focus on different feature subspaces of the image (such as texture, shape, and location). By aggregating the attention of all heads, the bias of a single viewpoint can be eliminated, providing a more stable, comprehensive, and downstream task-independent visual token importance measurement method, laying the foundation for subsequent accurate cropping.

[0017] 2) A semantic complementarity-based selection strategy based on dual feature similarity. To address the problem of traditional methods easily losing background and multi-object information in multi-object information selection and retention, this invention designs a complementary token selection strategy. This strategy combines two dimensions: "low similarity with the key token" and "low redundancy with other tokens." Through the proposed dual similarity complementary selection strategy, this invention not only measures the difference between candidate tokens and the key token but also the uniqueness of the candidate token itself, thereby accurately recalling underestimated background and multi-object information. This represents a technological leap from "retaining the most salient" to "retaining the most diverse." It can automatically identify and retain visual tokens that, although their attention scores are not high, contain unique semantic information (complementary to the key object), thereby maximizing the compressed information coverage.

[0018] 3) Residual Fusion Architecture for Multi-Granularity Global Semantics. Regarding the preservation of global multi-granularity information, this invention maximizes information retention through a multi-granularity visual token fusion strategy. This design effectively preserves the spatial structural relationships of multiple objects in vision, achieving superior visual token pruning. This invention proposes a unique token fusion mechanism: extracting multi-granularity information from all original visual tokens and obtaining fusion features through global average pooling, then superimposing (addition operation) these features onto the filtered token sequence to form the final input of the large language model. This design significantly reduces the number of tokens (reducing computational load) while forcibly injecting global contextual information of the image into the model. It is equivalent to adding a "global background" to fragmented local features, effectively preventing semantic breaks caused by pruning and significantly improving the model's inference performance under extremely low retention rates. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the steps of a visual token pruning method based on semantic complementarity according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart illustrating the steps of a multimodal large language model inference method according to an embodiment of the present invention.

[0021] Figure 3 This is a block diagram of a visual token clipping device based on semantic complementarity according to an embodiment of the present invention.

[0022] Figure 4 This is a block diagram of a multimodal large language model inference device according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] In this invention, a token refers to a basic representation unit obtained after discretizing or segmenting input information, used as the smallest unit for model processing and computation. For example, a visual token refers to a visual feature representation obtained by segmenting and extracting features from an image, while a text token refers to a word-level or sub-word-level semantic representation obtained by word segmentation or sub-word partitioning of text.

[0025] In one embodiment, the present invention provides a visual token pruning method based on semantic complementarity, such as... Figure 1 As shown, it includes the following steps: The image is encoded using a visual encoder to obtain an initial sequence of visual tokens; Based on the multi-head attention score of the visual encoder, key visual tokens are selected from the initial sequence of visual tokens; Based on feature similarity, complementary visual tokens of key visual tokens are selected from the initial visual token sequence; The initial visual token sequence is pooled to obtain the fused visual token; The key visual tokens and complementary visual tokens are concatenated and a fusion visual token is added to obtain the final visual token sequence.

[0026] In one embodiment, selecting key visual tokens from an initial sequence of visual tokens based on the multi-head attention score of the visual encoder includes: The attention score for each visual token is calculated in each attention head of the visual encoder, and the scores of the visual tokens in all attention heads are averaged to obtain the multi-head attention score for each visual token. The top k visual tokens, ranked from highest to lowest based on multi-head attention scores, are selected as key visual tokens.

[0027] In one embodiment, selecting complementary visual tokens of key visual tokens from an initial sequence of visual tokens based on feature similarity includes: The similarity between the feature vectors of the remaining visual tokens in the initial visual token sequence, excluding the key visual token, is calculated to obtain the first similarity. The second similarity score is obtained by calculating the similarity between the key visual token and the other visual tokens. By combining the first similarity score and the second similarity score, the complementary token score is obtained; The top k visual tokens are selected from the remaining visual tokens based on their complementary token scores, and these are used as complementary visual tokens.

[0028] In one embodiment, the pooling calculation is global average pooling.

[0029] In one embodiment, the present invention provides a multimodal large language model reasoning method, such as... Figure 2 As shown, it includes the following steps: The visual token sequence and text token sequence obtained by the above method are input into the multimodal large language model for inference; The multimodal large language model integrates visual information from visual token sequences with textual information from text token sequences to obtain inference results.

[0030] In one embodiment, the multimodal large language model is constructed using the following steps: Build a visual encoder to receive visual input, including images or videos; Build a large language model to receive text input; By combining a visual encoder with a large language model, a multimodal large language model is formed.

[0031] In one embodiment, this invention proposes a visual token pruning method for a multimodal large language model based on semantic complementarity, comprising six parts: multimodal dataset construction, multimodal large model construction, important token selection, complementary token selection, token fusion, and model testing. In the multimodal dataset construction stage, target data from different scenarios is collected, and the data is labeled with question-answer pairs. In the model construction stage, a visual model and a large language model are mainly constructed. In the important token selection stage, the importance of visual tokens is determined by the multi-head attention score of the visual model. In the complementary token selection stage, the semantic complementarity of tokens is determined by calculating the feature similarity between tokens. In the token fusion stage, all visual tokens are fused into a single visual token using global average pooling to reduce the loss of visual information. In the model testing stage, the previously retained important and complementary tokens are concatenated and added to the fused token, and finally fed into the large language model to achieve multimodal understanding. In this invention, the important token selection based on multi-head attention effectively preserves key visual information; in addition, the complementary token selection based on similarity proposed in this invention effectively selects semantically complementary visual information, ensuring information diversity; furthermore, the token fusion strategy effectively preserves the complete semantics of the vision by fusing visual information.

[0032] In one embodiment, the present invention provides a visual token pruning method for a multimodal large language model based on semantic complementarity, comprising the following steps: S1. Construct a visual question-answering dataset. For different question-answering scenarios, use a camera to capture images or videos of multiple scenes to obtain visual data. Then, ask questions by viewing the images or videos, and combine the questions with multiple people's answers to form the final visual question-answering dataset.

[0033] S2. Model Building. Building the visual encoder. It is used to receive visual input, including images or videos, and to build large language models. It is used to receive text input. The visual model and the text model are combined to form the final multimodal large language model.

[0034] S3. Key visual token selection.

[0035] Image encoding: Visual images or video frames are embedded into an initial sequence of visual tokens and then input into the visual encoder. middle; Multi-head attention scoring: The importance of a visual token is quantified by calculating the attention score of each visual token in each attention head of the visual encoder and averaging its scores across all attention heads. Aggregation based on multi-head attention scores provides a more stable and comprehensive evaluation of the importance of visual tokens. Top-k selection: After obtaining the attention score for each visual token, the top k visual tokens are selected from high to low scores as key visual tokens to retain the part that contributes the most to visual semantics, thus achieving efficient and informative token selection.

[0036] S4. Complementary visual token selection.

[0037] Similarity calculation: First, based on the initial visual token sequence, the similarity of the feature vectors among the remaining visual tokens (excluding the key visual token) is calculated to quantify their semantic association. Then, the similarity between the key token and the remaining visual tokens is calculated. By combining these two types of similarity, the complementary token score for each of the remaining visual tokens can be obtained.

[0038] Top-k selection: After obtaining the complementary token scores, select the top k visual tokens from the remaining visual tokens according to the scores from low to high. These are visual tokens that are complementary to the key visual tokens in S3. The aim is to retain diverse visual information that is complementary to the key regions and improve the model's ability to express global semantics.

[0039] S5. Token Fusion.

[0040] Visual feature extraction: Inputting visual information such as visual images or video frames into the visual encoder. Visual features are obtained from them; Pooling computation: The extracted visual features are subjected to global average pooling, that is, all visual feature vectors are fused to obtain a fused token.

[0041] S6. Model Testing.

[0042] Forming the visual token sequence: The key visual tokens and complementary visual tokens mentioned above are concatenated, and then a fusion token obtained through pooling calculation is added to form the final visual token sequence. Here, concatenation refers to connecting multiple tokens along the sequence dimension to form a longer token sequence; addition refers to adding multiple tokens element by element while keeping the original dimensions unchanged.

[0043] Multimodal large language model inference: The visual token sequence obtained by concatenation and the text token sequence obtained by text input encoding are input together into the large language model for inference. The model generates the final output based on the fusion of visual and text information, realizing rapid understanding and inference of multimodal information.

[0044] In one embodiment, the visual question-answering dataset is constructed through a visual question-answering acquisition and annotation process, including two stages: acquiring visual images or videos and question-answer annotation. First, image samples are acquired using a public image set, and each image is annotated. Generate the corresponding natural language question Forming a set of problems ,in First, determine the number of questions; second, for each question, collect multiple candidate answers and determine the final answer using a majority consensus rule. Its calculation form is: in, The number of candidate answers. It is an exponential function. For the predicted answer, This represents the set of true answers. Ultimately, the multimodal data sample, consisting of images and their corresponding question-answer pairs, can be formally represented as: The multimodal data samples are used for fast inference in the multimodal large language model.

[0045] In one embodiment, the multimodal large language model is derived from a visual encoder. With large language models Composition; visual encoder Input image or video frame Convert to visual token sequence Text input The resulting text token sequence is encoded. Finally, the visual token sequence and the text token sequence are concatenated in order and then input into the large language model. This forms a multimodal large language model, whose formal representation is as follows: in This indicates the concatenation of a visual token sequence and a text token sequence.

[0046] In one embodiment, the selection of the key visual token is determined based on the multi-head attention score of the visual model. This is first achieved through the visual encoder. Multi-head self-attention calculates the average attention score for each visual token: in, Indicates the first Each attention head's attention value, To focus on the number of attention heads, the K visual tokens with the highest average importance scores are then selected to form the key visual token set. This is done to preserve the most representative key areas in the image.

[0047] In one embodiment, after the key visual token selection is completed, the complementarity of each token to the key region is evaluated by calculating the semantic similarity between the key visual token and the remaining visual tokens: in, Cosine similarity representing token features. and These represent the remaining visual tokens and the key visual token, respectively. This metric measures the semantic overlap between the remaining visual tokens and the selected key visual token; a lower value indicates stronger semantic complementarity.

[0048] Then, calculate the minimum similarity within the set of the remaining visual tokens: in, and This represents the distinct tokens within the set of the remaining visual tokens. This similarity reflects the uniqueness of a token within the remaining set; a lower value indicates that its semantic information is less overridden by other tokens.

[0049] Subsequently, combining these two similarity metrics, the complementary token score is defined: Finally, based on the obtained complementary token scores Perform a Top-K selection, choosing the top K visual tokens from lowest to highest to obtain a set of complementary visual tokens. .

[0050] In one embodiment, the initial token sequence output by the visual encoder is used. Perform global average pooling to generate a single fused visual token. : in For the i-th visual token, the visual tokens are merged through this pooling operation. It preserves all the information in the image and reduces information loss.

[0051] In one embodiment, during the model testing and inference phase, the aforementioned key visual tokens and complementary visual tokens are concatenated and pooled to obtain a fused visual token, forming the final visual token sequence: Subsequently, the visual token sequence and the text token sequence obtained by encoding the text input are input together into the large language model. Inference is performed within the model. This method enables the model to rapidly understand and infer multimodal information by fusing visual and textual information.

[0052] To verify the effectiveness of the proposed technology, performance tests were conducted on the multimodal visual question answering datasets POPE, SQA, VizWiz, and the video understanding datasets MSVD-QA and TGIF-QA. Specifically, the test sets of the datasets were extracted, and then the multimodal large language model was tested in the order described above. The experimental results are shown in Tables 1 and 2.

[0053] Table 1 Comparison of Test Performance Table 2 Comparison of inference time on the POPE dataset As shown in Tables 1 and 2, the method of this invention achieves performance improvements of 21.7%, 1.9%, and 1.4% on the public visual question answering datasets POPE, SQA, and VizWiz, and 6.1% and 4.0% on the video understanding datasets MSVD-QA and TGIF-QA, respectively, compared to the comparative methods, with an average performance improvement of 7.1%. Furthermore, it reduces inference time by 41.6% compared to the basic large model, using only 8.18 minutes. This demonstrates that the proposed visual token pruning method for multimodal large language models based on semantic complementarity can significantly improve the inference speed of the model and bring about significant performance improvements.

[0054] Another embodiment of the present invention provides a visual token clipping device based on semantic complementarity, such as Figure 3 As shown, it includes: The image encoding module is used to encode the image using a visual encoder to obtain an initial visual token sequence; The key visual token selection module is used to select key visual tokens from an initial sequence of visual tokens based on the multi-head attention score of the visual encoder. A complementary visual token selection module is used to select complementary visual tokens of key visual tokens from an initial sequence of visual tokens based on feature similarity. The token fusion module is used to perform pooling calculations on the initial visual token sequence to obtain a fused visual token; The token concatenation module is used to concatenate key visual tokens and complementary visual tokens and add a fused visual token to obtain the final visual token sequence.

[0055] Another embodiment of the present invention provides a multimodal large language model inference device, such as... Figure 4 As shown, it includes: The token acquisition module is used to acquire the visual token sequence and text token sequence obtained by the above method of the present invention, and input them into the multimodal large language model for inference; The multimodal large language model module is used to fuse visual information from the visual token sequence and textual information from the text token sequence to obtain inference results.

[0056] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be referred to the corresponding process in the aforementioned method embodiments, and will not be repeated here. Each of the above modules can be implemented entirely or partially through software, hardware, or a combination thereof.

[0057] Another embodiment of the present invention provides a computer device (computer, server, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0058] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0059] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0060] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual token clipping method based on semantic complementarity, characterized in that, Includes the following steps: The image is encoded using a visual encoder to obtain an initial sequence of visual tokens; Based on the multi-head attention score of the visual encoder, key visual tokens are selected from the initial sequence of visual tokens; Based on feature similarity, complementary visual tokens of key visual tokens are selected from the initial visual token sequence; The initial visual token sequence is pooled to obtain the fused visual token; The pooling calculation is global average pooling; The key visual tokens and complementary visual tokens are concatenated and a fused visual token is added to obtain the final visual token sequence. Concatenation refers to joining multiple tokens along the sequence dimension to form a longer token sequence; addition refers to adding multiple tokens element by element while keeping the original dimensions unchanged.

2. The method according to claim 1, characterized in that, The multi-head attention score based on the visual encoder selects key visual tokens from the initial sequence of visual tokens, including: The attention score for each visual token is calculated in each attention head of the visual encoder, and the scores of the visual tokens in all attention heads are averaged to obtain the multi-head attention score for each visual token. The top k visual tokens, ranked from highest to lowest based on multi-head attention scores, are selected as key visual tokens.

3. The method according to claim 1, characterized in that, The step of selecting complementary visual tokens for key visual tokens from an initial visual token sequence based on feature similarity includes: The similarity between the feature vectors of the remaining visual tokens in the initial visual token sequence, excluding the key visual token, is calculated to obtain the first similarity. The second similarity score is obtained by calculating the similarity between the key visual token and the other visual tokens. By combining the first similarity score and the second similarity score, the complementary token score is obtained; The top k visual tokens are selected from the remaining visual tokens based on their complementary token scores, and these are used as complementary visual tokens.

4. A multimodal large language model reasoning method, characterized in that, Includes the following steps: The visual token sequence and text token sequence obtained by the method described in any one of claims 1 to 3 are input into the multimodal large language model for inference; The multimodal large language model integrates visual information from visual token sequences with textual information from text token sequences to obtain inference results.

5. The method according to claim 4, characterized in that, The multimodal large language model is constructed using the following steps: Build a visual encoder to receive visual input, including images or videos; Build a large language model to receive text input; By combining a visual encoder with a large language model, a multimodal large language model is formed.

6. A visual token clipping device based on semantic complementarity, characterized in that, The apparatus for performing the method according to any one of claims 1 to 3 comprises: The image encoding module is used to encode the image using a visual encoder to obtain an initial visual token sequence; The key visual token selection module is used to select key visual tokens from an initial sequence of visual tokens based on the multi-head attention score of the visual encoder. A complementary visual token selection module is used to select complementary visual tokens of key visual tokens from an initial sequence of visual tokens based on feature similarity. The token fusion module is used to perform pooling calculations on the initial visual token sequence to obtain a fused visual token; The token concatenation module is used to concatenate key visual tokens and complementary visual tokens and add a fused visual token to obtain the final visual token sequence.

7. A multimodal large language model reasoning device, characterized in that, include: The token acquisition module is used to acquire the visual token sequence and text token sequence obtained by the method of any one of claims 1 to 3, and input them into the multimodal large language model for inference; The multimodal large language model module is used to fuse visual information from the visual token sequence and textual information from the text token sequence to obtain inference results.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 5.

Images