Multi-modal large language model training and visual task processing method and device, and medium

Abstract

The invention relates to a multi-modal large language model training and visual task processing method, equipment and a medium. The model training method comprises the following steps: constructing a visual task instruction fine tuning scheme of a multi-modal large language model; aiming at different visual tasks, constructing a multi-modal question and answer data set by utilizing an existing data set based on a visual task instruction fine tuning scheme; and constructing a multi-modal big language model supporting understanding dialogues of multiple user visual interaction modes and visual feature decoding, and training the multi-modal big language model by using a multi-modal question and answer data set. Compared with the prior art, the method has the advantages of capability of simultaneously processing multi-vision tasks, high reasoning speed, low training cost, capability of supporting task expansion and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a multimodal large language model training and visual task processing method, device, and medium. Background Art

[0002] Large multimodal models have brought breakthrough progress to various advanced visual understanding tasks by compressing image and language information into a single autoregressive model based on the Transformer architecture. In existing work, multimodal large language models similar to the LLAVA architecture build linear layers to achieve the mapping relationship between image and text, so that large language models can achieve image understanding; multimodal large models such as Shikra, Ferret, and Kosmos-2 use technologies similar to Pix2Seq to convert image position information into sequence information by adding new words from the existing vocabulary or directly inputting text in a formal manner, thereby achieving the function of target detection; multimodal large models such as GLaMM, PSALM, and GroundHog use multiple visual encoders and use the SAM model or Mask2Former architecture as a decoder to achieve the function of image segmentation based on image understanding. However, existing large multimodal models can often only achieve one downstream visual task (target detection or image segmentation) based on image understanding, or can only call various existing downstream models through the agent capabilities of the large model itself. They lack end-to-end training and implementation capabilities, and lack an explicit and unified visual representation, making it difficult to achieve the integration of multiple visual tasks. Summary of the invention

[0003] The purpose of the present invention is to provide a multimodal large language model training and visual task processing method, device, and medium. By constructing a new data logic, a set of explicit visual representations is formed on the model side, so that the multimodal large language model can simultaneously realize image description (Image captioning), visual question answering (Visual question answering, VQA), object detection (Object detection), referring expression generation (Referring Expression Generation, REG), referring expression comprehension (Referring Expression Comprehension, REC), referring expression segmentation (Referring Expression Segmentation, RES), instance segmentation (Instance Segmentation), conversation and positioning detection (Grounded Conversation Generation Detection, GCG Detection), conversation and positioning segmentation (Grounded Conversation Generation Segmentation, GCG Segmentation), key point detection (Keypoint Detection), monocular depth estimation (Depth Estimation) and other functions.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] A multimodal large language model training method for visual task processing comprises the following steps:

[0006] A scheme for fine-tuning visual task instructions to build a multimodal large language model;

[0007] For different visual tasks, we use existing datasets to construct multimodal question-answering datasets based on visual task instruction fine-tuning schemes.

[0008] A multimodal large language model that supports multiple user visual interaction modes and understands dialogues and decodes visual features is constructed, and the multimodal large language model is trained using a multimodal question-answering dataset.

[0009] The visual task instruction fine-tuning scheme for building a multimodal large language model includes the following steps:

[0010] Construct different visual tasks, and design multiple question templates with different expressions for each task based on GPT-4. During the construction of the corresponding task data, randomly select questions from the templates to construct conversations. The visual tasks include image description, visual question answering, referential expression understanding, referential expression generation, referential expression segmentation, and dialogue and localization;

[0011] Design special words for the visual prompts input by the user for reference, and set that the special words are not tokenized in the text encoder;

[0012] Construct a visual decoding thought chain for generating thought chain content to further generate decoded text based on the thought chain content. The generated thought chain content includes at least one or more of phrases, units, and quantities;

[0013] Construct a general and extensible model answer scheme for the visual decoding task. Specifically, in the model answer scheme, the categories predicted by the model are wrapped before and after with "<phrase>" and "< / phrase>", and the model decoding types are wrapped with "<unit>" and "< / unit>"; <ref>” refers to the target of model decoding and is used for subsequent decoding.

[0014] The construction of the multimodal question-answering dataset is specifically as follows:

[0015] In the image description task, the question of the corresponding task is selected from the constructed question template, and the annotation of the image content in the existing image description dataset is used as the answer to form an image description question-answering dialogue;

[0016] In the visual question answering task, question and answer pairs in the existing visual question answering dataset are used to form an image-based visual question answering dialogue;

[0017] In the referential expression understanding task, the corresponding question is selected from the constructed question template, the user visual input is encoded using the designed special words, and the annotation of the image content in the existing referential expression understanding dataset is used as the answer to form an image description question-answering dialogue;

[0018] In the referential expression segmentation task, the question corresponding to the task is selected from the constructed question template, the designed special words are used to encode the user's visual input, and the constructed model answer scheme is used to generate answers based on the existing referential expression segmentation dataset, forming an image segmentation content question-answering dialogue;

[0019] In the referential expression generation task, the question corresponding to the task is selected from the constructed question template, the designed special words are used to encode the user's visual input, and the constructed model answer scheme is used to generate answers based on the existing referential expression generation dataset to form an image object detection content question-answering dialogue;

[0020] In the dialogue and positioning tasks, questions corresponding to the tasks are selected from the constructed question templates, and the constructed model answering scheme is used to generate answers based on the existing dialogue and positioning datasets to form image dialogue and positioning question and answer.

[0021] The multimodal large language model includes a visual encoder, a visual cue encoder, a visual feature mapper, a basic language model, a visual representation optimizer, and a visual content decoder, wherein:

[0022] The visual encoder adopts the CLIP-VIT model, and obtains global visual features by uniformly scaling the input image to a preset size and encoding it;

[0023] The visual cue encoder uniformly converts different types of visual cue inputs into masks of different shapes, pools the masks into the same size as the global visual features, and performs point-by-point multiplication operations with the global visual features;

[0024] The visual feature mapper maps the features output by the visual encoder and the visual cue encoder to the same dimension as the multimodal large language model text features to perform multimodal information alignment;

[0025] The basic language model adopts the Vicuna-7B model;

[0026] The visual representation optimizer optimizes the global visual features obtained in the visual encoder and reconstructs the decoded area of ​​the current task by introducing a learnable query token at the end of the question to restore large-scale details.

[0027] The visual content decoder adopts different designs for different visual tasks, and decodes the decoding target in the model decoding scheme by combining the global visual features and the visual representation optimized by the visual representation optimizer.

[0028] The training of the multimodal large language model is divided into two stages, wherein the first stage uses text and image data to train the basic language model and the visual feature mapper for aligning visual and text features; the second stage trains all models in the multimodal large language model except the visual encoder.

[0029] A visual task processing method based on a multimodal large language model comprises the following steps:

[0030] Based on the model training method as described above, a trained multimodal large language model is obtained;

[0031] Obtain visual task information input by the user;

[0032] Use the trained multimodal large language model to process visual tasks and output the processing results.

[0033] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the model training method described above when executing the program.

[0034] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the visual task processing method as described above when executing the program.

[0035] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the model training method as described above.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the visual task processing method as described above.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] (1) The present invention can integrate multiple common visual tasks, such as detection, segmentation, key point detection, depth estimation, etc., through a large language model, and supports low-cost task expansion.

[0039] (2) The present invention constructs a large number of multimodal question-answering datasets that can be directly used, and the data can be rapidly and continuously expanded according to the construction scheme.

[0040] (3) The present invention optimizes the performance of visual representation, encoding and other processes, making the model reasoning faster and the training cost lower. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2 It is a schematic diagram of the structure of a multimodal large language model of the present invention;

[0043] Figure 3 It is a schematic diagram of the structure of the visual cue encoder of the present invention;

[0044] Figure 4 It is a schematic diagram of the structure of the visual representation optimizer of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] This embodiment provides a multimodal large language model training method for visual task processing, such as Figure 1 As shown, the following steps are included:

[0047] S1, a fine-tuning scheme for visual task instructions to build a multimodal large language model.

[0048] This embodiment constructs an instruction tuning scheme that ensures that a large multimodal language model can achieve common visual tasks such as detection, segmentation, key point detection, and depth map estimation during a conversation, and supports extension to other visual-language multimodal tasks. Specifically, the following steps are included:

[0049] S101, constructs a variety of different visual tasks such as image description, visual question answering, referring expression comprehension (REC), referring expression generation (REG), referring expression segmentation (RES), and grounded conversation generation (GCG). Based on GPT-4, 100-150 question templates with different expressions are designed for each task. In the process of constructing the corresponding task data, questions are randomly extracted from the templates to construct dialogues.

[0050] S102, a special token "[VPT]" is designed to refer to the visual prompts input by the user, such as a point, a target box, a mask, and a scribble, and "[VPT]" is set in the text encoder as a separate token without further word segmentation.

[0051] S103, constructing a visual decoding thought chain. According to the current problem, the model needs to first generate the thought chain content format, and then generate the specific decoding text based on the content. The thought chain content that the model needs to generate includes at least one or more of phrases, units, and quantities.

[0052] In one embodiment, the thought chain of the instance segmentation task is as follows:

[0053] <task>

[0054] Unit decode(True).Class name,target unit and number:

[0055] -Name:a small stone building Unit: <unit> mask< / unit> Num:1

[0056] -Name:a serene japanese garden with a rock garden Unit: <unit> mask< / unit> Num:1

[0057] < / task>

[0058] S104, build a general and scalable model answer solution for visual decoding tasks such as Box, Mask, Keypoint, Depth Map, etc. Specifically, design " <phrase> "and"< / phrase> "Classes for model predictions for front and back wrapping," <unit> "and"< / unit> "Used to wrap model decoding type, " <ref>" is used to refer to the target of model decoding and is used for subsequent decoding. At this time, the answer example of the instance segmentation task can be expressed as: <phrase> a small stone building< / phrase> ( <unit> mask< / unit> [0] <ref> ), <phrase> a serene japanese garden with arock garden< / phrase> ( <unit> mask< / unit> [0] <ref>).

[0059] S2, for different visual tasks, we use existing datasets to construct a multimodal question-answering dataset based on a visual task instruction fine-tuning scheme.

[0060] Specifically:

[0061] S201, in the image description task, use COCO2014, llava-pretrain, llava-instruct, Grand caption, GRIT caption and other datasets as existing image description datasets, select the question of the corresponding task from the question template constructed in step S101, and use the annotation of the image content in the existing image description dataset as the answer to form an image description question-answering dialogue.

[0062] S202, in the visual question answering task, use PointQA, VQAv2, VQAE, and VQAX datasets as existing visual question answering datasets, and adopt question and answer pairs in the existing visual question answering datasets to form an image-based visual question answering dialogue.

[0063] S203, in the referential expression understanding task, use RefCOCO, RefCOCO+, RefCOCOg, GRIT REC, Grand REC, COCO Interact Box and other datasets as existing referential expression understanding datasets, select the question of the corresponding task from the question template constructed in step S101, use the special words designed in step S102 to encode the user's visual input, and use the annotations of the image content in the existing referential expression understanding dataset as answers to form an image description question-answering dialogue.

[0064] S204, in the referential expression segmentation task, use Grand RES, COCO Interact Mask and other datasets as existing referential expression segmentation datasets, select the question of the corresponding task from the question template constructed in step S101, use the special words designed in step S102 to encode the user's visual input, and use the model answer scheme constructed in step S104 to generate an answer based on the existing referential expression segmentation dataset, forming an image segmentation content question and answer dialogue.

[0065] S205, in the referential expression generation task, use GRIT REG, Grand REG, LLAVAg REG, COCOInteract REG, RefCOCO, RefCOCO+, RefCOCOg and other datasets as existing referential expression generation datasets, select the question of the corresponding task from the question template constructed in step S101, use the special words designed in step S102 to encode the user visual input, and use the model answer scheme constructed in step S104 to generate an answer based on the existing referential expression generation dataset, thereby forming an image object detection content question and answer dialogue.

[0066] S206, in the dialogue and positioning task, use LLAVAg GCG, PNG, GRIT GCG, Grand GCG and other datasets as existing dialogue and positioning datasets, select the question of the corresponding task from the question template constructed in step S101, use the model answer scheme constructed in step S104 to generate answers according to the existing dialogue and positioning datasets, and constitute image dialogue and positioning question and answer.

[0067] S3, build a multimodal large language model that can support understanding dialogues in various user visual interaction modes and decoding of visual features such as bounding boxes, masks, key points, depth maps, etc., and use a multimodal question-answering dataset to train the multimodal large language model.

[0068] like Figure 2 As shown, the multimodal large language model includes a visual encoder, a visual cue encoder, a visual feature mapper, a basic language model, a visual representation optimizer, and a visual content decoder.

[0069] 1. Visual encoder: This embodiment uses the CLIP-VIT model as the visual encoder. By uniformly scaling the input image to a size of 336*336, encoding is performed to obtain a global visual feature with a shape of [576,1024], where 576 represents the feature length, which is equivalent to flattening the 24*24 image feature.

[0070] 2. Visual cue encoder: For different types of visual cue inputs, such as point, target box, mask, and scribble, the visual cue encoder uniformly converts them into masks of different shapes. Its structure is as follows: Figure 3 As shown in the figure, for Point, it is converted into a circular mask with a radius of 10 and a point as the center; for Scribble, it is converted into a strip mask with a width of 10 and an unchanged shape; for Box, it is converted into a rectangular mask of the same area. The mask patch pooling scheme is adopted to pool the mask into the same size as the global visual feature, that is, 24*24, and multiply it with the global visual feature point by point.

[0071] 3. Visual feature mapper: The visual feature mapper maps the features output by the visual encoder and the visual cue encoder to the same dimension as the text features of the multimodal large language model to align the multimodal information.

[0072] 4. Basic language model: In this embodiment, the basic language model adopts the Vicuna-7B model.

[0073] 5. Visual representation optimizer: The global features obtained in the visual encoder may cause loss of detail information for fine visual decoding tasks (such as Mask, Depth Map, etc.). This embodiment uses a visual representation optimizer to optimize the global visual features obtained in the visual encoder. By introducing a learnable query token at the end of the problem, the decoding area of ​​the current task is reconstructed to restore large-scale details. Its structure is as follows: Figure 4 shown.

[0074] 6. Visual content decoder: For different visual tasks, the visual content decoder adopts different designs and uniformly uses the query token generated by the model as a learnable query token for subsequent decoding. The visual content decoder decodes the decoding target in the model decoding scheme by combining the global visual features and the visual representation optimized by the visual representation optimizer. For the Box and Keypoint decoding tasks, the visual content decoder uses Detr-Decoder, takes CLIP-VIT features as visual features, and fuses the visual representation optimized by the visual representation optimizer to decode the " <ref>" is decoded; for Mask and DepthMap decoding tasks, the visual content decoder uses MaskFormer-Decoder, takes CLIP-VIT features as visual features, and fuses the visual representation optimized by the visual representation optimizer to decode" <ref>" to decode.

[0075] The training of the multimodal large language model is divided into two stages. In the first stage, text and image data are used to train the basic language model and visual feature mapper to align visual and text features. In the second stage, all models in the multimodal large language model except the visual encoder are trained. Through the data set constructed in step S2, it is ensured that the large language model has multimodal understanding capabilities and responds to the visual decoding task in a preset paradigm.

[0076] This embodiment also provides a visual task processing method based on a multimodal large language model, comprising the following steps:

[0077] Based on the model training method as described above, a trained multimodal large language model is obtained;

[0078] Obtain visual task information input by the user;

[0079] Use the trained multimodal large language model to process visual tasks and output the processing results.

[0080] In one embodiment, the electronic device includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0081] Many components in an electronic device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0082] The computing unit may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing units include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit performs the various methods and processes described above, such as the above-mentioned model training method / visual task processing method. For example, in some embodiments, the above-mentioned model training method / visual task processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the computing unit, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit may be configured to perform the above-mentioned method in any other appropriate manner (e.g., by means of firmware).

[0083] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0084] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0085] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0086] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.< / ref> < / ref> < / ref> < / ref> < / ref> < / ref>

Claims

1. A multimodal large language model training method for visual task processing, characterized in that: The following steps are involved: A scheme for fine-tuning visual task instructions to build a multimodal large language model; For different visual tasks, we use existing datasets to construct multimodal question-answering datasets based on visual task instruction fine-tuning schemes. A multimodal large language model that supports multiple user visual interaction modes and understands dialogues and decodes visual features is constructed, and the multimodal large language model is trained using a multimodal question-answering dataset.

2. The scalable visual task processing method based on a multimodal large language model according to claim 1, characterized in that: The visual task instruction fine-tuning scheme for building a multimodal large language model includes the following steps: Construct different visual tasks, and design multiple question templates with different expressions for each task based on GPT-4. During the construction of the corresponding task data, randomly extract questions from the templates to construct dialogues. The visual tasks include image description, visual question answering, referring expression understanding, referring expression generation, referring expression segmentation, and dialogue and positioning. Designing a special word to refer to the visual prompt input by the user, and setting the special word not to be segmented in the text encoder; Constructing a visual decoding thought chain to generate thought chain content to further generate a decoding text according to the thought chain content, wherein the generated thought chain content includes at least one or more of phrases, units and quantities; Build a general and scalable model answering solution for visual decoding tasks. Specifically, in the model answering solution, the categories predicted by the model are wrapped by "<phrase>" and "< / phrase>", the model decoding types are wrapped by "<unit>" and "< / unit>", with " <ref> ” refers to the target of model decoding and is used for subsequent decoding.< / ref> 3. The scalable visual task processing method based on a multimodal large language model according to claim 2, characterized in that: The construction of the multimodal question-answering dataset is specifically as follows: In the image description task, the question of the corresponding task is selected from the constructed question template, and the annotation of the image content in the existing image description dataset is used as the answer to form an image description question-answering dialogue; In the visual question answering task, question and answer pairs in the existing visual question answering dataset are used to form an image-based visual question answering dialogue; In the referential expression understanding task, the corresponding question is selected from the constructed question template, the user visual input is encoded using the designed special words, and the annotation of the image content in the existing referential expression understanding dataset is used as the answer to form an image description question-answering dialogue; In the referential expression segmentation task, the question corresponding to the task is selected from the constructed question template, the designed special words are used to encode the user's visual input, and the constructed model answer scheme is used to generate answers based on the existing referential expression segmentation dataset, forming an image segmentation content question-answering dialogue; In the referential expression generation task, the question corresponding to the task is selected from the constructed question template, the designed special words are used to encode the user's visual input, and the constructed model answer scheme is used to generate answers based on the existing referential expression generation dataset to form an image object detection content question-answering dialogue; In the dialogue and positioning tasks, questions corresponding to the tasks are selected from the constructed question templates, and the constructed model answering scheme is used to generate answers based on the existing dialogue and positioning datasets to form image dialogue and positioning question and answer.

4. The scalable visual task processing method based on a multimodal large language model according to claim 1, characterized in that: The multimodal large language model includes a visual encoder, a visual cue encoder, a visual feature mapper, a basic language model, a visual representation optimizer, and a visual content decoder, wherein: The visual encoder adopts the CLIP-VIT model, and obtains global visual features by uniformly scaling the input image to a preset size and encoding it; The visual cue encoder uniformly converts different types of visual cue inputs into masks of different shapes, pools the masks into the same size as the global visual features, and performs point-by-point multiplication operations with the global visual features; The visual feature mapper maps the features output by the visual encoder and the visual cue encoder to the same dimension as the multimodal large language model text features to perform multimodal information alignment; The basic language model adopts the Vicuna-7B model; The visual representation optimizer optimizes the global visual features obtained in the visual encoder and reconstructs the decoded area of ​​the current task by introducing a learnable query token at the end of the question to restore large-scale details. The visual content decoder adopts different designs for different visual tasks, and decodes the decoding target in the model decoding scheme by combining the global visual features and the visual representation optimized by the visual representation optimizer.

5. The scalable visual task processing method based on a multimodal large language model according to claim 4, characterized in that: The training of the multimodal large language model is divided into two stages, wherein the first stage uses text and image data to train the basic language model and the visual feature mapper for aligning visual and text features; the second stage trains all models in the multimodal large language model except the visual encoder.

6. A visual task processing method based on a multimodal large language model, characterized in that: The following steps are involved: Obtaining a trained multimodal large language model based on the method described in any one of claims 1 to 5; Obtain visual task information input by the user; Use the trained multimodal large language model to process visual tasks and output the processing results.

7. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method as claimed in claim 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method as claimed in claim 6 is implemented.

Images