Data generation method and task platform

Abstract

The embodiment of the invention provides a data generation method and a task platform, and the data generation method comprises the steps: obtaining a target image and seed question and answer information corresponding to the target image, and obtaining image description information corresponding to the target image according to the target image; according to the target image, the image description information, the seed question and answer information and a preset prompt text template, a question and answer prompt text is generated, the preset prompt text template comprises a prompt evolution type, and the prompt evolution type is used for increasing question and answer diversity; and inputting the question and answer prompt text into a question and answer prediction model to obtain predicted question and answer information for the target image output by the question and answer prediction model, the predicted question and answer information comprising a predicted question and a predicted answer corresponding to the predicted question. Through the method, the problems of high manual annotation cost, uneven quality and simple model generation data in a current processing mode are solved, and the quality of predicting question and answer information is improved.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a data generation method. Background Technology

[0002] With the development of computer technology, multimodal large language models have also made great strides. The quality of image and text instruction data plays an important role in the ability of multimodal large language models to perform multiple visual language tasks.

[0003] The current text-image instruction data suffers from problems of limited quantity and low quality, which has become a bottleneck in the development of multimodal large language models. The current methods for obtaining text-image instruction data include manual annotation and model generation. Manually annotated data is costly and of inconsistent quality, while model-generated data is affected by shallow inference and is relatively simple, failing to guarantee the complexity and richness of text-image instruction data. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data generation method. One or more embodiments of this specification also relate to a task platform, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data generation method is provided, comprising: Obtain the target image and the seed question-and-answer information corresponding to the target image, and obtain the image description information corresponding to the target image based on the target image; The question-and-answer prompt text is generated based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template. The preset prompt text template includes a prompt evolution type, which is used to increase the diversity of questions and answers. The question-and-answer prompt text is input into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question.

[0006] According to a second aspect of the embodiments of this specification, a data generation method is provided, applied to a cloud-side device, comprising: The receiving end device sends a target image and seed question-and-answer information corresponding to the target image, and obtains image description information corresponding to the target image based on the target image; The question-and-answer prompt text is generated based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template. The preset prompt text template includes a prompt evolution type, which is used to increase the diversity of questions and answers. The question-and-answer prompt text is input into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question; The predicted question-and-answer information is sent to the edge device.

[0007] According to a third aspect of the embodiments of this specification, a task platform is provided, including a request interface and a response unit; The request interface is used to receive the target image and the seed question and answer information corresponding to the target image sent by the end device, and to obtain the image description information corresponding to the target image based on the target image; The response unit is configured to generate question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and a preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, which is used to increase question-and-answer diversity; input the question-and-answer prompt text into a question-and-answer prediction model to obtain predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes a predicted question and a predicted answer corresponding to the predicted question.

[0008] According to a fourth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above-described data generation method.

[0009] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the data generation method described above.

[0010] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the data generation method described above.

[0011] One embodiment of this specification provides a data generation method, comprising: acquiring a target image and seed question-and-answer information corresponding to the target image, and acquiring image description information corresponding to the target image based on the target image; generating question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and a preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, the prompt evolution type being used to increase question-and-answer diversity; inputting the question-and-answer prompt text into a question-and-answer prediction model to obtain predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes a predicted question and a predicted answer corresponding to the predicted question.

[0012] The method provided in the embodiments of this specification, in scenarios where prompt evolution types are used to predict questions and answers, combined with a question-and-answer prediction model, can significantly improve the diversity and complexity of data by leveraging the capabilities of the question-and-answer prediction model and the prompt evolution types. This solves the problems of high cost and inconsistent quality of manual annotation in current processing methods, as well as the simple data generated by the model, thereby improving the quality of predicted question-and-answer information. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a data generation method provided in one embodiment of this specification; Figure 2 This is a schematic diagram of a data generation method provided in one embodiment of this specification; Figure 3 This is a flowchart illustrating a data generation method provided in another embodiment of this specification; Figure 4 This is a schematic diagram of a task platform provided in one embodiment of this specification; Figure 5 This is a schematic diagram of the structure of a data generation device provided in one embodiment of this specification; Figure 6 This is an architecture diagram of a data generation system provided in one embodiment of this specification; Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0014] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0015] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0016] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0017] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0018] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0019] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0020] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0021] Multimodal Large Language Models (MLLMs) are complex artificial intelligence systems based on computer vision and natural language processing techniques. They possess a large number of parameters that enable the models to learn and process complex visual-language structures.

[0022] Instruction evolution is a process of progressively improving and optimizing instruction data, aiming to generate more complex, diverse, and efficient instruction formats. By simulating evolutionary mechanisms, instruction evolution can introduce new features, increase data complexity, or enhance reasoning capabilities. This process may include generating new instruction types, optimizing the performance of existing instructions, and eliminating inefficient or harmful instruction data, thereby improving the overall quality and applicability of the dataset.

[0023] The quality of text-based instruction data plays a crucial role in the capabilities of multimodal large language models across various visual language tasks. Existing text-based instruction data suffers from both limited quantity and low quality, becoming a bottleneck in the development of multimodal large language models. Currently, there are two main methods for acquiring text-based instruction data: 1. Manual annotation by human annotators results in inconsistent quality of graphic and textual data instructions due to the influence of the annotator's experience. The forms of graphic and textual data instructions are also limited. Furthermore, since manual annotation makes it difficult to think deeply, the difficulty of the annotated data is mainly distributed at a low to medium level.

[0024] 2. Using black-box commercial large models to generate graphic and text instruction data, however, this type of method is affected by shallow reasoning and tends to generate simple samples in a single generation. At the same time, it is limited by predefined task types, so the complexity of the generated data and the diversity of instruction forms cannot be guaranteed.

[0025] Based on this, a data generation method is provided in this specification. This specification also relates to a task platform, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0026] See Figure 1 , Figure 1 A flowchart of a data generation method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0027] Step 102: Obtain the target image and the seed question-and-answer information corresponding to the target image, and obtain the image description information corresponding to the target image based on the target image.

[0028] The data generation method provided in this specification is a method for generating other question-and-answer information related to a target image based on a target image and the corresponding question-and-answer information using a question-and-answer prediction model. This method requires first obtaining the target image and the corresponding seed question-and-answer information.

[0029] The target image can be understood as the image on which the method provided in this embodiment of the specification is based to generate question-and-answer information, and the seed question-and-answer information can be understood as the existing question-and-answer information for the target image. The question-and-answer information includes a question and the answer corresponding to the question.

[0030] In one specific embodiment provided in this specification, the target image and the corresponding seed question-and-answer information can be uploaded by the user; alternatively, they can be pre-stored in a designated storage location on the terminal, and the target image and seed question-and-answer information can be retrieved from the designated storage location according to a retrieval instruction. The method provided in this specification does not limit the specific method of retrieving the target image and the corresponding seed question-and-answer information; the actual application shall prevail.

[0031] After obtaining the target image, further image description information can be acquired. Image description information can be understood as information that describes the content within the target image. This can include overall descriptions of the target image, local descriptions, and the location information of objects within the image. It is important to note that image description information refers to the objectively existing descriptions of the target image, and does not include descriptive information extended from the target image.

[0032] There are several methods for obtaining the corresponding image description information from a target image. For example, the target image can be input into a pre-trained image recognition model, which then outputs the image description information of the target image. Alternatively, the target image and its corresponding image description information can be pre-saved, so that the image description information can be obtained along with the target image when it is acquired. In the embodiments provided in this specification, the specific process for obtaining the image description information is not limited; the actual application shall prevail.

[0033] In one specific embodiment provided in this specification, the example of a user uploading a target image and its corresponding seed question-and-answer information is used for explanation. The terminal executing this method displays a data upload interface to the user, who uploads the target image and its corresponding seed question-and-answer information on this interface. After acquiring the target image, the terminal inputs it into a pre-trained image recognition model to obtain the image description information output by the image recognition model.

[0034] Step 104: Generate question and answer prompt text based on the target image, the image description information, the seed question and answer information, and the preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, which is used to increase the diversity of question and answer.

[0035] After obtaining the target image, image description information, and seed question-and-answer information, these three data are combined with a preset prompt text template to generate the question-and-answer prompt text.

[0036] Question-and-answer prompt text can be understood as the text that needs to be input into the question-and-answer prediction model for processing. In other words, the prompt text can be seen as a prompt input into the question-and-answer prediction model (also known as the AI ​​large model). In the AI ​​large model, the prompt's role is to provide the model with contextual information about the input and the model's parameters. The prompt helps the large model better understand the intent of the input and respond accordingly, improving the interpretability and accessibility of the large model.

[0037] A preset prompt text template can be understood as a pre-set text template used to generate question-and-answer prompt text. The preset prompt text template pre-sets the positions of the image, image description information, and seed question-and-answer information. After obtaining the target image, image description information, and seed question-and-answer information, these three pieces of information can be filled into the corresponding positions in the preset prompt text template. That is, the target image is filled into the image position in the preset prompt text template, the image description information is filled into the image description information position in the preset prompt text template, and the seed question-and-answer information is filled into the question and answer positions in the preset prompt text template.

[0038] In practical applications, the preset prompt text template also includes a prompt evolution type, which is used to enhance the diversity of question-and-answer information. The prompt evolution type defines the coastal direction expected by the user in this instruction evolution. The question-and-answer prediction model can generate richer and more diverse question-and-answer information based on this prompt evolution type than the seed question-and-answer information. In a specific embodiment provided in this specification, the prompt evolution type includes any one of a cognitive reasoning evolution type, an interactive evolution type, and a fine-grained perceptual evolution type.

[0039] In the method provided in the embodiments of this specification, multiple iterations of the question-answering prediction model are used to automatically generate a large number of question-answering information of various categories. The generated question-answering information is used to improve the performance of the multimodal large language model. Therefore, question-answering information with diverse complexity is required. Instruction evolution is performed based on the target image and seed question-answering information, and the evolution direction is divided into three directions: cognitive reasoning evolution, interactive evolution, and fine-grained perception evolution. Among them, cognitive reasoning evolution aims to improve the reasoning operation steps of the instructions and increase the complexity of the instructions; interactive evolution aims to improve the diversity of instructions and provide a variety of instruction forms; fine-grained perception evolution aims to mine the visual information in the image and provide new instructions involving more fine-grained targets.

[0040] Cognitive reasoning evolution focuses on reasoning ability, which is one of the key functions of multimodal large language models. However, most existing instruction data consists of simple question-and-answer formats, lacking detailed reasoning processes. This makes it difficult for trained models to complete complex tasks requiring reasoning ability, such as multimodal agents and visual reasoning. In the methods provided in the embodiments of this specification, the cognitive reasoning evolution type focuses on generating question-and-answer information that includes detailed reasoning processes.

[0041] The purpose of fine-grained perceptual evolution is to extract as much visual information as possible from the target image, especially those overlooked non-subject visual objects. In current question-answering information, most instruction data, during instruction evolution, tends to construct questions targeting the main object in the image, ignoring non-subject objects, especially those with low frequency of occurrence. This results in almost no instruction data related to objects at the tail of a long-tailed distribution. Training a model with such data can easily lead to visual illusions in the model, and also results in poor generalization and robustness. In the method provided in the embodiments of this specification, fine-grained perceptual evolution focuses on the visual target problem, extracting as much usable and easily overlooked visual information as possible.

[0042] The purpose of interactive evolution is to combine visual and reasoning information to create an interactive experience. Currently, the instruction data has limited instruction formats, making it difficult for trained models to follow complex and diverse user-specified instructions or goals. This limits their practicality and applicability in real-world scenarios. To evolve instruction data with rich task formats and provide a better interactive experience for users, the methods provided in the embodiments of this specification also offer an interactive evolution type that can automatically generate instruction data with rich task formats.

[0043] In the preset prompt text template, these three types of specifications can be achieved through designated prompt parameters. In a specific embodiment provided in this specification, the method further includes: Obtain information on the target evolution requirements; The prompt evolution type is determined based on the target evolution requirement information, and a preset prompt text template is generated based on the prompt evolution type.

[0044] The target evolutionary requirement information can be understood as the expected direction of evolution in the current evolutionary task. After determining the target evolutionary requirement information, the corresponding prompt evolution type can be further determined, and a preset prompt text template can be generated based on the prompt evolution type. For example, if the current goal is to enrich fine-grained perceptual evolution, the target evolutionary requirement information could be "mining visual information"; or, if the current goal is to enrich "reasoning ability," the target evolutionary requirement information could be "enriching reasoning ability."

[0045] In practical applications, to ensure the diversity of generated question-and-answer information and obtain information on the evolving needs of the target, the following are required: Obtain at least one reference evolution requirement; The target evolution requirement information is randomly determined from at least one reference evolution requirement information.

[0046] Specifically, in the method provided in the embodiments of this specification, the aim is to automatically generate question-and-answer information using a question-and-answer prediction model. To ensure the diversity of the generated question-and-answer information, the evolutionary direction of this evolutionary task needs to be specified according to the corresponding evolutionary requirement information. In practical applications, several reference evolutionary requirement information will be preset. For example, reference evolutionary requirement information 1 could be to mine visual information in an image and provide instructions for more fine-grained targets, then its corresponding prompt evolution type is fine-grained perceptual evolution; another example is that reference evolutionary requirement information 2 could be to improve the operational reasoning steps of the instruction and increase the complexity of the instruction, then its corresponding prompt evolution type is reasoning cognitive evolution, and so on.

[0047] After determining multiple reference evolutionary requirement information, the target evolutionary requirement information for the current iteration can be randomly selected from among these reference information. In practical applications, the method provided in this specification's embodiments can generate multiple question-and-answer messages through multiple evolutions. For any given evolutionary process, the target evolutionary requirement information for that process must be determined. The target evolutionary requirement information defines an evolutionary basis for the current processing iteration. Based on the target evolutionary requirement information, a corresponding prompt evolution type is generated, thereby generating a corresponding preset prompt text template.

[0048] In one specific embodiment provided in this specification, generating a preset prompt text template according to the prompt evolution type includes: Obtain a preset evolutionary capability library, wherein the evolutionary capability library includes at least one preset evolutionary capability information; Select at least one target evolutionary capability from at least one preset evolutionary capability information; A preset prompt text template is generated based on the prompt evolution type, the preset evolution capability library, and the evolution capability information of each target.

[0049] The preset evolutionary capability library can be understood as a repository containing multiple preset evolutionary capability information. This preset evolutionary capability information can be understood as the capability information used during data processing of the target image. Specifically, the types of preset evolutionary capability information include image recognition capability and text reasoning capability.

[0050] Image recognition capability refers to the capabilities used in the image recognition process. For example, image recognition capability includes localization capability, referencing capability, computational capability, character recognition capability, and existence judgment capability. Localization capability refers to the ability to locate objects in an image; referencing capability refers to the ability to indicate objects in an image; computational capability refers to the ability to perform calculations between objects in an image; character recognition capability refers to the ability to recognize characters in an image; and existence judgment capability refers to the ability to determine whether a certain object exists in an image.

[0051] Textual reasoning ability refers to language-centric skills, such as the ability to describe relationships, understand scenes, predict behavior, and associate knowledge. Relationship description ability refers to the ability to describe the relationships between objects in an image; scene understanding ability refers to the ability to understand and describe scene information in an image; behavior prediction ability refers to the ability to predict the behavior of objects in an image; and knowledge association ability refers to the ability to connect knowledge and common sense gleaned from images.

[0052] The preset evolution capability library specifies various preset evolution capability information. In the current instruction evolution, one or more of these evolution capability information can be used; that is, at least one target evolution capability information is randomly selected from at least one preset evolution capability information. Since the above-mentioned preset evolution capability information will inevitably be used in the image recognition process, in order to make the instruction evolution more targeted, at least one target evolution capability information is randomly selected from multiple preset evolution capability information. This ensures that the target evolution capability information will definitely be used, but at the same time, it does not restrict the use of other preset evolution capability information.

[0053] Once the target evolution capability information is determined, a preset prompt text template can be generated based on the prompt evolution type, the preset evolution capability library, and the evolution capability information of each target.

[0054] In practical applications, the prompt evolution type specifies the evolution type of the instruction evolution process. However, within the same prompt evolution type, there may be different specific evolution directions. That is, each prompt evolution type corresponds to at least one prompt evolution parameter. For example, the prompt evolution parameters corresponding to the cognitive reasoning evolution type include "complex reasoning," etc.; the prompt evolution parameters corresponding to the fine-grained perception evolution type are "coarse-grained," "standard," and "fine-grained," etc.; and the prompt evolution parameters corresponding to the interactive evolution type include "region selection," "text conversion," "depth order," and "relative distance," etc. The prompt evolution parameters specify the specific evolution content for the prompt evolution type.

[0055] Based on this, in a specific embodiment provided in this specification, a preset prompt text template is generated according to the prompt evolution type, the preset evolution capability library, and the evolution capability information of each target, including: Determine the cue evolution parameters corresponding to the cue evolution type; By combining the aforementioned prompt evolution parameters, the preset evolution capability library, and the evolution capability information of each target, a preset prompt text template is generated.

[0056] After determining the prompt evolution type, the corresponding prompt evolution parameters are further determined. Finally, a preset prompt text template is generated by concatenating the prompt evolution parameters, the preset evolution capability library, and the evolution capability information of each target.

[0057] In one specific embodiment provided in this specification, a preset prompt text template is provided for further explanation. The preset prompt text template includes: Target image; #Image description information 1: ***** (Image description content); #Image description information 2: ***** (Location information of each object in the image); #Preset evolution capability library (including preset evolution capability information); #Object: *** (Object identified from image description information 2); #Skills: Target evolution ability information; #Evolutionary Format: Prompts for evolutionary parameters; #Question: Question information related to the seed Q&A; #Example: *****; #Answer: Answer information for seed question and answer information.

[0058] } The above is an example of a preset prompt text template. The evolution format is determined by the prompt evolution type. Based on the prompt evolution parameters in the evolution format, the corresponding prompt evolution type can be uniquely determined. The prompt evolution type is used to improve the diversity of question-and-answer information. After determining the prompt evolution parameters, the preset evolution capability library, and the information of each target evolution capability, the above information is concatenated in sequence, reserving space for target images, image description information, seed question-and-answer information, etc., to generate the preset prompt text template.

[0059] By filling the target image, image description information, seed question and answer information, etc. into the preset prompt text template, question and answer prompt text can be generated.

[0060] Step 106: Input the question-and-answer prompt text into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question.

[0061] After obtaining the question-and-answer prompt text, it can be input into the question-and-answer prediction model. After processing by the model, predicted question-and-answer information for the target image can be output. Specifically, the predicted question-and-answer information includes the predicted question for the target image and the predicted answer to that question.

[0062] In the method provided in the embodiments of this specification, the question-answering prediction model makes predictions based on the prompt evolution parameters corresponding to the prompt evolution type in the question-answering prompt text, as well as the content in the question-answering prompt text, to generate a prediction question for the target image and a prediction answer to the prediction question. Here, the capability of the question-answering prediction model is used. The question-answering prediction model can be a large language model, a machine learning model, or a deep learning model, and is not limited to this in the embodiments provided in this specification.

[0063] The question-answering prediction model outputs predicted question-answer information for the target image. This predicted question is more complex and diverse than simple seed question-answer information because it is influenced by a preset evolutionary capability library, target evolutionary capability information, and prompt evolutionary parameters.

[0064] In one specific embodiment provided in this specification, it further includes: The predicted question-and-answer information and the target image are input into a question-and-answer information verification model, wherein the question-and-answer information verification model is used to determine whether the predicted answer is correct; Obtain the verification result output by the question-and-answer information verification model; If the verification result is correct, the predicted question-and-answer information is determined to be the target question-and-answer information of the target image.

[0065] In this embodiment, the predicted question-and-answer information is generated by a question-and-answer prediction model. However, in practical applications, question-and-answer prediction models inevitably suffer from model illusion problems. To further improve the accuracy of the generated predicted question-and-answer information, the predicted question-and-answer information and the target image can be input into a question-and-answer information verification model for validation. The question-and-answer information verification model is used to verify whether the predicted answer to the predicted question is correct. In practical applications, the question-and-answer information verification model can be a large language model or other pre-trained machine learning models. To reduce computational resources and processing steps, preferably, the question-and-answer information verification model and the question-and-answer prediction model that generates the predicted question are both large language models.

[0066] The question-and-answer information verification model outputs two results based on the target image and predicted question-and-answer information: a correct verification result and a failed verification result. If the verification result is correct, it means the predicted question-and-answer information is accurate and can be used as the target question-and-answer information for the target image. If the verification result is failed, it means there are incorrect answers in the predicted question-and-answer information, and the predicted question-and-answer information is deleted and discarded.

[0067] Furthermore, in yet another embodiment of the method provided in this specification, the method further includes: The target question-and-answer information is used as seed question-and-answer information, and the operation of generating question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template is continued.

[0068] The method provided in the embodiments of this specification aims to upgrade simple instructions into more complex and diverse instructions or to create new instructions. Therefore, after determining the target question-and-answer information of the target image, this target question-and-answer information can be used as new seed question-and-answer information, and the operation of generating question-and-answer prompt text based on the target image, target image description information, seed question-and-answer information, and preset prompt text template in the above steps can be continued, and multiple iterations can be performed based on this. The number of iterations can be set according to the actual situation, such as 3 rounds, 4 rounds, etc. In the method provided in the embodiments of this specification, the number of iterations is not limited.

[0069] See Figure 2 , Figure 2 A schematic diagram of a data generation method provided in one embodiment of this specification is shown, such as... Figure 2 As shown, after obtaining the target image and seed question-and-answer information, the above steps generate predicted question-and-answer information 1, predicted question-and-answer information 2, and predicted question-and-answer information 3. If the verification result of predicted question-and-answer information 1 is found to be incorrect, it is discarded. If the verification results of predicted question-and-answer information 2 and predicted question-and-answer information 3 are correct, they are identified as target question-and-answer information and continue to be used as seed question-and-answer information. These are then combined with the target image for a second round of instruction evolution. Predicted question-and-answer information 4 can be obtained from predicted question-and-answer information 2, and predicted question-and-answer information 5 and predicted question-and-answer information 6 can be obtained from predicted question-and-answer information 3. If the verification result of predicted question-and-answer information 4 is found to be incorrect, it is discarded. The verification results of predicted question-and-answer information 5 and predicted question-and-answer information 6 are correct. Both are identified as target question-and-answer information and used as seed question-and-answer information. These are then combined with the target image for a third round of instruction evolution to obtain predicted question-and-answer information 7 and predicted question-and-answer information 8. After verification, the prediction results of both predicted question-and-answer information 7 and predicted question-and-answer information 8 are correct, so they can be used as target question-and-answer information. Finally, six target question-and-answer information pieces (predicted question-and-answer information 2, 3, 5, 6, 7, and 8) are obtained through the method provided in the embodiments of this specification.

[0070] In another specific embodiment provided in this specification, the method further includes: A text-image question-and-answer data pair is formed based on the target image and the target question-and-answer information; A multimodal large language model was trained using text and image question-and-answer data.

[0071] In the method provided in the embodiments of this specification, the purpose of obtaining target question-and-answer information is to enrich the image-text instruction data. Therefore, image-text question-and-answer data pairs can be formed based on the target image and the target question-and-answer information. After obtaining a preset number of image-text question-and-answer data pairs, a multimodal large language model can be trained using the image-text question-and-answer data pairs to improve the model capability of the multimodal large language model.

[0072] The method provided in the embodiments of this specification can significantly improve the diversity and complexity of data in scenarios where prompt evolution types are used to predict questions and answers. It solves the problems of high cost and inconsistent quality of manual annotation in current processing methods, as well as the simple data generated by the model, thereby improving the quality of predicted question and answer information.

[0073] Meanwhile, a verification step for the predicted question-and-answer information is introduced. By verifying the predicted question-and-answer information through a question-and-answer information verification model, erroneous data that may exist in the predicted question-and-answer information can be effectively removed, avoiding model illusions during the training of multimodal large language models. This effectively improves the quality of evolutionary data, thereby ensuring the training effect of subsequent multimodal large language models, especially the anti-illusion ability.

[0074] Furthermore, experiments have shown that the question-and-answer information obtained through the data generation method provided in the embodiments of this specification effectively improves the performance of the multimodal large language model in subsequent training. This indirectly demonstrates that the quality of question-and-answer information is more important than the quantity. The question-and-answer data obtained through this method can be used to train a multimodal large language model with the same or better performance using a smaller data training set.

[0075] Figure 3 This specification shows a flowchart illustrating a data generation method according to another embodiment. This data generation method is applied to cloud-side devices and specifically includes: Step 302: Receive the target image and the seed question-and-answer information corresponding to the target image sent by the receiving end device, and obtain the image description information corresponding to the target image based on the target image.

[0076] Step 304: Generate question and answer prompt text based on the target image, the image description information, the seed question and answer information, and the preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, which is used to increase the diversity of question and answer.

[0077] Step 306: Input the question-and-answer prompt text into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question.

[0078] Step 308: Send the predicted question-and-answer information to the end-side device.

[0079] In another specific embodiment provided in the embodiments of this specification, the method further includes: The predicted question-and-answer information and the target image are input into a question-and-answer information verification model, wherein the question-and-answer information verification model is used to determine whether the predicted answer is correct; Obtain the verification result output by the question-and-answer information verification model; If the verification result is correct, the predicted question-and-answer information is determined to be the target question-and-answer information of the target image; The target question and answer information is sent to the end device.

[0080] In this embodiment, if the question-answering prediction model is a large language model, due to the large number of model parameters and the large model size, there are certain requirements for the terminal deploying the large language model. If the terminal configuration is low, it may be unable to deploy and run the large language model and perform the corresponding processing. Therefore, in this embodiment, the large language model can be deployed on a cloud-side device. In actual application, it is only necessary to receive the target image and the corresponding seed question-answering information uploaded by the end-side device. The cloud-side device, using the method provided in the embodiments of this specification, can execute the above-mentioned question-answering information data generation method according to the target image and seed question-answering information to obtain the corresponding question-answering data, and then feed the obtained question-answering data back to the end-side device.

[0081] The data generation method running on cloud-side devices expands the application scenarios of this method, enabling even low-configuration terminals to use this method to obtain diverse question-and-answer data, thereby improving the user experience.

[0082] Figure 4 This is a schematic diagram of a task platform provided in one embodiment of this specification. The task platform includes a request interface 402 and a response unit 404, wherein: The request interface 402 is used to receive the target image and the seed question and answer information corresponding to the target image sent by the end device, and to obtain the image description information corresponding to the target image based on the target image; The response unit 404 is configured to generate question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and a preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, which is used to increase question-and-answer diversity; input the question-and-answer prompt text into a question-and-answer prediction model to obtain predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes a predicted question and a predicted answer corresponding to the predicted question.

[0083] Corresponding to the above method embodiments, this specification also provides embodiments of a data generation apparatus. Figure 5 A schematic diagram of a data generation apparatus according to one embodiment of this specification is shown. Figure 5 As shown, the device includes: The acquisition module 502 is configured to acquire a target image and seed question-and-answer information corresponding to the target image, and acquire image description information corresponding to the target image based on the target image; The text generation module 504 is configured to generate question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and a preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, which is used to increase the diversity of questions and answers; The prediction module 506 is configured to input the question-and-answer prompt text into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question.

[0084] Optionally, the device further includes a verification module configured to: The predicted question-and-answer information and the target image are input into a question-and-answer information verification model, wherein the question-and-answer information verification model is used to determine whether the predicted answer is correct; Obtain the verification result output by the question-and-answer information verification model; If the verification result is correct, the predicted question-and-answer information is determined to be the target question-and-answer information of the target image.

[0085] Optionally, the device further includes an iteration module configured to: The target question-and-answer information is used as seed question-and-answer information, and the operation of generating question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template is continued.

[0086] Optionally, the device further includes a template generation module, configured to: Obtain information on the target evolution requirements; The prompt evolution type is determined based on the target evolution requirement information, and a preset prompt text template is generated based on the prompt evolution type.

[0087] Optionally, the generation module is further configured to: Obtain at least one reference evolution requirement; The target evolution requirement information is randomly determined from at least one reference evolution requirement information.

[0088] Optionally, the generation module is further configured to: Obtain a preset evolutionary capability library, wherein the evolutionary capability library includes at least one preset evolutionary capability information; Select at least one target evolutionary capability from at least one preset evolutionary capability information; A preset prompt text template is generated based on the prompt evolution type, the preset evolution capability library, and the evolution capability information of each target.

[0089] Optionally, the generation module is further configured to: Determine the cue evolution parameters corresponding to the cue evolution type; By combining the aforementioned prompt evolution parameters, the preset evolution capability library, and the evolution capability information of each target, a preset prompt text template is generated.

[0090] Optionally, the preset evolutionary capability information types include image recognition capability and text reasoning capability.

[0091] Optionally, image recognition capabilities include localization capabilities, referential capabilities, computational capabilities, character recognition capabilities, and existence judgment capabilities; Textual reasoning ability includes the ability to describe relationships, the ability to understand scenarios, the ability to predict behavior, and the ability to associate knowledge.

[0092] Optionally, the prompting evolution type includes any one of the cognitive reasoning evolution type, the interactive evolution type, and the fine-grained perception evolution type.

[0093] Optionally, the device further includes a training module configured to: A text-image question-and-answer data pair is formed based on the target image and the target question-and-answer information; A multimodal large language model was trained using text and image question-and-answer data.

[0094] The apparatus provided in the embodiments of this specification can significantly improve the diversity and complexity of data in scenarios where prompt evolution types are used to predict questions and answers. It solves the problems of high cost and inconsistent quality of manual annotation in current processing methods, as well as the simple data generated by the model, thereby improving the quality of predicted question and answer information.

[0095] Meanwhile, a verification step for the predicted question-and-answer information is introduced. By verifying the predicted question-and-answer information through a question-and-answer information verification model, erroneous data that may exist in the predicted question-and-answer information can be effectively removed, avoiding model illusions during the training of multimodal large language models. This effectively improves the quality of evolutionary data, thereby ensuring the training effect of subsequent multimodal large language models, especially the anti-illusion ability.

[0096] Furthermore, experiments have shown that the question-and-answer information obtained through the data generation method provided in the embodiments of this specification effectively improves the performance of the multimodal large language model in subsequent training. This indirectly demonstrates that the quality of question-and-answer information is more important than the quantity. The question-and-answer data obtained through this method can be used to train a multimodal large language model with the same or better performance using a smaller data training set.

[0097] The above is an illustrative scheme of a data generation apparatus according to this embodiment. It should be noted that the technical solution of this data generation apparatus and the technical solution of the data generation method described above belong to the same concept. For details not described in detail in the technical solution of the data generation apparatus, please refer to the description of the technical solution of the data generation method described above.

[0098] See Figure 6 , Figure 6 This specification illustrates an architecture diagram of a data generation system provided in one embodiment of the specification. The data generation system may include a client 100 and a server 200. Client 100 is used to send a target image and the corresponding seed question and answer information to server 200; Server 200 is configured to: obtain image description information corresponding to the target image; generate question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and a preset prompt text template, wherein the preset prompt text template includes a prompt evolution type, which is used to increase question-and-answer diversity; input the question-and-answer prompt text into a question-and-answer prediction model to obtain predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes a predicted question and a predicted answer corresponding to the predicted question; and send the predicted question-and-answer information to client 100. Client 100 is also used to receive predicted question-and-answer information sent by server 200.

[0099] The data generation system may include multiple clients 100 and a server 200. Clients 100 can be referred to as edge devices, and server 200 can be referred to as cloud devices. Multiple clients 100 can establish communication connections through server 200. In the data generation scenario, server 200 is used to provide data generation services between multiple clients 100. Each client 100 can act as a sender or receiver, communicating through server 200.

[0100] Users can interact with server 200 through client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the data generation scenario, users can publish data streams to server 200 through client 100, server 200 can generate predicted question-and-answer information based on the data stream, and push the predicted question-and-answer information to other clients that have established communication.

[0101] In this system, client 100 and server 200 establish a connection via a network. The network provides the medium for communication between client 100 and server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by client 100 may need to undergo encoding, transcoding, compression, or other processing before being published to server 200.

[0102] Client 100 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 100 can be developed based on the software development kit (SDK) of the corresponding service provided by server 200, such as a real-time communication (RTC) SDK. Client 100 can be deployed on electronic devices and depends on the device or certain apps on the device to run. Electronic devices may have displays and support information browsing, such as personal mobile terminals like mobile phones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platform software.

[0103] Server 200 may include servers providing various services, such as servers providing communication services to multiple clients, servers supporting backend training of models used on clients, and servers processing data sent by clients. It should be noted that server 200 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0104] It is worth noting that the data generation method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the data generation method provided in the embodiments of this specification. In other embodiments, the data generation method provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0105] Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0106] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0107] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0108] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0109] The processor 720 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-described data generation method.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computing device embodiments are basically similar to the data generation method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data generation method embodiments.

[0111] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the data generation method described above.

[0112] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are fundamentally similar to the data generation method embodiments; relevant parts can be referred to in the description of the data generation method embodiments.

[0113] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the data generation method described above.

[0114] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data generation method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data generation method described above.

[0115] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0116] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0117] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0118] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0119] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data generation method, comprising: Obtain the target image and the seed question-and-answer information corresponding to the target image, and obtain the image description information corresponding to the target image based on the target image; The question-and-answer prompt text is generated based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template. The preset prompt text template includes a prompt evolution type, which is used to increase the diversity of questions and answers. The question-and-answer prompt text is input into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question.

2. The method of claim 1, further comprising: The predicted question-and-answer information and the target image are input into a question-and-answer information verification model, wherein the question-and-answer information verification model is used to determine whether the predicted answer is correct; Obtain the verification result output by the question-and-answer information verification model; If the verification result is correct, the predicted question-and-answer information is determined to be the target question-and-answer information of the target image.

3. The method of claim 2, further comprising: The target question-and-answer information is used as seed question-and-answer information, and the operation of generating question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template is continued.

4. The method of claim 1, further comprising: Obtain information on the target evolution requirements; The prompt evolution type is determined based on the target evolution requirement information, and a preset prompt text template is generated based on the prompt evolution type.

5. The method as described in claim 4, wherein obtaining target evolution requirement information includes: Obtain at least one reference evolution requirement; The target evolution requirement information is randomly determined from at least one reference evolution requirement information.

6. The method as described in claim 4, wherein generating a preset prompt text template according to the prompt evolution type includes: Obtain a preset evolutionary capability library, wherein the evolutionary capability library includes at least one preset evolutionary capability information; Select at least one target evolutionary capability from at least one preset evolutionary capability information; A preset prompt text template is generated based on the prompt evolution type, the preset evolution capability library, and the evolution capability information of each target.

7. The method as described in claim 6, wherein generating a preset prompt text template based on the prompt evolution type, the preset evolution capability library, and the evolution capability information of each target includes: Determine the cue evolution parameters corresponding to the cue evolution type; By combining the aforementioned prompt evolution parameters, the preset evolution capability library, and the evolution capability information of each target, a preset prompt text template is generated.

8. The method as described in claim 6, wherein the information types of the preset evolutionary capability information include image recognition capability and text reasoning capability.

9. The method as described in claim 8, wherein the image recognition capability includes positioning capability, referential capability, computational capability, character recognition capability, and existence judgment capability; Textual reasoning ability includes the ability to describe relationships, the ability to understand scenarios, the ability to predict behavior, and the ability to associate knowledge.

10. The method of claim 1, wherein the prompting evolution type includes any one of the cognitive reasoning evolution type, the interactive evolution type, and the fine-grained perception evolution type.

11. The method of claim 2, further comprising: A text-image question-and-answer data pair is formed based on the target image and the target question-and-answer information; A multimodal large language model was trained using text and image question-and-answer data.

12. A data generation method, applied to cloud-side devices, comprising: The receiving end device sends a target image and seed question-and-answer information corresponding to the target image, and obtains image description information corresponding to the target image based on the target image; The question-and-answer prompt text is generated based on the target image, the image description information, the seed question-and-answer information, and the preset prompt text template. The preset prompt text template includes a prompt evolution type, which is used to increase the diversity of questions and answers. The question-and-answer prompt text is input into the question-and-answer prediction model to obtain the predicted question-and-answer information for the target image output by the question-and-answer prediction model, wherein the predicted question-and-answer information includes the predicted question and the predicted answer corresponding to the predicted question; The predicted question-and-answer information is sent to the edge device.

13. The method of claim 12, further comprising: The predicted question-and-answer information and the target image are input into a question-and-answer information verification model, wherein the question-and-answer information verification model is used to determine whether the predicted answer is correct; Obtain the verification result output by the question-and-answer information verification model; If the verification result is correct, the predicted question-and-answer information is determined to be the target question-and-answer information of the target image; The target question and answer information is sent to the end device.

14. A task platform, comprising a request interface and a response unit; The request interface is used to receive the target image and the seed question and answer information corresponding to the target image sent by the end device, and to obtain the image description information corresponding to the target image based on the target image; The response unit is configured to generate question-and-answer prompt text based on the target image, the image description information, the seed question-and-answer information, and a preset prompt text template, wherein... The preset prompt text template includes a prompt evolution type, which is used to increase the diversity of question and answer; the question and answer prompt text is input into the question and answer prediction model to obtain the predicted question and answer information for the target image output by the question and answer prediction model, wherein the predicted question and answer information includes the predicted question and the predicted answer corresponding to the predicted question.

15. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 13.

16. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images