Training data generation method, device, medium and product

Abstract

The application discloses a kind of training data generation method, equipment, medium and product, comprising: obtaining target answer;Based on the attribute information of target answer, candidate set is constructed, candidate set includes target answer and at least one interference candidate;For candidate set, corresponding evidence information of each candidate is obtained from data source, and evidence information is classified, evidence information includes at least one of supporting evidence, excluding evidence, bridging evidence or conflict evidence;Based on candidate set and evidence information, the association structure between candidate and evidence is constructed to represent the support relationship and excluding relationship between each candidate;Based on association structure, generate question text, question text does not contain explicit information directly pointing to target answer;Based on candidate set and association structure, generate analysis process data corresponding to question text;Target answer, candidate set, question text, association structure and analysis process data are encapsulated to generate training data.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a training data generation method, device, medium, and product. Background Technology

[0002] As large language models gradually acquire agent capabilities such as retrieval, browsing, planning, and verification, relying solely on conventional question-answering corpora or shallow retrieval corpora is no longer sufficient to effectively improve the model's deep analysis capabilities in complex and open environments. In evaluation scenarios involving a large number of web pages, cross-site clues, and fuzzy descriptions, the task objective typically requires the model to continuously search through massive amounts of information, gradually narrow down the scope, and lock in a unique answer. This places extremely high demands on the model's ability to propose candidates, compare and eliminate them, and verify candidates.

[0003] However, existing data synthesis schemes still have significant shortcomings: on the one hand, some schemes mainly rely on the complexity of knowledge graph paths or entity ambiguity to create difficulty, with less explicit modeling of the competitive relationships between candidate answers; on the other hand, most schemes focus on how to "find the answer," while failing to adequately express the analytical trajectory of how erroneous paths are formed, how they are eliminated, and why the final answer is valid. This makes it difficult for models to learn complete analytical closed-loop capabilities during training, and they cannot simulate the systematic thinking process of human experts in complex scenarios. Summary of the Invention

[0004] One objective of this application is to provide a training data generation method, device, medium, and product, which at least addresses the problem that existing synthetic data schemes fail to adequately express the game-theoretic relationships between candidate answers and lack explicit characterization of the erroneous branch elimination trajectory, making it difficult for models to form a complete deep analysis loop in complex retrieval scenarios.

[0005] To achieve the above objectives, some embodiments of this application provide the following aspects:

[0006] This application provides a training data generation method, the method comprising:

[0007] Obtain the target answer;

[0008] A candidate set is constructed based on the attribute information of the target answer, the candidate set including the target answer and at least one interfering candidate;

[0009] For the candidate set, evidence information corresponding to each candidate is obtained from the data source, and the evidence information is classified. The evidence information includes at least one of supporting evidence, excluding evidence, bridging evidence, or conflicting evidence.

[0010] Based on the candidate set and the evidence information, a relationship structure between candidates and evidence is constructed to characterize the support and exclusion relationships between candidates.

[0011] Question text is generated based on the aforementioned association structure, and the question text does not contain explicit information that directly points to the target answer;

[0012] Based on the candidate set and the association structure, generate analysis process data corresponding to the question text;

[0013] The target answer, the candidate set, the question text, the association structure, and the analysis process data are encapsulated to generate training data.

[0014] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0015] Thirdly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the steps of the method described above.

[0016] Fourthly, some embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0017] Compared with related technologies, the solution provided in this application constructs a candidate set containing the target answer and interfering candidates, and introduces supporting evidence, excluding evidence, bridging evidence, and conflicting evidence to systematically model the relationship between candidates and evidence, thereby forming an association structure that can characterize the competition relationship between candidates and the constraint relationship between evidence. The question text generated based on this association structure avoids directly revealing the target answer. At the same time, combined with the candidate screening and exclusion mechanism, the generated data can effectively reflect the analysis process of multi-candidate competition and gradual convergence, thereby improving the model's reasoning and discrimination capabilities in complex scenarios.

[0018] Furthermore, by generating analysis process data that includes candidate branch generation, elimination, and convergence paths, and combining this with a multi-level verification mechanism to filter the generated data, the training data can be guaranteed to possess both difficulty in finding and verifiability. This method not only improves the structure and information completeness of the training data, but also guides the model to learn analysis paths and error correction mechanisms, thereby enhancing the model's stability and generalization ability while improving its accuracy. Attached Figure Description

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 A flowchart illustrating a training data generation method provided as an exemplary embodiment of this disclosure;

[0021] Figure 2 An exemplary structural diagram of the electronic device provided for some embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Figure 1 An exemplary flowchart of a training data generation method provided for an exemplary embodiment of this disclosure, the method comprising:

[0024] S101. Obtain the target answer.

[0025] Specifically, a seed answer pool can be formed by extracting candidate facts that are highly stable, objective, concise, and unique from multiple data sources, such as structured knowledge bases, semi-structured web pages, encyclopedic data sources, academic indexes, industry databases, or historical documents. Each seed in the seed answer pool can include the target answer ontology and its associated structured description information. The structured description information includes at least the answer type, domain label, time label, geographic label, and several verifiable attributes to characterize the semantic features and constraints of the target answer.

[0026] Based on this, candidate facts in the seed answer pool can be screened to determine the target answer. Specifically, candidate facts can be evaluated based on indicators such as multi-source data consistency, semantic stability, historical frequency of occurrence, or the credibility of structured knowledge, thereby selecting candidate facts that meet preset conditions as the target answer. The target answer can be an entity, event, or factual description with clear semantic orientation and uniqueness, and can be verified through evidence information, thereby ensuring the effectiveness and reliability of subsequent candidate construction and reasoning processes.

[0027] S102. Construct a candidate set based on the attribute information of the target answer, the candidate set including the target answer and at least one interfering candidate.

[0028] Specifically, attribute information of the target answer can be extracted, including but not limited to category attributes, domain attributes, time attributes, or spatial attributes. Based on this attribute information, other candidate options with similar attribute features to the target answer can be screened from the data source or candidate library as interference candidates, thereby forming a candidate set. The interference candidates may be semantically similar to the target answer, but differ in key attributes or constraints, in order to construct a multi-candidate competition relationship.

[0029] S103. For the candidate set, obtain evidence information corresponding to each candidate from the data source, and classify the evidence information, which includes at least one of supporting evidence, excluding evidence, bridging evidence, or conflicting evidence.

[0030] Specifically, for each candidate, textual, structured, or multimodal information related to that candidate can be retrieved from the data source, and the acquired evidence information can be organized and normalized. Based on this, the evidence information is classified according to the semantic or logical relationship between the evidence information and the candidates. The evidence information includes at least one or more of supporting evidence, excluding evidence, bridging evidence, or conflicting evidence. Different types of evidence are used to characterize the supporting, excluding, or informational connections between candidates.

[0031] S104. Construct a relationship structure between candidates and evidence based on the candidate set and the evidence information to characterize the support and exclusion relationships between candidates.

[0032] Specifically, candidates can be treated as nodes, and evidence information as connecting or relational nodes, establishing associations between candidates and evidence to form a structured representation. This association structure can be used to describe the supporting, excluding, and competing relationships between candidates and evidence. In different implementations, this association structure can be represented using graph structures, relational networks, or other forms.

[0033] S105. Generate question text based on the association structure, wherein the question text does not contain explicit information that directly points to the target answer.

[0034] Specifically, the original information can be reorganized and rewritten based on the evidence information and candidate constraints contained in the association structure to generate question text for training. During the generation process, explicit information that directly points to the target answer can be processed so that the question text is presented in an indirect manner, thereby avoiding directly revealing the target answer while retaining necessary semantic clues to support the subsequent reasoning process.

[0035] S106. Generate analysis process data corresponding to the question text based on the candidate set and the association structure.

[0036] Specifically, an analysis path can be constructed around the candidate set, and the candidates can be progressively screened and verified by combining the evidence information in the association structure, thereby forming analysis process data. The analysis process data is used to describe the reasoning path from the candidate set to the target answer, and can reflect the changes in candidate screening, evidence support, and exclusion relationships.

[0037] S107. Encapsulate the target answer, the candidate set, the question text, the association structure, and the analysis process data to generate training data.

[0038] Specifically, the aforementioned information can be organized and stored according to a preset data structure to form training data for model training or evaluation. The training data may include question text, answer information, candidate information, and corresponding analysis process information to support multi-candidate inference and path learning during model training.

[0039] In this embodiment, a candidate set containing the target answer and interfering candidates is constructed. Supporting evidence, exclusionary evidence, bridging evidence, and conflicting evidence are introduced to systematically model the relationship between candidates and evidence, thus forming an association structure that characterizes candidate competition and evidence constraints. The question text generated based on this association structure avoids directly revealing the target answer. Furthermore, combined with candidate filtering and exclusion mechanisms, the generated data effectively reflects the analysis process of multi-candidate competition and gradual convergence. This improves the structure and information completeness of the training data and enhances the model's analytical capabilities in multi-candidate filtering and complex reasoning scenarios.

[0040] In one embodiment, obtaining the target answer includes:

[0041] Extract candidate fact entities from multi-source data;

[0042] Based on the consistency of multi-source information, time-varying characteristics, and the credibility of structured knowledge, a comprehensive evaluation is conducted on each candidate fact entity to obtain a stability score.

[0043] Candidate fact entities whose stability scores are greater than a preset score threshold are identified as the target answer.

[0044] Specifically, the multi-source data can include various data sources such as structured knowledge bases, semi-structured web page data, encyclopedic data sources, academic indexes, industry databases, and historical documents. The system first performs unified data access and preprocessing on the above data sources, including text parsing, format standardization, entity alignment, and redundant information removal, thereby obtaining a set of candidate information that can be used for subsequent processing. Based on this, candidate fact entities with high stability, strong objectivity, concise answer expression, and uniqueness are extracted from the multi-source data to form a candidate answer pool. Each candidate fact entity in the candidate answer pool can be associated with structured descriptive information, which includes at least answer type, domain label, time label, geographic label, and several verifiable attributes to characterize the semantic features and constraints of the candidate fact entity.

[0045] Furthermore, for each candidate fact entity, the system evaluates its consistency based on the degree of consistency across multiple information sources. Specifically, it can statistically analyze the frequency of occurrence and consistency of expression of the candidate fact across different data sources, and quantify this based on consistency ratios or consistency scores. Simultaneously, it analyzes the temporal variation characteristics of the candidate fact, such as assessing its stability over different time periods to reflect whether there is significant temporal drift or changing trends. In addition, it can combine the credibility of structured knowledge to evaluate the confidence information of the candidate fact entity in knowledge graphs or structured databases, for example, by comprehensively judging based on the authority of the knowledge source, the stability of relationships, or the connectivity of nodes.

[0046] Furthermore, a stability score model can be constructed to quantitatively evaluate each candidate fact entity. Specifically, a stability score can be defined. for:

[0047]

[0048] in, The graph confidence score represents the structural credibility of the candidate fact entity in the authoritative knowledge graph, and can be evaluated based on the connection relationship features of the entity in the knowledge graph. This represents the degree of consensus among multiple sources, which characterizes the consistency of a candidate fact across multiple independent data sources. It can be obtained by extracting information from multiple data sources and calculating the consistency ratio. This represents the time-dependent variability, reflecting the changes in key attributes of a candidate fact within a certain timeframe. A higher frequency of change indicates lower stability. Parameters α, β, and γ are preset weights used to adjust the influence of different factors in the stability assessment.

[0049] In practical implementation, at least three independent data sources can be selected to extract and compare information from candidate fact entities. When the consistency ratio meets a preset condition, its consensus score is increased. Simultaneously, statistical analysis is performed on the attribute changes of candidate fact entities within a preset time window to determine their timeliness stability. Based on this, a stability score is calculated by comprehensively evaluating various indicators, and the stability score is compared with a preset score threshold. Only when the stability score is greater than the preset score threshold (e.g., ...) is the consensus score increased. The corresponding candidate fact entities are then identified as the target answer. The preset score threshold can be set according to application requirements to ensure that the selected target answer has high reliability and stability.

[0050] In this embodiment, by introducing multi-dimensional information such as multi-source consistency, time-varying characteristics, and the credibility of structured knowledge to comprehensively evaluate candidate fact entities, it is possible to effectively screen out target answers with high stability and reliability, thereby improving the basic quality of training data. Furthermore, this method can reduce the risk of bias caused by a single data source and reduce the impact of noisy data on subsequent candidate construction and inference processes, thereby improving the accuracy and robustness of the overall data generation process.

[0051] In one embodiment, constructing the candidate set based on the attribute information of the target answer includes:

[0052] Extract the attribute information of the target answer to generate a target feature representation;

[0053] Based on structured data acquisition, sibling node entities with the same preset parent class node as the target answer are obtained;

[0054] The similarity between each sibling node entity and the target feature representation is calculated based on feature similarity.

[0055] Sibling node entities whose similarity falls within a preset similarity range are selected as interference candidates.

[0056] Specifically, the target answer can first undergo attribute parsing, extracting its corresponding attribute information from multi-source data or a structured knowledge base. This attribute information may include category attributes, functional attributes, temporal attributes, spatial attributes, or semantic descriptive features, and a unified target feature representation can be constructed based on this attribute information. This target feature representation can be expressed using vectorization, or described using attribute sets, label sets, or other structured representations to reflect the characteristics of the target answer in a multi-dimensional attribute space.

[0057] Furthermore, sibling node entities with the same preset hierarchical parent node as the target answer (e.g., "belonging to the same category of drugs for treating cardiovascular diseases") can be obtained based on structured data. This structured data can originate from knowledge graphs or other hierarchically organized data structures. By locating the parent node of the target answer in the hierarchical structure and obtaining other child node entities under that parent node, a set of candidate entities with category similarity to the target answer can be obtained. This approach ensures that the interfering candidates maintain semantic category consistency with the target answer, thereby forming an effective candidate competition space.

[0058] Based on this, the similarity between each sibling node entity and the target feature representation is calculated. Specifically, the target feature representation can be compared with the feature representations of each candidate entity based on feature matching or similarity calculation methods to obtain a similarity measurement result. The similarity can be evaluated by comprehensively considering multiple attribute dimensions, thereby reflecting the semantic and attribute-wise closeness between the candidate entity and the target answer.

[0059] Subsequently, candidate entities are filtered according to a preset similarity interval, and sibling nodes whose similarity falls within this interval are identified as interference candidates. The preset similarity interval can be set according to actual application needs to achieve a balance between the number of candidates and the difficulty of differentiation. For example, when the similarity is too high, candidate entities may be too close to the target answer to be distinguishable; when the similarity is too low, effective competition may not be possible. By reasonably setting the similarity interval, candidate entities that possess both similarity and differences can be filtered out, thereby constructing a high-quality candidate set.

[0060] In one specific implementation, the similarity between candidate entities and the target answer can be calculated based on cosine similarity. The similarity can be expressed as:

[0061]

[0062] in, This represents the feature vector of the i-th candidate entity. The feature vector representing the target answer. Represents the vector dot product. These represent the norms of the corresponding vectors. This method allows us to quantify the similarity between candidate entities and the target answer in a multi-dimensional feature space.

[0063] Furthermore, a similarity filtering range can be set to filter candidate entities. In one example, the similarity range can be set between 0.65 and 0.85, meaning that candidate entities meeting the following conditions are filtered out as interference candidates:

[0064]

[0065] When the similarity is below the lower threshold, it indicates that the candidate entity differs significantly from the target answer, which is not conducive to building an effective competitive relationship. When the similarity is above the upper threshold, the candidate entity is too similar to the target answer, which may lead to difficulty in differentiation. By setting the above intervals, a balance can be achieved between similarity and discriminability, thereby obtaining a set of candidate entities with higher interference resistance.

[0066] In this embodiment, by filtering sibling node entities based on hierarchical structure and combining them with feature similarity constraints, it is possible to effectively construct interfering candidates that are semantically similar to the target answer but differ in key attributes, thereby enhancing the discriminative power and competitiveness of the candidate set. Furthermore, this method can improve the effectiveness of candidate screening and elimination in subsequent data generation processes, thus helping to enhance the guiding role of training data in the model's reasoning ability.

[0067] In one embodiment, constructing the association structure between candidates and evidence based on the candidate set and the evidence information includes:

[0068] Based on semantic reasoning, the relationship between the evidence information and the corresponding candidates in the candidate set is determined to identify the supporting or excluding relationship.

[0069] If the relationship is determined to be supportive, the corresponding evidence information is marked as the supporting evidence, and a positive association is established in the association structure.

[0070] If it is determined to be an exclusion relationship, the corresponding evidence information is marked as the exclusion evidence, and a negative association is established in the association structure;

[0071] Evidence information used to connect multiple pieces of evidence information is marked as bridging evidence, and evidence information with descriptive conflicts is marked as conflicting evidence.

[0072] Specifically, for each candidate in the candidate set, relevant evidence information can be obtained from the data source, and this evidence information can be uniformly represented and standardized, such as through text cleaning, entity alignment, and semantic standardization, to facilitate subsequent relationship determination. Subsequently, the relationship between the evidence information and the candidates is analyzed through semantic reasoning. Based on semantic matching, logical consistency judgment, or contextual semantic constraints, the type of role the evidence information plays in relation to the candidates can be determined. In one embodiment, a Natural Language Inference (NLI) model combined with Semantic Role Labeling (SRL) can be used to classify the retrieved evidence information.

[0073] Furthermore, in a specific implementation, supporting evidence can be evaluated through semantic reasoning to assess the degree of support for the candidate's validity. When the supporting evidence can semantically deduce the validity of the candidate, it is marked as supporting evidence. In one example, when the corresponding semantic reasoning probability meets a preset support threshold (…), the supporting evidence is considered to be supported. When a relationship is established, it can be determined as a supporting relationship, and a positive association between the candidate and the evidence can be established in the association structure.

[0074] For exclusionary evidence, semantic reasoning can be used to determine whether there is a semantic conflict or mutual exclusion relationship between the evidence information and the candidate. When the evidence information can refute the candidate, it is marked as exclusionary evidence. In one example, when the corresponding semantic conflict probability meets a preset exclusion threshold (e.g., When a negative association is established between a candidate and the evidence, it can be determined as an exclusion relationship, and a negative association between the candidate and the evidence can be established in the association structure.

[0075] Bridging evidence identifies evidence that, when used alone, cannot directly support or exclude candidates, but can be combined with other evidence to form a complete reasoning chain. This bridging evidence is used to establish connections between different pieces of evidence, thereby enabling cross-evidence information integration and reasoning path construction.

[0076] Conflicting evidence can be identified within the same search result set by providing different or even contradictory descriptions of the same object, and marked as conflicting evidence. This conflicting evidence reflects information differences between different data sources, providing a basis for subsequent screening based on evidence credibility or consistency.

[0077] In a further embodiment, the association structure can be represented using a graph structure, where candidates are nodes, evidence information is an attribute of the nodes or edges, supporting relationships correspond to positive connections, exclusion relationships correspond to negative connections, bridging evidence is used to connect multiple evidence nodes, and conflicting evidence is used to identify contradictory relationship edges. This structure allows for a unified expression of supporting and exclusion relationships between candidates and reflects the associations and conflicts between pieces of evidence.

[0078] In this embodiment, by classifying evidence information into multiple types and constructing structured relationships, it is possible to model the complex relationships between candidates in a systematic way. This allows the training data to contain not only supporting information but also exclusionary information and the relationship information between evidence, thereby improving the data's ability to express and constrain the model's reasoning ability and helping to enhance the model's accuracy and robustness in multi-evidence reasoning scenarios.

[0079] In one embodiment, before generating the question text based on the association structure, the method further includes:

[0080] Calculate the validation asymmetric index between the target answer and the interference candidates based on the association structure;

[0081] The verification asymmetric index is obtained by weighting the indicators of difficulty to find, ease of verification, competition, and analysis requirements.

[0082] Determine whether the verification asymmetric index meets the preset conditions;

[0083] In response to the verification asymmetric index satisfying the preset condition, the step of generating question text based on the association structure is executed.

[0084] Specifically, the overall characteristics of the candidate set can be quantitatively analyzed based on the aforementioned association structure to calculate the verification asymmetry index. This verification asymmetry index measures the asymmetric characteristics of generated samples in terms of "difficulty in directly retrieving answers" versus "ease of verification given candidates and evidence," thereby selecting training samples that are both challenging and verifiable.

[0085] Furthermore, the difficulty-to-find index is used to characterize whether the target answer is easily found directly in an open environment. Specifically, the ranking position or hit probability of the target answer in the search results can be evaluated by simulating a keyword-based retrieval process or an initial retrieval process based on semantic matching. When the target answer is difficult to retrieve directly under conditions without candidate constraints, its difficulty-to-find index is relatively high.

[0086] The verifiability metric is used to characterize whether, given a set of candidates and supporting evidence, the target answer can be verified with relatively certainty through an evidence-based reasoning process. Specifically, candidates can be progressively screened based on supporting and excluding evidence within the association structure, and the convergence to a unique candidate can be evaluated. A higher verifiability metric indicates a clear verification process and a unique result.

[0087] The competitiveness metric is used to characterize the similarity and difficulty of distinguishing between distractor candidates and the target answer. Specifically, the difficulty of distinguishing the candidate set can be evaluated based on the feature similarity or attribute overlap between candidates. When distractor candidates are semantically similar to the target answer but differ in key attributes, the competitiveness metric is high.

[0088] The analytical requirement index is used to characterize the reasoning complexity required to complete the candidate screening and answer determination process. Specifically, it can be evaluated based on factors such as the length of the evidence chain in the association structure, the number of evidence types involved, and the existence of bridging or conflicting evidence. The analytical requirement index is relatively high when multi-step reasoning or cross-evidence integration is required.

[0089] After obtaining the above indicators, they can be combined using a weighted approach to obtain the verification asymmetric indicators. The weights of different indicators can be set according to specific application scenarios to achieve a balance between the difficulty and verifiability of the training data. Subsequently, the verification asymmetric indicators are compared with preset conditions. When the verification asymmetric indicators meet the preset conditions, the current candidate set and associated structure are considered to have the conditions for generating high-quality question text, and the subsequent question text generation steps are executed; otherwise, the candidate set or evidence information can be adjusted, or the current sample can be discarded.

[0090] Furthermore, in one embodiment, when calculating the verification asymmetric index, the difficulty index, verifiability index, competition index, and analysis requirement index can be quantified and a comprehensive score can be obtained through a weighted method. Specifically, a comprehensive score can be defined. The calculation method is as follows:

[0091]

[0092] in, This indicates that the sex index is difficult to find. Indicates verifiability index, Indicates a competitive indicator. This indicates the analytical demand indicators. to This sets the preset weight parameters. In one example, w1=0.3, w2=0.3, w3=0.2, and w4=0.2.

[0093] Specifically, the difficulty index H is used to characterize the semantic distance between the question text and the target answer, and its calculation method can be expressed as follows:

[0094]

[0095] in, A vector representation of the question text. The vector representation of the target answer. This represents the cosine similarity between the two.

[0096] The verifiability index V is used to characterize the ease with which the target answer can be verified given the evidence path, and its calculation method can be expressed as follows:

[0097]

[0098] in, This represents the cross-entropy loss of the verification model for the target answer, given the evidence path. (·) represents an exponential function. This method converts the loss value during the verification process into a positive indicator, so that the lower the loss, the higher the corresponding verifiability indicator.

[0099] The competitiveness index C is used to characterize the interference strength of the interfering candidates. In one embodiment, it can be quantitatively calculated based on misleading evidence information pointing to the interfering candidates in the association structure, for example, by accumulating the weights of supporting or misleading evidence associated with each interfering candidate, thereby obtaining the competitiveness index:

[0100]

[0101] in, The evidence weights associated with the interfering candidates can be set based on the importance or credibility of the evidence.

[0102] The analytical requirement index R is used to characterize the complexity of the reasoning process, and its calculation method can be expressed as follows:

[0103]

[0104] in, Indicates the number of bridging pieces of evidence in the related structure. This represents the number of conflicting pieces of evidence. This method allows for a compressed representation of the complexity of the reasoning path, resulting in an increasing trend in the metric as complexity increases.

[0105] After obtaining the above indicators, a comprehensive score is obtained through weighted calculation. The comprehensive score is then compared with a preset threshold. If the value exceeds a preset threshold, the currently generated sample is retained for training data; otherwise, the candidate set or evidence structure is adjusted or the sample is discarded. In one example, the preset threshold can be set to... .

[0106] In this embodiment, by introducing asymmetric verification metrics to screen the candidate set and its associated structures, the difficulty and quality of generating training data can be effectively controlled, making the data both challenging and clearly verifiable through the evidence reasoning process. This enhances the guiding effect of training data on the model's reasoning ability and strengthens the model's generalization ability and stability in complex scenarios.

[0107] In one embodiment, generating the question text based on the association structure includes:

[0108] The misleading feature information related to the interference candidates is incorporated into the question text;

[0109] The constraints used to exclude interfering candidates are implicitly embedded in the question text;

[0110] The explicit entity information in the question text is hidden so that the question text does not contain entity identifier information that directly points to the target answer or the interference candidate.

[0111] Specifically, evidence information can be reorganized and transformed based on the aforementioned association structure to generate question text that meets training requirements. First, evidence information related to interfering candidates can be extracted from the association structure, and misleading features can be screened and processed. This type of feature information is then integrated into the question text, so that the question provides some support for the interfering candidates on a surface semantic level, thereby increasing the degree of competition among the candidates.

[0112] Furthermore, key constraints for excluding interfering candidates can be extracted from the association structure, such as attribute restrictions or logical relationships related to excluded evidence, and these constraints can be implicitly embedded in the question text. This implicit embedding can be achieved through semantic restatement, condition combination, or information rearrangement, so that the constraints are not presented as explicit rules but are implicitly expressed in the question description through natural language, thus providing necessary evidence for subsequent reasoning without directly revealing the answer.

[0113] During the generation of the question text, the original information can be rewritten and integrated to avoid directly quoting the original evidence and to hide explicit entity information related to the target answer or distracting candidates. This hiding process can include replacing entity names, generalizing expressions, or deleting explicit referential information, so that the question text does not contain entity identifiers that can directly locate the target answer or distracting candidates, while retaining key semantic clues for reasoning.

[0114] In a further implementation, the question text can be optimized based on a preset language generation strategy to maintain semantic coherence and integrity, while also meeting the training data's requirements for diversity and naturalness. Furthermore, the generated question text can be standardized, for example, by controlling text length, sentence structure, and semantic integrity, to improve data consistency.

[0115] In one example, the following prompt template can be constructed:

[0116] The system generates question text based on the given target answer, interference candidates, and the associated structure. The question text must meet the following constraints: First, the first half of the question text contains pseudo-supporting evidence features pointing to interference candidates. These features semantically support the interference candidates, thereby inducing shallow models to make misjudgments. Second, the second half of the question text implicitly embeds boundary conditions that can trigger bridging evidence and evidence that excludes interference candidates, enabling interference candidates to be excluded and the target answer to be pointed to when combined with the complete evidence chain. Third, the entity names of the target answer or interference candidates do not appear directly in the question text to avoid explicit pointing.

[0117] In a specific example, the text example is:

[0118] - Target answer: Pluto; Distractor: Neptune.

[0119] - The generated question: "This celestial object was once considered a key factor in calculating orbital perturbations in our galaxy (inducing Neptune), but it was later found that its mass was too small to cause such a large gravitational anomaly, and its orbital plane has a significant inclination to the ecliptic plane and at times crosses the orbits of nearby giant planets (burying bridging and exclusion evidence). What is this celestial object?"

[0120] In this embodiment, by simultaneously introducing misleading feature information and implicit constraints into the question text and hiding explicit entity information, questions with multiple candidate competition relationships can be constructed without directly revealing the answer. This enhances the difficulty of distinguishing the questions and the depth of reasoning, and helps to improve the effectiveness and expressive power of training data in complex reasoning tasks.

[0121] In one embodiment, prior to generating the training data, the method further includes:

[0122] Multi-level verification is performed on the question text and the analysis process data. The multi-level verification includes at least one of the following: shallow search failure verification, tool-less solution failure verification, explicit candidate verification success verification, uniqueness verification, and consistency verification.

[0123] In response to the multi-level verification of the question text and the analysis process data, the target answer, the candidate set, the question text, the association structure, and the analysis process data are encapsulated to generate training data.

[0124] Specifically, after generating the problem text and analysis process data, multi-level automatic verification can be performed to filter training samples that meet the quality requirements. The multi-level verification can be executed in a preset order or in parallel as needed to improve verification efficiency and reliability.

[0125] The shallow search failure check is used to determine whether the question text can be directly matched to the target answer using simple search methods. Specifically, the question text can be searched based on keyword matching or semantic similarity, and the ranking of the target answer in the search results can be analyzed. When the target answer is difficult to match directly in the shallow search results or is ranked low, the shallow search failure check is passed, thus ensuring that the question has a certain degree of difficulty to find.

[0126] The tool-free solution failure check is used to evaluate whether the correct answer can be obtained directly from the question text without relying on external tools or complex reasoning mechanisms. Specifically, the question can be solved using basic question-answering models or rule-based reasoning methods. When such simple methods fail to obtain the correct answer, the check is passed, thus ensuring that the question has the necessary depth of reasoning.

[0127] The explicit candidate validation success check is used to determine whether, given a candidate set, the candidates can be correctly selected based on the evidence information. Specifically, the candidate set can be progressively filtered based on supporting and excluding evidence in the association structure, and it can be verified whether it can converge to the target answer. When the filtering process can consistently obtain the target answer, the validation can be considered successful.

[0128] The uniqueness check is used to determine whether the target answer is unique under the current question text and associated structure. Specifically, it can detect whether there are multiple candidates that all meet the conditions under the evidence constraints. When only one candidate meets all the constraints, the uniqueness check can be passed, thereby avoiding the generation of ambiguous training data.

[0129] The consistency check is used to evaluate the semantic consistency between the question text, evidence information, and analysis process data. Specifically, it determines whether the information described in the question text matches the evidence information in the associated structure, and whether the analysis process data reasonably reflects the candidate selection process. When there are no obvious contradictions among the three, the consistency check is considered passed.

[0130] During the multi-level verification process described above, the generated data can be filtered based on various verification results. When the question text and the analysis process data pass a preset number or all verification conditions, the corresponding data is determined as a valid sample; otherwise, the question text or candidate set can be adjusted, or the sample can be discarded. After passing the verification, the target answer, the candidate set, the question text, the association structure, and the analysis process data are encapsulated according to a preset data structure to generate the final training data.

[0131] In this embodiment, by introducing a multi-level verification mechanism to screen the generated data layer by layer, it is possible to effectively ensure that the training data meets the preset requirements in terms of difficulty to find, verifiability and consistency, thereby improving the overall quality of the training data and enhancing the stability and generalization ability of the model in complex reasoning tasks.

[0132] Furthermore, in one embodiment, after the training data passes the multi-level verification, the training data can be graded by difficulty level to achieve hierarchical management and progressive use of the training data.

[0133] Specifically, the overall difficulty of the training data can be quantitatively calculated based on the complexity of the association structure and the difficulty of the verification process. The overall difficulty... It can be represented as:

[0134]

[0135] in, The graph depth represents the evidence relationship graph corresponding to the association structure, used to characterize the minimum number of reasoning jumps required to complete the reasoning; This represents the highest competitive score among the interfering candidates, reflecting the intensity of competition between them. This represents the probability that the untuned base model will directly provide the correct answer under zero-sample conditions; λ1, λ2, and λ3 are preset weight parameters. In one example, λ1=0.4, λ2=0.3, and λ3=0.3 can be set.

[0136] Furthermore, the graph depth The maximum competitiveness score can be calculated based on the shortest reasoning path length from candidate to target answer in the association structure; the longer the path, the more complex the reasoning process. The accuracy can be calculated based on the weights of misleading evidence corresponding to the interfering candidates, for example, by comparing the competitive indicators of each interfering candidate and taking the maximum value; the zero-sample accuracy... The answer can be obtained by inputting the question text into the base model for prediction and by calculating the probability that the model output matches the target answer.

[0137] After obtaining the overall difficulty level, the training data can be categorized according to a preset range. In one embodiment, the training data can be divided into multiple difficulty levels, such as basic reasoning level, complex elimination level, and deep reasoning level. Specifically, when the difficulty level is in a low range, the data can be classified as basic reasoning level to train the model's basic reasoning ability; when the difficulty level is in a medium range, the data can be classified as complex elimination level to train the model's filtering ability in a multi-candidate competitive environment; and when the difficulty level is in a high range, the data can be classified as deep reasoning level to train the model's analytical ability in complex evidence relationships and adversarial scenarios.

[0138] Furthermore, in one implementation, the training data can be used in stages based on the difficulty grading results. For example, training data can be gradually introduced in order of increasing difficulty, thereby achieving a progressive training process. In this way, the model can gradually adapt to inference tasks from simple to complex, thereby improving training efficiency and model performance.

[0139] Through the above implementation methods, by introducing a comprehensive difficulty assessment mechanism based on graph structure complexity, candidate competitiveness, and zero-shot prediction capability, it is possible to achieve refined hierarchical management of training data, thereby supporting the progressive training process of the model and helping to improve the learning effect and stability of the model in complex reasoning tasks.

[0140] In one embodiment, generating analysis process data corresponding to the question text based on the candidate set and the association structure includes:

[0141] Generate candidate sequences based on the candidate set;

[0142] The candidate sequences are screened based on the evidence information and the association structure, and candidates that do not meet the conditions are gradually eliminated according to the exclusion evidence in the association structure, so as to form the convergence process of the candidate branches.

[0143] Perform answer verification processing based on the filtered candidates;

[0144] The analysis process data records the generation, elimination, and convergence path of candidate branches, and includes at least one eliminated candidate branch and its corresponding elimination basis.

[0145] Specifically, a candidate sequence can first be constructed based on the candidate set. The candidate sequence can be arranged in a preset order, such as based on the similarity between the candidates and the target answer, the importance of the candidates, or a random strategy, thereby forming an initial candidate list. This candidate sequence serves as the starting state for subsequent screening and reasoning processes.

[0146] Furthermore, the candidate sequences can be progressively filtered based on the evidence information and the association structure. During the filtering process, supporting and excluding evidence in the association structure can be used to conditionally judge the candidates. When a candidate corresponds to excluding evidence, the candidate is removed from the candidate sequence; when a candidate is supported by supporting evidence, it can be retained for the next round of filtering. Through multiple rounds of filtering, the candidate range can be gradually narrowed, causing the candidate set to converge from a multi-branch state to a single result.

[0147] During the screening process, multiple candidate branch paths can be generated. Each candidate branch corresponds to a possible reasoning path and includes information on the candidate's state changes at each step. When a candidate is eliminated during the screening process, the specific basis for its elimination can be recorded, such as the corresponding exclusion evidence or unmet constraints, thus forming a branch record with the reason for elimination. By retaining at least one eliminated candidate branch and its exclusion basis, the analysis process data can contain complete reasoning path information.

[0148] After candidate selection is complete, answer verification can be performed based on the remaining candidates. Specifically, the final candidate can be verified for consistency by combining the evidence information in the association structure to confirm that the candidate satisfies all constraints and corresponds to the target answer. When the verification passes, the candidate is determined as the final result.

[0149] Furthermore, in one embodiment, the entire candidate screening process can be recorded in a structured manner, unifying the generation of candidate sequences, the step-by-step screening process, and the final convergence result into analysis process data. This analysis process data may include candidate state changes, evidence usage, and exclusion path information, thereby forming a complete reasoning trajectory.

[0150] In this embodiment, by constructing analysis process data based on candidate screening and evidence constraints, and recording the generation, elimination and convergence path of candidate branches, the training data can contain not only result information, but also the complete reasoning process, thereby enhancing the data's ability to express the model's reasoning ability and helping to improve the model's analysis and decision-making capabilities in complex multi-candidate scenarios.

[0151] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0152] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as cellular phones, smartphones, wearable devices, and other similar computing devices.

[0153] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 2 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0154] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103 and output device 1104 may be connected by a bus or other means, as shown in the figure, which is connected by a bus.

[0155] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0156] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0157] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0158] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0159] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0160] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0161] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0162] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0163] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0164] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0165] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0166] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural.

[0167] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for generating training data, characterized in that, The method includes: Obtain the target answer; A candidate set is constructed based on the attribute information of the target answer, the candidate set including the target answer and at least one interfering candidate; For the candidate set, evidence information corresponding to each candidate is obtained from the data source, and the evidence information is classified. The evidence information includes at least one of supporting evidence, excluding evidence, bridging evidence, or conflicting evidence. Based on the candidate set and the evidence information, a relationship structure between candidates and evidence is constructed to characterize the support and exclusion relationships between candidates. Question text is generated based on the aforementioned association structure, and the question text does not contain explicit information that directly points to the target answer; Before generating the question text based on the association structure, the method further includes: Calculate the validation asymmetric index between the target answer and the interference candidates based on the association structure; The verification asymmetric index is obtained by weighting the indicators of difficulty to find, ease of verification, competition, and analysis requirements. Determine whether the verification asymmetric index meets the preset conditions; In response to the verification asymmetric index satisfying the preset condition, the generation of question text based on the association structure is performed; Based on the candidate set and the association structure, generate analysis process data corresponding to the question text; The target answer, the candidate set, the question text, the association structure, and the analysis process data are encapsulated to generate training data.

2. The training data generation method according to claim 1, characterized in that, Obtaining the target answer includes: Extract candidate fact entities from multi-source data; Based on the consistency of multi-source information, time-varying characteristics, and the credibility of structured knowledge, a comprehensive evaluation is conducted on each candidate fact entity to obtain a stability score. Candidate fact entities whose stability scores are greater than a preset score threshold are identified as the target answer.

3. The training data generation method according to claim 1, characterized in that, The construction of the candidate set based on the attribute information of the target answer includes: Extract the attribute information of the target answer to generate a target feature representation; Based on structured data acquisition, sibling node entities with the same preset parent class node as the target answer are obtained; The similarity between each sibling node entity and the target feature representation is calculated based on feature similarity. Sibling node entities whose similarity falls within a preset similarity range are selected as interference candidates.

4. The training data generation method according to claim 1, characterized in that, The construction of the association structure between candidates and evidence based on the candidate set and the evidence information includes: Based on semantic reasoning, the relationship between the evidence information and the corresponding candidates in the candidate set is determined to identify a supporting relationship or an exclusion relationship. If the relationship is determined to be supportive, the corresponding evidence information is marked as the supporting evidence, and a positive association is established in the association structure. If it is determined to be an exclusion relationship, the corresponding evidence information is marked as the exclusion evidence, and a negative association is established in the association structure; Evidence information used to connect multiple pieces of evidence information is marked as bridging evidence, and evidence information with descriptive conflicts is marked as conflicting evidence.

5. The training data generation method according to claim 1, characterized in that, The generation of question text based on the association structure includes: The misleading feature information related to the interference candidates is incorporated into the question text; The constraints used to exclude the interfering candidates are implicitly embedded in the question text; The explicit entity information in the question text is hidden so that the question text does not contain entity identifier information that directly points to the target answer or the interference candidate.

6. The training data generation method according to claim 1, characterized in that, Prior to generating the training data, the method further includes: Multi-level verification is performed on the question text and the analysis process data. The multi-level verification includes at least one of the following: shallow search failure verification, tool-less solution failure verification, explicit candidate verification success verification, uniqueness verification, and consistency verification. In response to the multi-level verification of the question text and the analysis process data, the target answer, the candidate set, the question text, the association structure, and the analysis process data are encapsulated to generate training data.

7. The training data generation method according to claim 1, characterized in that, The step of generating analysis process data corresponding to the question text based on the candidate set and the association structure includes: Generate candidate sequences based on the candidate set; The candidate sequences are screened based on the evidence information and the association structure, and candidates that do not meet the conditions are gradually eliminated according to the exclusion evidence in the association structure, so as to form the convergence process of the candidate branches. Perform answer verification processing based on the filtered candidates; The analysis process data records the generation, elimination, and convergence path of candidate branches, and includes at least one eliminated candidate branch and its corresponding elimination basis.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images