Method, apparatus, device, medium and program product for processing generated content

Abstract

Embodiments of the present disclosure relate to methods, devices, equipment, media and program products for processing generated content, and relate to the fields of large model technology and large language model technology. The method includes generating a plurality of candidate generated content corresponding to the input content based on the user's input content. The method also includes using a first similarity algorithm to determine a group of candidate generated content that is not similar to the input content from a plurality of candidate generated content. The method also includes using a second similarity algorithm to determine a second similarity between the input content and the candidate generated content in a group of candidate generated content. The method also includes determining a combined similarity for the candidate generated content based on the first similarity and the second similarity. The method also includes determining a group of target generated content from a group of candidate generated content based on the combined similarity. Through this method, the comprehensive similarity in the generated content can be comprehensively evaluated, the diversity of the generated content is improved, and the accuracy of the quality inspection results is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of data processing, and more particularly to methods, devices, apparatuses, media, and program products for processing generated content. Background Art

[0002] At present, the development of the machine learning industry is becoming faster and faster. More and more different types of machine learning models have emerged and are widely used in different industries and fields. For example, vision-related models can be applied to visual inspection and intelligent driving, and language-related models can be applied to text processing and knowledge question-answering. There are also multimodal machine learning models that integrate various models with different focuses, which can be used in more complex operations that connect multiple different fields.

[0003] With the accelerated development of the machine learning industry, new technologies related to machine learning are emerging one after another, and the application scenarios of machine learning models are becoming more and more numerous. Especially in the field of text processing, the application of machine learning models is becoming more and more extensive. When processing data, there are more and more discussions on how to apply machine learning models and how to apply machine learning models more efficiently. For large amounts of data, applying machine learning models when processing will also encounter considerable challenges. Therefore, when faced with massive amounts of data, how to flexibly and efficiently use machine learning models to handle related data processing tasks has become a hot research topic. Summary of the invention

[0004] Embodiments of the present disclosure provide a method, apparatus, device, medium, and program product for processing generated content.

[0005] According to a first aspect of the present disclosure, a method for processing generated content is provided. The method includes generating a plurality of candidate generated content corresponding to the input content based on the user's input content. The method also includes determining a group of candidate generated content that is dissimilar to the input content from the plurality of candidate generated content using a first similarity algorithm, wherein the first similarity between the input content and the candidate generated content in the group of candidate generated content is less than a first threshold similarity. The method also includes determining a second similarity between the input content and the candidate generated content in the group of candidate generated content using a second similarity algorithm. The method also includes determining a combined similarity for the candidate generated content based on the first similarity and the second similarity. The method also includes determining a group of target generated content from the group of candidate generated content based on the combined similarity, wherein the combined similarity between the target generated content in the group of target generated content and the input content is less than a second threshold similarity.

[0006] In a second aspect of the present disclosure, a device for processing generated content is provided. The device includes a plurality of candidate generated content generation modules, configured to generate a plurality of candidate generated content corresponding to the input content based on the user's input content; a group of candidate generated content determination modules, configured to determine a group of candidate generated content that is not similar to the input content from the plurality of candidate generated content using a first similarity algorithm, wherein the first similarity between the input content and the candidate generated content in the group of candidate generated content is less than a first threshold similarity; a similarity determination module, configured to determine a second similarity between the input content and the candidate generated content in the group of candidate generated content using a second similarity algorithm; a combined similarity determination module, configured to determine a combined similarity for the candidate generated content based on the first similarity and the second similarity; and a group of target generated content determination modules, configured to determine a group of target generated content from the group of candidate generated content based on the combined similarity, wherein the combined similarity between the target generated content in the group of target generated content and the input content is less than a second threshold similarity.

[0007] In a third aspect of the present disclosure, an electronic device is provided, comprising at least one processor; and a storage device for storing at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements the method according to the first aspect of the present disclosure.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0009] In a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the method according to the first aspect of the present disclosure when executed by a processor.

[0010] It should be understood that the contents described in this content section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0012] Figure 1 A schematic diagram illustrating an example environment in which the apparatus and / or methods of some embodiments of the present disclosure may be implemented;

[0013] Figure 2A schematic diagram illustrating an example method for processing generated content according to some embodiments of the present disclosure is illustrated;

[0014] Figure 3 A schematic diagram illustrating an example of a system architecture for processing generated content according to some embodiments of the present disclosure;

[0015] Figure 4 A schematic diagram illustrating an example of adding input content according to some embodiments of the present disclosure is illustrated;

[0016] Figure 5 A schematic diagram illustrating an example of displaying a set of target-generated content according to some embodiments of the present disclosure is illustrated;

[0017] Figure 6 A schematic block diagram of an apparatus for processing generated content according to some embodiments of the present disclosure is illustrated;

[0018] Figure 7 A schematic block diagram of an example device suitable for implementing various embodiments of the present disclosure is illustrated.

[0019] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0020] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0021] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0022] For example, when receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0023] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0024] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0025] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0026] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0027] In the process of processing data, there are still many problems that need to be solved. For example, questions about the data source are generated based on the Large Language Model (LLM) to assist users in asking questions about the data source, so that users can implement data manipulation, indicator query, multi-dimensional statistics, etc. However, this faces the problem of low quality and high repetition rate of the input source, resulting in the generated text being highly repetitive and lacking in diversity. In addition, the quality detection method for the input data source is relatively simple, and it is impossible to comprehensively evaluate the quality and diversity of the generated text, especially in the scenario of large-scale text generation. It is impossible to efficiently conduct inter-processing and optimization, and it is impossible to ensure the timeliness and performance of the system.

[0028] In traditional solutions, the following methods can be used to implement data quality inspection. One method is a rule-based quality inspection method, which can use keyword matching to detect whether the generated text contains specific keywords through a pre-defined keyword list to evaluate the quality of the text. Regular expressions can also be used to match specific text patterns to detect the format and structure of the generated text. Another method is a statistical and machine learning method, which can use machine learning models for processing and use pre-trained machine learning models to evaluate and optimize the quality of the generated text. It is also possible to calculate cosine similarity, such as generating an embedding vector through a model, and then calculating the cosine similarity between the generated text and the reference text in the repository, and using this to evaluate the text similarity between the generated text and the reference text. Another method is an information retrieval-based method, which can use the BM-25 (Best Matching 25) retrieval algorithm to perform text retrieval and text similarity evaluation, and detect the repetitiveness and relevance of the generated text based on the retrieval results and the calculation results of the similarity. Although the above-mentioned methods can solve the quality inspection and optimization problems of the generated text to a certain extent, there are still some defects. For example, due to the limitations of a single algorithm, rule-based algorithms (such as keyword matching, regular expressions, etc.) are too dependent on predetermined rules, lack flexibility and adaptability, and are difficult to handle complex text generation scenarios. At the same time, machine learning-based methods have high computational complexity when dealing with large-scale text generation, and are difficult to meet real-time requirements. Although information retrieval-based methods (such as BM-25) perform well in text similarity assessment, a single algorithm is difficult to fully assess the diversity and quality of text. Moreover, the quality inspection standards for generated texts are relatively simple, and it is impossible to fully assess the diversity and quality of generated texts, which can easily lead to inaccurate quality inspection results and low quality inspection efficiency. The long quality inspection process is also difficult to meet the real-time and performance requirements of the system, which ultimately leads to a reduced user experience.

[0029] At least to solve the above and other potential problems, an embodiment of the present disclosure proposes a method for processing generated content. In this method, a computing device can first generate multiple candidate generated content corresponding to the input content based on the user's input content. Then the computing device uses a first similarity algorithm to determine a group of candidate generated content that is not similar to the input content from multiple candidate generated content. Subsequently, after determining a group of candidate generated content, the computing device uses a second similarity algorithm to determine the second similarity between the input content and the candidate generated content in a group of candidate generated content. Then, the computing device can determine the combined similarity for the candidate generated content according to the determined first similarity and second similarity. Finally, after determining the combined similarity, a group of target generated content is determined from a group of candidate generated content. Through this method, when performing quality inspection on multiple generated texts, by combining multiple different algorithms, the comprehensive similarity and repetitiveness in the generated text can be comprehensively evaluated, the diversity of the generated text is improved, and the accuracy and comprehensiveness of the quality inspection results are improved by combining different types of searches, and the quality of the generated text can be comprehensively evaluated from multiple dimensions, and the generated text is optimized in this way, which significantly improves the user experience.

[0030] The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. Figure 1 An example environment in which the device and / or method of an embodiment of the present disclosure can be implemented is shown. In environment 100, a computing device 102 is used to process input content 104, which can be content previously input by a user for query. The input content 104 can be completely consistent with the user's input text, and the input content 104 can also be a text after preprocessing the user's input text. Then, the computing device 102 processes the input content to obtain a plurality of different generated contents corresponding thereto.

[0031] Examples of computing device 102 include, but are not limited to, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant (PDA), a media player, etc.), a multiprocessor system, consumer electronics, a minicomputer, a mainframe computer, a distributed computing environment including any of the above systems or devices, and the like.

[0032] It is understandable that the present application is only described in combination with relevant embodiments, and the protection scope of the present application is not limited by the relevant embodiments. Any technical solutions in other applications that fall within the protection scope of the present application should be protected.

[0033] like Figure 1 As shown, first, in Figure 1In the environment 100, the computing device 102 first generates a plurality of candidate generated content 106 corresponding to the input content 104 based on the user's input content 104. The input content 104 may be pre-set and may be automatically applied to the text generation process by an automated script. After receiving the user's input content 104, the computing device 102 may also pre-process the input content 104, and then apply the pre-processed input content 104 to the machine learning model to generate a plurality of candidate generated content 106 for the input content 104.

[0034] After the plurality of candidate generated contents 106 are generated, the computing device 102 also uses the first similarity algorithm to calculate the similarity between the plurality of candidate generated contents 106 and the input content 104. If the first similarity 110 between the candidate content in the plurality of candidate generated contents 106 and the input content 104 is less than the first threshold similarity, the candidate content may be added to a group of candidate generated contents 108 that are not similar to the input content. If the first similarity 110 between the candidate content in the plurality of candidate generated contents 106 is greater than or equal to the first threshold similarity, the candidate content is not added to a group of candidate generated contents 108 that are not similar to the input content.

[0035] Subsequently, the computing device 102 further uses a second similarity algorithm to further determine a second similarity 112 between the input content 104 and a candidate generated content in a set of candidate generated content 108, and then further calculates and determines a combined similarity 114 for a candidate generated content in the set of candidate generated content based on the second similarity 112 determined for the candidate generated content in combination with the first similarity 110 corresponding thereto. Additionally, a set of combined similarities corresponding thereto can be calculated for the set of candidate generated content. Each candidate generated content in the set of candidate generated content has a corresponding combined similarity.

[0036] Finally, when the combined similarity 114 for a candidate generated content is less than the second threshold similarity, the candidate generated content is added to a group of target generated content as the target generated content. Therefore, at least one candidate generated content whose combined similarity is less than the second threshold similarity can be determined from the group of candidate generated content. Thus, the at least one candidate generated content is a group of target generated content 116 determined from the group of candidate generated content 108.

[0037] In some embodiments, the input content may be text, and the corresponding candidate generated content is the candidate generated text, so a group of target generated texts may be determined from a plurality of candidate generated texts corresponding to the input text. In some embodiments, the input content may be audio, and the corresponding candidate generated content is the candidate generated audio, wherein a group of target generated audios are determined from a plurality of candidate generated audios corresponding to the input audio. In some embodiments, the input content may be an image, and the corresponding candidate generated content is a candidate generated image, wherein a group of target generated images are determined from a plurality of candidate generated images corresponding to the input image. Through this method, when quality inspection is performed on a plurality of generated texts, by combining a plurality of different algorithms, the comprehensive similarity and repetitiveness in the generated content can be comprehensively evaluated, the diversity of the generated content is improved, and by combining a plurality of similarity algorithms, the accuracy and comprehensiveness of the quality inspection results are improved, and the quality of the generated content can be comprehensively evaluated from multiple dimensions, and the generated content can be optimized thereby, significantly improving the user experience.

[0038] Combined with the above Figure 1 A schematic diagram of an example environment in which the devices and / or methods of some embodiments of the present disclosure may be implemented is described below. Figure 2 A schematic diagram depicting an example method 200 for processing generated content according to some embodiments of the present disclosure. The example method may be Figure 1 Executed by computing device 102 or any suitable computing device.

[0039] like Figure 2 As shown, in the example method 200, at box 202, the computing device 102 generates a plurality of candidate generated contents corresponding to the input contents based on the user's input contents. For example, in order to increase the data available for training the model, the input contents that have been used may be processed to generate candidate generated contents corresponding thereto.

[0040] The input content may be input by the user into the computing device. In one example, the input content is a complete sentence or several phrases. In another example, the input content may be an image or a video. In yet another example, the input content may be audio content.

[0041] After the computing device receives the input content, it uses the machine learning model to process the input content and then generates multiple candidate content for the input content. In some embodiments, when the input content is text, the machine learning model can be a language model. In other embodiments, when the input content is an image or video, the machine learning model can be a visual model. In some other embodiments, when the input content is audio content, the machine learning model can be an audio model. Additionally, the machine learning model can be a multimodal model that combines text, image, and audio data processing capabilities.

[0042] At box 204, the computing device 102 uses a first similarity algorithm to determine a group of candidate generated content that is dissimilar to the input content from a plurality of candidate generated content, wherein a first similarity between the input content and the candidate generated content in the group of candidate generated content is less than a first threshold similarity.

[0043] When determining multiple candidate generated contents, a first similarity algorithm may be used to determine multiple similarities between the multiple candidate generated contents and the input content. Additionally, when the input content is an image or audio, the first similarity algorithm is a hierarchical navigable small-world graph algorithm; when the input content is text, a Boyer-Moore (BM) algorithm may be used to calculate the similarity.

[0044] In some embodiments, when the data volume of the input content is very large, for example, the data volume of the input content reaches a scale of tens of billions, the first similarity algorithm can use the inverted file index (IVF) part based on the vector computing library to perform preliminary screening on the data volume of tens of billions. It is understandable that any other similarity algorithm can be used as the first similarity algorithm, and this application does not limit it here.

[0045] In one example, after multiple similarities are calculated using the first similarity algorithm, the similarities among the multiple similarities can be compared with the size of the first threshold similarity. Then, a group of similarities that is less than the first threshold similarity is selected from the multiple similarities. Then, the computing device 102 determines a group of candidate generated content corresponding to the group of similarities from the multiple candidate generated content. In another example, after multiple similarities are calculated using the first similarity algorithm, the multiple similarities can be sorted, and then a group of similarities with a predetermined number of the smallest similarities is selected, and the first threshold similarity is a larger similarity behind the group of similarities.

[0046] In some embodiments, when the input content is one of an image, a video or an audio, a machine learning model can be used to first determine a first quantized representation of the input content and multiple quantized representations of multiple candidate generated contents. Then, a first similarity algorithm is used to determine a group of quantized representations that are dissimilar to the first quantized representation from the multiple candidate quantized representations, and then a group of candidate generated contents corresponding to the group of quantized representations is determined from the multiple generated contents.

[0047] In some embodiments, when the input content is text and the multiple candidate generated contents are multiple candidate generated texts, the first similarity algorithm may be a first text similarity algorithm. In this case, the computing device 102 may first determine the first text similarity of the input text and the candidate generated text in the multiple candidate generated texts according to the first text similarity algorithm. After the first text similarity is calculated using the first text similarity algorithm, the similarity in the first text similarity and the size of the first threshold similarity may be compared. Similarly, for the other candidate texts of the multiple candidate generated texts, the corresponding similarity will also be calculated with the input text. Therefore, there are multiple similarities corresponding to the multiple candidate texts. When the similarity in the multiple similarities is less than the first threshold similarity, the candidate generated text with the similarity less than the first threshold similarity is determined from the multiple candidate generated texts as the text in a group of candidate generated texts that are not similar to the input text. For example, the first threshold similarity is 0.3, and when the first similarity of the candidate text in the multiple candidate generated texts and the input text is less than 0.3, it is determined as the text in a group of candidate generated texts that are not similar to the input text.

[0048] At block 206, the computing device 102 determines a second similarity between the input content and the candidate generated content in the set of candidate generated content using a second similarity algorithm. After determining a set of candidate generated content, the computing device 102 further uses another similarity algorithm to calculate the similarity between the input content and the set of candidate generated content. In some embodiments, the first similarity algorithm is used for coarse-grained retrieval, while the second similarity algorithm is used for more fine-grained retrieval.

[0049] For example, as described above, after a group of candidate texts in a plurality of candidate texts is determined, the computing device may also use a second text similarity algorithm to determine a second text similarity between the input text and a candidate generated text in a group of candidate generated texts. Additionally, the computing device 102 may also use a third text similarity algorithm to calculate a third text similarity between the input text and a candidate generated text in a group of candidate generated texts.

[0050] Additionally, the second text similarity algorithm is a character granularity similarity algorithm, and the third text similarity algorithm is a word granularity similarity algorithm. In one example, the character granularity similarity algorithm may be a Jaro-winkler algorithm based on edit distance. In another example, the word granularity similarity algorithm may be a METEOR (metric for evaluation of translation with explicit ordering) algorithm.

[0051] Alternatively, the word-granularity similarity algorithm may also be a Levenshtein distance algorithm, which allows for inserting, deleting, and replacing characters in the process of calculating word-granularity similarity. The word-granularity similarity algorithm may also be a BLEU (Bilingual Evaluation Understudy, BLEU) algorithm, which evaluates the calculation quality by calculating the degree of overlap of n-grams between the machine translation result and one or more reference translations when calculating word-granularity similarity.

[0052] In some embodiments, when the input content is an image, the second similarity algorithm may be a hierarchical navigable small-world graph algorithm. In this case, the first similarity algorithm and the second similarity algorithm are the same. Additionally, the second dimension corresponding to the quantized representation of the candidate generated content using the second similarity algorithm is greater than the first dimension corresponding to the quantized representation of the candidate generated content obtained using the first similarity algorithm. In other words, the accuracy calculated by the first similarity algorithm is less than the accuracy calculated by the second similarity algorithm.

[0053] In some embodiments, when the input content is audio, the second similarity algorithm may be a Mel-spectrogram coefficient (MFCC) feature matching combined with dynamic time warping (DTW), wherein MFCC is used to extract features of the audio signal, and DTW is used to compare the similarity of two feature sequences, which is particularly suitable for processing audio signals with variable speed and pitch.

[0054] At block 208 , a combined similarity for the candidate generated content is determined based on the first similarity and the second similarity.

[0055] After the computing device calculates the first similarity and the second similarity by using the first similarity algorithm and the second similarity algorithm, it further determines the combined similarity for the candidate generated content according to the first similarity and the second similarity.

[0056] In some embodiments, a set of combined similarities for the candidate generated content may be determined by performing a weighted operation on the calculated first similarity and the second similarity, wherein the set of combined similarities also includes the previously calculated combined similarity.

[0057] In some embodiments, when the input content is text, after calculating the second text similarity and the third text similarity using the character granularity similarity algorithm and the word granularity similarity algorithm, the computing device further combines the previously calculated first text similarity to determine the combined similarity for the candidate generated texts. Then, a group of target texts is determined from a group of candidate generated texts based on the combined similarity.

[0058] Additionally, the combined similarity for a candidate generated text in a group of candidate generated texts may be determined by weighting the first text similarity, the second text similarity and the third text similarity corresponding to the candidate generated texts in the group of candidate generated texts.

[0059] At block 210 , based on the combined similarity, a group of target generated contents is determined from the group of candidate generated contents, wherein the combined similarity between the target generated contents and the input content in the group of target generated contents is less than a second threshold similarity.

[0060] After determining the combined similarity for a group of candidate generated content, the computing device 102 may filter the candidate generated content in the group of candidate generated content according to the size of the combined similarity, and determine the retained candidate generated content as a group of target generated content.

[0061] In some embodiments, the computing device 102 may filter a group of candidate generated content corresponding to a group of combined similarities according to a preset rule. For example, the candidate generated content corresponding to the combined similarity in the group of combined similarities being greater than the second threshold similarity is filtered and removed, thereby retaining the remaining dissimilar candidate generated content in the group of candidate generated content.

[0062] For example, candidate generated content corresponding to a group of combined similarities with a combined similarity greater than or equal to 0.5 is filtered and removed, thereby retaining the remaining candidate generated content with a combined similarity less than 0.5 in the group of candidate generated content.

[0063] After a set of target generated content is determined, it is also necessary to perform quality assessment on the determined set of target generated content. The main content of the quality assessment includes at least one of the following: logical consistency assessment, omission assessment, semantic consistency assessment or feedback assessment.

[0064] It is understandable that the method and basis of quality assessment can be set and changed by the user, and this application does not limit this. In addition, the target generated content that passes the quality assessment can be stored in a database, and the stored target generated content can be used for subsequent input content, thereby making it possible to generate new target generated content.

[0065] Through this method, when quality inspection is performed on multiple generated texts, by combining multiple different algorithms, the comprehensive similarities and repetitions in the generated texts can be comprehensively evaluated, thereby improving the diversity of the generated texts. In addition, the combination of different algorithms for searching improves the accuracy and comprehensiveness of the quality inspection results. The quality of the generated content can also be comprehensively evaluated from multiple dimensions, thereby optimizing the generated content and significantly improving the user experience.

[0066] Combined with the above Figure 2 A schematic diagram of an example method for processing generated content according to some disclosed embodiments is described. Figure 3 A schematic diagram depicting an example of a system architecture for processing generated content according to some embodiments of the present disclosure.

[0067] like Figure 3 As shown in example 300, taking the input content as text content as an example, after the user enters text in the search engine, the computing device starts a processing flow for the input text. First, the computing device inputs the input content into the text generation module 302, and then uses the language model to process the input content, thereby generating multiple generated texts.

[0068] In some embodiments, the computing device is generated based on input text for the user, and after receiving the input text for the user, the computing device may apply the input text to a machine learning model to generate a plurality of candidate generated texts for the input text.

[0069] After the computing device generates multiple candidate generated texts for the input text, it will perform a preliminary search 304 on the multiple generated texts. At the preliminary search module 306, a first similarity algorithm such as the BM25 algorithm can be used to process the input text to determine the first similarity between the input text and a set of candidate generated texts.

[0070] Subsequently, the computing device performs candidate text screening 308 on a set of candidate generated texts. Text screening is to screen a set of candidate generated texts determined by fine search 310 from a more fine-grained aspect.

[0071] For example, the second similarity algorithm may be a character granularity similarity algorithm targeting character granularity, and the third similarity algorithm may be a word granularity similarity algorithm targeting word granularity.

[0072] In the fine search module 312 , a group of candidate generated texts are finely searched from two search aspects: character granularity search 314 and word granularity search 316 .

[0073] In some embodiments, the second similarity algorithm may use the Jaro-Winkler algorithm 318 based on edit distance, and the third similarity algorithm may use the METEOR algorithm 320. Subsequently, a set of similarities of a set of candidate generated texts is determined according to the second similarity, and a set of similarities of a set of candidate generated texts is determined according to the third similarity algorithm.

[0074] Among them, the third similarity algorithm (METEOR algorithm) relies on finding reference words from the vocabulary 322 for comparison to determine a set of similarities for a set of candidate texts.

[0075] In some embodiments, the second similarity algorithm may also be a Levenshtein distance algorithm, which allows for inserting, deleting, and replacing characters in the process of calculating word-granularity similarity. The third similarity algorithm may also be a BLEU (Bilingual Evaluation Understudy, BLEU) algorithm, which evaluates the calculation quality by calculating the n-gram overlap between the machine translation result and one or more reference translations when calculating word-granularity similarity.

[0076] In some embodiments, the second similarity algorithm may also be a Damerau-Levenshtein distance algorithm, which adds an operation of exchanging two adjacent characters in addition to the basic operation of the Levenshtein distance algorithm. The third similarity algorithm may also be a Translation Edit Rate (TER) algorithm, which calculates the number of editing operations (such as insertion, deletion, replacement, and movement) required to convert the machine translation result into a reference translation when calculating the word granularity similarity.

[0077] In some embodiments, the second similarity algorithm may also be a Hamming distance algorithm, which is applicable to equal-length strings, represents the number of different characters at corresponding positions of two strings, and allows replacement operations. The third similarity algorithm may also be a ROUGE algorithm (Recall-Oriented Understudy for Gisting Evaluation, ROUGE), which is mainly used for summary evaluation, but can also be used for translation evaluation, calculating the overlap of n-grams, word order, and the longest common subsequence.

[0078] In some embodiments, the second similarity algorithm may also be the Needleman-Wunsch algorithm, which is mainly used for sequence alignment in bioinformatics, allows insertion, deletion and replacement operations, and can customize the weight of each operation. The third similarity algorithm may also be the chrF algorithm, which, because it is based on the character F-score, takes into account the n-gram overlap at the character level.

[0079] In some embodiments, the second degree similarity algorithm may also be the Smith-Waterman algorithm, which is also used for local sequence alignment in bioinformatics, allows insertion, deletion and substitution operations, and is suitable for finding similar subsequences in two sequences.

[0080] Subsequently, after a set of target generated texts are determined, the set of target generated texts will be quality checked and optimized 324 .

[0081] First, a set of target generated texts are applied to the quality inspection and optimization module 326, and then a comprehensive similarity evaluation 328 is performed. In this process, the computing device performs a weighted operation based on the first similarity, the second similarity and the third similarity to determine a set of combined similarities for a set of candidate generated texts.

[0082] Then, a repetitive filtering 330 is performed to filter the corresponding candidate generated texts whose combined similarity in the determined set of combined similarities is greater than or equal to the second threshold similarity (for example, 0.3), and retain the corresponding candidate generated texts whose combined similarity in the determined set of combined similarities is less than the second threshold similarity.

[0083] After filtering out duplicate texts in a set of candidate generated texts, the computing device determines the retained candidate generated texts as a set of target generated texts, and performs quality assessment 332 on the set of target generated texts.

[0084] In some embodiments, the logical consistency evaluation for a set of target generated content may include scoring the set of target generated content using a machine learning model. For example, the target generated content in a set of target generated content is "How is the weather in Beijing today?" and is applied to a machine learning model (e.g., a large language model) to convert the target generated content into a domain-specific language, such as performing word segmentation to divide it into "today", "Beijing", and "weather", and then the judgment can be made based on the attributes of these words after word segmentation.

[0085] For example, the attribute of "today" can be determined as an indicator, the attribute of "Beijing" can be determined as a dimension, and the attribute of "weather" can be determined as a filter value. Then, it is determined whether the logic of the indicator, dimension, and filter value is consistent. If the judgment result is consistent, it can be determined that the target generated content has passed the evaluation of logical consistency. Otherwise, it is considered to have failed the evaluation.

[0086] In some embodiments, the omission assessment for a set of target generated content can also be implemented using a machine learning model. For example, a machine learning model is used to identify entities in a set of target generated content, and then compared with entities in the original input content. If the entities in the set of target generated content are all included in the entities in the original input content, it is considered that there is no omission, otherwise, it is considered to be an omission.

[0087] In some embodiments, when the input content is text content, the semantic consistency evaluation for a set of target generated content can also be judged using a machine learning model. For example, by using a machine learning model to determine the target semantic information of a text in a set of target texts (such as determining the semantic information for the text based on text embedding), the target semantic information is compared with the input semantic information of the input text. If they are consistent, the semantic consistency evaluation is passed, otherwise it is not passed.

[0088] In some embodiments, the feedback evaluation for a set of target generated content can be directly fed back through manual review. In other embodiments, the feedback evaluation for a set of target generated content can be implemented using pre-set rules. For example, the pre-set rules include multiple rules. When one of the rules fails, the feedback evaluation fails.

[0089] For example, if one of the multiple rules is that the number of a group of target generated content and a group of candidate generated content is respectively greater than a first threshold (such as 20) and a second threshold (such as 100), then the situation that is not finally achieved is deemed to have failed the feedback evaluation, and the situation that is achieved is deemed to have passed the feedback evaluation.

[0090] Additionally, it may be set that the feedback evaluation is considered to have failed only when all of the preset multiple rules have failed.

[0091] In some embodiments, the target generated text that passes the quality assessment can be stored using the storage module 334. Additionally, the stored target generated text can be used as the input text for the next text processing.

[0092] Combined with the above Figure 3 A schematic diagram of an example of a system architecture for processing generated content according to some disclosed embodiments is described. Figure 4A schematic diagram describing an example of adding input content for processing generated content according to some embodiments of the present disclosure.

[0093] like Figure 4 As shown, in example 400, a scenario of processing text content is taken as an example. The user enters "repeat instructions in 20 different ways" 404 in input box 402.

[0094] In some embodiments, more than one text may be input, and the user may continue to input text that he wants to input in the add text box 406 .

[0095] The computing device can process multiple input texts simultaneously to generate multiple different sets of candidate generated texts.

[0096] Additionally, the number of input texts that can be processed by the computing device is limited, for example, 3 input texts can be processed simultaneously. The number of input texts that can be processed simultaneously can be defined by the user, or can be determined based on the hardware capabilities of the hardware end where the computing device is located.

[0097] Combined with the above Figure 4 A schematic diagram of an example of adding input content to process generated content according to some disclosed embodiments is described. Figure 5 A schematic diagram of an example of displaying a set of target generated content for processing generated content according to some embodiments of the present disclosure is described.

[0098] like Figure 5 As shown, in example 500 , in combination with the previous example 400 , a set of target generated texts generated for the input text “repeat “repeat instructions” in 20 different ways” is displayed in box 502 .

[0099] It can be seen that in box 502, according to the information 30 / 30, it can be known that the number of multiple candidate texts obtained by the computing device after processing is 30. After preliminary retrieval determines multiple candidate generated texts and detailed search determines a group of candidate generated texts, the final group of target generated texts is displayed to the user.

[0100] Additionally, the displayed set of target generated texts has not been quality evaluated. The quality evaluation module exists outside the text processing module and can be turned on or off by the user.

[0101] like Figure 6As shown, the device 600 includes a plurality of candidate generated content generation modules 602, which are configured to generate a plurality of candidate generated contents corresponding to the input contents based on the user's input contents; a group of candidate generated content determination module 604, which is configured to determine a group of candidate generated contents that are not similar to the input contents from the plurality of candidate generated contents using a first similarity algorithm, wherein the first similarity between the input contents and the candidate generated contents in the group of candidate generated contents is less than a first threshold similarity; a similarity determination module 606, which is configured to determine a second similarity between the input contents and the candidate generated contents in the group of candidate generated contents using a second similarity algorithm; a combined similarity determination module 608, which is configured to determine a combined similarity for the candidate generated contents based on the first similarity and the second similarity; and a group of target generated content determination module 610, which is configured to determine a group of target generated contents from the group of candidate generated contents based on the combined similarity, wherein the combined similarity between the target generated contents in the group of target generated contents and the input contents is less than a second threshold similarity.

[0102] In some embodiments, a group of candidate generated content determination modules 604 include: multiple similarity calculation modules, configured to use a first similarity algorithm to calculate multiple similarities between the input content and multiple candidate generated content; a group of similarity selection modules, configured to select a group of similarities less than a first threshold similarity from multiple similarities; and a group of candidate generated content determination modules, configured to determine a group of candidate generated content corresponding to a group of similarities from multiple candidate generated content.

[0103] In some embodiments, the input content is input text, the multiple candidate generated contents are multiple candidate generated texts, a group of candidate generated contents is a group of candidate generated texts, and the multiple similarity calculation modules include: a first text similarity algorithm utilization module, configured to utilize a first text similarity algorithm to determine multiple similarities between the input text and the multiple candidate generated texts.

[0104] In some embodiments, the similarity determination module 606 includes: a second text similarity determination module, configured to use a second text similarity algorithm to determine the second text similarity between the input text and a candidate generated text in a set of candidate generated texts; the combined similarity determination module 608 includes a third text similarity module, configured to use a third text similarity algorithm to determine the third text similarity between the input text and a candidate generated text in a set of candidate generated texts; and a first combined similarity determination module, configured to determine the combined similarity for the candidate generated text based on the first text similarity, the second text similarity and the third text similarity.

[0105] In some embodiments, the second text similarity algorithm is a character granularity similarity algorithm, and the third text similarity algorithm is a word granularity similarity algorithm.

[0106] In some embodiments, the first combined similarity determination includes: a weighting module configured to determine the combined similarity for the candidate generated text by performing a weighting operation on the first text similarity, the second text similarity, and the third text similarity.

[0107] In some embodiments, a set of target generated content determination modules 610 include: a set of combined similarity determination modules, configured to determine a set of combined similarities for a set of candidate generated content, a set of combined similarities including combined similarities; and a filtering module module, configured to generate a set of target generated content by filtering a set of candidate generated content based on a set of combined similarities.

[0108] In some embodiments, the filtering module includes: a filtering module with a similarity greater than a second threshold value, configured to filter out candidate generated content with a combined similarity greater than a second threshold value in a group of candidate generated content based on a group of combined similarities; and a group of target generated content determination module, configured to determine the filtered group of candidate generated content as a group of target generated content.

[0109] In some embodiments, the apparatus 600 further includes: a quality assessment module configured to perform quality assessment on a set of target generated contents, wherein the quality assessment includes at least one of the following: logical consistency assessment, omission assessment, semantic consistency assessment or feedback assessment.

[0110] In some embodiments, the multiple candidate generated content generation module 602 includes: a machine learning model application module, configured to use a machine learning model to generate multiple candidate generated content corresponding to the input content.

[0111] In some embodiments, the input content is one of audio or image, and a group of candidate generated content determination module 604 includes: a first quantization representation and multiple quantization representation determination module, configured to determine a first quantization representation of the input content and multiple quantization representations corresponding to multiple candidate generated contents; a group of quantization representation determination module, configured to determine a group of quantization representations that are dissimilar to the first quantization representation from multiple quantization representations using a first similarity algorithm; and a group of candidate generated content determination module, configured to determine a group of candidate generated contents corresponding to a group of quantization representations from multiple generated contents.

[0112] In some embodiments, the first similarity algorithm is a hierarchical navigable small-world graph algorithm.

[0113] In some embodiments, where the input content is an image, the second similarity algorithm is a hierarchical navigable small-world graph algorithm, and a first dimension of a quantized representation of the candidate generated content for the first similarity algorithm is smaller than a second dimension of a quantized representation of the candidate generated content for the second similarity algorithm.

[0114] In some embodiments, where the input is audio, the second similarity algorithm includes Mel-spectrogram coefficient feature matching and dynamic time warping.

[0115] Figure 7 A schematic block diagram of an example device 700 that may be used to implement embodiments of the present disclosure is shown. Figure 1 The computing device 102 in the embodiment can be implemented by using the device 700. As shown in the figure, the device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or computer program instructions loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0116] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks. High pitch technique, voice training, articulation, breathing, falsetto and correct singing method 2.

[0117] The various processes and processing described above, such as the example method 200 and examples 300 to 500, may be performed by the processing unit 701. For example, in some embodiments, the example method 200 and examples 300 to 500 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU 701, one or more actions of the example method 200 and examples 300 to 500 described above may be performed.

[0118] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0119] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive lists) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer-readable storage medium used herein is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0121] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0122] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0125] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0126] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing generated content, comprising: Based on the user's input content, generating a plurality of candidate generated content corresponding to the input content; Determine a group of candidate generated contents that are dissimilar to the input content from the plurality of candidate generated contents by using a first similarity algorithm, wherein a first similarity between the input content and the candidate generated contents in the group of candidate generated contents is less than a first threshold similarity; Determining a second similarity between the input content and the candidate generated content in the set of candidate generated content by using a second similarity algorithm; Determining a combined similarity for the candidate generated content based on the first similarity and the second similarity; as well as Based on the combined similarity, a group of target generated contents is determined from the group of candidate generated contents, wherein the combined similarity between the target generated contents in the group of target generated contents and the input content is less than a second threshold similarity.

2. The method according to claim 1, wherein determining a group of candidate generated contents that are dissimilar to the input content from the plurality of candidate generated contents using a first similarity algorithm comprises: Calculating multiple similarities between the input content and the multiple candidate generated contents using the first similarity algorithm; selecting a set of similarities from the plurality of similarities that are less than a first threshold similarity; as well as The set of candidate generated contents corresponding to the set of similarities is determined from the plurality of candidate generated contents.

3. The method according to claim 2, wherein the input content is an input text, the multiple candidate generated contents are multiple candidate generated texts, the group of candidate generated contents is a group of candidate generated texts, and using the first similarity algorithm to calculate multiple similarities between the input content and the multiple candidate generated contents comprises: A first text similarity algorithm is used to determine a first text similarity between the input text and the plurality of candidate generated texts.

4. The method according to claim 3, wherein determining the second similarity between the input content and the candidate generated content in the set of candidate generated content using a second similarity algorithm comprises: Determine, using a second text similarity algorithm, a second text similarity between the input text and the candidate generated text in the set of candidate generated texts; and Wherein, based on the first similarity and the second similarity, determining the combined similarity for the candidate generated content includes: Determine a third text similarity between the input text and the candidate generated text in the set of candidate generated texts by using a third text similarity algorithm; as well as Based on the first text similarity, the second text similarity and the third text similarity, a combined similarity for the candidate generated text is determined. 5 . The method according to claim 4 , wherein the second text similarity algorithm is a character granularity similarity algorithm, and the third text similarity algorithm is a word granularity similarity algorithm.

6. The method according to claim 4, wherein determining the combined similarity for the candidate generated text based on the first text similarity, the second text similarity and the third text similarity comprises: The combined similarity for the candidate generated text is determined by performing a weighted operation on the first text similarity, the second text similarity and the third text similarity.

7. The method according to claim 1, wherein determining a set of target generated contents from the set of candidate generated contents comprises: determining a set of combined similarities for the set of candidate generated content, the set of combined similarities comprising the combined similarity; as well as The set of target generated contents is generated by filtering the set of candidate generated contents based on the set of combined similarities.

8. The method according to claim 7, wherein generating the set of target generated contents by filtering the set of candidate generated contents comprises: Based on the set of combined similarities, filtering out candidate generated content having a combined similarity greater than a second threshold similarity from the set of candidate generated content; as well as The filtered set of candidate generated contents is determined as the set of target generated contents.

9. The method according to claim 7, further comprising: A quality assessment is performed on the set of target generated contents, wherein the quality assessment includes at least one of the following: a logical consistency assessment, an omission assessment, a semantic consistency assessment, or a feedback assessment.

10. The method according to claim 1, wherein generating a plurality of candidate generated contents corresponding to the input contents comprises: The plurality of candidate generated contents corresponding to the input contents are generated using a machine learning model.

11. The method according to claim 1, wherein the input content is one of audio or image, and determining a group of candidate generated content that is dissimilar to the input content from the plurality of candidate generated content using a first similarity algorithm comprises: determining a first quantized representation of the input content and a plurality of quantized representations corresponding to the plurality of candidate generated content; Determining, using a first similarity algorithm, from the plurality of quantized representations a set of quantized representations that are dissimilar to the first quantized representation; as well as A set of candidate generated content corresponding to the set of quantized representations is determined from the plurality of generated content.

12. The method of claim 11, wherein the first similarity algorithm is a hierarchical navigable small-world graph algorithm.

13. The method according to claim 12, wherein the input content is an image, the second similarity algorithm is a hierarchical navigable small-world graph algorithm, and a first dimension of the quantized representation of the candidate generated content for the first similarity algorithm is smaller than a second dimension of the quantized representation of the candidate generated content for the second similarity algorithm.

14. The method according to claim 12, wherein the input content is audio, and the second similarity algorithm includes Mel-spectrogram coefficient feature matching and dynamic time warping.

15. An apparatus for processing generated content, comprising: A plurality of candidate generated content generating modules, configured to generate a plurality of candidate generated content corresponding to the input content based on the user's input content; a group of candidate generated content determination module, configured to determine a group of candidate generated content that is dissimilar to the input content from the plurality of candidate generated content using a first similarity algorithm, wherein a first similarity between the input content and the candidate generated content in the group of candidate generated content is less than a first threshold similarity; a similarity determination module configured to determine a second similarity between the input content and the candidate generated content in the set of candidate generated content by using a second similarity algorithm; a combined similarity determination module, configured to determine a combined similarity for the candidate generated content based on the first similarity and the second similarity; as well as A group of target generated content determination module is configured to determine a group of target generated content from the group of candidate generated content based on the combined similarity, wherein the combined similarity between the target generated content in the group of target generated content and the input content is less than a second threshold similarity.

16. An electronic device, comprising: at least one processor; as well as A storage device, used to store at least one program, when the at least one program is executed by the at least one processor, so that the at least one processor implements the method according to any one of claims 1-14.

17. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 14 when executed by a processor.

18. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 14.

Images