Information processing device, information processing method, and program

Abstract

The present technology pertains to an information processing device, an information processing method, and a program that make it possible to reduce the risk of copyright infringement. In the present invention, generated text composed of tokens generated by sampling is corrected by replacement or resampling on the basis of similarity in terms of copyright infringement on the existing literary work. The present technology can be applied, for example, to the case of generating lyrics or the like.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program, and in particular to an information processing device, an information processing method, and a program that can reduce the risk of copyright infringement, for example.

[0002] For example, Patent Document 1 describes a technology for generating lyrics that harmonize with a melody using a generative model (generative AI (artificial intelligence)).

[0003] International Publication No. 2023 / 112534

[0004] There are concerns that content generated using generative models may pose a risk of copyright infringement.

[0005] Since text such as lyrics generated using generative models may also be at risk of copyright infringement, there is a demand for proposals for technologies to generate text that does not infringe copyright, mechanisms for checking copyright infringement, and other technologies to reduce the risk of copyright infringement.

[0006] This technology was developed in light of these circumstances, and makes it possible to reduce the risk of copyright infringement.

[0007] The information processing device or program of the present technology is an information processing device that has a correction unit that corrects generated text composed of tokens generated by sampling by replacement or resampling based on the similarity of the generated text to existing works in terms of copyright infringement, or a program for causing a computer to function as such an information processing device.

[0008] The information processing method of the present technology is an information processing method that includes modifying a generated text, which is composed of tokens generated by sampling, by replacement or resampling based on the similarity of the generated text to an existing work in terms of copyright infringement.

[0009] In this technique, generated text made up of tokens generated by sampling is modified by substitution or resampling based on the similarity of the generated text to existing copyrighted works in terms of copyright infringement.

[0010] The information processing device may be an independent device or an internal block constituting a single device.

[0011] The program can be provided by transmitting it via a transmission medium or by recording it on a recording medium.

[0012] 1 is a block diagram showing an example of a configuration of an embodiment of an information processing device to which the present technology is applied. FIG. 2 is a flowchart illustrating an example of automatic correction processing of the information processing device 10. FIG. 3 is a block diagram showing an example of a configuration of another embodiment of an information processing device to which the present technology is applied. FIG. 4 is a flowchart illustrating an example of interactive processing of the information processing device 10. FIG. 5 is a diagram illustrating sampling performed by a sampling unit 25. FIG. 6 is a diagram illustrating a substitution dictionary stored in a substitution dictionary storage unit 24. FIG. 7 is a diagram illustrating an example of correction of generated text by substitution. FIG. 8 is a flowchart illustrating an example of a process performed by a similarity evaluation unit 26 to calculate a similarity that represents the similarity of generated text to existing lyrics in terms of copyright infringement. FIG. 9 is a flowchart illustrating an example of a process performed by a forbidden expression evaluation unit 27 to evaluate whether generated text contains forbidden expressions. FIG. 10 is a diagram illustrating examples of forbidden expressions stored in a forbidden expression storage unit 17. FIG. 11 is a diagram illustrating an example of a display of a presentation UI generated by a presentation control unit 29. FIG. 12 is a diagram illustrating an example of a display of a presentation UI 50 in a state where a portion of interest is selected in generated text as lyrics displayed in a lyrics field 52. FIG. 13 is a diagram illustrating an example of a display of details regarding the similarity of similar melody portions. FIG. 14 is a diagram illustrating an example of a display of a similarity information field 56. FIG. 15 is a block diagram showing an example of a configuration of a computer to which the present technology is applied.

[0013] <Information Processing Apparatus to Which the Present Technology is Applied> FIG. 1 is a block diagram showing an example of the configuration of an embodiment of an information processing apparatus to which the present technology is applied.

[0014] In FIG. 1 , the information processing device 10 functions as a device for generating text, in this case, lyrics, using various input information from a user, and assists the user in creating lyrics. Lyric text generated by the information processing device 10 is also referred to as generated text. The information processing device 10 reduces the risk of copyright infringement of the generated text by modifying the generated text through substitution or resampling based on the similarity of the generated text as lyrics to existing works in terms of copyright infringement. Note that the information processing device 10 can also be applied to generating text other than lyrics, such as poetry. Furthermore, the information processing device 10 can also be applied to generating content such as music (audio) and images.

[0015] The information processing device 10 has a manual similarity evaluation result storage unit 11, a similarity evaluation function learning unit 12, an artist information storage unit 13, a song information storage unit 14, a lyrics information storage unit 15, a prohibited expression determination unit 16, a prohibited expression storage unit 17, and a generation unit 20.

[0016] The manual similarity evaluation result storage unit 11 stores evaluation result information related to the evaluation results of similarity evaluations made by various people regarding the similarity of various phrases from the viewpoint of copyright infringement when the phrases are compared with existing works, such as existing lyrics. The evaluation result information can be labeled to distinguish between evaluation results by experts in lyrics, such as lyricists, composers, and music producers, and evaluation results by non-experts (non-experts).

[0017] For simplicity of explanation, we will use existing lyrics as the existing work here, but in addition to existing lyrics, existing novels, movies (titles, etc.), and other content can also be used as the existing work.

[0018] The similarity evaluation function learning unit 12 uses the information stored in the manual similarity evaluation result storage unit 11 to train a learning model as a similarity evaluation function for evaluating similarity based on a measure of similarity perceived by humans, and supplies the model to (the similarity evaluation unit 26 of) the generation unit 20. In addition to the information stored in the manual similarity evaluation result storage unit 11, the processing of the similarity evaluation function learning unit 12 can use information stored in the artist information storage unit 13, the song information storage unit 14, and the lyric information storage unit 15, etc.

[0019] The artist information storage unit 13 stores artist information about existing artists who provide music (including music and lyrics, if any). The artist information includes meta information about the artist, such as the artist's name, the artist's popularity, and the artist's relationship to the music they provide. The artist's relationship to the music they provide may, for example, indicate whether the artist is the lyricist or composer of the music, or whether the artist is a singer.

[0020] The music information storage unit 14 stores music information about existing music pieces. The music information includes meta information about the music pieces, such as the name of the music piece, musical score, and part information about the music piece (such as melody, verse, chorus, etc.).

[0021] The lyrics information storage unit 15 stores lyrics information relating to lyrics of existing songs. The lyrics information includes not only text as lyrics but also meta-information about lyrics such as section information.

[0022] The prohibited expression determination unit 16 determines text as prohibited expressions using the lyric information stored in the lyric information storage unit 15 as needed, and supplies the determined text to the prohibited expression storage unit 17. For example, text that is estimated to be highly likely to infringe copyright, such as a neologism that is dependent on a specific song or artist (a neologism that evokes a specific song or artist), can be determined to be prohibited expressions. In addition, text such as ethically inappropriate words or phrases (including words) that should not appear in generated text can be determined to be prohibited expressions.

[0023] The forbidden expression storage unit 17 stores the forbidden expressions from the forbidden expression determination unit 16 .

[0024] The generation unit 20 uses the learning model from the similarity evaluation function learning unit 12, and the information stored in the artist information storage unit 13 to the lyrics information storage unit 15, and the prohibited expression storage unit 17 as needed, and generates generated text as lyrics using a generation model (generation AI).

[0025] The generation unit 20 includes a prompt input unit 21 , a melody input unit 22 , a token embedding storage unit 23 , a substitution dictionary storage unit 24 , a sampling unit 25 , a similarity evaluation unit 26 , a forbidden expression evaluation unit 27 , a replacement unit 28 , and a presentation control unit 29 .

[0026] The prompt input unit 21 and melody input unit 22 accept input information from the user and supply it to the sampling unit 25. Examples of input information include prompts to be given to the generative model, and vocal melody (musical score data) of a song to which lyrics (generated text) generated by the generation unit 20 will be added. The prompt input unit 21 is an IF (interface) that inputs prompts to be given to the generative model, accepts the prompts, and supplies them to the sampling unit 25. The melody input unit 22 is an IF that inputs melodies such as vocals, accepts the melodies, and supplies them to the sampling unit 25.

[0027] The token embedding storage unit 23 stores tokens generated by sampling using a generative model and their token embeddings, i.e., vectors (embedded representations) that represent the tokens in a predetermined vector space, in association with each other. The sampling unit 25 and the replacement unit 28 refer to the token embedding storage unit 23 as necessary, and convert the tokens into vectors for processing.

[0028] The substitution dictionary storage unit 24 stores a substitution dictionary, which will be described later.

[0029] The sampling unit 25 generates tokens by using a generative model configured, for example, by LLMs (large language models) and performing sampling in response to information input by the user from the prompt input unit 21 and the melody input unit 22. A token is a unit generated by sampling using a generative model, such as one character, one word, or one morpheme. The sampling unit 25 supplies generated text configured from the tokens generated by sampling to the similarity evaluation unit 26.

[0030] The sampling unit 25 also functions as a correction unit that corrects the generated text by resampling, based on the similarity of the generated text to existing lyrics in terms of copyright infringement. That is, the sampling unit 25 corrects the generated text by resampling based on the evaluation results of the similarity evaluation of the generated text from the similarity evaluation unit 26, which will be described later. The sampling unit 25 supplies the corrected generated text to the similarity evaluation unit 26.

[0031] The sampling unit 25 can also correct the generated text by resampling based on the evaluation result of the forbidden expression evaluation from the forbidden expression evaluation unit 27, which will be described later. For example, if the evaluation result of the forbidden expression evaluation indicates that the generated text contains a forbidden expression, the sampling unit 25 corrects the generated text by resampling so that the forbidden expression is not included in the generated text.

[0032] The similarity evaluation unit 26 is supplied with the generated text (including the generated text after correction) from the sampling unit 25. The similarity evaluation unit 26 is also supplied with the generated text from the replacement unit 28, that is, the generated text after correction by replacement, as will be described later.

[0033] The similarity evaluation unit 26 calculates a similarity that indicates the similarity of the generated text from the sampling unit 25 or the replacement unit 28 to existing lyrics in terms of copyright infringement, as a similarity evaluation. The similarity calculation is performed using the learning model from the similarity evaluation function learning unit 12 and the information stored in the artist information storage unit 13 to the lyric information storage unit 15 as needed. The similarity evaluation unit 26 supplies the generated text and the similarity as an evaluation result of the similarity evaluation to the sampling unit 25, the forbidden expression evaluation unit 27, and the replacement unit 28. Here, the larger the similarity value, the higher the similarity.

[0034] Here, if the similarity of the generated text indicates a high risk of copyright infringement of existing lyrics by that generated text, for example, if the similarity is equal to or greater than a threshold, the sampling unit 25 corrects by resampling the generated text from the similarity evaluation unit 26 (the generated text whose similarity has been calculated) so that expressions with a similarity equal to or greater than the threshold are not included in the generated text. Similarly, the replacement unit 28, which will be described later, corrects by replacement the generated text from the similarity evaluation unit 26 if the similarity of the generated text indicates a high risk of copyright infringement of existing lyrics by that generated text.

[0035] The forbidden expression evaluation unit 27 performs a forbidden expression evaluation to evaluate whether or not a forbidden expression is included in the generated text from the similarity evaluation unit 26. The forbidden expression evaluation is performed using the forbidden expressions stored in the forbidden expression storage unit 17. The forbidden expression evaluation unit 27 supplies the generated text and the evaluation result of the forbidden expression evaluation of the generated text (whether or not a forbidden expression is included in the generated text, and if so, the forbidden expression included) to the sampling unit 25, the forbidden expression evaluation unit 27, and the presentation control unit 29.

[0036] The replacement unit 28 functions as a correction unit that corrects the generated text by substitution using a substitution dictionary stored in the replacement dictionary storage unit 24, based on the similarity of the generated text to existing lyrics in terms of copyright infringement. That is, when the similarity evaluation unit 26 indicates that the similarity of the generated text is equal to or greater than a threshold, the replacement unit 28 corrects the generated text by substitution so that expressions with a similarity equal to or greater than the threshold are not included in the generated text. The replacement unit 28 supplies the corrected generated text to the similarity evaluation unit 26.

[0037] The replacement unit 28 can also modify the generated text by replacement based on the evaluation result of the forbidden expression evaluation from the forbidden expression evaluation unit 27. For example, when the evaluation result of the forbidden expression evaluation indicates that the generated text contains a forbidden expression, the replacement unit 28 modifies the generated text by replacement so that the generated text does not contain a forbidden expression.

[0038] The presentation control unit 29 generates a presentation UI (user interface) for presenting the generated text from the prohibition expression evaluation unit 27, and controls the presentation to be presented to the user.

[0039] Here, each block constituting the information processing device 10 can refer to necessary information of other blocks not indicated by arrows in addition to the information indicated by the arrows.

[0040] FIG. 2 is a flowchart illustrating an example of the automatic correction process of the information processing device 10.

[0041] The processing of the information processing device 10 includes, for example, processing of an automatic correction version, processing of an interactive version, and processing of an on-demand version.

[0042] In the auto-correction process, the generated text is corrected by resampling and / or substitution without user instruction, and the corrected generated text is presented. In the interactive process, candidates for correction by resampling and / or substitution (revision candidates) are presented, and the generated text is corrected with the revision candidates upon user instruction. In the on-demand process, revision candidates are presented upon user instruction, and the generated text is corrected with the revision candidates upon further user instruction.

[0043] In the information processing device 10, which of the automatically corrected version processing, the interactive version processing, and the on-demand version processing to perform can be set in advance in the information processing device 10 or can be set by the user, for example.

[0044] FIG. 2 shows the process of the automatic correction version of the information processing device 10.

[0045] In step S11, the prompt input unit 21 and the melody input unit 22 accept input information from the user and supply it to the sampling unit 25, and the process proceeds to step S12.

[0046] The input information depends on the generative model used for sampling. Examples of generative models that can be used include a general LLM and a melody-conditioned lyric language model that uses vocal melody as prior information for conditioning. Melody-conditioned lyric language models are described, for example, in Watanabe, K., et al., “A Melody-Conditioned Lyrics Language Model,” NAACL, 2018. (https: / / aclanthology.org / N18-1015 / ).

[0047] When a general LLM is used as the generative model, input information can include the atmosphere and melody of the lyrics to be generated as generated text, keywords, the name of the artist or song you want to evoke, etc. When a melody-conditional lyrics language model is used as the generative model, the vocal melody can also be used as input information.

[0048] In step S12, the sampling unit 25 generates tokens by sampling using a generative model in response to information input by the user from the prompt input unit 21 and the melody input unit 22. For example, when the sampling unit 25 generates a new token by sampling, it supplies generated text constructed by adding the new token to the end of the tokens generated up to that point to the similarity evaluation unit 26, and the process proceeds from step S12 to step S14.

[0049] In step S14, the similarity evaluation unit 26 calculates a similarity that indicates the similarity of the generated text to existing lyrics from the perspective of copyright infringement. The generated text that is the target of the similarity calculation in the similarity evaluation unit 26 is the generated text supplied from the sampling unit 25 in step S12, or the generated text supplied from the replacement unit 28 in step S13, which will be described later. The similarity evaluation unit 26 calculates, as the similarity that indicates the similarity of the generated text to existing lyrics from the perspective of copyright infringement, an index value that indicates, for example, the similarity between the entire or partial character string of the generated text and the entire or partial character string of existing lyrics stored in the lyrics information storage unit 15. The similarity evaluation unit 26 supplies the generated text and the similarity to the sampling unit 25, the prohibited expression evaluation unit 27, and the replacement unit 28, and the process proceeds from step S14 to step S15.

[0050] Here, as the similarity indicating the similarity of the generated text to the existing lyrics from the viewpoint of copyright infringement, multiple types (scales) of similarity can be calculated.

[0051] In step S15, the sampling unit 25 and / or the replacement unit 28 determines whether to perform correction by resampling and / or replacement on the generated text from the similarity evaluation unit 26, based on the similarity of the generated text. For example, if the similarity of the generated text is equal to or greater than a threshold, it can be determined that correction by resampling and / or replacement is to be performed, and if the similarity of the generated text is not equal to or greater than the threshold, it can be determined that correction by resampling and / or replacement is not to be performed.

[0052] The threshold value for the similarity of the generated text can be set in advance or by the user. When the similarity evaluation unit 26 calculates multiple types of similarity as the similarity of the generated text, a threshold value can be set for each type of similarity. When multiple types of similarity are calculated, a composite value of the multiple types of similarity, such as a weighted sum (average value), can be used as the final similarity. The weights used in the weighted sum of the multiple types of similarity can be set, for example, by the user. The weights used in the weighted sum of the multiple types of similarity can be calculated in advance by linear regression using the multiple types of similarity so that a similarity representing a manually evaluated similarity is obtained as the final similarity. For example, the weights can be calculated in advance by linear regression using the multiple types of similarity as explanatory variables so that the similarity output by the learning model serving as a similarity evaluation function for evaluating (calculating) similarity, which is trained by the similarity evaluation function learning unit 12, is obtained as the final similarity.

[0053] If it is determined in step S15 that the generated text should be corrected by resampling, the process returns to step S12. When the process returns from step S15 to step S12, the sampling unit 25 corrects the generated text by resampling the tokens that make up the generated text from the similarity evaluation unit 26, starting from a token located at a position obtained by tracing back from the end of the generated text to the beginning of the generated text. The sampling unit 25 supplies the generated text corrected by resampling to the similarity evaluation unit 26. Thereafter, the process proceeds from step S12 to step S14, and the same process is repeated thereafter. As described above, correcting the generated text by resampling can reduce the risk of copyright infringement for the generated text.

[0054] Furthermore, if it is determined in step S15 that the generated text should be corrected by replacement, the process proceeds to step S13. In step S13, the replacement unit 28 corrects the generated text by replacing part or all of the generated text from the similarity evaluation unit 26. The replacement unit 28 supplies the generated text corrected by replacement to the similarity evaluation unit 26. Thereafter, the process proceeds from step S13 to step S14, and the same process is repeated thereafter. As described above, by correcting the generated text by replacement, the risk of copyright infringement for the generated text can be reduced.

[0055] On the other hand, if it is determined in step S15 that the generated text will not be corrected by resampling or by replacement, the process proceeds to step S16.

[0056] In step S16, the forbidden expression evaluation unit 27 performs a forbidden expression evaluation to evaluate whether the generated text from the similarity evaluation unit 26 contains any forbidden expressions. The forbidden expression evaluation is performed by comparing the generated text with the forbidden expressions stored in the forbidden expression storage unit 17 on a string matching basis. The forbidden expression evaluation unit 27 supplies the generated text and the evaluation result of the forbidden expression evaluation, including whether all or a partial string of the generated text matches a forbidden expression, to the sampling unit 25, the forbidden expression evaluation unit 27, and the presentation control unit 29. Thereafter, the processing proceeds from step S16 to step S17.

[0057] In step S17, the sampling unit 25 and / or the replacement unit 28 determine whether to perform correction by resampling and / or replacement on the generated text based on the evaluation result of the forbidden expression evaluation from the forbidden expression evaluation unit 27. For example, if the evaluation result of the forbidden expression evaluation indicates that the generated text contains a forbidden expression, it can be determined that correction by resampling and / or replacement will be performed. Also, for example, if the evaluation result of the forbidden expression evaluation indicates that the generated text does not contain a forbidden expression, it can be determined that correction by resampling and / or replacement will not be performed.

[0058] If it is determined in step S17 that the generated text should be corrected by resampling, the process returns to step S12. When the process returns from step S17 to step S12, the sampling unit 25 corrects the generated text from the forbidden expression evaluation unit 27 by resampling. The sampling unit 25 supplies the generated text corrected by resampling to the similarity evaluation unit 26. Thereafter, the process proceeds from step S12 to step S14, and the same process is repeated thereafter.

[0059] Furthermore, if it is determined in step S17 that the generated text should be corrected by replacement, the process proceeds to step S13. In step S13, the replacement unit 28 corrects the generated text by replacing part or all of the generated text from the forbidden expression evaluation unit 27. The replacement unit 28 supplies the generated text corrected by replacement to the similarity evaluation unit 26. Thereafter, the process proceeds from step S13 to step S14, and the same process is repeated thereafter.

[0060] Whether to perform correction by resampling or correction by replacement as the correction of the generated text can be set in advance or can be set by the user. For example, the priority of correction by resampling and correction by replacement can be set in advance. Specifically, for example, the priority can be set in advance so that if correction by replacement is possible, correction by replacement is performed, and if correction by replacement is not possible, correction by resampling is performed. Also, for example, the user can set so that one of correction by resampling and correction by replacement, for example, correction by resampling is always performed. Furthermore, for example, in the generated text, correction by replacement can be performed for parts that can be corrected by replacement, and correction by resampling can be performed for parts that cannot be corrected by replacement. In this case, the generated text is exchanged between the sampling unit 25 and the replacement unit 28.

[0061] If it is determined in step S17 that the generated text will not be modified by resampling or by replacement, the process proceeds to step S18.

[0062] In step S18, the sampling unit 25 determines whether to terminate (complete) the generation of the generated text, i.e., the generation of tokens by sampling. For example, in step S18, if a specific condition is met, it is determined that the generation of the generated text is to be terminated, and if the specific condition is not met, it is determined that the generation of the generated text is not to be terminated. The specific condition may be, for example, that the generated text has reached a certain length (a certain number of tokens) or that the reading length of the generated text exceeds the number of notes in the vocal melody as input information. Other specific conditions may be, for example, that a special token indicating the end of the generated text has been generated.

[0063] If it is determined in step S18 that the generation of the generated text should not be terminated, the process returns to step S12. When the process returns from step S18 to step S12, the sampling unit 25 generates a new token by sampling, and supplies the generated text constructed by adding the new token to the end of the tokens generated up to that point to the similarity evaluation unit 26. The process then proceeds from step S12 to step S14, and the same process is repeated thereafter.

[0064] If it is determined in step S18 that the generation of the generated text is to be ended, the process proceeds to step S19.

[0065] In step S19, the presentation control unit 29 generates a presentation UI for presenting the generated text from the proscription expression evaluation unit 27, and performs presentation control to present the UI to the user.

[0066] FIG. 3 is a block diagram showing an example configuration of another embodiment of an information processing device to which the present technology is applied.

[0067] In the figure, parts corresponding to those in FIG. 1 are given the same reference numerals, and the description thereof will be omitted below as appropriate.

[0068] In Figure 3, the information processing device 10 has a manual similarity evaluation result storage unit 11 to a prohibited expression storage unit 17, and a generation unit 20, and the generation unit 20 has a prompt input unit 21 to a presentation control unit 29, and a correction candidate storage unit 31.

[0069] 3 is the same as the case of Fig. 1 in that it includes a manual similarity evaluation result storage unit 11 to a prohibited expression storage unit 17, and a generation unit 20, and the generation unit 20 includes a prompt input unit 21 to a presentation control unit 29. However, the information processing device 10 of Fig. 3 differs from the case of Fig. 1 in that a correction candidate storage unit 31 is newly provided in the generation unit 20.

[0070] The correction candidate storage unit 31 receives the correction contents of the generated text by resampling and replacement from the sampling unit 25 and the replacement unit 28. The correction candidate storage unit 31 stores the correction contents from the sampling unit 25 and the replacement unit 28 as candidates (correction candidates) for correction by resampling and replacement of the generated text before correction generated by the sampling unit 25.

[0071] The information processing device 10 in FIG. 1 processes the auto-corrected version, whereas the information processing device 10 in FIG. 3 can process any of the auto-corrected version, the interactive version, and the on-demand version.

[0072] 3, when interactive processing or on-demand processing is performed, the presentation control unit 29 generates and presents a presentation UI that presents the generated text before correction and the correction candidates stored in the correction candidate storage unit 31. When a user who views the presentation UI selects a correction candidate and issues a correction instruction, the generated text before correction is corrected with the correction candidate selected by the user.

[0073] FIG. 4 is a flowchart illustrating an example of interactive processing by the information processing device 10.

[0074] The on-demand version differs from the interactive version in that correction candidates are displayed (presented) after a user instruction, and correction candidates are displayed without a user instruction, in that the on-demand version process is different. Furthermore, in the interactive version process of Figure 4, the process that is the same as the automatic correction version process of Figure 2 will not be described as appropriate.

[0075] In steps S21, S22, S23, and S24, the same processes as in steps S11, S12, S14, and S15 in FIG. 2 are performed, respectively.

[0076] If it is determined in step S24 that the generated text should be corrected by resampling based on the similarity of the generated text, the process proceeds to step S25. In step S25, the sampling unit 25 performs resampling, but does not correct the generated text from the similarity evaluation unit 26 using the resampled token (series). Instead, the resampled token is supplied to and stored (saved) in the correction candidate storage unit 31 as a correction candidate. Thereafter, the process proceeds from step S25 to step S27.

[0077] Furthermore, if it is determined in step S24 based on the similarity of the generated text that the generated text should be corrected by replacement, the process proceeds to step S26. In step S26, the replacement unit 28 selects candidate character strings (replacement candidates) to be used to replace part or all of the generated text as a correction to the generated text from the similarity evaluation unit 26. The replacement unit 28 supplies the replacement candidates as correction candidates to the correction candidate storage unit 31 to store (save), and the process proceeds from step S26 to step S27.

[0078] On the other hand, if it is determined in step S24 that the generated text will not be corrected by resampling or by replacement, the process proceeds to step S27.

[0079] In step S27, the same forbidden expression evaluation as in step S16 in FIG. 2 is performed, and the process proceeds to step S28.

[0080] In step S28, similarly to step S17 in FIG. 2, it is determined whether or not to perform correction by resampling and / or replacement based on the evaluation result of the forbidden expression evaluation in step S27.

[0081] If it is determined in step S28 based on the evaluation result of the forbidden expression evaluation that the generated text should be corrected by resampling, the process proceeds to step S29. In step S29, similar to step S25, the sampling unit 25 performs resampling, and supplies the resampled tokens as correction candidates to the correction candidate storage unit 31 for storage. Thereafter, the process proceeds from step S29 to step S31.

[0082] Furthermore, if it is determined in step S28 based on the evaluation result of the forbidden expression evaluation that the generated text should be corrected by replacement, the process proceeds to step S30. In step S30, similar to step S26, the replacement unit 28 selects a replacement candidate to be used in replacing the generated text, and supplies the replacement candidate as a correction candidate to be stored in the correction candidate storage unit 31. Thereafter, the process proceeds from step S30 to step S31.

[0083] On the other hand, if it is determined in step S28 that the generated text will not be modified by resampling or by replacement, the process proceeds to step S31.

[0084] In step S31, similar to step S18 in FIG. 2, it is determined whether or not the generation of the generated text is to be ended (completed).

[0085] If it is determined in step S31 that the generation of the generated text is not to be ended, the process returns to step S22, and the generation of the generated text, that is, the generation of tokens by sampling, continues.

[0086] If it is determined in step S31 that the generation of the generated text is to be ended, the process proceeds to step S32.

[0087] In step S32, a presentation UI is generated and presented, similarly to step S19 in FIG.

[0088] <Sampling> FIG. 5 is a diagram for explaining the sampling performed by the sampling unit 25. As shown in FIG.

[0089] The sampling unit 25 uses a generative model (generative AI) to generate a token sequence as generated text. For example, any LLM can be used as the generative model. The LLM used as the generative model may be an LLM fine-tuned using existing lyrics. Furthermore, for example, the melody-conditional lyrics language model described in FIG. 2 can be used as the generative model.

[0090] The generative model autoregressively receives a token sequence y(1), y(2), ..., y(N-1) as input data, and outputs the probability distribution o(N) = (o(N,1), o(N,2), ..., o(N,K)) of the next token y(N) as output data. The probability distribution o(N) is a vector (o(N,1), o(N,2), ..., o(N,K)), where o(N,i) represents the probability that the i-th candidate token appears as the next token y(N). K represents the total number of candidate tokens.

[0091] The sampling unit 25 treats the token sequence y(1), y(2), ..., y(N-1), which serves as input data for the generative model, as a sequence of vectors by referring to the token embedding storage unit 23. When a melody-conditional lyrics language model is adopted as the generative model, the vocal melody (time-series data) is provided to the generative model as input data along with the token sequence y(1), y(2), ..., y(N-1).

[0092] The sampling unit 25 generates generated text y(1), y(2), ..., y(N) with a token length (number) of N by sampling (selecting) the token y(n) N times based on the output (data) of the generative model.

[0093] For each position n = 1, 2, ..., N of each token constituting the generated text, the output (data) of the generation model is represented as Model(y<n). Model(y<n) represents the output data of the generation model when the token sequence y(1), y(2), ..., y(n-1) is given as input data to the generation model. In this case, the probability distribution o(n) = (o(n, 1), o(n, 2), ..., o(n, K)) of the token y(n) is represented as o(n) = Model(y<n). Note that when the generation model is a melody-conditioned lyric language model, and the output data of the generation model when the token sequence y(1), y(2), ..., y(n-1) and the melody Melody are given as input data to the generation model which is the melody-conditioned lyric language model is represented as Model(y<n, Melody), the probability distribution o(n) = (o(n, 1), o(n, 2), ..., o(n, K)) of the token y(n) is represented as o(n) = Model(y<n, Melody).

[0094] The probability that the i-th candidate of the token is the token y(n) at the position n of the token, that is, the probability P(y(n)|y<n)(i) that the i-th candidate of the token is observed after the token sequence y(1), y(2), ..., y(n-1) is represented by the formula P(y(n)|y<n)(i) = e^o(n, i) / Σe^o(n, j). The summation Σ in the denominator represents the sum when j is changed from 1 to K.

[0095] The sampling unit 25 performs sampling to select the token y(n) from the K candidates of the token according to the probability P(y(n)|y<n)(i). Note that in the correction of the generated text by resampling, one or more of the candidates of the token can be used as correction candidates according to the probability P(y(n)|y<n)(i). ​In the sampling unit 25, the calculation of the probability P(y(n)|y<n)(i) when selecting the token y(n) can be adjusted using the temperature parameter T>0. In this case, the probability P(y(n)|y<n)(i) is calculated according to the formula P(y(n)|y<n)(i) = e^(o(n, i) / T) / Σe^(o(n, j) / T).

[0097] When the probability P(y(n)|y<n)(i) is calculated using the temperature parameter T, when T>1, candidates for tokens with a low probability o(n, i) are likely to be selected as the token y(n), and the range of expressions in the generated text can be widened.

[0098] When the sampling unit 25 determines that it is necessary to correct the generated text by resampling, the sampling unit 25 corrects the generated text by performing resampling, which resamples the tokens from the token at the position traced from the last token to the first token that constitutes the generated text.

[0099] The resampling position, that is, the position of the token to be resampled, is set to trace back one token at a time, for example, from the position N of the latest (last) token in the generated text. Then, resampling is performed starting from the resampling position. The setting of the resampling position and the resampling are repeated until it is determined that no correction of the generated text by resampling is required. That is, the setting of the resampling position and the resampling are repeated until no word (correction word) that should be corrected based on the similarity of the generated text after correction by resampling and the evaluation result of the prohibited expression evaluation appears in the generated text after correction by resampling.

[0100] Here, in resampling, it is possible to suppress the selection of the same token as before resampling. Suppression of the selection of the same token as before resampling can be achieved, for example, by setting the temperature parameter T to a large value in calculating the probability P(y(n)|y<n)(i) of a token candidate that matches the same token as before resampling. Alternatively, suppression of the selection of the same token as before resampling can be achieved, for example, by setting the probability P(y(n)|y<n)(i) to 0 for a token candidate that matches the same token as before resampling.

[0101] As described above, when the position of resampling is set to trace tokens one by one and resampling is repeated until it is determined that the generated text is not corrected by resampling, the calculation cost may increase.

[0102] Therefore, the calculation cost can be reduced by dynamically determining the position of resampling.

[0103] As a dynamic determination method for dynamically determining the position of resampling, for example, there is a method of setting the position INT(a * N / 100) traced back by (100 - a)% of the token length N of the generated text from the end to the beginning of the generated text as the position of resampling. INT() represents truncation of the decimal part of the value in the parentheses.

[0104] Also, as a dynamic determination method, for example, there is a method of setting the position traced back by the number of tokens of the latest phrase (correction word) whose similarity in the generated text is above the threshold or the latest phrase determined to be a prohibited expression in the prohibited expression evaluation as the position of resampling.

[0105] Other dynamic determination methods include, for example, setting the position of the most recent token containing a specific part of speech as the resampling position, among tokens contained in the most recent phrase whose similarity in the generated text is equal to or exceeds a threshold value, or the most recent phrase determined to be a forbidden expression in the forbidden expression evaluation. The specific part of speech may be a part of speech whose meaning may change significantly when resampled, such as a noun or proper noun. In this case, it is necessary to store the parts of speech contained in the tokens of the generated text (the tokens that make up the generated text).

[0106] <Replacement> FIG. 6 is a diagram illustrating the replacement dictionary stored in the replacement dictionary storage unit 24. As shown in FIG.

[0107] The substitution dictionary is generated using a substitution table.

[0108] In the substitution table, a replacement target word, which is a vocabulary to be replaced, and a replacement word to replace the replacement target word are registered in association with each other. The substitution table can be generated, for example, by extracting pairs of synonyms with the same number of mora from a Japanese thesaurus (synonym dictionary) and treating these pairs as pairs of a replacement target word and a replacement word. In a substitution dictionary, multiple replacement words may be associated with one replacement target word.

[0109] The replacement table in FIG. 6 stores target words and replacement words (character strings), as well as the number of moras and vowel sequences of the target words and replacement words. Furthermore, the replacement table stores the edit distance of the vowel sequence, information on the token sequence (target token sequence) constituting the target word, and information on the token sequence (replacement word token sequence) constituting the replacement word. The information on the target token sequence is token IDs that serve as identification information for identifying the tokens constituting the target word, and the information on the replacement word token sequence is token IDs that serve as identification information for identifying the tokens constituting the replacement word. In FIG. 6 , for example, the information on the target token sequence

[0245] indicates that the target word for replacement is composed of the token "sappari" (refreshing) with a token ID of 245. Furthermore, for example, the information on the replacement word token sequence [456, 235] indicates that the replacement word is composed of the token "sou" (sou) with a token ID of 456 and the token "so" (refreshing) with a token ID of 235, in sequence.

[0110] When correcting a generated text by replacing a target word in the generated text with a replacement word, the meaning of the corrected generated text may change from the meaning of the generated text before correction. For example, if the target word in the generated text "I completely forgot" is "refreshingly," and the generated text is corrected by replacing the target word with the replacement word "souso" that is associated with the target word in the replacement table, the corrected generated text will become "I completely forgot."

[0111] As described above, a replacement dictionary is generated to prevent the meaning of the generated text after correction by replacement from changing from the meaning of the generated text before correction.

[0112] In generating a replacement dictionary, for example, phrases of a certain length or less are extracted from each line of text in which lyrics stored in the lyrics information storage unit 15 are expressed in lyrics format (as written on a lyrics card, etc.). Furthermore, for each phrase extracted from the lyrics, if the phrase contains a replacement target word (or a word that matches a replacement target word) in the replacement table, the replacement target word included in the phrase is replaced with the replacement word associated with that replacement target word, and a replaced phrase is generated. If multiple replacement words are associated with one replacement target word in the replacement table, the replacement target word is replaced with each of the multiple replacement words, and a plurality of replaced phrases are generated.

[0113] Thereafter, a similarity indicating the similarity between the phrase before replacement and the phrase after replacement is calculated. As the similarity between the phrase before replacement and the phrase after replacement, for example, the cosine similarity between vectors obtained by converting each of the phrase before replacement and the phrase after replacement into a vector representation can be used. The conversion of the phrase into a vector representation can be performed using, for example, a model that has undergone expression training, such as Japanese-trained BERT.

[0114] After calculating the similarity between the pre-replacement phrase and the post-replacement phrase, a replacement dictionary is generated in which pairs of pre-replacement phrases and post-replacement phrases whose similarity is equal to or greater than a predetermined threshold are associated and registered as pairs of replacement target words (replacement target phrases) and replacement words (replacement phrases). Alternatively, a replacement dictionary is generated in which, for pairs of pre-replacement phrases and post-replacement phrases whose similarity is equal to or greater than a predetermined threshold, the replacement target words included in the pre-replacement phrases and the replacement words included in the post-replacement phrases are associated and registered. Therefore, the replacement dictionary is a dictionary in which replacement target words and replacement words are associated with each other.

[0115] When a replacement target word is replaced with a replacement word using a replacement dictionary generated in the manner described above, it is possible to prevent the meaning of the generated text after correction by the replacement from changing from the meaning of the generated text before correction.

[0116] The replacement unit 28 identifies words (sequences of one or more words) such as phrases that have a similarity to existing lyrics that is equal to or exceeds a threshold value, indicating the similarity in terms of copyright infringement, and prohibited expressions included in the generated text, as recommended modifications, and performs replacement to modify the generated text. The generated text is modified by replacement, for example, by replacing part or all of the modification word with a replacement word associated with a replacement target word that matches part or all of the modification word. The generated text that has been modified by replacement is again subjected to similarity evaluation and prohibited expression evaluation.

[0117] Here, the sampling unit 25 also identifies modified words in the generated text and modifies the generated text by resampling so that some or all of the modified words included in the generated text are changed to other words. The generated text modified by resampling is again subjected to similarity evaluation and forbidden expression evaluation.

[0118] The replacement unit 28 can perform replacement of the generated text if the generated text can be corrected by replacement. Whether the generated text can be corrected by replacement is determined based on whether the replacement dictionary contains a replacement target word that matches part or all of the modifying word, or a replacement target word that matches a word in the generated text that contains part or all of the modifying word (hereinafter also referred to as a modifying word-containing word). If the replacement dictionary contains a replacement target word that matches part or all of the modifying word, or a replacement target word that matches a modifying word-containing word, it is determined that the generated text can be corrected by replacement. Below, examples of correcting the generated text by replacement will be described for cases where the entire modifying word matches the replacement target word, and cases where a modifying word-containing word that includes the entire modifying word matches the replacement target word.

[0119] FIG. 7 is a diagram illustrating an example of correcting generated text by substitution.

[0120] FIG. 7A shows an example of correction of generated text by replacement when a replacement target word that matches the modified word is registered in the replacement dictionary, i.e., when the entire modified word matches the replacement target word.

[0121] In Figure 7A, the generated text consists of 12 tokens y(1), y(2), ..., y(12), and the latest four tokens y(9) to y(12) = "●●●●" are the modified words.

[0122] When a replacement target word "●●●●" that matches the entirety of the modifying word "●●●●" is registered in the replacement dictionary, it is determined that the generated text can be modified by replacement. For example, when a replacement target word "●●●●" that matches the entirety of the modifying word "●●●●" is associated with the replacement word "★★★★" in the registration dictionary, it is determined that the generated text can be modified by replacement. In this case, the modifying word "●●●●" in the generated text is replaced with the replacement word "★★★★" that is associated with the replacement target word "●●●●" in the modifying word "●●●●". Note that even when a replacement target word that matches part of the modifying word is registered in the replacement dictionary, it is determined that the generated text can be modified by replacement. Therefore, for example, when the replacement target word "●●●●" is associated with the replacement word "★★★★" in the replacement dictionary and the modifying word is "●●●●××" that includes the replacement target word "●●●●", it is determined that the generated text can be modified by replacement. In this case, part of the modifying word "●●●●××", i.e., the replacement target words "●●●●" of four tokens in the modifying word "●●●●××" of six tokens, are replaced with the replacement word "★★★★" associated with the replacement target word "●●●●".

[0123] FIG. 7B shows an example of correction of generated text by replacement when a replacement target word that matches a modified word-containing word that includes all of the modified word is registered in the replacement dictionary, i.e., when a modified word-containing word that includes all of the modified word matches the replacement target word.

[0124] In Figure 7B, the three most recent tokens y(10) to y(12) = "●●●" in the generated text, which consists of 12 tokens y(1), y(2), ..., y(12), are modifiers. For example, the five most recent tokens y(8) to y(12) = "■■●●●" in the generated text are modifier-container words that contain all of the modifiers "●●●".

[0125] If the replacement target word "■■●●●" that matches the modifier inclusion word "■■●●●" is registered in the replacement dictionary, it is determined that the generated text can be corrected by replacement. For example, if the replacement target word "■■●●●" and the replacement word "△△△△" are associated in the registration dictionary, it is determined that the generated text can be corrected by replacement. In this case, the modifier inclusion word "■■●●●" in the generated text is replaced with the replacement word "△△△△" that is associated with the replacement target word "■■●●●" that matches the modifier inclusion word "■■●●●". It is also determined that the generated text can be corrected by replacement if a replacement target word that matches a modifier inclusion word that contains part of the modifier is registered in the replacement dictionary. Therefore, for example, if the generated text is "■■●●●××" and the five tokens "●●●××" in the generated text "■■●●●××" are modifiers, and the replacement target word "■■●●●" in the generated text "■■●●●××" matches the modifier containment word "■■●●●" which contains the three tokens "●●●" that are part of the modifier "●●●××", and is registered in the replacement dictionary in association with the replacement word "△△△△", the modifier containment word "■■●●●" in the generated text "■■●●●××" will be replaced with the replacement word "△△△△" associated with the replacement target word "■■●●●" that matches the modifier containment word "■■●●●".

[0126] <Calculation of similarity representing the similarity of the generated text to existing lyrics from the perspective of copyright infringement> Figure 8 is a flowchart that explains an example of the process performed by the similarity evaluation unit 26 to calculate the similarity representing the similarity of the generated text to existing lyrics from the perspective of copyright infringement.

[0127] The similarity evaluation unit 26 extracts, from the generated text, a target character string for which a similarity that indicates similarity to existing lyrics from the viewpoint of copyright infringement is to be calculated, and calculates the similarity for the target character string.

[0128] In step S51, the similarity evaluation unit 26 extracts (cuts out) a target string c(y) for which similarity is to be calculated (a target string to be evaluated for similarity evaluation) from the generated text y = y(1), y(2), ..., Y(N), and the process proceeds to step S52.

[0129] For example, the similarity evaluation unit 26 can extract all character strings from the first token y(1) of the generated text y to the most recently generated token y(N) as the target character string c(y). For example, the similarity evaluation unit 26 can extract a character string consisting of a predetermined number M (<=N) of the latest tokens y(N-M+1), y(N-M+2), ..., Y(N) of the generated text y as the target character string c(y). For example, when the generated text y is expressed as lyrics (in the format used on a lyrics card, for example), the similarity evaluation unit 26 can extract a character string consisting of the latest tokens in a predetermined number L of lines as the target character string c(y). The number of characters in one line of the generated text y expressed as lyrics can be, for example, a value based on the number of characters in one line of existing lyric expressions, for example, a value exceeding the average number of characters in one line of existing lyric expressions. In addition, when the generated text y is expressed as lyrics, the number of characters in a line can be determined based on other factors, such as when a special token indicating the end of a line (end of line) is generated when generating tokens using a generation model.

[0130] In step S52, the similarity evaluation unit 26 selects existing lyrics (hereinafter also referred to as target lyrics) x for which the similarity of (the target character string of) the generated text is to be calculated, and the process proceeds to step S53.

[0131] For example, the similarity evaluation unit 26 can sequentially select all lyrics whose lyric information is stored in the lyric information storage unit 15 as target lyrics x.

[0132] Alternatively, for example, the similarity evaluation unit 26 can sequentially select, as target lyrics x, only lyrics of songs by a specific artist from among the lyrics whose lyric information is stored in the lyric information storage unit 15. In this case, the similarity evaluation unit 26 can select, as specific artists, the target lyrics x from among the top R artists whose representative vectors, generated based on vectors that embed the lyrics of the artist's songs, are closest to the vector that embeds the target string c(y). The artist's representative vector can be an average vector obtained by converting the lyrics of each of the artist's songs into vectors that embed the lyrics using an arbitrary language model and calculating the average of those vectors. The number R (a natural number) of the top R can be determined in advance.

[0133] In step S53, the similarity evaluation unit 26 extracts (cuts out) one or more lyric strings c(x) from the target lyrics x, which are strings to be used to calculate the similarity of the target string c(y) in the generated text y, and the process proceeds to step S54. For example, the similarity evaluation unit 26 can sequentially extract, from the beginning of the target lyrics x, units of the same order as the target string c(y) as lyric strings c(x).

[0134] In step S54, the similarity evaluation unit 26 calculates a similarity between the target character string c(y) and the lyric character string c(x), which represents the similarity of the generated text to the target lyrics x as an existing work in terms of copyright infringement. The process then proceeds from step S54 to step S55. The similarity between the target character string c(y) and the lyric character string c(x) can be calculated using a predetermined function f. The similarity calculated using the function f can be expressed as f(c(y), c(x)). The similarity evaluation unit 26 can also calculate multiple types of similarity using multiple functions.

[0135] In step S55, the similarity evaluation unit 26 determines whether or not the calculation of similarity has been completed for all lyrics for which similarity should be calculated.

[0136] If it is determined in step S55 that calculation of similarity for all lyrics for which similarity calculation is to be performed has not yet been completed, the process returns to step S52, where one of the lyrics for which similarity calculation has not yet been performed is selected as a new target lyric x, and the same process is then repeated.

[0137] If it is determined in step S55 that the calculation of similarity has been completed for all lyrics for which similarity should be calculated, the process ends.

[0138] Various types of similarity can be used for the similarity between the target string c(y) and the lyric string c(x), which represents the similarity of the generated text to the target lyrics x as an existing work in terms of copyright infringement. That is, various types of similarity can be calculated by using different calculation methods, different information used for calculating similarity, and different combinations of the calculation methods and the information used for calculating similarity.

[0139] The similarity between the target string c(y) and the lyric string c(x) can be determined by, for example, one or more of the following first to seventh index values: The first index value represents the character-by-character identity between the target string c(y) and the lyric string c(x), the second index value represents the semantic similarity between the target string c(y) and the lyric string c(x), the third index value represents the similarity between the target string c(y) and the lyric string c(x) taking into account the melody, and the fourth index value represents the similarity between the target string c(y) and the lyric string c(x) taking into account the composition of the song, the fifth index value represents the similarity between the target string c(y) and the lyric string c(x) taking into account meta information about the song and / or artist, and the sixth index value represents the similarity between the target string c(y) and the lyric string c(x) taking into account information input by the user. The seventh index value is an index value that represents the similarity between the target string c(y) and the lyrics string c(x), and is output by a learning model that has learned a measure of similarity perceived by humans.

[0140] The first index value, which represents the character-by-character identity between the target string c(y) and the lyrics string c(x), is calculated by performing a superficial string comparison between the target string c(y) and the lyrics string c(x).

[0141] The first index value may be, for example, the length of the longest matching string between the target string c(y) and the lyrics string c(x), or the edit distance.

[0142] When calculating the first index value, the target string c(y) and the lyrics string c(x) can be strings containing a mixture of kanji and kana, or they can be strings written entirely in hiragana.

[0143] Furthermore, when calculating the first index value, personal nouns and endings in the target string c(y) and the lyrics string c(x) can be replaced with special characters (including symbols). This prevents a low similarity value from being calculated as the first index value even when there is similarity from the perspective of copyright infringement between the target string c(y) and the lyrics string c(x) simply because the only difference between them is whether a noun is in the first or second person, or because the endings are different.

[0144] Furthermore, when calculating the first index value, a value that monotonically decreases with respect to the occurrence probability or frequency of the lyric string c(x) can be weighted and calculated as the final first index value. The occurrence probability or frequency of the lyric string c(x) can be calculated using, for example, perplexity or n-gram occurrence frequency or probability. In this case, for example, when the lyric string c(x) is a characteristic expression that occurs rarely, a value indicating high similarity is calculated as the first index value. When the target string c(y) is similar to (or matches) a characteristic expression that occurs rarely, there is a high possibility that it will be suspected of being a scam. Therefore, when the target string c(y) is similar to a characteristic expression that occurs rarely, a value indicating high similarity is calculated as the first index value, making it easier to modify the target string c(y) in the generated text by resampling and / or replacement. As a result, suspicion of scamming can be reduced and the risk of copyright infringement can be reduced.

[0145] The second index value representing the semantic similarity between the target string c(y) and the lyric string c(x) can be, for example, calculated using vectors (embedding representations) by using an LLM such as BERT to convert the target string c(y) and the lyric string c(x), and then using these vectors to calculate values ​​such as Euclidean distance or cosine similarity. In this case, a value representing semantic similarity can be calculated as the second index value, regardless of the word order, slight differences in vocabulary, differences in string length, etc. between the target string c(y) and the lyric string c(x).

[0146] The third index value representing the similarity between the target string c(y) and the lyric string c(x) taking into account the melody can be calculated using only the target string c(y) and the lyric string c(x). The third index value can be calculated using the string similarity, which represents the similarity between the target string c(y) and the lyric string c(x), and the melody similarity, which represents the similarity between the melody corresponding to the target string c(y) and the melody corresponding to the lyric string c(x). For example, the third index value can be calculated as a weighted sum of the string similarity and the melody similarity. The melody similarity can be calculated, for example, using the technique described in Typke, R., et al., “Using Transportation Distances for Measuring Melodic Similarity,” ISMIR, 2003. According to the third index value, the similarity as character strings is not particularly high, but the similarity of the melody is high. Therefore, when listening to a song with lyrics containing the generated text, if the generated text is perceived to be highly similar to existing lyrics, the target character string c(y) of the generated text is more likely to be modified by resampling and / or replacement. As a result, the risk of copyright infringement can be reduced.

[0147] As a fourth index value that represents the similarity between the target character string c(y) and the lyric character string c(x) taking into account the composition of the song, for example, a value that is weighted according to which part (composition) of the song the target character string c(y) and the lyric character string c(x) belong to can be used.

[0148] For example, weights can be determined in advance for each part of a song, such as the verse, bass line, and chorus, such as {verse, bass line, chorus} = {1.5, 1.0, 3.0}, and the fourth index value can be calculated by adding a weight corresponding to the part of the lyrics string c(x) (the part of the song that corresponds to the lyrics string c(x)) to another index value that indicates the similarity between the target string c(y) and the lyrics string c(x). The part of the lyrics string c(x) can be identified from the music information stored in the music information storage unit 14.

[0149] Regarding the weights corresponding to parts, for example, a large weight may be assigned to the chorus, and a small weight may be assigned to the non-chorus parts, such as the verse and bridge. In this case, when the target string c(y) is similar to the chorus of existing lyrics, a value representing a higher similarity is calculated as the fourth index value compared to when the target string c(y) is similar to other parts of the existing lyrics, such as the verse and bridge. The fourth index value calculates a value representing a high similarity for target string c(y) that is similar to the chorus of existing lyrics and has a strong impact on the author (lyricist) of the existing lyrics and general users, making it easier to modify the target string c(y) in the generated text through resampling and / or replacement. As a result, the risk of copyright infringement can be reduced. Furthermore, the fourth index value prevents a value representing a high similarity from being calculated for target string c(y) that is similar to parts of the existing lyrics that are not prominent in the entire song, such as the verse and bridge. As a result, it is expected that excessive modification of the target string c(y) of the generated text due to resampling and / or substitution will be suppressed.

[0150] The fourth index value can be a value weighted according to the part of the lyrics string c(x), or a value weighted by dynamically setting weights corresponding to both the part of the lyrics string c(x) and the part of the target string c(y) (the part of the song corresponding to the target string c(y)). Information about the part of the target string c(y) can be included in information input by the user, for example, the melody (data).

[0151] When weights corresponding to parts of the lyrics string c(x) and parts of the target string c(y) are dynamically set and the weighted value is calculated as the fourth index value, it is possible to appropriately deal with cases such as, for example, where in a song whose lyrics are generated text, it is acceptable for the A-melody (lyrics) to be similar to the A-melody (lyrics) of an existing song, or where it is acceptable for the chorus to be somewhat similar to the A-melody or B-melody of an existing song, but it is desirable to avoid it being similar to the chorus of an existing song.

[0152] That is, if the target string c(y) is a chorus and the lyric string c(x) is also a chorus, a large weight, for example, a weight greater than 1, can be set. In this case, when the target string c(y) that is a chorus is similar to the lyric string that is a chorus, a value representing high similarity is calculated as the fourth index value. As a result, it becomes easier to correct the target string c(y) in the generated text by resampling and / or replacement, and it is possible to prevent the target string c(y) that is a chorus from becoming similar to the chorus (lyrics) of an existing song.

[0153] Furthermore, if the target string c(y) is a chorus and the lyric string c(x) is a part other than the chorus, a small weight, for example, a weight smaller than 1, can be set. In this case, even if the target string c(y) that is a chorus is somewhat similar to a lyric string in a part other than the chorus, for example, the verse or bridge, a value representing that the similarity is not very high is calculated as the fourth index value. As a result, for target string c(y) that is a chorus and is somewhat similar to the verse or bridge (lyrics) of an existing song, modification of the target string c(y) in the generated text by resampling and / or replacement is suppressed.

[0154] Furthermore, if the target string c(y) is a verse verse and the lyric string c(x) is a verse verse, a small weight can be set. In this case, even if the target string c(y) is a verse verse and the lyric string is a verse verse, a value indicating that the similarity is not very high is calculated as the fourth index value. As a result, for target string c(y) that is a verse verse and is similar to the verse verse (lyrics) of an existing song, modification of the target string c(y) in the generated text by resampling and / or replacement is suppressed.

[0155] The fifth index value, which represents the similarity between the target string c(y) and the lyric string c(x) taking into account meta information of the song and / or artist, can be calculated using meta information of the song and / or artist of an existing song that includes the lyrics x from which the lyric string c(x) was extracted.

[0156] For example, weights can be set according to the popularity of existing songs, artists, lyricists, and composers, and the weights can be added to the other index values ​​to calculate the fifth index value. The weights corresponding to popularity can be weights that increase as the popularity increases. The popularity of songs, artists, etc. can be quantified based on, for example, the song's past sales or the number of search hits on the Internet, and a weight, such as one of five levels, can be set according to the popularity.

[0157] The lyrics of existing songs with high name recognition, such as songs or artists, tend to give the impression of similarity. According to the fifth index value, even if the target string c(y) is slightly similar to the lyrics of existing songs with high name recognition, such as songs or artists, a value indicating high similarity is calculated. This makes it easier to modify the target string c(y) in the generated text through resampling and / or replacement. As a result, the impression of similarity to the lyrics of existing songs with high name recognition, such as songs or artists, can be suppressed, thereby reducing the risk of copyright infringement.

[0158] The sixth index value, which represents the similarity between the target string c(y) and the lyric string c(x) taking into account the information input by the user, can be calculated using the information input by the user, such as the text that serves as a keyword entered as a prompt, various parameters, etc.

[0159] For example, if a prompt contains information (direct recall information) that directly evokes a specific song, such as a characteristic phrase contained in existing lyrics, or the name of an artist, song title, or lyricist, it is assumed that the user has a strong intention to generate lyrics that are similar to the existing lyrics. However, generating generated text that is similar to existing lyrics is not appropriate from the perspective of reliance. In this case, strict checks for the similarity of the generated text to existing lyrics should be conducted to prevent copyright infringement.

[0160] Therefore, when direct recall information is input as a prompt, a value obtained by weighting the other index values ​​more heavily than when other information is input as a prompt can be calculated as the sixth index. For example, the sixth index can be calculated based on the number of times the keyword input as a prompt is included in the lyric string c(x), the lyrics x from which the lyric string c(x) was extracted, or all lyrics written by the lyricist of lyrics x. The greater the number of times, the greater the weighting. Furthermore, for example, when the prompt matches the artist name of the artist who provided the music with lyrics x or the title of the music with lyrics x, the sixth index can be calculated as a value with a greater weighting.

[0161] The seventh index value representing the similarity between the target string c(y) and the lyric string c(x) is output by a learning model that has learned a measure of similarity perceived by humans and can be calculated by a learning model that learns a measure of similarity perceived by humans and outputs a similarity that represents the similarity between the target string c(y) and the lyric string c(x) in accordance with that measure. Learning a measure of similarity perceived by humans is described, for example, in Brian Kulis, "Metric Learning: A Survey," Foundations and Trends in Machine Learning: Vol. 5, No. 4, pp. 287-364, 2013.

[0162] The seventh index value represents how a person perceives the similarity between the target string c(y) and the lyric string c(x). For a learning model that learns a measure of similarity perceived by humans, learning can be performed by separating the learning data into similarity labels representing the similarity perceived by experts and similarity labels representing the similarity perceived by general users (non-experts). In this case, the learning model can obtain the similarity representing the similarity perceived by experts and the similarity representing the similarity perceived by general users as the seventh index value. An expert is, for example, someone involved in providing music, such as an artist, lyricist, composer, or music producer.

[0163] As the similarity between the target character string c(y) and the lyrics character string c(x), for example, one or more of the first to seventh index values ​​as described above can be adopted.

[0164] Specifically, for example, the similarity between the target string c(y) and the lyrics string c(x) can be determined by using one of the first to seventh index values, or by using multiple index values. When multiple index values ​​are used, the weighted sum of the multiple index values ​​is used as the final similarity, and a decision can be made on whether to modify the generated text by resampling and / or replacement based on this final similarity. Furthermore, each of the multiple index values ​​can be compared with an individual threshold value as the final similarity, and a decision can be made on whether to modify the generated text by resampling and / or replacement based on whether the number of similarities that are equal to or greater than the threshold value is a predetermined number or more.

[0165] Furthermore, the similarity between the target string c(y) and the lyrics string c(x) can be calculated using a possible combination of the first through seventh index values, such as a value calculated by combining one of the first through seventh index values ​​with one of the weightings used in calculating the first through seventh index values.

[0166] Specifically, for example, the second index value representing the semantic similarity between the target string c(y) and the lyric string c(x) can be weighted by the fourth index value according to which part (composition) of the song the target string c(y) and the lyric string c(x) belong to, and this weighted value can be used as the similarity between the target string c(y) and the lyric string c(x).

[0167] <Prohibited Expression Evaluation> FIG. 9 is a flowchart illustrating an example of the process of the prohibited expression evaluation performed by the prohibited expression evaluation unit 27 to evaluate whether or not a generated text contains a prohibited expression.

[0168] In step S61, the forbidden expression evaluation unit 27 acquires the generated text from the similarity evaluation unit 26, and the process proceeds to step S62.

[0169] In step S62, the forbidden expression evaluation unit 27 selects an unselected forbidden expression from the forbidden expressions stored in the forbidden expression storage unit 17 as a forbidden expression of interest, and the process proceeds to step S63.

[0170] In step S63, the forbidden expression evaluation unit 27 performs a forbidden expression evaluation (forbidden expression inclusion check) to check (evaluate) whether or not the generated text contains a notable forbidden expression, and the process proceeds to step S64.

[0171] In step S64, the forbidden expression evaluation unit 27 determines whether or not the process is complete. For example, in step S64, it is determined that the process is complete when all of the forbidden expressions stored in the forbidden expression storage unit 17 have been selected as the forbidden expressions of interest.

[0172] If it is determined in step S64 that the process is not to be completed, the process returns to step S62, where a new attention-prohibited expression is selected, and the same process is repeated thereafter.

[0173] If it is determined in step S64 that the process is to be completed, the process ends.

[0174] The forbidden expressions used in the forbidden expression evaluation are determined in advance by the forbidden expression determination unit 16 and stored in the forbidden expression storage unit 17 .

[0175] The forbidden expression determination unit 16 determines, for example, ethically inappropriate words and phrases that should not appear in the generated text as forbidden expressions. For example, the forbidden expression determination unit 16 determines, as forbidden expressions, expressions that are considered to be broadcast-prohibited words. Broadcast-prohibited words can be identified, for example, from publications such as broadcast-prohibited word dictionaries that are publicly available on the Internet. Examples of broadcast-prohibited word dictionaries that are publicly available on the Internet include the Broadcast Prohibited Word List (http: / / monoroch.net / kinshi / ) and the e-book version "Moekin Complete Edition: The Moe Broadcast-Prohibited Word Dictionary" (http: / / monoroch.net / moekin / ).

[0176] Furthermore, the prohibited expression determination unit 16 determines, as prohibited expressions, texts and the like that are estimated to have a high risk of copyright infringement, such as coined words that evoke a specific song or artist. When determining coined words that evoke a specific song or artist as prohibited expressions, the prohibited expression determination unit 16 can be configured with two modules, an extractor and a determiner (not shown).

[0177] The extractor performs a morphological analysis of the text as lyrics stored in the lyrics information storage unit 15, and based on the results of the morphological analysis, extracts part-of-speech n-grams (sequences of n parts of speech) from the text as lyrics as candidate expressions that are candidates for prohibited expressions. The number of parts of speech n in the part-of-speech n-grams can be set in advance or by the user. For example, the extractor uses a morphological analysis algorithm such as MeCab that uses a dictionary containing new words and proper nouns, such as the NEolog dictionary, to extract part-of-speech n-grams containing specific parts of speech (sequences of these) as candidate expressions. The NEolog dictionary is described at neologd / mecab-ipadic-neologd: Neologism dictionary based on the language resources on the Web for mecab-ipadic (github.com) (https: / / github.com / neologd / mecab-ipadic-neologd). MeCab is described in MeCab: Yet Another Part-of-Speech and Morphological Analyzer (taku910.github.io)(https: / / taku910.github.io / mecab / ).

[0178] The determiner determines whether or not the candidate expression extracted by the extractor is determined to be a prohibited expression. For example, the determiner determines whether or not the candidate expression satisfies a predetermined (necessary) condition for being a prohibited expression. The predetermined condition may be, for example, that the candidate expression is included in the title of a song, or that the candidate expression includes the name of a song, a person's name, a character's name, or a building name.

[0179] The determiner counts the number of times that a candidate expression that satisfies a predetermined condition appears on a predetermined internet site (for example, a term site such as Wikipedia) and in existing lyrics. If the number of times that the candidate expression appears on the predetermined site and in existing lyrics is equal to or less than a threshold, the determiner determines that the candidate expression is to be a prohibited expression.

[0180] The forbidden expression determining unit 16 converts the forbidden expressions into tokens (sequences) using, for example, a morphological analyzer used in the language model, and stores the tokens in the forbidden expression storage unit 17 .

[0181] FIG. 10 is a diagram showing examples of forbidden expressions stored in the forbidden expression storage unit 17. As shown in FIG.

[0182] 10, the forbidden expression "Only one flower in the world" is converted into the tokens "in the world," "only one," "of," and "flower" and stored. Each token is associated with a token ID and linguistic information about the token, such as the part of speech (of the word) that constitutes the token.

[0183] <Presentation UI> FIG. 11 is a diagram showing a display example of a presentation UI generated by the presentation control unit 29. As shown in FIG.

[0184] The presentation UI 50 presents the generated text as lyrics, the melody to be attached to the generated text as lyrics (the melody accepted by the melody input unit 22), the evaluation result of the similarity evaluation by the similarity evaluation unit 26, etc. Therefore, the presentation UI 50 allows the user to check the generated text as lyrics, the melody, the evaluation result of the similarity evaluation, etc.

[0185] In FIG. 11, the presentation UI 50 has a note count field 51, a lyrics field 52, a threshold field 53, a correction suggestion field 54, a piano roll field 55, and a similarity information field 56, etc.

[0186] The syllable count field 51 is located, for example, in the upper left of the presentation UI 50. The syllable count field 51 displays the number of syllables for each division unit into which the generated text as lyrics displayed in the lyrics field 52 is divided.

[0187] The lyrics field 52 is located, for example, to the right of the syllable count field 51. In the lyrics field 52, generated text containing a mixture of kanji and kana is displayed as lyrics, separated into predetermined units (such as "dream," "if," and "which" in FIG. 11).

[0188] The threshold field 53 is located, for example, to the right of the lyrics field 52. A slide bar for setting a similarity threshold is displayed in the threshold field 53. The user can set the similarity threshold by operating the slide bar. Therefore, the presentation UI 50 is configured to allow the setting of the similarity threshold. The similarity threshold can be a value set by operating the slide bar, or a preset value. Furthermore, in the interactive version processing or the on-demand version processing, when a user instructs the generated text to be corrected by resampling and / or replacement, a log of the similarity of the portion of the generated text to be corrected is kept. Learning using the similarity described in the log allows the similarity threshold to be set as an appropriate similarity threshold, based on a similarity that is likely to lead the user to instruct the generated text to be corrected.

[0189] The suggestion field 54 is located, for example, to the right of the lyrics field 52 and below the threshold field 53. The suggestion field 54 displays suggestions to be used to correct the generated text by resampling or replacement. The user can select a suggestion displayed in the suggestion field 54 and correct the generated text with the selected suggestion.

[0190] The piano roll field 55 is disposed, for example, below the note count field 51, the lyrics field 52, and the correction candidate field 54. The piano roll field 55 displays a piano roll with lyrics. For example, the piano roll field 55 displays a piano roll in which rectangles representing the pitch and duration of notes are arranged at positions (vertically) corresponding to the pitch of the notes and with widths (horizontally) corresponding to the duration of the notes. The rectangles representing the pitch and duration of the notes display text in hiragana as the lyrics corresponding to the notes. Therefore, the piano roll field 55 allows the user to confirm the correspondence between the generated text as lyrics and the melody accepted by the melody input unit 22, i.e., to confirm the lyrics (hiragana) corresponding to the notes.

[0191] In addition, instead of a piano roll, the piano roll field 55 can display, for example, a staff on which notes indicating the pitch and duration of notes and text as lyrics corresponding to those notes are arranged.

[0192] The similarity information field 56 is located, for example, below the piano roll field 55. The similarity information field 56 displays information about the similarity of the generated text as lyrics displayed in the lyrics field 52 to existing lyrics in terms of copyright infringement.

[0193] The user can select a portion of the generated text as lyrics displayed in the lyrics field 52 of the presentation UI 50 that the user wants to focus on.

[0194] FIG. 12 is a diagram showing an example of the display of the presentation UI 50 in a state where a portion of interest is selected in the generated text as lyrics displayed in the lyrics field 52.

[0195] In FIG. 12, in the text of lyrics displayed in the lyrics field 52, the parts "dream" and "if" are selected as the parts of interest.

[0196] In the piano roll field 55, lyrics including the portion of interest, for example, the lyrics of the line of interest in the lyrics field 52 (hereinafter also referred to as the line of interest), are displayed in hiragana within a rectangle representing the pitch and duration of the notes. In Figure 12, in the lyrics field 52, the lyrics of the line of interest, which is the line of interest "If it were a dream," are displayed in hiragana within a rectangle representing the pitch and duration of the notes. The correspondence between the sounds and the characters constituting the lyrics does not necessarily correspond to one character per sound, and there may be cases where one sound corresponds to two or more characters. For example, in Figure 12, the "tta" and "sho" in the lyrics "Yokatta desho" correspond to one sound.

[0197] Here, in the generated text as lyrics displayed in the lyrics field 52, for example, when the last line "of old memories" is selected as the part of interest, the line containing the part of interest "of old memories" becomes the line of interest in the piano roll field 55, and the lyrics of the line of interest "to wipe away the dust of old memories" are displayed in hiragana within a rectangle representing the pitch and length of the notes.

[0198] If the similarity of the portion of interest is equal to or greater than a threshold, the correction candidate field 54 displays a correction candidate for correcting the portion of interest. The user can correct the portion of interest using the correction candidate by selecting a correction candidate displayed in the correction candidate field 54. If the correction candidate selected by the user is a correction candidate obtained by resampling, the portion of the generated text as lyrics that follows the portion of interest may also be corrected by resampling. The presentation UI 50 displays the generated text after correction. The user can confirm the similarity of the corrected generated text, and can further correct the corrected generated text as needed, and can also correct the melody input into the melody input unit 22 as needed, to complete the song.

[0199] When the user changes the threshold by operating the slide bar in the threshold field 53, the content presented in the presentation UI 50 may change. For example, if the threshold is changed to a lower value and the number of parts in the generated text as lyrics with a similarity above the threshold increases, the presentation of parts with a similarity above the threshold increases. In the presentation UI 50, parts of the generated text as lyrics with a similarity above the threshold can be displayed in a different display format from the other parts, for example, in a different color from the other parts.

[0200] In the piano roll field 55, an alert can be displayed indicating that the similarity is above a threshold for portions of the generated text as lyrics or portions where the melody similarity (similarity indicating the similarity to the melody of an existing musical piece) is above a threshold. For example, an alert mark 61 indicating that the similarity is above a threshold can be displayed at the beginning of portions of the lyrics of a focused line where the similarity (third index value) indicating the similarity taking melody into consideration is above a threshold, or portions where the melody similarity to the melody corresponding to the lyrics of the focused line is above a threshold (hereinafter, these portions are also referred to as melody-similar portions). Furthermore, melody-similar portions can be displayed in a different display format from other portions. In FIG. 12 , in the piano roll field 55, the portion "It must have been nice" in the lyrics of the focused line (the first line of the lyrics field 52) "If it were a dream, how nice would it have been?" is identified as a melody-similar portion. An alert mark 61 is displayed at the beginning of the melody-similar portion "It must have been nice." Furthermore, the melody-similar portion "It must have been nice" is displayed in a different display format from the other portion "If it were a dream, how nice would it have been." In FIG. 12, the other part "If it's a dream, how much?" is displayed in white letters, while the melody-like part "Isn't it nice?" is displayed in black letters.

[0201] As described above, in the piano roll field 55, the alert mark 61 is displayed for the melody-similar portion of the lyrics of the focused line, and is displayed in a different display format from the other portions. This alerts the user to the presence of a melody-similar portion in the lyrics of the focused line. Furthermore, the user can check on the piano roll (or musical staff) the similarity of the focused line's lyrics to existing music, taking into account both the lyrics of the focused line and the melody corresponding to those lyrics.

[0202] When an alert mark 61 in the piano roll field 55 or a similar melody portion is selected, the presentation UI 50 can display details about the similarity of the similar melody portion, such as the degree of similarity of the similar melody portion, thereby allowing the user to check the details about the similarity of the similar melody portion.

[0203] Furthermore, when a part other than the melody-similar part is selected in the piano roll field 55, the presentation UI 50 can display the similarity of the selected part, thereby allowing the user to confirm the similarity of the part other than the melody-similar part.

[0204] The similarity information field 56 displays the lyrics of the line of interest, and displays information about the similarity of the lyrics of the line of interest to existing lyrics. For example, in the similarity information field 56, parts of the lyrics of the line of interest that have a similarity above a threshold are displayed in a different display format from the other parts. This makes it possible to alert the user to the existence of parts that are similar to existing lyrics. In FIG. 12 , the similarity of the part "if it were a dream" in the lyrics of the line of interest, "How nice would it be if it were a dream?", is above the threshold, and the part "if it were a dream" is displayed in a different display format from the other parts, such as a different color, as shown in bold in the figure.

[0205] In the similarity information field 56, for the lyrics of the line of interest, the parts whose similarity is above a threshold are displayed in a different display format from the other parts, and even if the similarity is not above the threshold, parts that are close to the threshold and have a certain degree of similarity to existing lyrics can be displayed in a different display format from the parts whose similarity is above the threshold and the other parts, for example in a different color.

[0206] The user can select any portion of the lyrics of the focused line displayed in the similarity information field 56. When any portion of the lyrics of the focused line displayed in the similarity information field 56 is selected, the presentation UI 50 can display the similarity of the selected portion. This allows the user to confirm the similarity of the selected portion. If the similarity evaluation unit 26 calculates multiple types of similarity, the presentation UI 50 can display each of the multiple types of similarity of the selected portion. When displaying multiple types of similarity, each type can be displayed in a different display format, for example, a different color. Furthermore, the presentation UI 50 can display details regarding the similarity, including the similarity, for the selected portion. In FIG. 12 , the portion "If it were a dream" in the lyrics of the focused line "How nice it would be if it were a dream" is selected, and a comment 62 regarding the similarity is displayed as details regarding the similarity of the portion "If it were a dream" In Figure 12, comments 62 regarding similarity include, "The lyrics are 80% the same as bbbb (song title) by aaaa (singer name)," "There is a similar item in category cccc (movie title)," and "There is a similar item in category dddd (novel title)."

[0207] Additionally, in the lyrics of the focused line, for a portion where the similarity to existing lyrics is equal to or greater than a threshold, information about the song corresponding to the existing lyrics (existing lyrics with a similarity equal to or greater than the threshold) can be displayed in the similarity information field 56. Information about the song can include, for example, the song title, an image of the album jacket on which the song is recorded, an image of the artist of the song, etc.

[0208] As described above, the presentation UI 50 can display the alert mark 61 in the piano roll field 55, or, when a similar melody portion is selected, details regarding the similarity of the similar melody portion.

[0209] FIG. 13 is a diagram showing an example of a display of details regarding the similarity of similar melody parts.

[0210] 13, a comment 71 on the similarity is displayed as a detailed description of the similarity of the melody similarity portion "Yokatta desho." The comment 71 on the similarity states, "The lyrics and melody are 70% similar to ffff (song title) by eeee (singer name)."

[0211] FIG. 14 is a diagram showing an example of the display of the similarity information field 56. As shown in FIG.

[0212] As described above, in the similarity information field 56, the portion of the lyrics of the focused line where the similarity is above a threshold can be displayed in a different display format from the other portions. As described above, the portion of the lyrics of the focused line where the similarity is above a threshold can be displayed in a different color from the other portions, and can also be displayed with a different background color. Furthermore, the background color of the portion of the lyrics of the focused line where the similarity is above a threshold can be shaded according to the similarity.

[0213] Furthermore, if the lyrics of the line of interest are a mixture of kanji and kana, and for example, the melody similarity changes in the middle of the reading of a kanji, the background of the kanji can be divided into left and right halves at the position where the melody similarity changes, and shading can be applied according to the melody similarity, as shown in Figure 14. In Figure 14, the melody similarity differs between the "i" and "ma" of the reading "ima" of the kanji "mi," and the background of the kanji "mi" is divided into two, left and right, and shading according to different melody similarities is applied to the background corresponding to the "i" and the background corresponding to the "ma" of the reading "ima" of the kanji "mi."

[0214] In the similarity information field 56, when the melody similarity of the melody corresponding to the lyrics of the focused line is equal to or greater than a threshold, the similarity of the melodies can be displayed differently from when the similarity of the lyrics of the focused line is equal to or greater than a threshold. For example, when the melody similarity of the melody corresponding to the lyrics of the focused line is equal to or greater than a threshold, the similarity information can be displayed in a different display format, such as a different color, depending on whether the melody similarity of the melody corresponding to the lyrics of the focused line is equal to or greater than a threshold. Furthermore, when the melody similarity of the melody corresponding to the lyrics of the focused line is equal to or greater than a threshold, details regarding the similarity of the melody can be displayed in the piano roll field 55. For example, a comment such as "The melody is 70% similar to ffff (song title) by eeee (singer name)" can be displayed as the details regarding the similarity of the melody. The "70%" in this comment refers to the melody similarity.

[0215] <Description of a Computer to which the Present Technology is Applied> Next, the above-described series of processes can be performed by hardware or software. When the series of processes are performed by software, a program constituting the software is installed in a general-purpose computer or the like.

[0216] FIG. 15 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0217] The program can be recorded in advance on the hard disk 905 or ROM 903 as a recording medium built into the computer.

[0218] Alternatively, the program can be stored (recorded) on a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 can be provided as a so-called package software. Here, examples of the removable recording medium 911 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0219] The program can be installed into the computer from the removable recording medium 911 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 905. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer via a wired network such as a LAN (Local Area Network) or the Internet.

[0220] The computer includes a CPU (Central Processing Unit) 902 , to which an input / output interface 910 is connected via a bus 901 .

[0221] When a user inputs a command via an input / output interface 910 by operating an input unit 907, the CPU 902 executes a program stored in a read-only memory (ROM) 903 in accordance with the command. Alternatively, the CPU 902 loads a program stored on a hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0222] As a result, the CPU 902 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 902 outputs the processing results from the output unit 906 via the input / output interface 910, or transmits them from the communication unit 908, or further records them on the hard disk 905, as necessary.

[0223] The input unit 907 is made up of a keyboard, a mouse, a microphone, etc. The output unit 906 is made up of an LCD (Liquid Crystal Display), a speaker, etc.

[0224] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0225] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0226] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0227] It should be noted that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the present technology.

[0228] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0229] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0230] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0231] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0232] The present technology can have the following configurations.

[0233] <1> An information processing device including a correction unit that corrects generated text composed of tokens generated by sampling by replacement or resampling, based on similarity of the generated text to existing works in terms of copyright infringement. <2> The information processing device described in <1>, wherein the correction unit corrects the generated text by replacement using a replacement dictionary in which replacement target words to be replaced are associated with replacement words that replace the replacement target words. <3> The information processing device described in <2>, wherein the correction unit defines a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a modification word, and if part or all of the modification word matches the replacement target word, replaces the replacement target word in the modification word with the replacement word associated with the replacement target word. <4> The information processing device described in <3>, wherein the correction unit, if a modification-word-containing word that includes part or all of the modification word matches the replacement target word in the generated text, replaces the modification-word-containing word with the replacement word associated with the replacement target word that matches the modification-word-containing word. <5> The information processing device according to any one of <2> to <4>, wherein the correction unit corrects the generated text by resampling. <6> The information processing device according to any one of <1> to <5>, wherein the correction unit, taking a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a modified word, corrects the generated text by repeatedly resampling tokens from a position going back from the end of the generated text to the beginning until the modified word no longer appears in the generated text. <7> The information processing device according to any one of <1> to <5>, wherein the correction unit, taking a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a modified word, corrects the generated text by resampling from a token at a position going back a number of tokens constituting the modified word, going back from the end of the generated text to the beginning. <8> The information processing device according to any one of <1> to <5>, wherein the correction unit corrects the generated text by resampling from a token at a position going back a number of tokens constituting the generated text, going back a number of tokens constituting the generated text, going back a number of tokens constituting the generated text.<9> The information processing device of any one of <1> to <5>, wherein the correction unit modifies the generated text by taking a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a modified word, and resampling from tokens that include a word of a predetermined part of speech among the tokens that make up the modified word. <10> The information processing device of any one of <1> to <9>, further comprising a similarity evaluation unit that calculates the similarity representing the similarity, wherein the correction unit modifies the generated text including the modified word, by taking a word in the generated text whose similarity is equal to or greater than a threshold as a modified word. <11> The information processing device of <10>, wherein the similarity evaluation unit extracts, from the generated text, a target string for which the similarity to existing lyrics is to be calculated, and calculates the similarity for the target string. <12> The information processing device of <11>, wherein the similarity evaluation unit calculates the similarity for lyrics of songs by the artist that are in the top R positions closest to a vector that embeds and represents the target string, when the artist's representative vector, generated based on a vector that embeds and represents the lyrics of the artist's songs, is located. <13> The information processing device according to <11> or <12>, wherein the similarity evaluation unit calculates, as the similarity, one or more of: an index value representing identicalness between the target string and the existing lyrics on a character-by-character basis; an index value representing semantic similarity between the target string and the existing lyrics; an index value representing similarity between the target string and the existing lyrics taking into account melody; an index value representing similarity between the target string and the existing lyrics taking into account song structure; an index value representing similarity between the target string and the existing lyrics taking into account meta information of the song and / or artist; an index value representing similarity between the target string and the existing lyrics taking into account information input by a user; and an index value representing similarity between the target string and the existing lyrics output by a learning model that has learned a scale of similarity perceived by humans.<14> The information processing device according to any one of <1> to <13>, further comprising a sampling unit that performs sampling to select the nth token from each candidate token based on the probability that each candidate token is the nth token of the generated text, wherein the sampling unit also functions as the correction unit that corrects the generated text by resampling. <15> The information processing device according to any one of <1> to <14>, wherein the correction unit corrects the generated text including a predetermined prohibited expression as a corrective word. <16> The information processing device according to any one of <1> to <15>, further comprising a presentation control unit that generates a presentation user interface (UI) that presents the generated text. <17> The information processing device according to <16>, wherein the presentation UI displays portions of the generated text where the similarity representing the similarity is equal to or greater than a threshold in a display format different from that of other portions. <18> The information processing device according to <17>, wherein the presentation UI is configured to allow the threshold to be set. <19> An information processing method including modifying generated text by replacement or resampling, based on the similarity of generated text made up of tokens generated by sampling to existing works in terms of copyright infringement. <20> A program for causing a computer to function as an information processing device, comprising a modification unit that modifies generated text by replacement or resampling, based on the similarity of generated text made up of tokens generated by sampling to existing works in terms of copyright infringement.

[0234] 10 Information processing device, 11 Manual similarity evaluation result storage unit, 12 Similarity evaluation function learning unit, 13 Artist information storage unit, 14 Music information storage unit, 15 Lyric information storage unit, 16 Prohibited expression determination unit, 17 Prohibited expression storage unit, 20 Generation unit, 21 Prompt input unit, 22 Melody input unit, 23 Token embedding storage unit, 24 Replacement dictionary storage unit, 25 Sampling unit, 26 Similarity evaluation unit, 27 Prohibited expression evaluation unit, 28 Replacement unit, 29 Presentation control unit, 31 Correction candidate storage unit, 50 Presentation UI, 51 Number of notes field, 52 Lyrics field, 53 Threshold field, 54 Correction candidate field, 55 Piano roll field, 56 Similarity information field, 61 Alert mark, 62, 71 Comments on similarity, 901 Bass, 902 CPU, 903 ROM, 904 RAM, 905 hard disk, 906 output unit, 907 input unit, 908 communication unit, 909 drive, 910 input / output interface, 911 removable recording medium

Claims

1. An information processing device comprising a correction unit that corrects generated text, which is composed of tokens generated by sampling, by substitution or resampling based on the similarity of the generated text to existing works in terms of copyright infringement.

2. The information processing device according to claim 1, wherein the correction unit corrects the generated text by substitution using a substitution dictionary in which target words to be replaced are associated with replacement words that replace the target words.

3. The information processing device according to claim 2, wherein the correction unit defines a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a modified word, and if part or all of the modified word matches the replacement target word, replaces the replacement target word in the modified word with the replacement word associated with the replacement target word.

4. The information processing device described in claim 3, wherein when a modified word containing a part or all of the modified word matches the replacement target word in the generated text, the correction unit replaces the modified word containing the modified word with the replacement word associated with the replacement target word that matches the modified word containing the modified word.

5. The information processing device according to claim 2, wherein the correction unit corrects the generated text by resampling.

6. The information processing device according to claim 1, wherein the correction unit corrects the generated text by repeatedly resampling tokens from a position going back from the end of the generated text to the beginning, using a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a correction word, until the correction word no longer appears in the generated text.

7. The information processing device according to claim 1, wherein the correction unit corrects the generated text by taking a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a correction word and performing resampling from a token located as far back as the number of tokens constituting the correction word, moving from the end of the generated text to the beginning.

8. The information processing device according to claim 1, wherein the correction unit corrects the generated text by resampling from a token located a predetermined percentage of the number of tokens constituting the generated text, going back from the end of the generated text to the beginning.

9. The information processing device according to claim 1, wherein the correction unit corrects the generated text by taking a word in the generated text whose similarity representing the similarity is equal to or greater than a threshold as a modified word and resampling from tokens that include a word of a predetermined part of speech among the tokens that make up the modified word.

10. An information processing device according to claim 1, further comprising a similarity evaluation unit that calculates a similarity indicating the similarity, wherein the correction unit corrects the generated text including a modified word in the generated text, the modified word being a word in the generated text whose similarity is equal to or greater than a threshold value.

11. The information processing device according to claim 10, wherein the similarity evaluation unit extracts, from the generated text, a target character string for which the similarity to existing lyrics is to be calculated, and calculates the similarity for the target character string.

12. The information processing device described in claim 11, wherein the similarity evaluation unit calculates the similarity for the lyrics of songs by the artist that are in the top R positions where the artist's representative vector, generated based on a vector that embeds and represents the lyrics of the artist's songs, is closest to the vector that embeds and represents the target character string.

13. The information processing device according to claim 11, wherein the similarity evaluation unit calculates, as the similarity, one or more of the following: an index value representing the identity of the target string and the existing lyrics on a character-by-character basis; an index value representing the semantic similarity between the target string and the existing lyrics; an index value representing the similarity between the target string and the existing lyrics taking into account melody; an index value representing the similarity between the target string and the existing lyrics taking into account song structure; an index value representing the similarity between the target string and the existing lyrics taking into account meta information about the song and / or artist; an index value representing the similarity between the target string and the existing lyrics taking into account information input by a user; and an index value representing the similarity between the target string and the existing lyrics output by a learning model that has learned a scale of similarity perceived by humans.

14. The information processing device of claim 1, further comprising a sampling unit that performs sampling to select the nth token from each candidate token based on the probability that each candidate token is the nth token of the generated text, and the sampling unit also functions as the correction unit that corrects the generated text by resampling.

15. The information processing device according to claim 1, wherein the correction unit corrects the generated text including a predetermined prohibited expression as a corrective word.

16. The information processing device according to claim 1, further comprising a presentation control unit that generates a presentation UI (user interface) that presents the generated text.

17. The information processing device according to claim 16, wherein the presentation UI displays portions of the generated text where the degree of similarity representing the similarity is equal to or greater than a threshold in a display format different from that of other portions.

18. The information processing device according to claim 17, wherein the presentation UI is configured to enable setting of the threshold value.

19. An information processing method comprising modifying a generated text, consisting of tokens generated by sampling, by substitution or resampling based on the similarity of the generated text to an existing work from the perspective of copyright infringement.

20. A program for causing a computer to function as an information processing device, which includes a correction unit that corrects generated text, which is composed of tokens generated by sampling, by substitution or resampling based on the similarity of the generated text to existing works in terms of copyright infringement.

Images