Server for providing multimedia content according to text analysis result based on artificial intelligence model, and operation method therefor

Abstract

The present disclosure relates to a server for providing multimedia content on the basis of text analysis, and an operation method therefor. A method for providing multimedia content by a server on the basis of text analysis according to an embodiment may comprise the steps of: acquiring a core keyword for each paragraph from an electronic device, the core keyword being determined on the basis of morpheme analysis on text including at least one paragraph; on the basis of whether the core keyword for each paragraph is a polysemous word or a homonymous word, determining a target meaning for the core keyword for each paragraph; on the basis of whether two or more paragraphs including the core keyword indicating the same target meaning are identified in the text, matching an image resource for each of the paragraphs; generating multimedia content by synthesizing the image resource matched for each of the paragraphs; and transmitting the generated multimedia content to the electronic device.

Description

A server providing multimedia content based on text analysis results based on an artificial intelligence model and its operation method.

[0001] The present invention relates to a server that provides multimedia content based on text analysis, and to a method for operating the same. More specifically, the present invention relates to a server that provides multimedia content by matching video resources based on core keywords determined for each paragraph of text, and to a method for operating the same.

[0002] Multimedia content is experiencing rapid growth in the digital media and entertainment sector, a trend accelerated particularly by the proliferation of the Internet and mobile devices. Multimedia plays a key role in diverse fields, including information delivery, education, advertising, and entertainment. Recently, a variety of multimedia content has been provided through personal channels.

[0003] However, existing multimedia content creation is limited to professionals or users who can handle editing software, so there is a problem that it takes a lot of time and money for general users to create multimedia content.

[0004] To address these issues, prompt-based multimedia content generation technologies have recently been developed, but they have limitations in accurately identifying the context of the prompt and providing the multimedia content desired by the user.

[0005] Therefore, there is a need for technology development to match context-appropriate video resources based on accurate semantic identification of polysemous words contained in input text and to prevent duplicate matching of matched video resources.

[0006] According to one embodiment, a server and its operating method for providing multimedia content based on text analysis may be provided. More specifically, a server and its operating method may be provided that identifies the precise meaning of core keywords in each paragraph of text and, based on this, provides multimedia content that does not include overlapping video resources.

[0007] According to one embodiment of the present disclosure for achieving the above-described technical problem, a method for generating multimedia content based on text analysis by a server connected to an electronic device providing a multimedia content generation service may be provided, the method including: obtaining a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device; determining a target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemy or a homonym; matching a video resource for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the core keyword based on whether two or more paragraphs including the core keyword indicating the same target meaning are identified in the text; and generating multimedia content by synthesizing the matched video resources for each paragraph; transmitting the generated multimedia content to the electronic device.

[0008] According to another embodiment of the present disclosure for solving the above technical problem, a server for generating multimedia content based on text analysis, comprising: a network interface; a database for storing one or more instructions; and at least one processor for executing the one or more instructions; wherein the at least one processor obtains a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device, determines a target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemy or a homonym, matches an image resource for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the core keyword and the target meaning based on whether two or more paragraphs including the core keyword indicating the same target meaning are identified in the text, and generates multimedia content by synthesizing the matched image resources for each paragraph, and transmits the generated multimedia content to the electronic device.

[0009] According to another embodiment of the present disclosure for achieving the above-described technical problem, a method for generating multimedia content based on text analysis by a server connected to an electronic device providing a multimedia content generation service may be provided, wherein a computer-readable recording medium storing a program for causing the method to be performed includes: obtaining a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device; determining a target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemous word or a homonym; matching an image resource for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the core keyword based on whether two or more paragraphs including the core keyword representing the same target meaning are identified in the text; and generating multimedia content by synthesizing the matched image resources for each paragraph; transmitting the generated multimedia content to the electronic device.

[0010] In one embodiment, the satisfaction of provided multimedia content can be improved by performing accurate image resource matching according to the context of a prompt input by a user based on text analysis.

[0011] In one embodiment, multimedia content intended by a user can be provided by performing image resource matching based on accurate semantic identification of polysemous or homonymous words.

[0012] In one embodiment, by not matching overlapping video resources per paragraph of a prompt, rich multimedia content can be provided.

[0013] FIG. 1 is a diagram schematically illustrating a process for providing multimedia content based on text analysis according to one embodiment.

[0014] FIG. 2 is a flowchart of a method for an electronic device to provide multimedia content based on text analysis according to one embodiment.

[0015] FIG. 3 is a flowchart specifically explaining a method for an electronic device to determine a core keyword according to one embodiment.

[0016] FIG. 4 is a flowchart specifically explaining a method for an electronic device to divide text into paragraphs according to one embodiment.

[0017] FIG. 5 is a flowchart of a method in which a server generates multimedia content based on core keywords obtained from an electronic device according to one embodiment.

[0018] FIG. 6 is a flowchart of a method by which a server determines the target meaning of a core keyword according to one embodiment.

[0019] FIG. 7 is a diagram illustrating an operation of a server according to one embodiment of the present invention to match different video resources for paragraphs having the same target meaning.

[0020] Figure 8 is a block diagram of an electronic device according to one embodiment.

[0021] Figure 9 is a block diagram of a server according to one embodiment.

[0022] FIG. 10 is a diagram for explaining a process of generating multimedia content by interworking between an electronic device and a server according to one embodiment.

[0023] In one embodiment, a method for a server to provide multimedia content based on text analysis may include: obtaining a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device; determining a target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemous word or a homonym; matching a video resource for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the core keyword based on whether two or more paragraphs including the core keyword indicating the same target meaning are identified in the text; and generating multimedia content by synthesizing the matched video resources for each paragraph; transmitting the generated multimedia content to the electronic device.

[0024] According to another embodiment, a server for providing multimedia content based on text analysis includes: a network interface; a database for storing one or more instructions; and at least one processor for performing the one or more instructions, wherein the at least one processor obtains a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device, determines a target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemy or a homonym, and matches an image resource for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the core keyword based on whether two or more paragraphs including the core keyword representing the same target meaning are identified in the text, and generates multimedia content by synthesizing the matched image resources for each paragraph, and transmits the generated multimedia content to the electronic device.

[0025] Hereinafter, terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0026] The terms used in this disclosure have been selected from widely used, current terms, taking into account the functions of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of the disclosure.

[0027] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used throughout the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0028] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted for clarity of description of the present disclosure, and similar parts are designated with similar reference numerals throughout the specification.

[0029]

[0030] FIG. 1 is a diagram schematically illustrating a process for providing multimedia content based on text analysis according to one embodiment.

[0031] According to one embodiment, the multimedia content providing system (10) can obtain text (prompt) based on user input, and generate and synthesize multimedia content by applying an artificial intelligence model to the obtained text. According to one embodiment, the multimedia content providing system (10) can resolve the problem of matching video resources that do not correspond to the context of the input text by determining core keywords for each paragraph of the obtained text, and identifying the exact meaning that fits the context depending on whether the determined core keywords are polysemous or homonyms.

[0032] In addition, according to one embodiment, the multimedia content provision system (10) matches video resources by considering sub-keywords that are dependent on the core keyword through structural analysis of sentences or paragraphs containing the core keyword when the meaning of the core keyword for each paragraph is the same, thereby preventing duplicate matching of the same video resource for each paragraph and providing multimedia content that is more similar to the context of the text.

[0033] According to one embodiment, the multimedia content providing system (10) may include an electronic device (1000) and a server (2000). However, the present invention is not limited to the above-described example, and according to another embodiment, the multimedia content providing system (10) may further include an electronic device (4000) connected via a network (3000). The electronic device (1000) and the electronic device (4000) illustrated in FIG. 1 may be terminals or computer devices used by different users who utilize the multimedia content providing service provided by the multimedia content providing system (10).

[0034] An electronic device (1000) may obtain text (prompt) (110) from a user, summarize the obtained text (120), and perform morphological analysis (130) on the summarized text, thereby determining (140) a core keyword for each paragraph. In one embodiment, the prompt may include paragraphs including one or more sentences, and text including the paragraphs.

[0035] The server (2000) can match (160) image resources for each paragraph included in the text based on the core keyword obtained from the electronic device (1000). According to one embodiment, when a core keyword is input, the server (2000) can match image resources using an artificial intelligence-based image generation model that matches image resources based on keywords for each paragraph within the text or summary text based on whether the core keyword is a polysemous word or a homonym (150). According to one embodiment, the server (2000) can generate multimedia content (170) by matching image resources to text. It goes without saying that the above-described operations may be performed by the server (2000) or the electronic device (1000).

[0036]

[0037] FIG. 2 is a flowchart of a method for an electronic device to provide multimedia content based on text analysis according to one embodiment.

[0038] In S210, the electronic device (1000) can obtain a text including at least one paragraph. According to one embodiment, the electronic device (1000) can obtain the text from a user input or an external device connected to the electronic device, and if the obtained signal is a voice signal, the obtained voice signal can be converted into text to obtain the text.

[0039] In S220, the electronic device (1000) may determine core keywords for each paragraph based on morphological analysis of the acquired text. According to one embodiment, the electronic device (1000) may summarize the text based on a user request obtained or the length of the text, and perform morphological analysis on the summarized text to determine core keywords for each paragraph.

[0040] According to one embodiment, the electronic device (1000) may determine a core keyword for each paragraph based on at least one of a keyword network including a keyword frequency for each paragraph, user feedback accumulated for each frequency, a keyword frequency determined for each paragraph, and information regarding a connection relationship of the keywords.

[0041] In S230, the electronic device (1000) can transmit the determined paragraph-by-paragraph core keyword to the server (2000).

[0042] In S240, the electronic device (1000) can obtain multimedia content generated by synthesizing image resources matched for each paragraph based on at least one sub-keyword determined based on whether the core keyword is a polysemous or homonymous word or whether two or more paragraphs containing the core keyword that represents the same target meaning as the target meaning in the text are identified from the server (2000).

[0043] In one embodiment, the target meaning may be one meaning among several candidate meanings of the core keyword that is determined according to the context within the text, and the sub-keyword may be a keyword that is dependent on the core keyword.

[0044] In S250, the electronic device (1000) can output the generated multimedia content.

[0045] As another example, it goes without saying that at least one of the operations S210 to S350 illustrated in FIG. 2 can be performed by a server (2000) connected to an electronic device (1000) or by a linkage between the electronic device (1000) and the server (2000).

[0046]

[0047] FIG. 3 is a flowchart specifically explaining a method for an electronic device to determine a core keyword according to one embodiment.

[0048] In S310, the electronic device (1000) may summarize the text based on a user input requesting a text summary or attention determined based on the length of the text. In one embodiment, the electronic device (1000) may generate a summarized text by applying a sequence-to-sequence model including a weight determined based on attention determined based on the length of the text to the text.

[0049] In S320, the electronic device (1000) can divide the summarized text into paragraphs. In one embodiment, the level of paragraph division can be adjusted by merging or dividing paragraphs based on the keyword network similarity of the paragraphs within the generated summarized text.

[0050] In S330, the electronic device (1000) can determine core keywords for each segmented paragraph. According to one embodiment, the electronic device (1000) can determine core keywords for each paragraph based on the frequency of use of a plurality of morpheme keywords obtained by performing morphological analysis on the summarized text and the correlation between the morpheme keywords. For example, the electronic device (1000) can perform morphological analysis on the text using the OKT morphological analyzer. However, the present invention is not limited thereto, and various morphological analyzers such as the Komoran, Hannanum, and Kkma morphological analyzers can be used as needed.

[0051]

[0052] FIG. 4 is a flowchart specifically explaining a method for an electronic device to divide text into paragraphs according to one embodiment.

[0053] In S410, the electronic device (1000) may determine a first paragraph division point within the text based on the number of keywords within the text. For example, the electronic device (1000) may determine a first paragraph division point for initially dividing the text based on the number of keywords used within the text.

[0054] In S420, the electronic device (1000) may determine a keyword network regarding at least one of the frequencies and connection relationships of the core keywords included in the first divided paragraphs divided according to the first paragraph division point. According to one embodiment, the electronic device (1000) may determine a keyword network including the keyword frequency for each first divided paragraph, user feedback cumulatively measured for each frequency, the frequency of keywords determined for each paragraph, and information regarding the connection relationships of the keywords.

[0055] In S430, the electronic device (1000) may determine the similarity of the keyword network for adjacent paragraphs among the first divided paragraphs. According to one embodiment, the electronic device (1000) may determine the similarity of the keyword network for each of the first divided paragraphs with adjacent paragraphs.

[0056] In S440, the electronic device (1000) can determine a second paragraph division point within the text by merging adjacent paragraphs whose keyword network similarity is greater than or equal to a threshold similarity. For example, if the keyword network similarity of two adjacent paragraphs is greater than or equal to a preset threshold similarity, the electronic device (1000) can identify that the contexts of the two paragraphs are similar and, by merging the two paragraphs, determine a second paragraph division point where the paragraph division point has been changed.

[0057] In S450, the electronic device (1000) can divide the text into paragraphs corresponding to second divided paragraphs determined based on the second paragraph division point.

[0058]

[0059] FIG. 5 is a flowchart of a method in which a server generates multimedia content based on core keywords obtained from an electronic device according to one embodiment.

[0060] In S510, the server (2000) can obtain a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from an electronic device.

[0061] In S520, the server (2000) can determine the target meaning of the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemy or a homonym. The operation of the server (2000) determining the target meaning of the core keyword based on whether the core keyword for each paragraph is a polysemy or a homonym will be described in detail with reference to FIG. 6 described below.

[0062] In S530, the server (2000) can match video resources for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the core keyword, depending on whether two or more paragraphs containing core keywords representing the same target meaning are identified in the text. For example, the server (2000) can determine at least one sub-keyword associated with a core keyword so as not to match the same video resources for paragraphs having the same target meaning of the core keyword, and can match video resources for each paragraph based on the determined sub-keyword.

[0063] In one embodiment, sub keywords may be determined based on the priority of morpheme keywords that are dependent on the core keyword and the number of times they are used within the paragraph.

[0064] In S540, the server (2000) can generate multimedia content by synthesizing video resources matched to each paragraph. According to one embodiment, the video resources can be labeled with labeling values ​​that include at least one of a core keyword, a target meaning, or sub-keywords that are dependent on the target meaning. For example, the server (2000) can generate multimedia content by matching video resources to text based on the labeling values ​​of the video resources.

[0065] In S550, the server (2000) can transmit the generated multimedia content to the electronic device.

[0066] As another example, it goes without saying that at least one of the operations S510 to S550 illustrated in FIG. 5 can be performed by an electronic device connected to a server or by a linkage between an electronic device and a server.

[0067]

[0068] FIG. 6 is a flowchart of a method by which a server determines the target meaning of a core keyword according to one embodiment.

[0069] In S610, the server (2000) can obtain paragraph-by-paragraph core keywords from the electronic device (1000).

[0070] In S620, the server (2000) can identify whether a core keyword for each paragraph is a polysemy or homonym. According to one embodiment, the server (2000) can identify whether a core keyword determined for each paragraph is a polysemy or homonym based on a preset list of polysemy or homonyms.

[0071] In S630, if the core keyword for each paragraph is not a polysemous or homonymous word, the server (2000) can determine a single meaning corresponding one-to-one to the core keyword as the target meaning of the core keyword.

[0072] In S640, the server (2000) can determine the paragraph-specific core keyword as a target keyword if the paragraph-specific core keyword is a polysemous word or a homonym.

[0073] In S650, the server (2000) can convert multiple meanings associated with the determined target keyword into vectors of a predetermined dimension. According to one embodiment, the server (2000) can convert multiple candidate meanings of the target keyword into multiple candidate meaning vectors using a predetermined word-to-vector conversion (Word2Vec) function.

[0074] In S660, the server (2000) can determine a sentence analysis range regarding a distance range in which sentences to be analyzed to identify the target meaning are separated from a sentence including the target keyword based on the distance between the transformed vectors.

[0075] According to one embodiment, the server (2000) may determine the sentence analysis range as the first sentence analysis range if the minimum value of at least one of the identified distances is greater than or equal to a preset threshold distance. For example, if the minimum value of at least one of the identified distances is greater than or equal to a threshold distance, the server (2000) may identify that the semantic distance between candidate meanings is large and determine the sentence analysis range as the first sentence analysis range.

[0076] According to another embodiment, the server (2000) may identify the number of distances between vectors less than the threshold distance when the minimum value of at least one identified distance is less than the threshold distance, and if the number of distances between the identified vectors is less than a preset threshold number, the server (2000) may determine the sentence analysis range as a second sentence analysis range. For example, even if the minimum value of the distance is less than the threshold distance, if the number of distances between vectors less than the threshold distance is less than the threshold number, the server (2000) may identify that the number of candidate meanings requiring comparison is small, and determine the sentence analysis range as a second sentence analysis range that is wider than the first sentence analysis range.

[0077] According to another embodiment, if the number of distances between the identified vectors is greater than or equal to a preset threshold, the server (2000) may determine the sentence analysis range as a third sentence analysis range. For example, if the number of distances between the identified vectors is greater than or equal to the threshold, the server (2000) may identify that there are many candidate meanings with close semantic distances, and determine the sentence analysis range as a third sentence analysis range that is wider than the second sentence analysis range.

[0078] In S670, the server (2000) can identify entity names for words included in at least one sentence determined based on the determined sentence analysis range. For example, the server (2000) can identify entity names related to unique characteristics of the words.

[0079] According to one embodiment, when the sentence analysis range of the target keyword is identified as the first sentence analysis range, the server (2000) can identify entity names for words included in a sentence including the target keyword. For example, when the sentence analysis range of the target keyword is identified as the first sentence analysis range, the server (2000) determines that one target meaning among candidate meanings can be identified only through sentences including the target keyword, and can identify entity names for words included in a sentence including the target keyword.

[0080] According to another embodiment, when the sentence analysis range of the target keyword is identified as the second sentence analysis range, the server (2000) can identify entity names for words included in a sentence including the target keyword and in the preceding and following adjacent sentences. For example, when the sentence analysis range of the target keyword is identified as the second sentence analysis range, the server (2000) determines that one of the candidate meanings can be identified through a sentence including the target keyword and the preceding and following sentences adjacent to the sentence, and can identify entity names for words included in a sentence including the target keyword and in the preceding and following adjacent sentences.

[0081] According to another embodiment, if the sentence analysis range of the target keyword is identified as the third sentence analysis range, the server (2000) can identify entity names for words included in a paragraph including the target keyword. For example, if the sentence analysis range of the target keyword is identified as the third sentence analysis range, the server (2000) determines that one of the candidate meanings can be identified through the paragraph including the target keyword, and can identify entity names for words included in the paragraph including the target keyword.

[0082] In S680, the server (2000) can determine the target meaning based on the dependency relationship determined according to the syntax tree representing the dominant-subordinate relationship between the identified entity names. For example, the server (2000) can accurately determine the target meaning by determining the sentence-by-sentence syntax tree for the identified entity names and determining the dominant-subordinate relationship between each entity name of the determined syntax tree. According to another embodiment, the target meaning can also be determined based on the relationship between words through the determination of the semantic role for the core keyword.

[0083] Although not illustrated in FIG. 6, if the minimum value of at least one identified distance is identified as being less than or equal to a preset minimum distance, the server (2000) may determine a paragraph analysis range regarding a search range of paragraphs adjacent to a paragraph including the core keyword, identify sentence similarity regarding structural similarity between sentences included in at least one paragraph determined based on the determined paragraph analysis range and sentences including the core keyword, and modify and update the identified sentence analysis range based on the identified sentence similarity. For example, if the minimum value of the distance is identified as being less than or equal to the preset minimum distance, the server (2000) may determine that a search of other adjacent paragraphs is necessary to determine the target meaning of the core keyword, and may additionally identify entity names for sentences having a similar structure to a sentence including the core keyword among the other adjacent paragraphs. For example, the sentence similarity may be determined by an arrangement of morpheme keywords determined based on morpheme analysis of the sentence.

[0084] The server (2000) according to the present disclosure determines the analysis scope of text for determining the target meaning in units of sentences and paragraphs according to the process described above in FIG. 6, thereby not only accurately identifying the target meaning but also efficiently using data required for the calculation, thereby improving the calculation speed.

[0085]

[0086] FIG. 7 is a diagram for explaining an operation of a server (2000) according to one embodiment of the present invention to match different video resources for paragraphs having the same target meaning.

[0087] According to one embodiment, the server (2000) identifies whether there is a paragraph having the same target meaning as the core keyword, and if it is identified that there is a paragraph having the same target meaning, determines at least one sub-keyword for each paragraph including the core keyword, and based on the core keyword and the at least one sub-keyword, matches different video resources for paragraphs having the same core keyword.

[0088] For example, the server (2000) can identify that the target meanings of 'wind (712)', which is determined as a core keyword in paragraph A (710), and 'wind (722)', which is determined as a core keyword in paragraph B (720), both represent 'movement of air'. In addition, for example, if the server (2000) identifies two or more paragraphs that include core keywords representing the same target meaning in the text, the server (2000) can determine at least one of 'weather (714)' or 'a lot (716)', which are dependent on 'wind (712)' in paragraph A (710), as a sub-keyword, and can determine at least one of 'fresh (724)' or 'calmly (726)', which are dependent on 'wind (722)', as a sub-keyword in paragraph B. Additionally, the server (2000) can match the image resource (732) from the labeled database (730) for each paragraph based on the determined target meaning and sub-keywords.

[0089] According to one embodiment, the video resource (732) may be tagged with element information (734) including at least one of background information, object information, or object action information for each section of the video resource, and labeling information including story information (736) regarding the story of the video generated based on the element information.

[0090] Although not shown in FIG. 7, according to one embodiment, the server (2000) may identify morpheme keywords that are dependent on the core keyword, generate a syntax tree including a plurality of hierarchical structures between the core keyword and the identified morpheme keywords, and, based on the generated syntax tree, determine at least one sub-keyword among morpheme keywords in a hierarchy adjacent to the core keyword, based on a priority determined according to the number of times they are used within the paragraph. For example, if a predetermined number or more of morpheme keywords that are dependent on the core keyword are identified, the server (2000) may generate a syntax tree for determining some of the morpheme keywords as sub-keywords. In addition, for example, the server (2000) may determine sub-keywords in the order of the number of times they are used within the paragraph among morpheme keywords that are most adjacent in a hierarchy between the core keyword and the morpheme keywords in the generated syntax tree.

[0091] According to one embodiment, when a first video resource tagged with labeling information including both the target meaning and the sub-keyword is identified, the server (2000) may match the identified first video resource to a paragraph including the core keyword based on the target meaning and the sub-keyword. For example, when the server (2000) identifies that a first video resource labeled with the core keyword 'wind (712)' and the sub-keywords 'weather (714)' and 'a lot (716)' of paragraph A (710) exists in the database (730), the server (2000) may match the first video resource to paragraph A (710).

[0092] According to another embodiment, when the first image resource is not identified, the server (2000) identifies a second image resource tagged with labeling information including the target meaning, and when a third image resource tagged with labeling information further including a synonym of the sub-keyword is identified among the second image resources, the server (2000) can match the identified third image resource to a paragraph including the core keyword based on the target meaning and the synonym of the sub-keyword.

[0093] For example, if the server (2000) identifies that the first video resource labeled with both the core keyword 'wind (722)' and the sub-keyword 'fresh (724)' of paragraph B (720) does not exist in the database (730), the server (2000) may first identify the second video resource labeled with only the core keyword 'wind (722)', identify the third video resource labeled with 'refreshing, clean', which is a synonym of 'fresh (724)' among the second video resources, and match the third video resource identified in paragraph B (720).

[0094] The server (2000) according to the present disclosure matches video resources by utilizing sub-keywords that are dependent on each other, even if the core keywords for each paragraph are the same, according to the process described above in FIG. 7, so that the same video resources for each paragraph are not matched, and multimedia content that is more similar to the context of the text can be provided to the user.

[0095]

[0096] Figure 8 is a block diagram of an electronic device according to one embodiment.

[0097] As illustrated in FIG. 8, an electronic device (1000) according to an embodiment may include a display (1210), a first processor (1300), a first network interface (1500), and a memory (1700). However, not all of the illustrated components are essential components. The electronic device (1000) may be implemented with more components than the illustrated components, or may be implemented with fewer components.

[0098] The display (1210) includes a screen for displaying and outputting information processed in the electronic device (1000). In addition, the screen can output text acquired by the electronic device (1000) or a rendered image generated by matching a partial video to text.

[0099] The first processor (1300) typically controls the overall operation of the electronic device (1000). For example, the first processor (1300) can control the overall operation of the electronic device (1000) including the first network interface (1500) by executing programs stored in the memory (1700). In addition, the first processor (1300) can perform the functions of the electronic device (1000) described in FIGS. 1 to 7 by executing programs stored in the memory (1700).

[0100] According to one embodiment, the first processor (1300) may be provided in multiple units. For example, the plurality of first processors (1300) may execute one or more instructions to obtain text including at least one paragraph, and determine core keywords for each paragraph based on morphological analysis of the obtained text.

[0101] The first network interface (1500) may include one or more components that enable the electronic device (1000) to communicate with other devices (not shown) and a server (2000). The other devices (not shown) may be computing devices such as the electronic device (1000) or sensing devices, but are not limited thereto. For example, the first network interface (1500) may include a wireless communication interface (1510), a wired communication interface (1520), and a mobile communication unit (1530). The wireless communication interface (1510) may include, but is not limited to, a short-range wireless communication unit, a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an infrared (IrDA, infrared Data Association) communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultra wideband) communication unit, etc.

[0102] The memory (1700) can store a program for processing and controlling the first processor (1300), and can also store data input to or output from the electronic device (1000). In addition, the memory (1700) can store a text summary model, a morphological analysis model, a word-to-vector model, an artificial intelligence model for determining core keywords, or an artificial intelligence model for generating a rendered image through text-based image resource matching used by the electronic device (1000).

[0103] For example, the memory (1700) may store weight values ​​regarding layers, nodes, and connection strengths of at least one neural network model. Furthermore, the electronic device (1000) may further store learning data generated by the electronic device (1000) to train the neural network model. Furthermore, the memory (1700) may further store information regarding the operating environments of cameras or servers connected to the electronic device.

[0104] The memory (1700) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0105]

[0106] Figure 9 is a block diagram of a server according to one embodiment.

[0107] According to one embodiment, the server (2000) may include a second network interface (2100), a database (2200), and a second processor (2300). The configuration of the server (2000) illustrated in FIG. 9 may correspond to the configuration of the electronic device described in FIG. 8. According to one embodiment, the second network interface (2100) may obtain prompt or text information from the electronic device (1000) and transmit image information generated based on the obtained prompt or text information to the electronic device.

[0108] In another embodiment, the second network interface (2100) can transmit and receive information about artificial intelligence models and neural network models learned by the server to and from an electronic device. In another example, the second network interface (2100) can also transmit and receive partial video information matching text or prompts, and tag information matching the partial video information.

[0109] Additionally, according to one embodiment, the database (2200) may correspond to the memory described above in FIG. 8. For example, the database (2200) may store text information obtained from the electronic device (1000), images that can match the text information, dictionary data for identifying polysemous or homonymous words, or image information.

[0110] According to one embodiment, the second processor (2300) can control the overall operation of the server (2000). For example, the second processor (2300) can control the second network interface (2100) and the database (2200), thereby performing all or part of the method for generating multimedia content based on text analysis performed by the electronic device (1000) described in FIGS. 1 to 7.

[0111] According to one embodiment, the following operations may be performed to accurately match a video resource to a text obtained by a user input of the server (2000). For example, the second processor (2300) may obtain a text (prompt) and a core keyword for each paragraph of the text from an electronic device, and may match the video resource to the prompt based on a target meaning determined based on whether the core keyword for each paragraph included in the text is a polysemous word or a homonym, or at least one sub-keyword determined based on whether two or more paragraphs including a core keyword representing the same target meaning in the prompt are identified, thereby generating multimedia content.

[0112] According to one embodiment, the second processor (2300) may generate multimedia content, characterized in that it summarizes the prompt based on a user input requesting the prompt summary obtained from the electronic device or attention determined based on the length of the prompt, divides the summarized prompt into paragraphs, and determines a core keyword for each divided paragraph. For example, the server (2000) may perform prompt summary using a sequence model that indicates the importance of each text element in the prompt with a predetermined weight according to the attention weight and outputs a summary text when a context vector according to the attention weight and text input is input.

[0113] According to one embodiment, the second processor (2300) determines a first paragraph division point based on the number of core keywords per paragraph, determines a keyword network regarding at least one of the frequencies and connection relationships of the core keywords included in first divided paragraphs divided according to the first paragraph division point, determines a similarity of the keyword network for adjacent paragraphs among the first divided paragraphs, and merges adjacent paragraphs whose similarity of the keyword network is equal to or greater than a threshold similarity, thereby determining a second paragraph division point within the text, and may divide the summarized prompt based on the second paragraph divisions determined based on the second paragraph division point.

[0114] For example, the server (2000) may improve the accuracy of image resource matching by optimizing paragraph segmentation based on the similarity between keyword networks by applying a similarity determination technique based on at least one of cosine similarity, Jaccard similarity, and graph similarity between keyword networks regarding the frequency and connection relationship of keywords per paragraph. According to one embodiment, if the core keyword per paragraph is not a polysemous or homonymous word, the second processor (2300) may determine a single meaning corresponding one-to-one to the core keyword as the target meaning of the core keyword.

[0115] According to one embodiment, when the core keyword for each paragraph is identified as a polysemous word or a homonym, the second processor (2300) determines the core keyword for each paragraph as a target keyword, converts a plurality of meanings related to the determined target keyword into vectors of a predetermined dimension, and determines a sentence analysis range in which sentences adjacent to a sentence including the target keyword, among the sentences to be analyzed to identify the target meaning, are separated from the sentence including the target keyword based on the distance between the converted vectors, identifies a noun for words included in at least one sentence determined based on the sentence analysis range, and determines the target meaning based on a dependency relationship determined according to a syntax tree representing a dominant-subordinate relationship between the identified nouns.

[0116] According to one embodiment, the target meaning may indicate one meaning identified as the most appropriate meaning among a plurality of candidate meanings included in polysemy or homonyms. According to one embodiment, the second processor (2300) may identify whether there is a paragraph having the same target meaning as the core keyword determined for each paragraph in the prompt, and if it is identified that there is a paragraph having the same target meaning, determine at least one sub-keyword for each paragraph including the core keyword, and, based on the target meaning determined for the core keyword for each paragraph and the at least one sub-keyword, match image resources labeled with labeling information including both the target meaning and the sub-keyword for each paragraph, thereby generating the multimedia content.

[0117]

[0118] FIG. 10 is a diagram for explaining a process of generating multimedia content by interworking between an electronic device and a server according to one embodiment.

[0119] In S1101, the server (2000) can pre-build and train an artificial intelligence-based image generation model. For example, when a core keyword for each paragraph of text is input, the server (2000) can pre-train an artificial intelligence-based image generation model that generates an image by matching image resources for each paragraph based on the input core keyword. In addition, when a prompt is input, the server (2000) can pre-train an artificial intelligence-based image generation model that generates an image by dividing the input prompt into predetermined paragraphs and matching image resources for each divided paragraph.

[0120] In step S1102, the electronic device (1000) may acquire text including at least one paragraph from a user input or an external device. In step S1104, the electronic device (1000) may determine core keywords for each paragraph based on a summary and morphological analysis of the acquired text. In step S1106, the electronic device (1000) may transmit the determined core keywords for each paragraph to the server (2000).

[0121] In step S1108, the server (2000) can identify whether the core keyword for each paragraph acquired from the electronic device is a polysemy or a homonym. In step S1110, the server (2000) can determine the target meaning based on whether the core keyword is a polysemy or a homonym. In step S1112, the server (2000) can perform paragraph-by-paragraph video resource matching based on the determined target meaning. In step S1114, the server (2000) can generate multimedia content by synthesizing the matched video resources. In step S1116, the server (2000) can transmit the generated multimedia content to the electronic device (1000). In step S1118, the electronic device (1000) can output the multimedia content transmitted from the server.

[0122]

[0123] The method for generating multimedia content based on text analysis according to the present disclosure may be implemented in the form of program commands that can be executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, and the like, either singly or in combination. The program commands recorded on the medium may be those specifically designed and constructed for the present invention, or may be known and usable by those skilled in the art of computer software.

[0124] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concepts of the present invention defined in the following claims also fall within the scope of the present invention.

Claims

1. A method for generating multimedia content based on text analysis by a server connected to an electronic device providing a multimedia content generation service, A step of obtaining a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device; A step of determining the target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemy or a homonym; A step of matching video resources for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the target meaning and the core keyword, depending on whether two or more paragraphs containing core keywords representing the same target meaning are identified in the text; and A step of generating multimedia content by synthesizing video resources matched to each paragraph above; A method comprising: a step of transmitting the generated multimedia content to the electronic device; 2. In the first paragraph, the core keyword is, by the electronic device, A method wherein the summary text is generated by dividing the generated summary text into paragraphs based on user input requesting a text summary for the electronic device or attention determined based on the length of the text, and the generated summary text is determined by the generated divided paragraphs.

3. In the second paragraph, the divided paragraphs are, by the electronic device, The first paragraph division point in the summary text is determined based on the number of keywords in the text, A keyword network is determined regarding at least one of the frequency and connection relationship of the keywords included in the first divided paragraphs divided according to the first paragraph division point, The similarity of the keyword network for adjacent paragraphs among the above first divided paragraphs is determined, When the second paragraph division point within the text is determined by merging adjacent paragraphs whose similarity in the keyword network is greater than the threshold similarity, A method characterized in that it corresponds to second divided paragraphs determined based on the second paragraph division point.

4. In the second paragraph, the core keyword for each paragraph is, by the electronic device, After performing morphological analysis on the above summarized text, multiple morphological keywords are obtained. A method characterized in that it is determined based on the frequency of use of the acquired plurality of morpheme keywords and the correlation between the morpheme keywords.

5. In the first paragraph, the step of determining the target meaning is: A method comprising: a step of determining a single meaning corresponding one-to-one to the core keyword as the target meaning of the core keyword, if the core keyword for each paragraph above is not a polysemous or homonymous word; 6. In the first paragraph, the step of determining the target meaning is: A step of determining the core keyword for each paragraph as a target keyword when the core keyword for each paragraph is identified as a polysemous word or a homonym; A step of converting a plurality of meanings related to the above-determined target keyword into vectors of a predetermined dimension; A step of determining a sentence analysis range in which sentences adjacent to a sentence containing the target keyword are separated from a sentence containing the target keyword to identify the target meaning, based on the distance between the transformed vectors; A step of identifying entity names for words included in at least one sentence determined based on the determined sentence analysis range; and A method comprising: determining the target meaning based on a dependency relationship determined according to a syntax tree representing a dominant-subordinate relationship between the identified entity names; 7. In paragraph 6, the step of determining the sentence analysis scope is: A step of identifying at least one distance between the transformed vectors; A step of determining the sentence analysis range as a first sentence analysis range when the minimum value of at least one distance identified above is greater than or equal to a preset threshold distance; A step of identifying the number of distances between vectors that are less than the threshold distance, if the minimum value of at least one distance identified above is less than the threshold distance; If the number of distances between the identified vectors is less than a preset threshold number, determining the sentence analysis range as a second sentence analysis range; and A method comprising: a step of determining the sentence analysis range as a third sentence analysis range when the number of distances between the identified vectors is greater than or equal to a preset threshold number; 8. In the 7th paragraph, the method, after the step of determining the sentence analysis range, A step of determining a paragraph analysis scope regarding a search scope of paragraphs adjacent to a paragraph containing the core keyword, when the minimum value of at least one distance identified above is identified as being less than or equal to a preset minimum distance; A step of identifying sentence similarity regarding structural similarity between sentences included in at least one paragraph determined based on the determined paragraph analysis scope and sentences including the core keyword; and A method further comprising: a step of modifying and updating the identified sentence analysis range based on the identified sentence similarity; 9. In paragraph 8, the step of identifying the entity name is: If the sentence analysis range of the target keyword is identified as the first sentence analysis range, identify the entity names for words included in the sentence including the target keyword, If the sentence analysis range of the target keyword is identified as the second sentence analysis range, identify the entity names for the words included in the sentence containing the target keyword and the adjacent sentences before and after it, A method characterized in that, when the sentence analysis range of the target keyword is identified as a third sentence analysis range, the entity name for words included in a paragraph including the target keyword is identified.

10. In the first paragraph, the step of matching video resources by paragraph is: A step of identifying whether there is a paragraph having the same target meaning as the core keyword; If it is identified that there is a paragraph having the same target meaning as the above, a step of determining at least one sub-keyword for each paragraph containing the above core keyword; and A method comprising: a step of matching different video resources for paragraphs having the same core keyword based on the core keyword and at least one sub-keyword; 11. In the 10th paragraph, the step of determining the sub-keyword is: A step of identifying morpheme keywords that are dependent on the above core keywords; A step of generating a syntax tree including a plurality of hierarchical structures between the core keyword and the identified morpheme keywords; A method comprising: a step of determining at least one sub-keyword based on a priority determined according to the number of times it is used in the paragraph among morpheme keywords in a layer adjacent to the core keyword based on the generated syntax tree; 12. In the 11th paragraph, the step of matching the different video resources is: When a first video resource tagged with labeling information including both the target meaning and the sub-keyword is identified, a step of matching the identified first video resource to a paragraph including the core keyword based on the target meaning and the sub-keyword; If the first image resource is not identified, a step of identifying a second image resource tagged with labeling information including the target meaning; and A method comprising: a step of matching the identified third video resource to a paragraph including the core keyword based on the target meaning and the synonym of the sub-keyword, when a third video resource tagged with labeling information including a synonym of the sub-keyword is identified among the second video resources; 13. In paragraph 12, the video resource A method characterized in that labeling information is tagged with element information including at least one of background information, object information, or object behavior information for each section of the video resource, and story information regarding the story of the video generated based on the element information.

14. In a server that generates multimedia content based on text analysis, network interface; A database that stores one or more instructions; At least one processor performing one or more of the above instructions; At least one processor above Obtaining paragraph-specific core keywords determined based on morphological analysis of a text containing at least one paragraph from an electronic device, Based on whether the core keyword for each paragraph above is a polysemy or homonym, the target meaning for the core keyword for each paragraph above is determined, Depending on whether two or more paragraphs containing core keywords representing the same target meaning are identified in the text, video resources are matched for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the core keyword, Multimedia content is created by synthesizing video resources matched to each paragraph above, A server that transmits the generated multimedia content to the electronic device.

15. A method for generating multimedia content based on text analysis by a server connected to an electronic device providing a multimedia content generation service, A step of obtaining a core keyword for each paragraph determined based on morphological analysis of a text including at least one paragraph from the electronic device; A step of determining the target meaning for the core keyword for each paragraph based on whether the core keyword for each paragraph is a polysemy or a homonym; A step of matching video resources for each paragraph based on the target meaning or at least one sub-keyword that is dependent on the target meaning and the target meaning and the core keyword, depending on whether two or more paragraphs containing core keywords representing the same target meaning are identified in the text; and A step of generating multimedia content by synthesizing video resources matched to each paragraph above; A computer-readable recording medium storing a program for performing a method, including a step of transmitting the generated multimedia content to the electronic device.

Images