System and Method for Generating Video Based on Cut-by-Cut Direction Control Using Webtoons

Abstract

The present invention includes a video generation system based on cut-by-cut direction control utilizing a webtoon, comprising: a cut extraction unit for extracting two or more individual cuts included in webtoon data; an analysis unit for analyzing the situation of the individual cuts extracted by the cut extraction unit based on text corresponding to the individual cuts; a prompt generation unit for automatically generating a prompt for video generation according to the situation of the individual cuts analyzed by the analysis unit; a video generation unit for generating an individual video corresponding to the individual cuts according to the prompt for video generation; and a video integration unit for integrating two or more individual videos generated from the video generation unit to produce a webtoon video corresponding to the webtoon data.

Description

Technology Field

[0001] The present invention relates to a video generation system and method based on cut-by-cut direction control utilizing webtoons. Background Technology

[0003] Webtoons distributed on digital platforms have high utility as story IPs, leading to a continuously increasing demand for conversion into video content, such as advertising and promotional shorts. However, conventional webtoon video adaptations tend to focus on producing short shorts of 20 to 60 seconds rather than realizing the entire episode as a complete narrative flow. Consequently, there are limitations in producing video units of 5 minutes or longer that maintain the context of the story.

[0004] In addition, short-form webtoon video production often relies heavily on manual work to ensure the quality of the final product, which repeatedly leads to prolonged production periods or excessive production costs. For example, it has been reported that the production cost per minute exceeds 2 million won or the average production period reaches 2 to 3 weeks, causing difficulties in mass production or simultaneous multi-production.

[0005] Meanwhile, technology has been proposed to provide webtoon-based digital comic content as a 3D viewer to enhance realism compared to 2D, or to support the author in setting animations through effect buttons without the assistance of a programmer. Additionally, technology is proposed to provide digital comic content that allows for user direction by storing camera work data generated by the user adjusting rotation angles, etc., into frames (see Patent Document 1).

[0006] However, there is still room for improvement in these conventional technologies regarding the efficient generation and production of "video results that are naturally connected in episode units rather than shorts," which is a key demand for webtoon adaptation, as well as in structurally alleviating the burden of manual work while consistently reflecting cut-level directing requirements throughout the entire production process.

[0007] Furthermore, there are growing concerns that core IPs, such as webtoon manuscripts, unique characters, and unreleased setting materials, could be transmitted to external networks or used for unauthorized learning in external environments, and such security risks act as a limiting factor in the adoption of webtoon adaptation into video content.

[0008] Therefore, in video generation using webtoons, there is a need for new technical means that can alleviate the problems of high costs and long production periods caused by the shorts-centered and manual-centered production structure, satisfy both the requirements for episode-based video production and cut-by-cut direction, and simultaneously reduce concerns about IP leakage and unauthorized learning. Prior art literature

[0010] Republic of Korea Registered Patent 10-1839018 Republic of Korea Published Patent 10-2025-0143582 The problem to be solved

[0011] The objective of the present invention is to provide a system and method for extracting two or more individual cuts included in webtoon data, analyzing the situation of the individual cuts based on text corresponding to the individual cuts, automatically generating a prompt for video generation according to the analyzed situation of the individual cuts, generating an individual video corresponding to the individual cuts according to the prompt for video generation, and integrating the two or more generated individual videos to produce a webtoon video corresponding to the webtoon data.

[0012] In addition, the objective of the present invention is to provide a system and method that ensures consistency between the content of each cut and the video result by automatically converting the analysis result of the individual cut situation into a prompt for video generation and reflecting it in the generation of individual videos.

[0013] In addition, the objective of the present invention is to provide a system and method that reduces the burden of repetitive manual work, such as creating prompts and producing and combining clips by cut by a worker, in the production of webtoon-based shorts. means of solving the problem

[0015] The present invention includes a video generation system based on cut-by-cut direction control utilizing a webtoon, comprising: a cut extraction unit for extracting two or more individual cuts included in webtoon data; an analysis unit for analyzing the situation of the individual cuts extracted by the cut extraction unit based on text corresponding to the individual cuts; a prompt generation unit for automatically generating a prompt for video generation according to the situation of the individual cuts analyzed by the analysis unit; a video generation unit for generating an individual video corresponding to the individual cuts according to the prompt for video generation; and a video integration unit for integrating two or more individual videos generated from the video generation unit to produce a webtoon video corresponding to the webtoon data.

[0016] Additionally, the webtoon data includes one or more image data arranged in a scrolling manner, and the cut extraction unit includes a video generation system based on cut-by-cut direction control utilizing a webtoon, which detects cut boundaries in the image data and extracts the individual cuts.

[0017] Additionally, the analysis unit analyzes the individual cut situation, which includes at least one of the emotion, action, and background of the individual cut, based on the text, and the prompt generation unit generates a video generation prompt that includes instruction information for directing the individual video based on the individual cut situation, thereby including a video generation system based on cut-by-cut direction control utilizing a webtoon.

[0018] Additionally, it further includes a preprocessing unit that restores the background after removing the text area corresponding to the speech bubble or subtitle included in the individual cut; and the image generation unit includes an image generation system based on cut-by-cut direction control utilizing a webtoon, which generates the individual image by expanding the outer area of ​​the individual cut when the individual cut does not conform to a preset image ratio.

[0019] Additionally, it further includes a shot manager that sets direction parameters for each individual cut according to user input; and the video generation unit includes a video generation system based on cut-by-cut direction control utilizing a webtoon that generates the individual video by reflecting the direction parameters in the video generation prompt.

[0020] In addition, the video generation unit includes a video generation system based on cut-by-cut direction control utilizing a webtoon, which generates the individual video with movement assigned based on the feature points of a person or object in the individual cut.

[0021] Additionally, the system includes a video generation system based on cut-by-cut direction control utilizing a webtoon, further comprising a post-processing integrated unit configured to produce the webtoon video after a user performs modification work on the individual video, and to apply sound effects and BGM to the webtoon video or perform upscaling.

[0022] In addition, the above system includes a video generation system based on cut-by-cut direction control utilizing webtoons, configured to operate in a closed network-based on-premise environment.

[0023] A method for generating video based on cut-by-cut direction control using a webtoon according to an embodiment of the present invention comprises: a cut extraction step in which a cut extraction unit extracts two or more individual cuts included in webtoon data; an analysis step in which an analysis unit analyzes the situation regarding the individual cuts extracted by the cut extraction unit based on text corresponding to the individual cuts; a prompt generation step in which a prompt generation unit automatically generates a prompt for video generation according to the situation of the individual cuts analyzed by the analysis unit; a video generation step in which a video generation unit generates an individual video corresponding to the individual cuts according to the prompt for video generation; and a video integration step in which a video integration unit integrates two or more individual videos generated from the video generation unit to produce a webtoon video corresponding to the webtoon data. Effects of the invention

[0025] The effect of the present invention is that the cut extraction unit extracts two or more individual cuts from webtoon data, thereby automatically defining the processing unit of the cut regardless of the scroll form or image arrangement form of the webtoon, which can reduce the burden on the operator of manually separating cuts or manually specifying the work range per cut. In addition, the effect of the present invention is that the analysis unit analyzes the situation of individual cuts based on text corresponding to individual cuts, thereby reducing the deviation in which a video producer arbitrarily interprets the situation and decides the direction, and the events, emotions, actions, or scene development indicated by the text are reflected in the video generation process, thereby improving the consistency between the content of each cut and the video result.

[0026] In addition, the effect of the present invention is that the prompt generation unit automatically generates a video generation prompt according to the analyzed individual cut situation, thereby reducing the work of manually creating or repeatedly modifying cut-specific prompts, standardizing the prompt generation process, and improving the productivity of repetitive production for multiple cuts.

[0027] In addition, the effect of the present invention is that the image generation unit generates individual images corresponding to individual cuts according to the image generation prompt, thereby automatically converting still image-based webtoon cuts into cut-unit images, so that the generation of cut-by-cut clips required in shorts production can be automated, and accordingly, the manpower and time required for the production of cut-by-cut clips can be reduced.

[0028] In addition, the effect of the present invention is that the video integration unit integrates two or more individual videos to produce a webtoon video corresponding to the webtoon data, so that individual videos generated for each cut are combined into a single result and provided, thereby reducing the work of sequentially arranging or manually combining clips for each cut in a separate editing tool, and the video result can be provided in a form corresponding to the cut flow of the webtoon.

[0029] In addition, the effect of the present invention is that individual cut situation analysis, automatic prompt generation, individual video generation, and integrated production are performed as a single linked processing flow, thereby enabling the output to be produced based on consistent standards even when repeated generation is required for the same webtoon data, and facilitating parallel processing for multiple works or multiple episodes, thus providing an automation foundation suitable for mass production.

[0030] In addition, since the situation for each cut is distinguished and reflected based on text in the present invention, the likelihood of the narrative progression between cuts being naturally reflected in the video result increases, thereby improving the story delivery power conveyed to the viewer compared to a simple arbitrary motion assignment method.

[0031] In addition, the effect of the present invention is that since the text-based situation analysis results are automatically generated as a prompt and reflected in the generation of individual videos, variations in the quality of the output due to the creator's proficiency are mitigated, and video quality above a certain level can be repeatedly provided.

[0032] In addition, the effect of the present invention is that since webtoon data is automatically processed on a cut-by-cut basis and a final integrated video is produced, the processing unit can be expanded not only to short production but also to content with a large number of cuts, thereby expanding the scope of application for webtoon-based video production. Brief explanation of the drawing

[0034] FIG. 1 is a configuration diagram showing a video generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention. FIG. 2 is an example diagram showing the process of a pre-analysis unit of a video generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention analyzing webtoon data. FIG. 3 is an example diagram showing the process of a cut extraction unit of a video generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention extracting two or more individual cuts included in webtoon data. FIG. 4 is an example diagram showing the process of removing a text area corresponding to a speech bubble or subtitle included in an individual cut by a preprocessing unit of a video generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention. FIG. 5 is an example diagram showing the process of expanding and generating the outer area of ​​an individual cut when the individual cut does not correspond to a preset video ratio, in an image generation unit of an image generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention. FIG. 6 is an example diagram illustrating a process in which an image generation unit of an image generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention generates an individual image corresponding to an individual cut according to a prompt for image generation, and a post-processing integration unit processes the individual image so that the individual image reflecting the modification work is confirmed as an integration target after the user's modification work is performed on the individual image. FIG. 7 is an example diagram illustrating the process in which an image integration unit of an image generation system based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention integrates two or more individual images generated from an image generation unit to produce a webtoon image corresponding to webtoon data, and a post-processing integration unit applies sound effects and BGM to the produced webtoon image and performs upscaling to improve output quality. FIG. 8 is a flowchart illustrating a method for generating video based on cut-by-cut direction control using a webtoon according to a preferred embodiment of the present invention. Specific details for implementing the invention

[0035] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0036] Hereinafter, the present invention will be described with reference to the drawings for explaining a video generation system based on cut-by-cut direction control utilizing webtoons according to embodiments of the present invention.

[0037] First, the webtoon data described in the present invention can secure a processing state for the webtoon data by selecting a target work to be converted into video and verifying the scope of permission for use and copyright-related conditions for the selected work.

[0038] In this way, webtoon videos can be produced using webtoon data for which copyright agreements have been reached.

[0039] Referring to FIG. 1, the video generation system based on cut-by-cut direction control using a webtoon according to the present invention includes a pre-analysis unit (40), a cut extraction unit (20), an analysis unit (40), a prompt generation unit (50), a video generation unit (60), and a video integration unit (70).

[0040] First, the analysis of the work concept and story can be performed by the preliminary analysis unit (40) before the analysis of individual cut units.

[0041] Here, referring to FIG. 2, the preliminary analysis unit (40) can be performed by aggregating text included in all episodes or selected sections of the webtoon data to derive main characters, the flow of events, worldview clues, atmosphere, and genre characteristics.

[0042] Additionally, the preliminary analysis unit (40) can be performed to estimate the overall directorial tone of the work based on recurring expressions, the tone of the dialogue, the perspective of the narrative, and the frequency of scene transitions.

[0043] In addition, the results of the preliminary analysis unit (40) are transmitted to the prompt generation unit (50), so that the prompt for video generation generated according to the individual cut situation can be reflected to be consistent with the concept and story flow of the entire work.

[0044] For example, if the result of the preliminary analysis unit (40) indicates that a particular work has a tension-centered development and short cut transitions, the prompt generation unit (50) can process the prompt for each individual cut to include an atmosphere that heightens tension, a duration suitable for fast cut transitions, and an expression corresponding to lighting with strong contrast.

[0045] Referring to FIG. 3, the cut extraction unit (20) extracts two or more individual cuts included in the webtoon data.

[0046] Here, webtoon data may include one or more image data arranged in a scrolling manner.

[0047] In addition, the image data may be a long-axis image in which the content of one episode of a webtoon is arranged continuously in a vertical direction, and even if multiple images are stored sequentially, they may be provided in a combined form according to the overall scroll order.

[0048] At this time, the cut extraction unit (20) normalizes the input image data to a processable resolution and extracts individual cuts by detecting cut boundaries based on the vertical direction, and the cut boundaries can be defined by reflecting the margin area formed between adjacent individual cuts, the partition line, and the discontinuity of the background pattern together.

[0049] More specifically, the cut extraction unit (20) calculates a feature quantity for each window section while moving the scan window in the vertical direction of the image data, and determines whether the feature quantity satisfies the properties of the margin area and the partition line, thereby setting it as a cut boundary candidate. At this time, the feature quantity can be calculated to include all of the following: whether it is a low-density section based on pixel density and brightness distribution, the scarcity of contour components based on edge (contour) intensity, and whether the possibility of being inside the cut is excluded based on the existence of a text area.

[0050] When multiple cut boundary candidates are detected, the cut extraction unit (20) determines the cut boundary by reflecting the continuous length (vertical width) of the candidate section, the amount of change in the image pattern before and after the candidate section, and the distribution of text areas around the candidate section, and automatically extracts two or more individual cuts by dividing the image data by defining the upper coordinates and lower coordinates of each individual cut based on the determined cut boundary.

[0051] Additionally, the cut extraction unit (20) can be configured to determine the cut boundary by reflecting the partition line detection result and the discontinuous point detection result of the background pattern together in the cut boundary candidate in preparation for cases where cut boundary is not detected or webtoon data with extremely limited margin is input, thereby enabling individual cut extraction even in works where margin-based detection is difficult.

[0052] The cut extraction unit (20) can be configured to allow the same individual cut to be reproduced in subsequent processing by assigning a cut index that reflects the webtoon progression order to each extracted individual cut, and by saving the individual cut image itself as a separate file or saving the coordinate range (top / bottom and left / right boundaries) within the original image data as metadata.

[0053] Meanwhile, webtoon data includes a PSD file, and the PSD file may be in a form where characters, backgrounds, speech bubbles, effects, etc. are separated into layers or layer groups.

[0054] When a PSD file is provided, the cut extraction unit (20) can first check area information divided into cut units in the layer / layer group or mask (including clipping / cut mask) of the PSD.

[0055] The cut extraction unit (20) can determine the range of an individual cut in the first step based on the area information when area information divided into cut units exists, generate scroll-type image data by compositing and rendering a PSD, perform the same cut boundary detection on the generated image data to determine the boundary of the individual cut in the second step, and determine the final range of the individual cut by comparing the cut unit area information determined in the first step with the cut boundary determined in the second step, and if there is a discrepancy area, correct the first area information based on the second cut boundary.

[0056] If cut unit area information does not exist or is insufficient, the cut extraction unit (20) may be configured to generate scroll-type image data by compositing and rendering the PSD, and to determine the range of individual cuts by performing cut boundary detection on the generated image data.

[0057] Thus, individual cut extraction can be performed using a processing pipeline where PSD-based area information and image-based cut boundary detection are applied simultaneously.

[0058] For example, when a webtoon episode 1 is provided as a single vertical long-axis image, the cut extraction unit (20) detects a candidate section corresponding to a margin area or a partition line by scanning in the vertical direction, determines whether the continuous length of the candidate section is within a range that can be recognized as a cut boundary and whether the image patterns before and after the candidate section are different from each other to determine the cut boundary, and then divides the image based on the determined cut boundary to sequentially generate each individual cut.

[0059] In addition, when multiple cut boundary candidates are detected because the margin length is short or the background is similar, the cut extraction unit (20) can determine the cut boundary by first selecting a section among the candidate sections where the contour component is low and the text area is relatively small as the cut boundary, or by first selecting a section with a large amount of pattern change before and after the candidate section, thereby reducing over-segmentation or under-segmentation.

[0060] In addition, even when the first episode of the webtoon is provided by being divided and stored as multiple images, the cut extraction unit (20) can extract individual cuts by detecting cut boundaries based on the combined form of images according to the scroll order, or by detecting cut boundaries for each image and then arranging them in the overall order to output individual cuts, thereby providing individual cuts in the same result form.

[0061] Accordingly, the webtoon data includes one or more image data arranged in a scrolling manner and further includes a PSD file, and the cut extraction unit (20) determines whether there is cut unit area information defined by a layer or mask from the PSD file, and if there is cut unit area information, extracts individual cuts based on said cut unit area information, and if there is no or insufficient cut unit area information, generates the image data by synthesizing the PSD and then detects cut boundary candidates by scanning the image data in the vertical direction. In addition, the cut extraction unit (20) determines whether the cut boundary candidate corresponds to a low-density section based on pixel density and brightness distribution, the sparseness of contour components based on edge intensity, and the existence of a text area, thereby determining the cut boundary, and the cut boundary is determined based on the margin area, partition line, and discontinuity of background pattern formed between adjacent individual cuts, and the cut extraction unit (20) can extract two or more individual cuts by correcting the determined cut boundary with said cut unit area information.

[0062] The analysis unit (40) can analyze the situation of the individual cut extracted from the cut extraction unit (20) based on the text corresponding to the individual cut.

[0063] Here, the analysis unit (40) can analyze an individual cut situation including at least one of the emotions, actions, and background of the individual cut based on the text.

[0064] More specifically, the analysis unit (40) can decompose the text corresponding to the individual cut into sentence units or utterance units, normalize unnecessary symbols, repetitive characters, overlapping of exclamation marks, and duplicate expressions of onomatopoeia and mimetic words, and then extract the emotional expressions, behavioral expressions, and background expressions included in each unit to derive the individual cut situation.

[0065] At this time, emotions can be derived based on the tone of speech, expressions of exclamation, expressions of evaluation, expressions of threat, refusal, or request, and expressions of tension in onomatopoeia and mimetic words; actions can be derived based on verb phrases, onomatopoeia and mimetic words indicating movement, and expressions of the direction of action according to demonstratives; and background can be derived based on place names, expressions of time of day, expressions of weather, and spatial cues.

[0066] In addition, if the text includes dialogue and narration by multiple speakers, the analysis unit (40) can calculate emotions, actions, and backgrounds on a speaker-by-speaker or sentence-by-sentence basis, integrate them on an individual cut basis to determine individual cut situations, and process the determined individual cut situations to include the degree of emphasis on emotions, actions, and backgrounds.

[0067] When integrating emotions, the analysis unit (40) can select the emotion with the highest tension, intensity, or negativity among the emotions produced for each speaker as the representative emotion of the individual cut, or determine the individual cut emotion by weighting the average of the emotions of multiple speakers if they are similar.

[0068] When integrating actions, the analysis unit (40) arranges the actions produced for each speaker in chronological order or order of importance, maintains actions occurring simultaneously within individual cuts in parallel, and selects the action with the highest priority as the representative action for actions occurring sequentially.

[0069] When integrating backgrounds, the analysis unit (40) selects the most specific place name, time zone expression, or weather expression among the spatial clues included in the narrative or dialogue as the representative background of the individual cut, and if multiple background information conflicts, it may prioritize the background information included in the narrative.

[0070] The analysis unit (40) can process the confirmed individual cut situation so that the degree of emphasis on emotion, action, and background is included.

[0071] For example, the text corresponding to an individual cut may include the line “Quiet, if you get caught it’s over,” the onomatopoeia “quickly,” and the description “at the end of the dark hallway.”

[0072] In this case, the analysis unit (40) can break down the text into sentence units or utterance units.

[0073] The analysis unit (40) can extract expressions of admiration, expressions of threat, and onomatopoeic expressions indicating tension contained in each decomposed unit as expressions of emotion.

[0074] Therefore, the analysis unit (40) can derive individual cut situations that include emotions corresponding to tension or fear.

[0075] Additionally, the analysis unit (40) can derive individual cut situations that include actions corresponding to rapid movement based on onomatopoeia and verb phrases indicating movement such as “hurriedly.”

[0076] Additionally, the analysis unit (40) can derive individual cut situations that include a background corresponding to a night or low-light indoor corridor based on spatial clues such as “dark,” “corridor,” and “end,” and low-light clues.

[0077] Additionally, if the same individual cut includes additional instructional lines such as “It’s over there, hide!”, the analysis unit (40) can determine the individual cut situation in which an action including movement in a specific direction or guidance of the gaze is derived based on the instructional words and directional expressions.

[0078] The prompt generation unit (50) can automatically generate a prompt for video generation based on the individual cut situation analyzed by the analysis unit (40).

[0079] Here, the prompt generation unit (50) can generate a video generation prompt that includes instruction information for directing individual video based on individual cut situations.

[0080] More specifically, the prompt generation unit (50) can organize individual cut situations by classifying them into emotions, actions, and backgrounds.

[0081] The prompt generation unit (50) can generate instruction information for direction to correspond to emotions, actions, and backgrounds, and can reflect the instruction information for direction in a prompt for video generation.

[0082] In this case, the direction information may include items such as camera composition, camera movement, scene rhythm, lighting, maintenance of background texture, and handling of character gaze.

[0083] The prompt generation unit (50) can determine each item by a predefined mapping rule based on the emotion, action, and background of the individual cut situation.

[0084] Additionally, the prompt generation unit (50) may apply a consistency verification rule to determine whether there is a contradiction between each item value determined by the mapping rule.

[0085] And, the prompt generation unit (50) can adjust the item value by applying the adjustment priority in the order of background, action, and emotion when a contradiction is determined.

[0086] Additionally, the prompt generation unit (50) can maintain the item value determined by the background during adjustment and redetermine the item value that causes a contradiction among the item values ​​determined by behavior and emotion according to priority.

[0087] And, the prompt generation unit (50) can reapply consistency verification rules to the re-determined item values ​​and repeat the process one or more times until the contradiction is resolved to determine the combination of item values.

[0088] Additionally, the prompt generation unit (50) can organize the combination of confirmed item values ​​into instruction information for direction and complete a prompt for video generation that includes instruction information for direction.

[0089] Here, the consistency verification rule may include whether the lighting item is determined to be low light and strong contrast when the background is analyzed as night or low light.

[0090] Consistency verification rules may include whether camera movement items are determined to be tracking or panning when the behavior is analyzed as rapid movement.

[0091] The consistency verification rule may include whether the scene rhythm item is determined to be rapid development when the emotion is analyzed as tension or fear.

[0092] For example, if the text corresponding to an individual cut includes the line “Quiet, if you get caught it’s over,” the onomatopoeia “quickly,” and the description “at the end of the dark hallway,” the analysis unit (40) may derive an individual cut situation in which the emotion is analyzed as tension, the action is analyzed as rapid movement, and the background is analyzed as a night room.

[0093] In this case, the prompt generating unit (50) can determine the lighting item as low light and strong contrast.

[0094] The prompt generation unit (50) determines the camera movement item as tracking or panning, and the prompt generation unit (50) can determine the scene rhythm item as rapid development, and can apply a consistency verification rule to the combination of determined item values ​​to check whether there is no contradiction.

[0095] And the prompt generation unit (50) can determine the combination of the item values ​​when it is determined that there is no contradiction, and can generate a prompt for video generation that includes instruction information for the determined direction.

[0096] Additionally, if the same individual cut includes additional instruction lines such as “It’s over there, it’s hidden!”, an individual cut situation in which the action is emphasized in a specific direction may be produced, and the prompt generation unit (50) can generate a prompt for video generation by adjusting the direction value of tracking or panning so that the direction of movement is revealed in the camera movement item.

[0097] Referring to FIG. 6, the image generation unit (60) can generate individual images corresponding to individual cuts according to the image generation prompt.

[0098] Here, the video generation unit (60) can generate individual videos with movement based on the feature points of the person or object in the individual cut.

[0099] More specifically, the image generation unit (60) can extract face landmarks, joint landmarks, and contour landmarks of a person included in an individual cut as feature points.

[0100] Additionally, the image generation unit (60) can extract corner points, edge intersection points, and representative contour points of objects included in individual cuts as feature points.

[0101] At this time, the image generation unit (60) can generate feature point-based motion information such that, if there is an action or motion instruction included in the prompt for image generation, the temporal change path of face landmarks, joint landmarks, contour landmarks, corner points, edge intersection points, and representative contour points is defined.

[0102] Meanwhile, the video generation unit (60) can generate or correct frames so that the position, posture, gaze, and shape of a person or object in an individual cut change on a frame-by-frame basis according to feature point-based motion information, thereby generating an individual video with motion.

[0103] In addition, the image generation unit (60) can generate a frame by restricting the relative arrangement relationship of face landmarks, joint landmarks, contour landmarks, corner points, edge intersection points, and representative contour points so that the consistency of the appearance of a person or object is maintained during the process of applying feature point-based motion information.

[0104] Additionally, the image generation unit (60) can process the movement so that it flows naturally by mitigating the change in feature point-based motion information or applying inter-frame interpolation when there are sections where the movement is excessively distorted or feature point tracking is unstable.

[0105] For example, if the video generation prompt corresponding to an individual cut includes “a person turns around and raises their hand to signal,” the video generation unit (60) can extract face landmarks, joint landmarks, and contour landmarks as feature points, and generate a feature point movement path corresponding to head rotation and arm raising to generate an individual video in which head and arm movements change over time.

[0106] Additionally, when a prompt for generating an image corresponding to an individual cut includes “the object moves quickly to the right,” the image generation unit (60) can extract corner points, edge intersection points, and representative contour points as feature points, and generate a feature point movement path corresponding to the direction of movement to generate an individual image of the object moving within the screen.

[0107] Referring to FIG. 6, the video integration unit (70) can integrate two or more individual videos generated from the video generation unit (60) to produce a webtoon video corresponding to the webtoon data.

[0108] Here, the video integration unit (70) can perform integration by matching the generation order of individual videos to the order of individual cuts of webtoon data and arranging the individual videos according to the corresponding order.

[0109] Additionally, the image integration unit (70) can determine the alignment order of individual images using a cut index assigned to each individual image.

[0110] At this time, the video integration unit (70) can define a cut transition section so that the start frame and end frame of the individual video are connected, and process the cut transition section so that fade, dissolve, and cut transition effects are applied.

[0111] Meanwhile, the video integration unit (70) can normalize the tone of individual videos when the color, brightness, and contrast between individual videos differ excessively, thereby reducing visual dissonance during scene transitions.

[0112] In addition, the video integration unit (70) can process the playback time and cut transition point of individual videos so that the scrolling direction and scene development flow of the webtoon data are maintained in the webtoon video.

[0113] At this time, the video integration unit (70) can process the playback time and cut transition point of the individual video based on the state in which the individual videos are sequentially arranged according to the individual cut order and cut index of the webtoon data.

[0114] Additionally, the video integration unit (70) can process the playback time and cut transition point of each individual video so that a preset playback time value is applied to each individual video and the length of the cut transition section is maintained within a preset range.

[0115] Additionally, the video integration unit (70) can process the individual cut order of the webtoon data in the event that the individual cut order does not match the playback order of the individual video by rearranging the arrangement order of the individual video based on the cut index and adjusting the cut transition point to reduce the discrepancy.

[0116] For example, if the order of individual cuts in the webtoon data is defined as the first individual cut, the second individual cut, and the third individual cut, the video integration unit (70) can produce a webtoon video by arranging the first individual video, the second individual video, and the third individual video in the same order, and then defining a cut transition section.

[0117] In addition, if the color tone, brightness, and contrast of the first individual image and the second individual image are different, the image integration unit (70) can apply tone normalization to reduce the sense of unfamiliarity when switching scenes.

[0118] Meanwhile, referring to FIG. 4, the video generation system based on cut-by-cut direction control using a webtoon according to the present invention may further include a preprocessing unit (30) that removes a text area corresponding to a speech bubble or subtitle included in an individual cut and then restores the background.

[0119] Here, the preprocessing unit (30) can detect speech bubble areas and subtitle areas in individual cuts and generate a mask so that text areas corresponding to the detected speech bubble areas and subtitle areas are removed.

[0120] At this time, the preprocessing unit (30) can perform background restoration so that the surrounding background pattern of the text area is continuous when the masked text area is removed.

[0121] Additionally, the preprocessing unit (30) can perform background restoration by correcting the boundaries of the text area so that they are smoothly connected, so that the character outline, background outline, and main object outline are not broken during the text area removal process.

[0122] In addition, the preprocessing unit (30) can reduce visual dissonance by applying boundary cleanup and texture correction when residual text traces, boundary artifacts, and blurred areas remain in the background restoration result.

[0123] At this time, the image generation unit (60), referring to FIG. 5, can generate an individual image by expanding the outer area of ​​the individual cut when the individual cut does not correspond to the pre-set image ratio.

[0124] Here, the image generation unit (60) can define a frame expansion area such that the upper, lower, left, and right outer areas are expanded based on the original frame of the individual cut.

[0125] Additionally, the image generation unit (60) can generate an outer area so that the expanded area naturally connects with the background pattern, color tone, and contour flow of the original individual cut.

[0126] Meanwhile, the image generation unit (60) checks whether the result of expanding the outer area satisfies a preset image ratio, and if it does not satisfy it, the expansion width of the outer area is readjusted so that an individual image corresponding to the preset image ratio is generated.

[0127] For example, if a speech bubble is placed at the top of the screen in an individual cut, the preprocessing unit (30) removes the text area corresponding to the speech bubble and subtitle and restores the top background so that it naturally continues, and then the video generation unit (60) can generate an individual video by creating a frame with an expanded top and bottom outer area to match the vertical output ratio.

[0128] Here, background restoration may mean processing so that the removed area is naturally filled in based on the surrounding pixels, color, texture, and contour flow of the removed area after the text area corresponding to the speech bubble or subtitle has been removed.

[0129] In this case, background restoration may include processing to correct the background information so that it is continuous when the original background information is clearly present, and may include processing to reconstruct the background so that it naturally continues based on surrounding information when the original background information is insufficient or partially obscured.

[0130] Meanwhile, the video generation system based on cut-by-cut direction control using a webtoon according to the present invention may further include a shot manager that sets individual cut-by-cut direction parameters according to user input.

[0131] Here, the shot manager can provide a user interface to input direction parameters for each individual shot.

[0132] In addition, the shot manager can enable direction parameters to be mapped and stored for each cut index corresponding to an individual cut.

[0133] At this time, the direction parameters may include camera angle, camera distance, viewpoint, screen composition, subject movement, background movement, cut duration, cut transition method, lighting, mood, aspect ratio, resolution, and frame rate.

[0134] In addition, the shot manager can save directing parameters that are repeatedly used in the same episode or the same work as presets, and allow the presets to be recalled and applied to individual cuts.

[0135] Additionally, the shot manager can suggest recommended production parameters based on individual cut situations derived from the analysis unit (40), and allow the user to modify and finalize the recommended production parameters.

[0136] At this time, if the input direction parameters conflict with the pre-set video aspect ratio, cut transition interval, or output format, the shot manager can correct the range of the direction parameters or provide a warning.

[0137] At this time, the video generation unit (60) can generate individual videos by reflecting the direction parameters in the video generation prompt.

[0138] Here, the video generation unit (60) can process the production parameters so that they are replaced with sentences or tokens of the video generation prompt and included, and so that individual videos are generated with the production parameters applied.

[0139] Additionally, the video generation unit (60) can apply basic direction parameters based on individual cut situations derived from the analysis unit (40) when direction parameters are not entered, and process the input direction parameters to take precedence over the basic direction parameters when direction parameters are entered.

[0140] For example, if a user inputs a camera distance of close range, a camera angle of high angle, a cut duration of short range, a subject movement of left range, strong lighting, and a tense atmosphere for a specific individual cut, the video generation unit (60) can process the individual video to be generated with the corresponding direction parameters reflected in the video generation prompt.

[0141] Meanwhile, the video generation system based on cut-by-cut direction control using a webtoon according to the present invention, with reference to FIGS. 6 and 7, may further include a post-processing integrated unit (80) that processes to produce a webtoon video after a user's modification work is performed on an individual video, and applies sound effects and BGM to the webtoon video or performs upscaling.

[0142] Here, the post-processing integration unit (80) can perform modification work to reflect trimming of playback sections, adjustment of playback speed, deletion and replacement of frame sections, and changes to cut transition effects for individual videos selected by the user.

[0143] Additionally, the post-processing integration unit (80) can process the individual image with the user's modification work reflected so that it is stored separately from the original individual image, and the versions before and after modification are managed as a history.

[0144] At this time, the post-processing integration unit (80) can process the individual video list to be updated so that the modified individual video is included in the integration target, and the webtoon video to be re-produced according to the updated list.

[0145] Additionally, the post-processing integration unit (80) can select sound effects and BGM to be applied to the webtoon video, and process the selected sound effects and BGM so that they are placed according to the cut transition section, scene development flow, and playback time.

[0146] In addition, the post-processing integration unit (80) can process the volume, start time, end time, and fade in / fade out of the sound effects and BGM so that the auditory dissonance during scene transitions is reduced.

[0147] In addition, the post-processing integration unit (80) can process the output quality to be improved by applying upscaling when the resolution of the webtoon video falls short of the target output specifications.

[0148] At this time, the post-processing integration unit (80) can process so that corrections are applied to ensure that the frame boundaries, contours, and textures before and after upscaling are not excessively distorted.

[0149] For example, if a user shortens the playback section of a specific individual video and changes the cut transition effect to a dissolve, and then places sound effects and BGM to give tension to the webtoon video, the post-processing integration unit (80) can rearrange the modified individual video so that it is reflected as an integration target, and process the webtoon video so that the sound effects and BGM are applied at the time of the cut transition.

[0150] Meanwhile, the video generation system based on cut-by-cut direction control utilizing webtoons according to the present invention can be configured to operate in a closed network-based on-premise environment.

[0151] Here, a closed network-based on-premises environment can be configured to perform webtoon data input, individual cut extraction, individual cut situation analysis, creation of prompts for video generation, individual video generation, and webtoon video integration within an internal network isolated from the external network.

[0152] In addition, the on-premises environment can be operated so that webtoon data, generated prompts, generated individual videos, and produced webtoon videos are stored in internal storage and not transmitted externally.

[0153] In this case, the on-premises environment can be processed so that operations are performed by authorized users by applying user account authentication, access control, and access logging.

[0154] In addition, even when the external export of work results is required, the on-premises environment is operated to apply restrictions on the selection of export targets, the recording of the export time, and export paths, thereby reducing the risk of webtoon data leakage.

[0155] Meanwhile, in the image generation system based on cut-by-cut direction control using a webtoon according to the present invention, at least some of the cut extraction unit (20), analysis unit (40), preprocessing unit (30), prompt generation unit (50), image generation unit (60), image integration unit (70), and post-processing integration unit (80) can be performed by an artificial intelligence model.

[0156] Here, the artificial intelligence model may include a natural language processing model, an image analysis model, and a generative model, and each model may be integrated into a single model or executed as multiple models separated by function.

[0157] Additionally, the analysis unit (40) may use a natural language processing model to process text corresponding to individual cuts, and the natural language processing model may be applied to perform sentence-unit decomposition, normalization, semantic interpretation, and extraction of emotions, behaviors, and backgrounds of the text.

[0158] In addition, the cut extraction unit (20) may use an image analysis model to detect cut boundaries in scroll-type image data, and the image analysis model may be applied to detect discontinuities in margin areas, partition lines, and background patterns.

[0159] Additionally, the preprocessing unit (30) may use an image segmentation model and an image restoration model to remove text areas corresponding to speech bubbles or subtitles and to perform background restoration processing. The image segmentation model may be applied to detect speech bubble areas and subtitle areas, and the image restoration model may be applied so that the removed areas are naturally filled in based on surrounding information.

[0160] Additionally, the prompt generation unit (50) may use a natural language generation model to automatically generate a video generation prompt corresponding to an individual cut situation, and the natural language generation model may be applied to generate a prompt sentence that includes emotions, actions, backgrounds, and direction parameters.

[0161] In addition, the video generation unit (60) may use a generative video model or an image-to-video model to generate individual videos based on a video generation prompt and individual cuts, and feature point-based motion generation may be performed using a pose estimation model or a feature point tracking model.

[0162] Additionally, the video integration unit (70) and the post-processing integration unit (80) may use an artificial intelligence-based correction model or an editing automation tool to automate the alignment of individual videos, the application of cut transition effects, the placement of sound effects and BGM, and upscaling processing.

[0164] The following describes a method for generating video based on panel-by-panel direction control using webtoons; however, the explanation of the aforementioned video generation system may be referenced, and redundant content may be omitted.

[0165] First, referring to FIG. 8, the target work selection and copyright consultation step (S10) selects a target work to be converted into video using webtoon data, and confirms the scope of permission to use and copyright-related conditions for the selected work to secure a state where the webtoon data can be processed.

[0166] The preliminary analysis step (S20) allows the preliminary analysis unit (10) to aggregate text included in all episodes or selected sections of the webtoon data to analyze main characters, the flow of events, worldview clues, atmosphere, and genre characteristics, and to generate preliminary analysis information so that the analysis results are reflected in the subsequent prompt generation step (S70).

[0167] The cut extraction step (S30) allows the cut extraction unit (20) to extract two or more individual cuts included in the webtoon data.

[0168] The preprocessing step (S40) allows the preprocessing unit (30) to process the background after removing the text area corresponding to the speech bubble or subtitle included in the individual cut, thereby reducing visual distractions in the subsequent image generation step (S80).

[0169] The image expansion generation step (S50) allows the image generation unit (60) to expand and generate the outer area of ​​an individual cut when the individual cut does not correspond to a preset image ratio, thereby ensuring an input state where subsequent individual image generation can be performed.

[0170] The analysis step (S60) allows the analysis unit (40) to analyze the situation of the individual cut extracted from the cut extraction unit (20) based on the text corresponding to the individual cut.

[0171] The prompt generation step (S70) allows the prompt generation unit (50) to automatically generate a prompt for video generation based on the individual cut situation analyzed by the analysis unit (40).

[0172] In the video generation step (S80), the video generation unit (60) can generate individual videos corresponding to individual cuts according to the video generation prompt.

[0173] The modification work reflection step (S90) allows the post-processing integration unit (80) to process the individual image to be confirmed as an integration target after the user's modification work is performed on the individual image.

[0174] In the target work selection and copyright consultation stage (S10)0, the video integration unit (70) can integrate two or more individual videos generated from the video generation unit (60) to produce a webtoon video corresponding to the webtoon data.

[0175] The post-processing integration step (S110) allows the post-processing integration unit (80) to apply sound effects and BGM to the produced webtoon video and perform upscaling to improve the output quality.

[0177] A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and their equivalents should be interpreted as being included within the scope of the present invention. Furthermore, the order of operation of the configurations described in the above process does not necessarily need to be performed in a chronological order, and it is understood that even if the order of execution of each configuration and step is changed, such a process may fall within the scope of the present invention as long as it satisfies the essence of the present invention. Explanation of the symbols

[0179] 10 : Pre-analysis unit 20 : Cut extraction unit 30 : Preprocessing Unit 40 : Analysis Unit 50 : Prompt generation unit 60 : Image generation unit 70 : Image Integration Unit 80 : Post-processing Integration Unit S10: Selection of Target Works and Copyright Negotiation Stage S20: Preliminary analysis stage S30: Cut extraction stage S40: Preprocessing step S50: Image expansion generation stage S60: Analysis stage S70: Prompt generation step S80: Image generation stage S90: Modification reflection stage S100: Video integration stage S110: Post-processing integration stage

Claims

Claim 1 A cut extraction unit for extracting two or more individual cuts included in webtoon data; an analysis unit for analyzing the situation regarding the individual cuts extracted by the cut extraction unit based on text corresponding to the individual cuts; a prompt generation unit for automatically generating a prompt for video generation according to the situation of the individual cuts analyzed by the analysis unit; and a video generation unit for generating an individual video corresponding to the individual cuts according to the prompt for video generation. and an image integration unit that integrates two or more individual images generated from the image generation unit to produce a webtoon image corresponding to the webtoon data; wherein the webtoon data includes one or more image data arranged in a scrolling manner, and the cut extraction unit is characterized by detecting cut boundaries in the image data to extract the individual cuts, and the analysis unit analyzes the individual cut situation including at least one of the emotion, action, and background of the individual cut based on the text, and the prompt generation unit generates the video generation prompt including instruction information for directing the individual image based on the individual cut situation, and wherein, if the webtoon data further includes a PSD file, the cut extraction unit determines whether cut unit area information defined by a layer or mask exists from the PSD file, and if cut unit area information exists, extracts the individual cut based on the cut unit area information, and if cut unit area information does not exist or is insufficient, generates the image data by composite rendering the PSD, and then detects cut boundary candidates by scanning the image data in the vertical direction, and for the cut boundary candidates, determines whether they correspond to a low-density section based on pixel density and brightness distribution, and edges A cut boundary is determined by judging both the scarcity of contour components based on intensity and the presence of a text area, said cut boundary is determined based on the margin area formed between adjacent individual cuts, the partition line, and the discontinuity of the background pattern, and two or more individual cuts are extracted by correcting the determined cut boundary with said cut unit area information.The above-described prompt generation unit organizes individual cut situations by classifying them into emotion, action, and background, generates instruction information for direction corresponding to the emotion, action, and background, and the instruction information for direction includes items such as camera composition, camera movement, scene rhythm, lighting, maintenance of background texture, and handling of character gaze, determines each item by a predefined mapping rule according to the emotion, action, and background of the individual cut situation, applies a consistency verification rule to determine whether there is a contradiction between each item value determined by the mapping rule, adjusts the item values ​​by applying adjustment priorities in the order of background, action, and emotion when a contradiction is determined, maintains the item value determined by the background, and redefines the item value among the item values ​​determined by action and emotion that causes a contradiction according to priority, applies the consistency verification rule again to the redeterminated item values, and confirms the combination of item values ​​by repeating this process one or more times until the contradiction is resolved, organizes the confirmed combination of item values ​​as instruction information for direction, completes a prompt for video generation containing the instruction information for direction, and is configured to operate in a closed network-based on-premise environment, and the above-described closed network-based on-premise A video generation system based on cut-by-cut direction control utilizing webtoons, characterized by an environment configured to perform webtoon data input, individual cut extraction, individual cut situation analysis, video generation prompt generation, individual video generation, and webtoon-video integration within an internal network isolated from an external network, and to operate such that webtoon data, generated prompts, generated individual videos, and produced webtoon videos are stored in an internal storage so as not to be transmitted externally, and to apply user account authentication, access control, and access log recording, and to configure the system to apply the selection of the export target, recording of the export time, and restriction of the export path when external export of the work results is required. Claim 2 delete Claim 3 delete Claim 4 A video generation system based on cut-by-cut direction control utilizing a webtoon, further comprising a preprocessing unit that restores the background after removing a text area corresponding to a speech bubble or subtitle included in the individual cut in claim 1, and wherein the video generation unit generates the individual video by expanding the outer area of ​​the individual cut when the individual cut does not conform to a preset video ratio. Claim 5 A video generation system based on cut-by-cut direction control utilizing a webtoon, further comprising, in claim 1, a shot manager that sets direction parameters for each individual cut according to user input; and wherein the video generation unit generates the individual video by reflecting the direction parameters in the video generation prompt. Claim 6 A video generation system based on cut-by-cut direction control utilizing a webtoon, characterized in that, in claim 1, the video generation unit generates the individual video to which movement is imparted based on the feature points of a person or object in the individual cut. Claim 7 A video generation system based on cut-by-cut direction control utilizing a webtoon, characterized in that, in claim 1, it further includes a post-processing integrated unit configured to produce the webtoon video after a user's modification work is performed on the individual video, and to apply sound effects and BGM to the webtoon video or perform upscaling. Claim 8 delete Claim 9 A cut extraction step in which a cut extraction unit extracts two or more individual cuts included in webtoon data; an analysis step in which an analysis unit analyzes the situation regarding the individual cuts extracted by the cut extraction unit based on text corresponding to the individual cuts; a prompt generation step in which a prompt generation unit automatically generates a prompt for video generation according to the situation of the individual cuts analyzed by the analysis unit; and a video generation step in which a video generation unit generates an individual video corresponding to the individual cuts according to the prompt for video generation. The method comprises a video integration step in which a video integration unit integrates two or more individual images generated from a video generation unit to produce a webtoon image corresponding to the webtoon data; wherein the webtoon data includes one or more image data arranged in a scrolling manner, and the cut extraction unit detects cut boundaries in the image data to extract the individual cuts, and the analysis unit analyzes the individual cut situation including at least one of the emotion, action, and background of the individual cut based on the text, and the prompt generation unit generates a prompt for video generation including instruction information for directing the individual image based on the individual cut situation, and wherein, if the webtoon data further includes a PSD file, the cut extraction unit determines whether cut unit area information defined by a layer or mask exists from the PSD file, and if cut unit area information exists, extracts the individual cut based on the cut unit area information, and if cut unit area information does not exist or is insufficient, generates the image data by synthesizing and rendering the PSD, detects cut boundary candidates by scanning the image data in the vertical direction, and for the cut boundary candidates, [corresponds] to the low-density section based on pixel density and brightness distribution A cut boundary is determined by determining the presence, the scarcity of contour components based on edge intensity, and the presence of a text area, and the cut boundary is determined based on the margin area formed between adjacent individual cuts, the partition line, and the discontinuity of the background pattern.Two or more individual cuts are extracted by correcting the confirmed cut boundaries with the above-mentioned cut unit area information, and the prompt generation unit organizes the individual cut situations by classifying them into emotion, action, and background, and generates instruction information for direction corresponding to the emotion, action, and background. The instruction information for direction includes items such as camera composition, camera movement, scene rhythm, lighting, maintenance of background texture, and character gaze processing. Each item is determined by a predefined mapping rule according to the emotion, action, and background of the individual cut situation. A consistency verification rule is applied to determine whether there is a contradiction between the item values ​​determined by the mapping rule. If a contradiction is determined, the item values ​​are adjusted by applying the adjustment priority in the order of background, action, and emotion. During adjustment, the item values ​​determined by the background are maintained, and among the item values ​​determined by action and emotion, the item values ​​causing contradictions are re-determined according to priority. The consistency verification rule is applied again to the re-determined item values, and the process is repeated one or more times until the contradiction is resolved to finalize the combination of item values. The finalized combination of item values ​​is organized as instruction information for direction, and a prompt for video generation containing the instruction information for direction is completed. Closed network A method for generating video based on cut-by-cut direction control utilizing a webtoon, characterized by being configured to operate in a closed network-based on-premises environment, wherein the closed network-based on-premises environment is arranged so that input of webtoon data, extraction of individual cuts, analysis of individual cut situations, generation of prompts for video generation, generation of individual videos, and integration of webtoon videos are performed in an internal network separated from an external network, and wherein the webtoon data, generated prompts, generated individual videos, and produced webtoon videos are stored in an internal storage so as not to be transmitted externally, user account authentication, access control, and access log recording are applied, and wherein, if external export of the work results is required, the selection of the export target, recording of the export time, and restriction of the export path are applied.

Images