A method for automatic line wrapping based on video script

Abstract

The present invention relates to the technical field of image subtitle insertion, and in particular to a method for automatic line wrapping based on video scripts. The method determines information of multiple dimensions based on the video script, and then uses the degree of pixel change in the video material for segmentation. The obtained action video frame segments can decompose the video segments into multiple action stages. For an action video frame segment, the interval of the text segmentation is determined by quantifying the necessity of subtitle segmentation, and the overall coverage of the subtitles can be determined by the matching degree of the text segmentation in the matching process, and the display duration of the subtitle content can be determined. The necessity of subtitle line wrapping is further quantified by changes in the shooting scene and the shooting angle, and the line wrapping strategy between adjacent screen contents can be judged by the necessity of subtitle line wrapping. The present invention makes the final generated subtitles have a clear correspondence with the picture, and the disappearance of the subtitles and the generated intervals can give the audience a deeper impression.

Description

Technical Field

[0001] The present invention relates to the technical field of image subtitle insertion, and in particular to a method based on automatic line wrapping of video scripts. Background Art

[0002] UGC medical videos refer to video content about medical science that is independently created and published by ordinary users. They aim to popularize daily medical knowledge to the audience in a vivid way through the combination of subtitles and object demonstrations. To produce this type of self-media video, a video script is first drawn up. The video script usually contains five aspects of video planning: shooting scene, copy content, duration, video segment, and shooting angle. The shooting scene is the planned location for video shooting, the copy content is the planned subtitle content for video display, the duration is the duration of video display, the video segment is the dynamic change information within the corresponding duration, and the shooting angle is the placement and angle of different cameras. According to the content planning of the video script, a large amount of video material is shot, and then the video material is edited and selected according to the video script, and the subtitles are matched to the edited and selected video material so that the subtitles can be automatically displayed in line breaks according to the changes in the video content.

[0003] For subtitles in videos, including dialogue subtitles and other subtitles, existing technologies can directly use voice recognition to determine the subtitle content and insert it intelligently. However, for other subtitles, because there is no voice information as a reference, a fixed video frame is generally selected manually to start inserting the subtitle information, so that the subtitle information starts to be displayed in line breaks at fixed time intervals. However, in actual practice, different video materials in the video script have different durations, different levels of importance, and other factors. The traditional subtitle line break display method cannot be well combined with the video segment of the video material, so the traditional subtitle display method may break the line too fast or too slow, which means that the subtitle content cannot be well matched with the video segment planned by the video script. Summary of the Invention

[0004] In order to solve the technical problem in the prior art that manually inserting subtitles and setting a line break strategy result in poor subtitle display effects, the present invention aims to provide a method for automatic line break based on video scripts. The technical solution adopted is as follows:

[0005] The present invention proposes a method for automatic line wrapping based on a video script, the method comprising:

[0006] Conduct text analysis on video scripts to determine the shooting scene, shooting angle, shooting duration, and copy content of each video segment;

[0007] For each video segment, segmenting is performed based on the degree of pixel change between adjacent frames in the video material to obtain multiple action video frame segments in the video material;

[0008] Match the text segmentation of the text content with each action video frame segment, and determine the necessity of subtitle segmentation for each action video frame segment based on the number of matched text segmentation and the length of the action video frame segment; determine the interval of the matched text segmentation based on the necessity of subtitle segmentation, and then determine the subtitle content;

[0009] Determine the overall coverage of subtitles for each action video frame segment based on the matching degree between the text segmentation and each video frame, and the necessity of the subtitle segmentation; determine the display duration of the subtitle content based on the overall coverage of the subtitles, and insert the subtitle content;

[0010] Between adjacent video segments, the necessity of subtitle line breaks between adjacent video segments is determined based on the degree of change in the shooting scene and shooting angle, as well as the difference in the necessity of subtitle word segmentation; based on the necessity of subtitle line breaks, it is determined whether to display the subtitle content in a line break.

[0011] Furthermore, the method for segmenting the action video frame segments includes:

[0012] For adjacent frames in the video material, pixel feature matching is performed between the adjacent frames, and the product of the number of pixels that produce position changes and the maximum position change distance is normalized to obtain the motion expression degree between the adjacent frames; according to the motion expression degree, the video frames in the video material are divided into large-scale motion video frames and small-scale motion video frames, and adjacent large-scale motion video frames and adjacent small-scale motion video frames constitute the motion video frame segment.

[0013] Furthermore, the method for obtaining the necessity of subtitle segmentation includes:

[0014] Obtain the duration ratio of the action video frame segment in the video segment; obtain the ratio of the number of text segmentations matched by the action video frame segment in the total number of text segmentations in the video segment; normalize the product of the duration ratio and the number ratio to obtain the necessity of the subtitle segmentation.

[0015] Furthermore, the method for determining the subtitle content includes:

[0016] For an action video frame segment, if the necessity of subtitle segmentation is greater than the preset necessity threshold, the matching text segmentation is directly used as the subtitle content; if the necessity of subtitle segmentation is not greater than the preset necessity threshold, the matching text segmentation is input into the preset language model to determine the concise sentence formed, and the concise sentence is used as the subtitle content.

[0017] Furthermore, a pre-trained neural network is used to match the text segment with each video frame in each action video frame segment.

[0018] Furthermore, the score value generated during the neural network matching process is used as the matching degree.

[0019] Furthermore, the method for obtaining the overall coverage of subtitles includes:

[0020] For each action video frame segment, the average matching degree of all video frames with the matching text segmentation in the matching process is obtained, and the product of the average matching degree and the necessity of the subtitle segmentation is normalized to obtain the overall coverage of the subtitle.

[0021] Furthermore, the step of determining the display duration of the subtitle content according to the overall coverage of the subtitles and inserting the subtitle content includes:

[0022] If the overall coverage of the subtitles is greater than the preset coverage threshold, the display duration of all subtitle contents is the same, and all subtitle contents are evenly inserted into the action video frame segments; if the overall coverage of the subtitles is not greater than the preset coverage threshold, the corresponding action video frame segments will be marked, and the staff will be notified to manually determine the display duration and insertion method.

[0023] Furthermore, the method for obtaining the necessity of subtitle line wrapping includes:

[0024] Between adjacent video segments, if the shooting scene or shooting angle changes, the change weight is set to a first preset value; if neither the shooting scene nor the shooting angle changes, the change weight is set to a second preset value; wherein the first preset value is greater than the second preset value; the subtitle word segmentation necessity difference between the last action video frame segment of the previous video segment and the first action video frame segment of the next video segment is obtained; the product of the subtitle word segmentation necessity difference and the change weight is normalized to obtain the subtitle line break necessity.

[0025] Furthermore, the determining whether to display the subtitle content in a line wrap according to the necessity of the subtitle line wrap includes:

[0026] If the necessity of subtitle line wrapping is less than the preset line wrapping necessity threshold, the subtitle clearing command will not be executed and the next subtitle content will be displayed directly; if the necessity of subtitle line wrapping is not less than the preset line wrapping necessity threshold, the subtitle clearing command will be executed to clear the subtitle content of the previous video segment and then display the next subtitle content.

[0027] The present invention has the following beneficial effects:

[0028] The present invention determines information of multiple dimensions based on the video script, and then uses the degree of pixel change in the video material to segment it. The obtained action video frame segment can decompose the video segment into multiple action stages, and then analyze an action video frame segment in a targeted manner. For an action video frame segment, the more obvious the distribution of its corresponding text segmentation and duration in the entire video segment is, the more important the action video frame segment is. Therefore, the interval of the text segmentation is determined by quantifying the necessity of subtitle segmentation. The overall coverage of the subtitle can be further determined by the matching degree of the text segmentation in the matching process, and the display duration of the subtitle content is determined, so that the subtitle content finally inserted can reduce the blank space of the video subtitle and deepen the audience's impression. Further considering the connection between adjacent video segments, the necessity of subtitle line breaking is further quantified by the changes in the shooting scene and the shooting angle. The line breaking strategy between adjacent switching content can be judged by the necessity of subtitle line breaking. While the final generated subtitles have a clear correspondence with the picture, the disappearance and generated interval of the subtitles can give the audience a deeper impression. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A flow chart of a method for automatic line wrapping based on a video script provided by one embodiment of the present invention;

[0031] Figure 2 A schematic diagram of a video script provided by one embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a subtitle insertion effect provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To further illustrate the technical means and effects employed by the present invention to achieve the intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a method for automatic line wrapping based on video scripts according to the present invention, including its specific implementation, structure, features, and effects. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0034] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0035] The following describes in detail a specific solution of a method for automatic line wrapping based on video scripts provided by the present invention in conjunction with the accompanying drawings.

[0036] See also Figure 1 , which shows a flow chart of a method for automatic line wrapping based on a video script provided by one embodiment of the present invention, the method comprising:

[0037] Step S1: Perform text analysis on the video script to determine the shooting scene, shooting angle, shooting duration, and text content in each video segment.

[0038] See also Figure 2 , which shows a schematic diagram of a video script provided by an embodiment of the present invention, Figure 2 This is a video script for a medical promotional video. The video script has clear copy content, shooting duration, shooting scene, shooting angle, and image content. Simple text analysis and classification can determine the information under each dimension. It should be noted that text recognition and classification analysis of video scripts are technical means well known to those skilled in the art and will not be elaborated here.

[0039] by Figure 2 For example, the information extracted from one video segment is:

[0040] (1) Video segment number: 1.

[0041] (2) Shooting scene: outdoor.

[0042] (3) Shooting duration: 3s.

[0043] (4) Shooting angle: long shot.

[0044] (5) Screen content: The commentator gestures with one hand.

[0045] (6) Copy content: What should we do if we find someone lying on the ground?

[0046] It should be noted that, after determining the content of the copy, the embodiment of the present invention needs to perform word segmentation on the content of the copy, and the jieba word segmentation tool can be used for word segmentation. For example, "The first step is to confirm the safety of the environment. Is it safe for us and the injured?" will be segmented into "The first step / we / need / to / confirm / the / safety / of / the / environment / for / us / and / the / injured / whether / it / is / safe."

[0047] In the embodiment of the present invention, each video segment needs to shoot multiple video materials, and then select the most suitable video material from the multiple video materials as the video material to be subsequently inserted with subtitles. This is a conventional method of video shooting and will not be described in detail here.

[0048] Step S2: For each video segment, segment it based on the degree of pixel change between adjacent frames in the video material to obtain multiple action video frame segments in the video material.

[0049] For a video segment, the medical science popularization video proposed in the embodiment of the present invention usually includes multiple character actions for emergency medical treatment. For example, in the artificial respiration science popularization video, the popularization personnel will make continuous and large pressing movements. For a video segment, if the generation of subtitles and line breaks can be associated with the importance of the actions generated in the video, it will give users a deep impression and better reflect the significance of science popularization and teaching. Therefore, for the video material of each video frame, it is necessary to determine each action therein, because the generation of actions will cause the pixel point information between video frames to change. Therefore, based on the degree of pixel change between adjacent frames in the video material, segmentation can be performed to obtain multiple action video frame segments in the video material.

[0050] Preferably, in an embodiment of the present invention, the method for segmenting action video frame segments includes:

[0051] For adjacent frames in the video material, pixel feature matching is performed between adjacent frames, and the product of the number of pixels that produce position changes and the maximum position change distance is normalized to obtain the motion expression between adjacent frames. That is, between adjacent frames, the more pixel changes there are and the greater the amplitude of the changes, the greater the motion expression between the two frames. It should be noted that pixel feature matching is a technical means well known to those skilled in the art, and is implemented by matching algorithms such as FLANN matching, which will not be elaborated here. It should be noted that the position change distance is the Euclidean distance between matching pixel points in the image coordinate system.

[0052] According to the motion expression degree, video frames in the video material are divided into large motion video frames and small motion video frames, and adjacent large motion video frames and adjacent small motion video frames constitute the motion video frame segment. In the embodiment of the present invention, the Otsu threshold algorithm can be used to adaptively perform threshold segmentation on the motion expression degree, thereby obtaining two types of video frames.

[0053] It should be noted that the normalization means adopted in the embodiments of the present invention may adopt linear normalization. Those skilled in the art may choose other normalization algorithms to achieve the normalization purpose, which will not be elaborated or limited here.

[0054] Step S3: Match the text segmentation of the text content with each action video frame segment, and determine the necessity of subtitle segmentation for each action video frame segment based on the number of matched text segmentation and the length of the action video frame segment; determine the interval of the matched text segmentation based on the necessity of subtitle segmentation, and then determine the subtitle content.

[0055] When creating self-media videos, such as UGC medical videos, each video segment corresponds to a single frame, and each frame corresponds to a corresponding textual content. However, the frame itself records the dynamic changes in information over a specified duration, captured across multiple video frames. Therefore, it is necessary to combine word segmentation combinations with video footage segments at different content stages to perform content matching. This identifies several word segmentation points within the same frame, which is then used to determine the subtitle content. The video footage for each segment contains the dynamic behavior corresponding to the frame content. For example, a video clip of a commentator gently tapping a dummy on the shoulder would be divided into two main content stages: the commentator bending over and then, while remaining bent over, tapping the dummy with both hands. Different content stages have different emphases, and the textual content contains dynamic information corresponding to the entire frame. Therefore, the textual content can be segmented by content stage. Therefore, the textual content is first matched against the textual content. The textual content is then matched against each action video frame. For each action video frame, all matching textual content segments are obtained.

[0056] For an action video frame segment, the longer its duration and the more text segmentations that are matched, the more important the information in the action video frame segment is, which means that the dynamic information of the action video frame segment needs to be explained with distinctive and memorable subtitles, thereby reducing the difficulty for the audience to understand the video content and deepening the impression of the content. Therefore, the embodiment of the present invention determines the necessity of subtitle segmentation for each action video frame segment based on the number of matched text segmentations and the length of the action video frame segment. The greater the necessity of subtitle segmentation, the more brief subtitle content is needed when forming the subtitle content to enhance the audience's memory. Therefore, the interval between matching text segmentations can be determined based on the necessity of subtitle segmentation, and then the subtitle content can be determined. That is, the greater the necessity of subtitle segmentation, the more frequent the intervals between text segmentations should be, thereby forming more and shorter subtitle content to deepen the audience's impression.

[0057] Preferably, in an embodiment of the present invention, the method for obtaining the necessity of subtitle segmentation includes:

[0058] The duration ratio of the action video frame segments in the video segment is obtained; the number ratio of the text segmentation words matching the action video frame segments in the total number of text segmentation words in the video segment is obtained; and the product of the duration ratio and the number ratio is normalized to obtain the necessity of subtitle segmentation words. Specifically, the embodiment of the present invention integrates the two types of proportion information in a positive correlation manner through multiplication to obtain the necessity of subtitle segmentation words. The necessity of subtitle segmentation words is essentially determined by comparing the subtitle segmentation words with the action video frame segments in the entire video segment.

[0059] Preferably, in an embodiment of the present invention, the method for determining subtitle content includes:

[0060] For an action video frame, if the subtitle segmentation necessity exceeds a preset necessity threshold, the matching text segmentation is directly used as the subtitle content. If the subtitle segmentation necessity does not exceed the preset necessity threshold, the matching text segmentation is input into a preset language model to determine the resulting concise sentence, which is used as the subtitle content. In this embodiment of the present invention, the necessity threshold is set to 0.25.

[0061] As an example, Figure 2 The fourth video segment in the video script contains the text "Sir, sir, can you hear me?". After word segmentation, the result is "Sir / Sir / Can / You / Hear / Me / Talk / Can?" This short text content is assigned to two action video frame segments, of which the first action video frame segment has obvious slapping movements and matches the content "Sir, sir". The second action video frame is a small movement and matches the content "Can you hear me?". If the necessity of subtitle segmentation for the first segment is greater than the preset necessity threshold, the final subtitle content is "Sir" and "Sir", that is, two relatively short subtitle contents; if the necessity of subtitle segmentation for the second segment is not greater than the preset necessity threshold, the concise sentence "Can you hear me?" is finally formed as the subtitle content.

[0062] The language model in the embodiment of the present invention is a deep generative model (Seq2Seq / Transformer). The specific construction and use of the model are technical means well known to those skilled in the art and will not be described in detail here.

[0063] Step S4: Determine the overall coverage of the subtitles for each action video frame segment based on the degree of matching between the text segmentation and each video frame, as well as the necessity of the subtitle segmentation; determine the display duration of the subtitle content based on the overall coverage of the subtitles and insert the subtitle content.

[0064] For an action video frame segment, after the subtitle content is determined, the subtitle content needs to be inserted into the video. The necessity of subtitle segmentation can serve as the basis for segmentation, and it may also represent the importance of the information in the action video frame segment. Further combined with the matching degree of the text segmentation and each video frame in the matching process, the greater the matching degree, the more the text segmentation matches the action video frame segment, and the more effective the information. Therefore, the fusion of these two features can obtain the overall coverage of subtitles. The greater the overall coverage of subtitles, the more effective coverage must be ensured when inserting the subtitle content to avoid the simultaneous appearance of two subtitle contents. Conversely, the smaller the overall coverage of subtitles, the more different subtitle contents can be allowed to overlap without affecting the video effect. Therefore, the display duration of the subtitle content can be determined based on the overall coverage of the subtitles and the subtitle content can be inserted.

[0065] Preferably, in an embodiment of the present invention, a pre-trained neural network is used to match the text segmentation with each video frame in each action video frame segment. The neural network is mainly used to detect actions and match them with corresponding segmentation, such as a network formed by combining a key point detection network with a fully connected layer. This is a technical means well known to those skilled in the art and will not be described in detail here. The neural network generates a score value when matching the segmentation and video frame, that is, a large score value indicates a successful match, and thus the score value can be used as the degree of match.

[0066] Preferably, in an embodiment of the present invention, the method for obtaining the overall coverage of subtitles includes:

[0067] For each action video frame segment, since it includes multiple video frames, the average matching degree of all video frames with the matching text segmentation words during the matching process is obtained. The product of the average matching degree and the necessity of the subtitle segmentation word is normalized to obtain the overall coverage of the subtitles.

[0068] Preferably, in an embodiment of the present invention, determining the display duration of subtitle content and inserting the subtitle content according to the overall coverage of the subtitles includes:

[0069] If the overall coverage of the subtitles is greater than the preset coverage threshold, it means that the subtitle content should not be overlapped and should be displayed one by one in sequence. Therefore, the display duration of all subtitle contents is set to be the same, and all subtitle contents are evenly inserted into the action video frame segments.

[0070] If the overall coverage of the subtitles is not greater than the preset coverage threshold, it means that the subtitles are allowed to be displayed overlapping. It should be noted that the coverage proposed in the embodiment of the present invention does not mean that the two subtitles overlap, but are displayed together in the video. The corresponding action video frame segments are marked and the staff is notified to manually determine the display duration and insertion method. Figure 3 As shown, it shows a schematic diagram of a subtitle insertion effect provided by an embodiment of the present invention. In the subtitle insertion effect at the head, two subtitle contents are displayed overlappingly.

[0071] In the embodiment of the present invention, the coverage threshold is set to 0.25.

[0072] Step S5: between adjacent video segments, the necessity of subtitle line wrapping between adjacent video segments is determined based on the degree of change in the shooting scene and shooting angle, as well as the difference in the necessity of subtitle word segmentation; and whether to display the subtitle content in a line wrap is determined based on the necessity of subtitle line wrapping.

[0073] Step S4 mainly provides a strategy for inserting subtitle content in a video segment. Between adjacent video segments, different screen contents have different expression strengths for keywords in their respective screen contents due to different changes in shooting scenes and angles, resulting in different subtitle coherence requirements. If the expression strengths are similar, when different screen contents are docked, there will not be a large background change between the different screens, and the core objects of the copy content between the docking screen contents are similar. At this time, the subtitle information between the docking screen contents is allowed to have a display delay within a certain range; if the expression strength differences are more obvious, when different screen contents are docked, there will be a large background change between the different screens, and the core objects of the copy content between the docking screen contents are too different. At this time, the subtitle information between the docking screen contents needs to be switched immediately for correspondence. Therefore, the embodiment of the present invention determines the necessity of subtitle line breaks between adjacent video segments based on the degree of change in shooting scenes and shooting angles, as well as the difference in the necessity of subtitle word segmentation. That is, the greater the degree of change, the greater the difference in the necessity of subtitle word segmentation, indicating that there are obvious core information differences between the video segments, and subtitle line break switching needs to be performed as soon as possible. Therefore, whether to perform line break display of subtitle content is determined based on the necessity of subtitle line break.

[0074] Preferably, in an embodiment of the present invention, the method for obtaining the necessity of subtitle line wrapping includes:

[0075] Between adjacent video segments, if the shooting scene or shooting angle changes, the change weight is set to a first preset value; if neither the shooting scene nor the shooting angle changes, the change weight is set to a second preset value; wherein the first preset value is greater than the second preset value; in an embodiment of the present invention, the first preset value is set to 1 and the second preset value is set to 0.5.

[0076] Because the last action video frame of the previous video segment is adjacent to the first action video frame of the next video segment in temporal order, the subtitle segmentation necessity difference between the last action video frame of the previous video segment and the first action video frame of the next video segment is obtained. The subtitle segmentation necessity difference is normalized by multiplying the product of the change weight to obtain the subtitle line break necessity. The subtitle segmentation necessity difference is the absolute value of the difference between the two subtitle segmentation necessity values.

[0077] Preferably, in an embodiment of the present invention, determining whether to display subtitle content in a line wrap mode according to the necessity of subtitle line wrapping includes:

[0078] If the subtitle line wrap necessity is less than the preset line wrap necessity threshold, the subtitle clearing command is not executed and the next subtitle content is displayed directly; if the subtitle line wrap necessity is not less than the preset line wrap necessity threshold, the subtitle clearing command is executed to clear the subtitle content of the previous video segment and then display the next subtitle content. Figure 3As shown, between adjacent video segments, the subtitle clearing command is directly executed, and the subtitle content of the previous segment is cleared and the subtitle content of the second video segment is directly displayed.

[0079] In summary, the present invention determines information of multiple dimensions based on the video script, and then uses the degree of pixel change in the video material for segmentation. The obtained action video frame segments can decompose the video segments into multiple action stages. For an action video frame segment, the interval of the text segmentation is determined by quantifying the necessity of subtitle segmentation, and the overall coverage of the subtitles can be determined by the matching degree of the text segmentation in the matching process, and the display duration of the subtitle content can be determined. Through the changes in the shooting scene and the shooting angle, the necessity of subtitle line breaks is further quantified, and the line break strategy between adjacent screen contents can be judged by the necessity of subtitle line breaks. The present invention makes the final generated subtitles have a clear correspondence with the picture, and the disappearance and generated intervals of the subtitles can give the audience a deeper impression.

[0080] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0081] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A method for automatic line wrapping based on video scripts, characterized in that: The method comprises: Conduct text analysis on video scripts to determine the shooting scene, shooting angle, shooting duration, and copy content of each video segment; For each video segment, segmenting is performed based on the degree of pixel change between adjacent frames in the video material to obtain multiple action video frame segments in the video material; Match the text segmentation of the text content with each action video frame segment, and determine the necessity of subtitle segmentation for each action video frame segment based on the number of matched text segmentation and the length of the action video frame segment; determine the interval of the matched text segmentation based on the necessity of subtitle segmentation, and then determine the subtitle content; Determine the overall coverage of subtitles for each action video frame segment based on the matching degree between the text segmentation and each video frame, and the necessity of the subtitle segmentation; determine the display duration of the subtitle content based on the overall coverage of the subtitles, and insert the subtitle content; Between adjacent video segments, the necessity of subtitle line wrapping is determined based on the degree of change in shooting scenes and shooting angles, as well as the difference in the necessity of subtitle word segmentation; and whether to display the subtitle content in a line wrap is determined based on the necessity of subtitle line wrapping. The method for obtaining the necessity of subtitle segmentation includes: Obtaining a duration ratio of the action video frame segment in the video segment; obtaining a ratio of the number of text segmentations matched by the action video frame segment in the total number of text segmentations in the video segment; normalizing the product of the duration ratio and the number ratio to obtain the necessity of the subtitle segmentation; The method for obtaining the overall coverage of subtitles includes: For each action video frame segment, obtain the average matching degree of all video frames with the matching text segmentation in the matching process, and normalize the product of the average matching degree and the necessity of the subtitle segmentation to obtain the overall coverage of the subtitle; The method for obtaining the necessity of subtitle line wrapping includes: Between adjacent video segments, if the shooting scene or shooting angle changes, the change weight is set to a first preset value; if neither the shooting scene nor the shooting angle changes, the change weight is set to a second preset value; wherein the first preset value is greater than the second preset value; the subtitle word segmentation necessity difference between the last action video frame segment of the previous video segment and the first action video frame segment of the next video segment is obtained; the product of the subtitle word segmentation necessity difference and the change weight is normalized to obtain the subtitle line break necessity.

2. The method for automatic line wrapping based on video script according to claim 1, characterized in that: The segmentation method of the action video frame segments includes: For adjacent frames in the video material, pixel feature matching is performed between the adjacent frames, and the product of the number of pixels that produce position changes and the maximum position change distance is normalized to obtain the motion expression degree between the adjacent frames; according to the motion expression degree, the video frames in the video material are divided into large-scale motion video frames and small-scale motion video frames, and adjacent large-scale motion video frames and adjacent small-scale motion video frames constitute the action video frame segment.

3. The method for automatic line wrapping based on video script according to claim 1, characterized in that: The method for determining the subtitle content includes: For an action video frame segment, if the necessity of subtitle segmentation is greater than the preset necessity threshold, the matching text segmentation is directly used as the subtitle content; if the necessity of subtitle segmentation is not greater than the preset necessity threshold, the matching text segmentation is input into the preset language model to determine the concise sentence formed, and the concise sentence is used as the subtitle content.

4. The method for automatic line wrapping based on video script according to claim 1, characterized in that: A pre-trained neural network is used to match the text segment with each video frame in each action video segment.

5. The method for automatic line wrapping based on video script according to claim 4, characterized in that: The score value generated during the neural network matching process is used as the matching degree.

6. The method for automatic line wrapping based on video script according to claim 1, characterized in that: The method of determining the display duration of the subtitle content according to the overall coverage of the subtitles and inserting the subtitle content includes: If the overall coverage of the subtitles is greater than the preset coverage threshold, the display duration of all subtitle contents is the same, and all subtitle contents are evenly inserted into the action video frame segments; if the overall coverage of the subtitles is not greater than the preset coverage threshold, the corresponding action video frame segments will be marked, and the staff will be notified to manually determine the display duration and insertion method.

7. The method for automatic line wrapping based on video script according to claim 1, characterized in that: The determining whether to display the subtitle content in a line wrap according to the necessity of the subtitle line wrap includes: If the necessity of subtitle line wrapping is less than the preset line wrapping necessity threshold, the subtitle clearing command will not be executed and the next subtitle content will be displayed directly; if the necessity of subtitle line wrapping is not less than the preset line wrapping necessity threshold, the subtitle clearing command will be executed to clear the subtitle content of the previous video segment and then display the next subtitle content.

Images