Information video synthesis method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides an information video synthesis method and device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the invention, the information website information and the material information corresponding to the video synthesis request are acquired, the information text is acquired according to the information website information, the text title and the text abstract are determined according to the information text, and the information voice information and the information subtitle information are generated according to the text abstract. The target information video is generated according to the information voice information, the information subtitle information, the text title and the material information, and the target information video is automatically generated according to the information text and the material information corresponding to the information website information provided by the user, so that the user does not need to manually edit materials, and the requirement on the video editing capability of the user is reduced; and the video generation efficiency is effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computing technology, and in particular to an information video synthesis method, device, equipment, storage medium and product. Background Art

[0002] With the development of Internet technology, the form of information dissemination is not limited to text and pictures, and video display has become one of the mainstream methods. Video can not only convey richer information, but also bring a more intuitive experience. However, combining pictures or video materials into a complete video requires professional skills and complex operations, and it takes a lot of time.

[0003] Currently, the commonly used video synthesis solution generally uses dedicated video editing software to load multiple images or videos and other materials into various areas of the display container according to preset screen layout configuration information, and renders them to generate video playback files. This video synthesis solution requires users to have strong video editing capabilities, has high professional requirements on users, and has low video generation efficiency. Summary of the invention

[0004] The embodiments of the present application provide an information video synthesis method, device, equipment, storage medium and product to solve the technical problems in the related art that the video synthesis solution has high professional requirements on users and the video generation efficiency is low, thereby effectively improving the video generation efficiency.

[0005] In a first aspect, an embodiment of the present application provides an information video synthesis method, comprising:

[0006] When receiving a video synthesis request, obtaining information website information and material information corresponding to the video synthesis request;

[0007] Acquire the information text according to the information website address information, and determine the text title and text summary according to the information text;

[0008] Generate news voice information and news subtitle information according to the text summary;

[0009] Generate a target information video according to the information voice information, the information subtitle information, the text title, and the material information.

[0010] In a second aspect, an embodiment of the present application provides an information video synthesis device, including a material acquisition module, a text analysis module, a summary processing module and a video synthesis module, wherein:

[0011] The material acquisition module is configured to, upon receiving a video synthesis request, acquire information website information and material information corresponding to the video synthesis request;

[0012] The text analysis module is configured to obtain the information text according to the information website address information, and determine the text title and text summary according to the information text;

[0013] The summary processing module is configured to generate news voice information and news subtitle information according to the text summary;

[0014] The video synthesis module is configured to generate a target information video based on the information voice information, the information subtitle information, the text title, and the material information.

[0015] In a third aspect, an embodiment of the present application provides an information video synthesis device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the information video synthesis method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the information video synthesis method as described in the first aspect.

[0019] In the fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the information video synthesis method described in the first aspect.

[0020] The embodiment of the present application obtains information on the website address and material information corresponding to a video synthesis request, obtains information text according to the information on the website address, determines a text title and a text summary according to the information text, generates information voice information and information subtitle information according to the text summary, and generates a target information video according to the information voice information, information subtitle information, text title and material information. The target information video is automatically generated according to the information text and material information corresponding to the information on the website address provided by the user, without the need for the user to manually edit the material, thereby reducing the requirement on the user's video editing ability and effectively improving the video generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of an information video synthesis method provided by an embodiment of the present application;

[0022] Figure 2is a flow chart of another information video synthesis method provided by an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of a flow chart of information video frame generation provided by an embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of displaying material information provided by an embodiment of the present application;

[0025] Figure 5 It is a display schematic diagram of an information video frame provided by an embodiment of the present application;

[0026] Figure 6 It is a structural schematic diagram of an information video synthesis device provided in an embodiment of the present application;

[0027] Figure 7 It is a structural schematic diagram of an information video synthesis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0029] The information video synthesis method provided in this application can be applied to the synthesis of information videos such as news videos, promotional videos, and educational videos. It supports users to quickly generate customized information videos. It aims to automatically generate target information videos based on the information text and material information corresponding to the information URL information provided by the user. Users do not need to manually edit the materials, which reduces the requirements for users' video editing capabilities and improves video generation efficiency.

[0030] In the existing video synthesis scheme, it is generally to obtain the preset screen layout configuration information, automatically adjust the display size and position of each area in the display container, load multiple local videos and real-time video according to the configuration information to the corresponding area, and render to generate playback files. These methods are suitable for special effects editing of multiple screens displayed in the same display container. The video playback effect on mobile terminals is poor, and users are required to have strong video editing capabilities, which requires high professionalism from users and has low video generation efficiency. Based on this, an information video synthesis method of an embodiment of the present application is provided to solve the technical problem that the existing video synthesis scheme has high professionalism requirements for users and low video generation efficiency.

[0031] Figure 1 A flowchart of an information video synthesis method provided in an embodiment of the present application is given. The information video synthesis method provided in an embodiment of the present application can be executed by an information video synthesis device, which can be implemented by hardware and / or software and integrated in an information video synthesis device (for example, a server that provides background services for online news platforms and social media).

[0032] The following description is made by taking an information video synthesis device executing an information video synthesis method as an example. Figure 1 , the information video synthesis method comprises:

[0033] S110: When a video synthesis request is received, information website information and material information corresponding to the video synthesis request are obtained.

[0034] In one embodiment, when it is necessary to synthesize an information video, a video synthesis request can be generated according to one or more consulting website information (such as a news website, a publicity consulting website, an education consulting website, etc., which record URLs of relevant information) and material information (the material information may include material video, material pictures, etc.) corresponding to the synthesized information video. For example, a user sends a video synthesis request to an information video synthesis device by writing an HTTP POST request and sending it to the information video synthesis device. The HTTP POST request may include consulting website information and material information. Optionally, the material information may be compressed and recorded in a video synthesis request by setting an encoding format (such as base64 encoding format). Optionally, the video synthesis request may also represent the material information by the storage address or identification information of the material information, and the material information may be obtained from a preset material information storage location according to the storage address or identification information. Optionally, the material information may be stored on a user terminal or in a server.

[0035] Exemplarily, when a video synthesis request is received, the video synthesis request is parsed to obtain the consulting website information and the material information. Optionally, when the material information is represented by a storage address or identification information, the material information can be obtained from a preset material information storage location according to the storage address or identification information.

[0036] S120: Acquire information text according to the information website address information, and determine a text title and a text summary according to the information text.

[0037] For example, after obtaining the information website information and material information corresponding to the video synthesis request, the information text corresponding to the consultation website information is obtained, and the text title and text summary (text summary) are determined according to the information text. The text title can be a concise sentence indicating the main content of the information text, such as a news title, a propaganda title, an education title, etc., and the text summary can be a short article that concisely and accurately describes the main content of the information text.

[0038] In one embodiment, the text title and text summary corresponding to the information text can be obtained based on natural language processing technology. For example, the information text is input into a trained natural language processing model (such as a large language model LLM), and the natural language processing model outputs the text title and text summary corresponding to the information text.

[0039] Optionally, a request library may be used to send an HTTP request (requests request) to obtain the information text, and a parsing library (such as the beautifulsoup library) may be used to parse the web page content to extract the information text. For example, through an HTTP GET request, a request library may be used to send a web page content acquisition request to the URL corresponding to the information website information. After receiving the web page content, the parsing library may be used to parse the web page content and extract the text content in the web page content as the information text.

[0040] S130: Generate information voice information and information subtitle information according to the text summary.

[0041] Exemplarily, after determining the text summary corresponding to the information text, the information voice information and the information subtitle information can be generated according to the text summary. Optionally, the text summary can be input into a trained language model (such as a TTS model), and the language model analyzes and processes the text summary and outputs the information voice information corresponding to the text summary. Optionally, information subtitle information in a preset subtitle text format (such as SRT, SSA, ASS, SUB, VTT, TXT, etc.) can be generated according to the text summary.

[0042] In one embodiment, a language model for processing texts in different languages ​​may be configured, and the corresponding language model may be determined according to the language corresponding to the text summary, and the text summary may be analyzed and processed by the language model of the corresponding language and the information voice information corresponding to the text summary may be output. Optionally, the language of the information voice information output by the language model may be consistent with the language corresponding to the text summary, or may be a preset language.

[0043] This solution inputs the information text into the natural language processing model, which analyzes and processes the information text and outputs the text title and text summary corresponding to the information text. By refining the information content through natural language processing technology and combining speech synthesis with subtitle synchronization, it can effectively ensure the accurate transmission of information, make the video content more refined, and improve the audience's viewing experience and information acquisition efficiency.

[0044] S140: Generate a target information video according to the information voice information, information subtitle information, text title, and material information.

[0045] Exemplarily, a target information video is generated based on the information voice information, information subtitle information, text title and material information determined above. For example, a preliminary video is generated based on the material information (for example, a video with a duration consistent with the information voice information is generated based on the material picture and / or material video), and the information voice information, information subtitle information and text title are added to the preliminary video to obtain the target information video (the information subtitle information and text title can be added to the set position of the video), and the target information video is returned to the user.

[0046] Optionally, when returning the target information video to the user, the video file corresponding to the target information video may be returned to the user, or the storage address or identification information corresponding to the target information video may be returned to the user). It is understandable that when playing the target information video, the information voice information is played synchronously, and the information subtitle information and text title are displayed synchronously in the video playback screen.

[0047] In the above, by obtaining the news website address information and material information corresponding to the video synthesis request, obtaining the news text according to the news website address information, and determining the text title and text summary according to the news text, generating news voice information and news subtitle information according to the text summary, and generating the target news video according to the news voice information, news subtitle information, text title and material information, the target news video is automatically generated according to the news text and material information corresponding to the news website address information provided by the user, without the need for the user to manually edit the material, thereby reducing the requirement on the user's video editing ability and effectively improving the video generation efficiency.

[0048] Based on the above embodiments, Figure 2A flowchart of another information video synthesis method provided in an embodiment of the present application is given, and the information video synthesis method is a specific embodiment of the above-mentioned information video synthesis method. Figure 2 , the information video synthesis method comprises:

[0049] S210: When a video synthesis request is received, obtain information on the website address and material information corresponding to the video synthesis request.

[0050] S220: Acquire information text according to the information website address information, and determine a text title and a text summary according to the information text.

[0051] S230: Segment the text summary according to sentence separators in the text summary to obtain a plurality of sentence texts.

[0052] S240: Generate multiple sentence voice information based on the multiple sentence texts, and generate information voice information based on the multiple sentence voice information.

[0053] S250: Generate information subtitle information according to the multiple sentence texts based on the speech lengths of the multiple sentence speech information.

[0054] Exemplarily, after obtaining the text summary, the sentence separators in the text summary are determined, and the text summary is processed into sentences according to the sentence separators to obtain multiple sentence texts corresponding to the text summary. For example, the sentence separator can be a comma "," period ".", question mark "?", semicolon ";", exclamation mark "!", etc. For example, for the text summary "The setting sun slowly sinks against the western hills, and the surging Yellow River rushes toward the East China Sea. If you want to see enough of the scenery of a thousand miles, you have to climb to a higher level of the city tower." The text summary can be divided into four sentence texts: "The setting sun slowly sinks against the western hills," "The surging Yellow River rushes toward the East China Sea," "If you want to see enough of the scenery of a thousand miles," and "You have to climb to a higher level of the city tower."

[0055] Optionally, different combinations of sentence separators may be preset according to different languages. When the text summary needs to be segmented according to the sentence separators in the text summary, the language corresponding to the text summary may be determined (for example, the language of the text summary may be determined using a preset language analysis tool), and the corresponding combination of sentence separators may be determined according to the language, and then the text summary may be segmented according to the combination of sentence separators of the corresponding language.

[0056] In one embodiment, after obtaining a plurality of sentence texts, a plurality of sentence voice information corresponding to the plurality of sentence texts is generated using a voice model, and the plurality of sentence voice information is sequentially spliced ​​into information voice information. After obtaining a plurality of sentence voice information, information subtitle information is generated according to the voice length of each sentence voice information and the sentence text corresponding to each sentence voice information.

[0057] For example, the information subtitle information records the texts and subtitle timestamps corresponding to multiple sentence texts recorded in sequence, wherein the subtitle timestamp corresponding to the sentence text is the cumulative voice length of the sentence voice information corresponding to the sentence text before the sentence text, so that when the voice corresponding to the sentence voice information is played, the text corresponding to the corresponding sentence text is displayed on the screen.

[0058] This scheme obtains multiple sentence texts by sentence-by-sentence segmentation according to sentence separators in the text summary, generates sentence voice information from each sentence text, generates information voice information based on multiple sentence voice information, and generates information subtitle information based on the voice length of each sentence voice information and each sentence text, thereby ensuring that the information subtitle information can be correctly synchronized with the information voice information and improving the quality of information video synthesis.

[0059] S260: Generate an information video frame according to the material information.

[0060] Exemplarily, an information video is generated based on the material information obtained above. For example, when an information video includes one or more material pictures, multiple information video frames can be generated based on some or all of the material pictures; when an information video includes one or more material videos, multiple information video frames can be generated based on some or all of the material video frames of the material videos; when an information video includes one or more material pictures and one or more material videos, multiple information video frames can be generated based on some or all of the material pictures and / or some or all of the material video frames of the material videos.

[0061] In one possible embodiment, Figure 3 As shown in the schematic diagram of a flow chart of information video frame generation, the information video synthesis method provided by the present solution generates information video frames according to material information, which may include:

[0062] S261: scaling the material information according to a preset target size to obtain a background frame covering the target size.

[0063] S262: scaling the material information according to the target size to obtain a foreground frame retaining complete content.

[0064] S263: Synthesize the foreground frame and the background frame into an information video frame.

[0065] Exemplarily, the material information obtained above is scaled according to a preset target size to obtain a background frame covering the target size. The target size may be the size corresponding to the synthesized target information video. For example, the picture or video frame corresponding to the material information is scaled proportionally according to the preset target size, so that the top or bottom of the scaled picture or video frame is greater than or equal to the top or bottom corresponding to the target size, and the left or right side of the scaled picture or video frame is greater than or equal to the left or right side corresponding to the target size. At this time, the target size is surrounded by the size of the scaled picture or video frame, and the scaled picture or video frame can cover the target size. The scaled picture or video frame can be cropped according to the target size to obtain the background frame corresponding to the picture or video frame.

[0066] In one embodiment, the material information provided by the present solution may be material pictures and / or material videos. When the material information includes material pictures, each material picture is scaled according to a preset target size to obtain a background frame covering the target size. When the material information includes material videos, each material video frame of the material video is scaled according to a preset target size to obtain a background frame covering the target size.

[0067] In a possible embodiment, the information video synthesis method provided by the present solution can further perform blur processing (such as Gaussian blur processing) on ​​the background frame after scaling the material information according to the preset target size to obtain the background frame covering the target size. By performing blur processing on the background frame, the present solution can effectively reduce the interference of the picture information of the background frame on the picture information of the foreground frame, thereby improving the quality of video generation.

[0068] In one embodiment, the acquired material information is scaled according to the target size to obtain a foreground frame that retains the complete content. For example, the picture or video frame corresponding to the material information is scaled proportionally according to the target size, so that the top or bottom of the scaled picture or video frame is less than or equal to the top or bottom corresponding to the target size, and the left or right side of the scaled picture or video frame is less than or equal to the left or right side corresponding to the target size. At this time, the size corresponding to the scaled picture or video frame is surrounded by the target size, the scaled picture or video frame can retain the complete content, and the scaled picture or video frame can be used as the corresponding foreground frame.

[0069] In one embodiment, the material information provided by the present solution may be material pictures and / or material videos. When the material information includes material pictures, each material picture is scaled according to a preset target size to obtain a foreground frame retaining the complete content. When the material information includes material videos, each material video frame of the material video is scaled according to a preset target size to obtain a foreground frame retaining the complete content.

[0070] After obtaining the background frame and the foreground frame, the background frame and the foreground frame corresponding to the same material information are synthesized into an information video frame. For example, the foreground frame is added to the background frame to obtain the information video frame. At this time, the foreground frame will block all or part of the content of the background frame. This solution scales the material information according to the target size to obtain a background frame covering the target size and a foreground frame retaining the complete content, and synthesizes the foreground frame and the background frame into an information video frame. While ensuring the consistency of the information video frame size, it ensures the accurate transmission of the material information picture content, thereby improving the quality of information video generation.

[0071] like Figure 4 A schematic diagram of displaying material information provided, and Figure 5 As shown in the schematic diagram of displaying a news video frame provided, in the case of Figure 4 The material information shown in FIG. 1 is scaled to obtain a background frame (the background frame is blurred) and a foreground frame, and the foreground frame is superimposed on the background frame to obtain the following: Figure 5 In the information video frame shown, it can be seen that the picture content of the foreground frame is superimposed on the corresponding position of the background frame, and the background frame has been blurred so as not to interfere with the content of the foreground frame, thereby ensuring the accurate transmission of the picture content of the foreground frame.

[0072] S270: Add a timestamp to the news video frame.

[0073] In one embodiment, the display duration of each information video frame is determined, and a timestamp is added to the information video frame according to the display duration of each information video frame, wherein the timestamp of each information video frame is the sum of the display durations corresponding to the previous information video frames.

[0074] In a possible embodiment, the material information provided by the present solution includes material pictures and / or material videos. Correspondingly, the information video synthesis method provided by the present solution adds a timestamp to the information video frame, including:

[0075] In the case where the information video frame is obtained based on the material picture, the target video frame is determined from the information video frame according to the number of pictures of the material picture and the picture number threshold, and a timestamp is added to each target video frame according to the voice duration of the information voice information and the number of target video frames. The picture number threshold is determined according to the voice duration of the information voice information and the preset minimum display duration of the picture;

[0076] When the information video frame is obtained based on the material video, the target video frame is determined from multiple information video frames according to the voice duration of the information voice information, and a timestamp is added to the target video frame.

[0077] Exemplarily, when the information video frame is obtained based on material pictures (that is, the only material information provided is material pictures), the picture quantity threshold can be determined based on the voice duration of the information voice information and the preset minimum picture display duration.

[0078] The picture quantity threshold can be understood as the maximum number of pictures that can be displayed under the constraints of the voice duration of the information voice information and the minimum display duration of the picture. Optionally, the ratio of the voice duration of the information voice information and the preset minimum display duration of the picture can be determined, and the result of rounding down the ratio is determined as the picture quantity threshold. For example, the picture quantity threshold can be expressed as p_max=t_total / / t_pmin, where t_total is the voice duration of the information voice information, and t_pmin is the minimum display duration of the picture.

[0079] In one embodiment, the target video frame is determined from the news video frame according to the number of material pictures and the picture number threshold, for example, the minimum value of the picture number (the number of material pictures obtained) and the picture number threshold is used as the actual number of pictures displayed (p = min (p_input, p_max), p_input is the number of pictures). When the picture number threshold is used as the actual number of pictures displayed, the corresponding number of news video frames can be determined as the target video frame from multiple news video frames in sequence, at set intervals or randomly.

[0080] In one embodiment, after determining a plurality of target video frames, a timestamp is added to each target video frame according to the voice duration of the information voice information and the number of target video frames. For example, the ratio of the voice duration of the information voice information to the number of target video frames is determined, and the ratio is used as the frame playback duration of each target video frame (t=t_total / p). The timestamp of each target video frame is determined according to the frame playback duration. For example, the timestamp corresponding to the first target video frame is 0:0:0, and the timestamp of subsequent target video frames is the sum of the frame playback durations of the previous target video frames. This solution determines the target video frame from the information video frame according to the number of material pictures and the picture number threshold when the material information provided only includes material pictures, thereby ensuring that the number of target video frames is within the supported number threshold, ensuring the consistency of the target information video with the information voice information, and ensuring the quality of information video generation.

[0081] Exemplarily, when the information video frame is obtained based on the material video (that is, the only material information provided is the material video), the target video frame can be determined from multiple information video frames according to the voice length of the information voice information, and a timestamp can be added to the target video frame.

[0082] For example, when the video duration of the material video is greater than the voice duration of the information voice information, some of the subsequent information video frames are discarded so that the total duration of the retained information video frames is not greater than the voice duration of the information voice information, and the retained information video frames are used as target video frames; when the video duration of the material video is less than the voice duration of the information voice information, some of the information video frames can be copied and added to the original multiple information video frames so that the total duration of the multiple information video frames is consistent with the voice duration of the information voice information.

[0083] In one embodiment, after determining a plurality of target video frames, a timestamp is added to each target video frame according to the voice duration of the information voice information and the number of target video frames. For example, the ratio of the voice duration of the information voice information to the number of target video frames is determined, and the ratio is used as the frame playback duration of each target video frame (t=t_total / p), or the playback duration of each information video frame in the information video frame is used as the frame playback duration of each target video frame. After determining the frame playback duration of each target video frame, the timestamp of each target video frame can be determined according to the frame playback duration, for example, the timestamp corresponding to the first target video frame is 0:0:0, and the timestamp of subsequent target video frames is the sum of the frame playback durations of the previous target video frames. When the material information provided by this solution is only the material video, the target video frame is determined from the plurality of information video frames according to the voice duration of the information voice information, so as to ensure that the number of target video frames is within the supported quantity threshold, ensure the consistency of the target information video with the information voice information, and ensure the quality of the information video generation.

[0084] In a possible embodiment, the information video synthesis method provided by the present solution adds a timestamp to the information video frame, including:

[0085] When the information video frame is obtained based on the material picture and the material video, and the total length of the minimum picture display time of the material picture and the video length of the material video is less than or equal to the voice length of the information voice information, a first remaining time is determined according to the voice length of the information voice information and the video length of the material video, and the picture display time of the information video frame corresponding to the material picture is determined according to the first remaining time, and a timestamp is added to each target video frame according to the video length of the material video and the picture display time;

[0086] When the information video frame is obtained based on the material picture and the material video, and the total duration of the minimum picture display duration of the material picture and the video duration of the material video is greater than the voice duration of the information voice information, the target video frame is determined from multiple information video frames according to the total minimum picture display duration of the material picture and the voice duration of the information voice information, and a timestamp is added to the target video frame.

[0087] Exemplarily, when the news video frame is obtained based on the material picture and the material video (i.e., the provided material information includes the material picture and the material video), the total minimum display time of the material picture and the total video time of the material video and the voice time of the news voice information are compared. Among them, the minimum display time of the picture is the product of the number of pictures corresponding to the material picture and the minimum display time of the picture (t_p_total=p_input*t_pmin).

[0088] In one embodiment, when the total minimum display time of the material pictures and the video time of the material video is less than or equal to the voice time of the information voice information (t_p_total+t_video≤t_total, where t_video is the video time of the material video), the material pictures and the information video frames corresponding to the material video can be used as target video frames, and all the material pictures can be played during the remaining time of the complete playback of the material video.

[0089] Exemplarily, the first remaining duration can be determined based on the voice duration of the information voice information and the video duration of the material video (for example, the difference between the voice duration of the information voice information and the video duration of the material video is determined, and the difference is used as the first remaining duration).

[0090] In one embodiment, after determining the first remaining time, the picture display time of the information video frame corresponding to the material picture can be determined according to the first remaining time (for example, the ratio of the first remaining time to the number of material pictures is used as the picture display time of the information video frame corresponding to each material picture, that is, the picture display time t_image = t_remain / p_input, and the first remaining time t_remain = t_total-t_video).

[0091] After determining the picture display duration of the information video frame corresponding to the material picture, a timestamp can be added to each target video frame according to the video duration of the material video and the picture display duration. For example, the ratio of the video duration of the material video to the number of target video frames can be determined, and the ratio can be used as the frame display duration of the target video frame corresponding to each material video. Alternatively, the playback duration of each information video frame in the information video frame can be used as the frame display duration of the target video frame. After determining the frame display duration of the target video frame, the timestamp of each target video frame can be determined according to the frame display duration or picture display duration corresponding to each target video frame.

[0092] For example, the timestamp corresponding to the first target video frame (which can be the target video frame corresponding to the material video or the target video frame corresponding to the material picture) is 0:0:0, and the timestamp of the subsequent target video frames is the sum of the frame display duration and / or picture display duration of the previous target video frame. Optionally, the target video frame corresponding to the material video can be arranged before or after the target video frame corresponding to the material picture, or can be arranged between the target video frames corresponding to the material picture. In this scheme, when the material information provided includes material pictures and material videos, and the sum of the minimum display duration of the picture and the video duration of the material video is less than or equal to the voice duration, the target video frame is determined from multiple information video frames according to the voice duration of the information voice information, to ensure that the number of target video frames matches the number of video frames supported by the voice duration, to ensure the consistency of the target information video with the information voice information, and to ensure the quality of information video generation.

[0093] In one embodiment, when the total duration of the minimum picture display time of the material picture and the video duration of the material video is greater than the voice duration of the information voice information (t_p_total+t_video>t_total), the product of the number of pictures corresponding to the material picture and the minimum picture display time can be determined, and the product result can be used as the total minimum picture display time of the material picture, and the target video frame is determined from multiple information video frames according to the minimum picture display time of the material picture and the voice duration of the information voice information, and a timestamp is added to the target video frame.

[0094] For example, the information video frames corresponding to some or all of the material images are used as target video frames, and the remaining time of the material video available for playback is determined according to the voice duration of the information voice information and the shortest display duration of the target video frames corresponding to the material images. The target video frames are determined from the information video frames corresponding to the material video according to the remaining time. For example, when the remaining time is less than the playback time of one frame of the material video, the target video frames are no longer determined from the information video frames corresponding to the material video. At this time, the target video frames only include the information video frames corresponding to the material images; when the remaining time is greater than or equal to the playback time of one frame of the material video, the target video frames are determined from the information video frames corresponding to the material video. At this time, the target video frames include the material images and the information video frames corresponding to the material video. After determining multiple target video frames, add timestamps to the target video frames according to the playback time corresponding to each target video frame. When the material information provided includes material pictures and material videos, and the sum of the minimum display time of the pictures and the video time of the material videos is greater than the voice time, this solution determines the target video frames from multiple information video frames according to the total minimum display time of the material pictures and the voice time of the information voice information, thereby ensuring that the number of target video frames matches the number of video frames supported by the voice time, ensuring the consistency of the target information video with the information voice information, and ensuring the quality of information video generation.

[0095] In a possible embodiment, the information video synthesis method provided by this solution determines target video frames from multiple information video frames according to the minimum total display duration of the material pictures and the voice duration of the information voice information, and adds timestamps to the target video frames, including:

[0096] When the minimum total display duration of the material pictures is less than the voice duration of the information voice information, determine the second remaining duration according to the preset minimum picture display duration and the voice duration of the information voice information, determine the target video frames from multiple information video frames according to the second remaining duration, and add timestamps to the target video frames;

[0097] When the minimum total display duration of the material pictures is greater than or equal to the voice duration of the information voice information, determine the target video frames from the information video frames according to the number of material pictures and the picture number threshold, and add timestamps to each target video frame according to the voice duration of the information voice information and the number of target video frames.

[0098] Exemplarily, compare the minimum total display duration of the material pictures and the voice duration of the information voice information, and determine whether to use the information video frames corresponding to the material video as the target video frames according to the comparison result.

[0099] In an embodiment, when the minimum total display duration of the material pictures is less than the voice duration of the information voice information (t_p_total < t_total), the information video frames corresponding to the material pictures can be used as the target video frames, and determine the second remaining duration according to the preset minimum picture display duration and the voice duration of the information voice information (for example, use the difference between the voice duration of the information voice information and the product of the number of material pictures and the minimum picture display duration as the second remaining duration), and determine the target video frames from multiple information video frames according to the second remaining duration.

[0100] For example, all the information video frames corresponding to the material pictures are taken as target video frames, and the ratio of the second remaining duration and the frame playback duration of each video frame in the material video is determined, and the ratio is rounded down to obtain the target number of target video frames determined from the information video frames corresponding to the material pictures, and the information video frames with the target number of frames are determined from the information video frames corresponding to the material pictures as the target video frames, for example, the information video frames with the front target number of frames in the information video frames corresponding to the material pictures are taken as the target video frames. At this time, the target video frames include the material images and the information video frames corresponding to the material videos. After determining the target video frames, timestamps are added to the target video frames according to the playback duration corresponding to each target video frame, wherein the playback duration of the target video frames corresponding to the material pictures is the minimum display duration of the pictures, and the playback duration of the target video frames corresponding to the material videos is the frame playback duration of each video frame in the material video. This solution determines the second remaining time according to the preset minimum image display time and the voice duration of the information voice information when the minimum total display time of the material image is less than the voice duration of the information voice information, and determines the target video frame from multiple information video frames according to the second remaining time, so as to ensure that the number of target video frames matches the number of video frames supported by the voice duration, ensure the consistency of the target information video and the information voice information, and ensure the quality of information video generation.

[0101] In one possible embodiment, when the minimum total display time of the material pictures is greater than or equal to the voice duration of the information voice information (t_p_total≥t_total), the target video frame is determined from the information video frame according to the number of pictures of the material pictures and the picture number threshold, and each target video frame is timestamped according to the voice duration of the information voice information and the number of target video frames, that is, the target video frame is determined based on the situation in which the information video frame is obtained based on the material pictures and the timestamp is added to each target video frame, which will not be described in detail in this scheme. This scheme determines the target video frame from the information video frame according to the number of pictures of the material pictures and the picture number threshold when the minimum total display time of the material pictures is greater than or equal to the voice duration of the information voice information, thereby ensuring that the number of target video frames matches the number of video frames supported by the voice duration, ensuring the consistency of the target information video with the information voice information, and ensuring the quality of information video generation.

[0102] S280: Generate a target information video according to a timestamp, information video frame, information voice information, information subtitle information and text title.

[0103] Exemplarily, the above-determined timestamp, information video frame, information voice information, information subtitle information and text title are input into a preset video generation tool (such as the MoviePy library), and the timestamp, information video frame, information voice information, information subtitle information and text title are integrated through the video generation tool to obtain the target information video.

[0104] In one embodiment, information video frames can be inserted sequentially in ascending order according to the timestamps corresponding to the information video frames, a blank video frame (the previous information video frame or an inserted frame generated based on the previous and next information video frames) can be inserted between two information video frames according to a preset video frame rate, and text titles and text corresponding to the information subtitle information are added to the preset title display position and subtitle display position to obtain a preliminary information video, and information voice information is added to the preliminary information video to obtain a target information video.

[0105] This solution generates news video frames based on material information, and adds timestamps to the news video frames. It can then generate target news videos based on timestamps, news video frames, news voice information, news subtitle information, and text titles, thereby integrating material information, news voice information, text titles, and news subtitle information. This improves the efficiency of news video generation, realizes real-time publishing of news content, shortens the process from news publishing to video generation, can quickly respond to hot events, reduces dependence on video producers, and reduces video production costs.

[0106] In the above, by obtaining the information website address information and material information corresponding to the video synthesis request, obtaining the information text according to the information website address information, and determining the text title and text summary according to the information text, generating the information voice information and information subtitle information according to the text summary, and generating the target information video according to the information voice information, information subtitle information, text title and material information, the target information video is automatically generated according to the information text and material information corresponding to the information website address information provided by the user, without the need for the user to manually edit the material, reducing the video editing ability requirements for the user, and effectively improving the video generation efficiency. At the same time, by dividing the text summary into sentences according to the sentence separators in the text summary to obtain multiple sentence texts, generating sentence voice information from the sentence texts respectively, generating information voice information according to the multiple sentence voice information, and generating information subtitle information according to the voice length of each sentence voice information and each sentence text, it is ensured that the information subtitle information can be correctly synchronized with the information voice information, thereby improving the quality of information video synthesis. And according to the material information, news video frames are generated, and timestamps are added to the news video frames. The target news video can be generated according to the timestamp, news video frames, news voice information, news subtitle information and text title, thereby realizing the integration of material information, news voice information, text title and news subtitle information, improving the generation efficiency of news videos, realizing the real-time release of news content, shortening the process from news release to video generation, being able to quickly respond to hot events, reducing dependence on video producers, and reducing video production costs.

[0107] Figure 6 Schematic diagram of the structure of an information video synthesis device provided by an embodiment of the present application. Figure 6 The information video synthesis device includes a material acquisition module 61, a text analysis module 62, a summary processing module 63 and a video synthesis module 64.

[0108] Among them, the material acquisition module 61 is configured to obtain the information website information and material information corresponding to the video synthesis request when receiving the video synthesis request; the text analysis module 62 is configured to obtain the information text according to the information website information, and determine the text title and text summary according to the information text; the summary processing module 63 is configured to generate information voice information and information subtitle information according to the text summary; the video synthesis module 64 is configured to generate the target information video according to the information voice information, information subtitle information, text title, and material information.

[0109] In the above, by obtaining the news website address information and material information corresponding to the video synthesis request, obtaining the news text according to the news website address information, and determining the text title and text summary according to the news text, generating news voice information and news subtitle information according to the text summary, and generating the target news video according to the news voice information, news subtitle information, text title and material information, the target news video is automatically generated according to the news text and material information corresponding to the news website address information provided by the user, without the need for the user to manually edit the material, thereby reducing the requirement on the user's video editing ability and effectively improving the video generation efficiency.

[0110] In a possible embodiment, the summary processing module 63 generates news voice information and news subtitle information according to the text summary, and is configured as follows:

[0111] Segment the text summary according to the sentence separators in the text summary to obtain multiple sentence texts;

[0112] Generate multiple sentence voice information based on multiple sentence texts, and generate information voice information based on multiple sentence voice information;

[0113] Based on the speech length of multiple sentence speech information, information subtitle information is generated according to multiple sentence texts.

[0114] In a possible embodiment, the video synthesis module 64 generates a target news video according to the news voice information, news subtitle information, text title, and material information, and is configured as follows:

[0115] Generate news video frames based on material information;

[0116] Add timestamps to news video frames;

[0117] Generate a target news video based on timestamp, news video frame, news voice information, news subtitle information and text title.

[0118] In a possible embodiment, the video synthesis module 64 generates an information video frame according to the material information, and is configured as follows:

[0119] Scaling the material information according to a preset target size to obtain a background frame covering the target size;

[0120] The material information is scaled according to the target size to obtain a foreground frame that retains the complete content;

[0121] The foreground frame and the background frame are combined into an information video frame.

[0122] In a possible embodiment, the video synthesis module 64 is further configured to perform blur processing on the background frame after scaling the material information according to a preset target size to obtain a background frame covering the target size.

[0123] In a possible embodiment, the material information includes material pictures and / or material videos, and the video synthesis module 64 adds a timestamp to the news video frame, and is configured as follows:

[0124] In the case where the information video frame is obtained based on the material picture, the target video frame is determined from the information video frame according to the number of pictures of the material picture and the picture number threshold, and a timestamp is added to each target video frame according to the voice duration of the information voice information and the number of target video frames. The picture number threshold is determined according to the voice duration of the information voice information and the preset minimum display duration of the picture;

[0125] When the information video frame is obtained based on the material video, the target video frame is determined from multiple information video frames according to the voice duration of the information voice information, and a timestamp is added to the target video frame.

[0126] In a possible embodiment, the material information includes material pictures and / or material videos, and the video synthesis module 64 adds a timestamp to the news video frame, and is configured as follows:

[0127] When the information video frame is obtained based on the material picture and the material video, and the total length of the minimum picture display time of the material picture and the video length of the material video is less than or equal to the voice length of the information voice information, a first remaining time is determined according to the voice length of the information voice information and the video length of the material video, and the picture display time of the information video frame corresponding to the material picture is determined according to the first remaining time, and a timestamp is added to each target video frame according to the video length of the material video and the picture display time;

[0128] When the information video frame is obtained based on the material picture and the material video, and the total duration of the minimum picture display duration of the material picture and the video duration of the material video is greater than the voice duration of the information voice information, the target video frame is determined from multiple information video frames according to the total minimum picture display duration of the material picture and the voice duration of the information voice information, and a timestamp is added to the target video frame.

[0129] In a possible embodiment, the video synthesis module 64 determines a target video frame from a plurality of news video frames according to the minimum total display duration of the material picture and the voice duration of the news voice information, and adds a timestamp to the target video frame, and is configured as follows:

[0130] When the minimum total display time of the material image is less than the voice time of the information voice information, a second remaining time is determined according to the preset minimum display time of the image and the voice time of the information voice information, and a target video frame is determined from multiple information video frames according to the second remaining time;

[0131] When the minimum total display time of the material pictures is greater than or equal to the voice duration of the information voice information, the target video frames are determined from the information video frames according to the number of pictures of the material pictures and the picture number threshold, and timestamps are added to each target video frame according to the voice duration of the information voice information and the number of target video frames.

[0132] It is worth noting that in the embodiment of the above-mentioned information video synthesis device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0133] The embodiment of the present application also provides an information video synthesis device, which can integrate the information video synthesis device provided in the embodiment of the present application. Figure 7 Schematic diagram of the structure of an information video synthesis device provided by an embodiment of the present application. Figure 7 The information video synthesis device includes: an input device 73, an output device 74, a memory 72 and one or more processors 71; the memory 72 is used to store one or more programs; when one or more programs are executed by one or more processors 71, the one or more processors 71 implement the information video synthesis method provided in the above embodiment. The above-provided information video synthesis device, equipment and computer can be used to execute the information video synthesis method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0134] The embodiments of the present application also provide a non-volatile storage medium storing computer executable instructions, which are used to execute the information video synthesis method provided in the above embodiments when executed by a computer processor. Of course, the non-volatile storage medium storing computer executable instructions provided in the embodiments of the present application, whose computer executable instructions are not limited to the information video synthesis method provided above, can also execute the related operations in the information video synthesis method provided in any embodiment of the present application. The information video synthesis device, equipment and storage medium provided in the above embodiments can execute the information video synthesis method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the information video synthesis method provided in any embodiment of the present application.

[0135] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, including a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the information video synthesis method provided in each embodiment of the present application.

Claims

1. A method for synthesizing information video, characterized in that: include: When receiving a video synthesis request, obtaining information website information and material information corresponding to the video synthesis request; Acquire the information text according to the information website address information, and determine the text title and text summary according to the information text; Generate news voice information and news subtitle information according to the text summary; Generate a target information video according to the information voice information, the information subtitle information, the text title, and the material information.

2. The information video synthesis method according to claim 1, characterized in that: The generating of information voice information and information subtitle information according to the text summary includes: Segment the text summary according to sentence separators in the text summary to obtain multiple sentence texts; Generate multiple sentence voice information according to the multiple sentence texts, and generate information voice information according to the multiple sentence voice information; Based on the speech lengths of the plurality of sentence speech information, information subtitle information is generated according to the plurality of sentence texts.

3. The information video synthesis method according to claim 1, characterized in that: The generating of the target information video according to the information voice information, the information subtitle information, the text title, and the material information includes: Generate an information video frame according to the material information; Adding a timestamp to the news video frame; Generate a target information video according to the timestamp, the information video frame, the information voice information, the information subtitle information and the text title.

4. The information video synthesis method according to claim 3, characterized in that: The step of generating an information video frame according to the material information comprises: Scaling the material information according to a preset target size to obtain a background frame covering the target size; Scaling the material information according to the target size to obtain a foreground frame retaining complete content; The foreground frame and the background frame are synthesized into an information video frame.

5. The information video synthesis method according to claim 4, characterized in that: After scaling the material information according to the preset target size to obtain a background frame covering the target size, the method further includes: The background frame is blurred.

6. The information video synthesis method according to claim 3, characterized in that: The material information includes material pictures and / or material videos, and the adding of timestamps to the news video frames includes: In the case where the information video frame is obtained based on the material picture, the target video frame is determined from the information video frame according to the number of pictures of the material picture and a picture number threshold, and a timestamp is added to each of the target video frames according to the voice duration of the information voice information and the number of the target video frames, and the picture number threshold is determined according to the voice duration of the information voice information and a preset minimum display duration of the picture; In the case where the information video frame is obtained based on the material video, a target video frame is determined from a plurality of the information video frames according to the voice duration of the information voice information, and a timestamp is added to the target video frame.

7. The information video synthesis method according to claim 3, characterized in that: The material information includes material pictures and / or material videos, and the adding of timestamps to the news video frames includes: In the case that the information video frame is obtained based on the material picture and the material video, and the total minimum display duration of the material picture and the video duration of the material video is less than or equal to the voice duration of the information voice information, a first remaining duration is determined according to the voice duration of the information voice information and the video duration of the material video, a picture display duration of the information video frame corresponding to the material picture is determined according to the first remaining duration, and a timestamp is added to each of the target video frames according to the video duration of the material video and the picture display duration; When the information video frame is obtained based on the material picture and the material video, and the total duration of the minimum picture display duration of the material picture and the video duration of the material video is greater than the voice duration of the information voice information, a target video frame is determined from multiple information video frames according to the total minimum picture display duration of the material picture and the voice duration of the information voice information, and a timestamp is added to the target video frame.

8. The information video synthesis method according to claim 7, characterized in that: The step of determining a target video frame from a plurality of the information video frames according to the minimum total display duration of the material picture and the voice duration of the information voice information, and adding a timestamp to the target video frame comprises: In the case where the minimum total display time of the material picture is less than the voice time of the information voice information, a second remaining time is determined according to the preset minimum display time of the picture and the voice time of the information voice information, and a target video frame is determined from the multiple information video frames according to the second remaining time; When the minimum total display time of the material pictures is greater than or equal to the voice time of the information voice information, the target video frames are determined from the information video frames according to the number of pictures of the material pictures and the picture number threshold, and timestamps are added to each of the target video frames according to the voice time of the information voice information and the number of the target video frames.

9. An information video synthesis device, characterized in that: It includes material acquisition module, text analysis module, summary processing module and video synthesis module, among which: The material acquisition module is configured to, upon receiving a video synthesis request, acquire information website information and material information corresponding to the video synthesis request; The text analysis module is configured to obtain the information text according to the information website address information, and determine the text title and text summary according to the information text; The summary processing module is configured to generate news voice information and news subtitle information according to the text summary; The video synthesis module is configured to generate a target information video based on the information voice information, the information subtitle information, the text title, and the material information.

10. An information video synthesis device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the information video synthesis method as described in any one of claims 1-8.

11. A non-volatile storage medium storing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the information video synthesis method as described in any one of claims 1-8.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the information video synthesis method described in any one of claims 1-8 is implemented.

Images