Financial news editing method based on large language model

Abstract

The invention relates to the technical field of news collection and editing, in particular to a financial news editing method based on a large language model, which comprises the following steps: processing original video content by using a hybrid expert model to obtain video segment information; splitting according to the video segmentation information to obtain segmented videos; acquiring news materials to be processed, inputting the news materials to the hybrid expert model, generating text editing contents according to the content abstract, and matching the segmented videos to form contents to be organized; and forming and outputting an edited manuscript according to the to-be-organized content. Aiming at the problem that a news auxiliary processing scheme in the prior art can only process text news contents, the method comprises the following steps of: firstly splitting original video contents to obtain segmented videos corresponding to different expression semantics, and inputting news materials of to-be-processed texts into a hybrid expert model on the basis, so as to extract content abstracts; and the segmented videos are matched and organized, so that the processing efficiency of the multimedia news is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of news editing and compiling, and in particular to a financial news editing method based on a large language model. Background Art

[0002] News editing is the process of identifying, collecting, processing and producing news information. The news editing process usually includes the corresponding radio and television material recording, editing, news release writing, broadcasting, and later directing and effect production. With the rise of online media, it usually also includes streaming video production and uploading, as well as the corresponding text news release production and typesetting process.

[0003] Financial news is a subcategory of news, focusing on collecting, reporting and publishing news in the financial field. Financial news can be divided into broad and narrow sense. Broad financial news, or pan-economic news, covers all social and economic life and economic-related fields, including production to consumption, cities to rural areas, macro to micro, safety production to service quality, and economic work to politics and related fields in social life. Narrow financial news focuses on various trading activities in the capital market. Since commercial activities themselves are extremely time-sensitive, the timeliness requirements for news collection and editing are also extremely high.

[0004] In order to improve the efficiency of news gathering and editing, some technical solutions for assisting news gathering and editing already exist in the prior art.

[0005] For example, Chinese patent CN201610154211.X discloses a method and device for automatically generating Internet news, which includes the following steps: using a distributed crawler to crawl the source code of a news page from a specific news website; extracting news information from the source code of each news page, and encapsulating each news information into structured news of the same structure; identifying the acquired structured news to obtain independent news events, each news event includes multiple structured news belonging to the same event, and each structured news in the same news event represents a different event stage; generating corresponding news summaries for each structured news in the news event; sorting multiple structured news in the news event based on the release time, and generating an event context structure with the news summaries corresponding to each structured news after sorting. The complete news event is presented with time as the main line, allowing users to easily understand the development process of the event.

[0006] For another example, Chinese patent CN202311353653.3 discloses an automatic news generation system based on intelligent writing. The system includes: a keyword entry unit, which is used to provide users with input keywords, and the keywords include at least: time, place and event keywords, and derive keywords for the input event keywords to obtain derived event keywords; a news verification part, which is used to search for news based on time, place and derived keywords to obtain target news, perform text content analysis on the target news, determine related keywords, and assign weights to each related keyword; a content generation part, which is used to generate content based on each related keyword and its corresponding weight, as well as time, place and derived event keywords. It realizes personalized, high-quality and efficient news report generation, and improves the quality and efficiency of information dissemination.

[0007] However, during the actual implementation process, the inventors found that this type of technical solution usually only focuses on the processing and auxiliary writing of news materials with single text content. With the popularity of network-type media, more press releases are issued in the form of pictures, texts, videos, and streaming media. The technical solutions in the prior art are not effective in processing this type of news materials. Summary of the invention

[0008] In view of the above problems existing in the prior art, a financial news editing method based on a large language model is provided.

[0009] The specific technical solutions are as follows:

[0010] A financial news editing method based on a large language model, comprising:

[0011] Step S1: processing the original video content using a hybrid expert model to obtain video segmentation information;

[0012] Step S2: splitting the original video content according to the video segmentation information to obtain segmented videos;

[0013] Step S3: obtaining the news material to be processed and inputting it into the hybrid expert model to obtain a content summary of the news material to be processed;

[0014] Step S4: Generate text editing content according to the content summary and match the segmented video to form content to be organized;

[0015] Step S5: forming an edited manuscript based on the content to be organized and outputting it.

[0016] On the other hand, the hybrid expert model comprises:

[0017] An initialization layer, wherein the initialization layer vectorizes the input information to form an initialization vector;

[0018] an attention layer, the attention layer being connected to the initialization layer;

[0019] The attention layer calculates the correlation for each of the initialization vectors and forms vector association information;

[0020] A first residual module, wherein a first input end of the first residual module is connected to the initialization layer, and a second input end of the first residual module is connected to an output end of the attention layer;

[0021] The first residual module superimposes the residual information of the initialization vector and the vector association information to form a vector to be processed;

[0022] A routing gateway, wherein an input end of the routing gateway is connected to an output end of the first residual module;

[0023] The output ends of the routing gateway are respectively connected to a plurality of expert models;

[0024] The routing gateway allocates the to-be-processed vector to the expert model based on a control gate mechanism;

[0025] A weighted fusion module, wherein the input end of the weighted fusion module is respectively connected to each of the expert models;

[0026] The weighted fusion module performs weighted fusion on the output results of the expert model according to the output probability of the expert model to obtain a hidden state tensor;

[0027] A second residual module, wherein a first input end of the second residual module is connected to an output end of the first residual module, and a second input end of the second residual module is connected to an output end of the weighted fusion module;

[0028] The second residual module forms a model output result after superimposing the residual information of the vector to be processed with the hidden state tensor and outputs it;

[0029] The model output result is the content summary or the video segmentation information.

[0030] On the other hand, before executing step S1, an expert model generation process is also included, and the expert model generation process includes:

[0031] Step A1: Collect news corpus corresponding to financial news and construct a financial data set;

[0032] Step A2: embed Chinese semantic units into the vocabulary of the Mixtral model to obtain a Chinese adjustment model;

[0033] Step A3: using the financial data set to train the Chinese adjustment model, and adjusting the Chinese adjustment model based on the LoRA fine-tuning technology during the training process to obtain a pre-trained model;

[0034] Step A4: Parameter fusion is performed on the pre-trained model and the Mixtral model to obtain the expert model.

[0035] On the other hand, the step S1 comprises:

[0036] Step S11: extracting video stream and audio stream from the original video content respectively;

[0037] Step S12: extracting a picture start marker from the video stream, and extracting an audio start marker from the audio stream;

[0038] Step S13: aligning the audio stream and the video stream according to the picture start mark and the audio start mark to obtain a processed video;

[0039] Step S14: inputting the audio stream into a speech transcription model to obtain text content;

[0040] Step S15: semantically identifying the text content and segmenting it to obtain segmented semantics, extracting picture frames from the video stream for each time interval of the audio stream corresponding to each paragraph of the segmented semantics, and assembling them with the segmented semantics to obtain multimodal input data;

[0041] Step S16: inputting the multimodal input data into the hybrid expert model for processing to obtain the video segmentation information.

[0042] On the other hand, the video segmentation information includes: content-intensive segment intervals, content themes, and content importance scores;

[0043] The step S2 comprises:

[0044] Step S21: extracting pre-cut content from the original video content according to the content-intensive segment interval;

[0045] Step S22: offsetting the nodes of the pre-cut content according to the volume amplitude interval of the audio stream to form a modified video content;

[0046] Step S23: labeling the modified video content according to the content theme and the content importance score to form the segmented video.

[0047] On the other hand, the step S3 comprises:

[0048] Step S31: preprocessing the news material to be processed to obtain preprocessing information;

[0049] Step S32: classifying the preprocessed information to add classification labeling information;

[0050] Step S33: inputting the classification annotation information into the hybrid expert model to obtain the content summary.

[0051] On the other hand, the step S31 comprises:

[0052] Step S311: identifying and removing noise information in the news material to be processed to obtain denoised information;

[0053] Step S312: Formatting the denoised information to obtain the pre-processed information;

[0054] The formatting process includes unified coding format and case conversion.

[0055] On the other hand, before executing step S3, a news collection process is also included, and the news collection process includes:

[0056] Step B1: Using crawler software to obtain original web page data from the target website;

[0057] Step B2: extracting the original news material from the original web page data.

[0058] On the other hand, the step S4 comprises:

[0059] Step S41: generating a semantic vector according to the content summary;

[0060] Step S42: extracting the news material to be processed based on the semantic vector to obtain related paragraphs and editing them to obtain the text editing content, and searching the material library based on the semantic vector to obtain the matching segmented video;

[0061] Step S43: generating the content to be organized according to the text editing content and the segmented video.

[0062] On the other hand, the step S5 comprises:

[0063] Step S51: searching for a corresponding news template according to the content summary;

[0064] Step S52: Filling the text editing content and the segmented video in the news template to form the edited manuscript.

[0065] The above technical solution has the following advantages or beneficial effects:

[0066] In view of the problem that the news auxiliary processing scheme in the prior art can only process text-based news content and has poor processing effect on news content involving video materials, this scheme introduces a news editing method based on a hybrid expert model. Specifically, for the original video content, the video segmentation information is first extracted and split through the hybrid expert model to obtain segmented videos corresponding to different expression semantics. On this basis, the text news material to be processed is input into the hybrid expert model to extract the content summary and match the segmented video for organization, and finally the news release is obtained and issued, thereby improving the processing efficiency of multimedia news. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0068] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of a hybrid expert model in an embodiment of the present invention;

[0070] Figure 3 This is a schematic diagram of the expert model generation process in an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram of step S1 in an embodiment of the present invention;

[0072] Figure 5 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0073] Figure 6 This is a schematic diagram of step S3 in an embodiment of the present invention;

[0074] Figure 7 This is a schematic diagram of step S31 in an embodiment of the present invention;

[0075] Figure 8 This is a schematic diagram of the news collection process in an embodiment of the present invention;

[0076] Fig. 9 This is a schematic diagram of step S4 in an embodiment of the present invention;

[0077] Fig.10 Schematic diagram of step S5 in an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0080] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0081] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0082] The present invention comprises:

[0083] A financial news editing method based on a large language model, such as Figure 1 As shown, including:

[0084] Step S1: processing the original video content using a hybrid expert model to obtain video segmentation information;

[0085] Step S2: splitting the original video content according to the video segmentation information to obtain segmented videos;

[0086] Step S3: obtaining the news material to be processed and inputting it into the hybrid expert model to obtain a content summary of the news material to be processed;

[0087] Step S4: Generate text editing content according to the content summary and match the segmented video to form the content to be organized;

[0088] Step S5: Generate an edited manuscript based on the content to be organized and output it.

[0089] Specifically, in view of the problem that the news auxiliary processing scheme in the prior art can only process text-based news content and has poor processing effect on news content involving video materials, this scheme introduces a news editing method based on a hybrid expert model. Specifically, for the original video content, the video segmentation information is first extracted and split through the hybrid expert model to obtain segmented videos corresponding to different expression semantics. On this basis, the news material of the text to be processed is input into the hybrid expert model to extract the content summary and match the segmented video for organization, and finally the news release is obtained and issued, thereby improving the processing efficiency of multimedia news.

[0090] In the implementation process, the above financial news editing method is mainly configured as a software embodiment in a computer device, such as a workstation, server, etc. of a news media, for processing the input original video content and the news material to be processed to obtain a news article and output it.

[0091] The original video content mainly refers to the relevant videos recorded during the news gathering and editing process, including the recorded videos of the on-site situation, the recorded videos of the interviewees, the rendered animations for auxiliary explanation, the explanation videos recorded by the journalists, and the interview videos with the interviewees, etc., which can all be called videos. In some embodiments, in order to improve the processing efficiency, the part of the video may have been preliminarily edited to remove useless empty shots.

[0092] The hybrid expert model is a large language model trained and fine-tuned for financial news corpus. It has a certain multimodal input capability and can recognize single-frame images contained in video frames and input text content, and understand their meaning in conjunction with input prompt words and other content, thereby outputting corresponding relevant information. Depending on the prompt words, the feedback information may include the meaning expressed in different time ranges in the video, time nodes corresponding to different semantics, a general summary of the text content, and so on.

[0093] Video segmentation information is a content summary containing time information obtained by the hybrid expert model after identifying the input video frame and the corresponding semantics. It mainly includes two parts: time node, start point and end point, and the content described by the corresponding video in the time segment. Based on this part of information, the video can be intercepted to obtain segmented video for storage.

[0094] Generally speaking, in order to achieve better editing and compiling efficiency, the above steps can be performed separately or in parallel. For example, after shooting a large amount of video material, it is processed through a hybrid expert model, split into segmented videos and stored in the material library. After the news material to be processed is obtained and the text editing content is generated, the corresponding segmented videos are re-matched from the material library.

[0095] The semantics of the news materials to be processed and the original video content input at the same time usually contain a many-to-many relationship.

[0096] The news materials to be processed are usually news content in text form, including written news articles, multiple data sources, external reprinted news sources, etc.

[0097] The content summary is a summary description text generated by the hybrid expert model after semantic recognition based on the news material to be processed. Based on this part of the text combined with a general large-scale language model, it can remove irrelevant and redundant content in the news material to be processed and assist the editor in obtaining the text editing content.

[0098] The edited manuscript may exist in various forms according to the needs. Taking the news content of a web page as an example, the news content in this scenario may be graphic content. The text part comes from the text editing content, and the source of the pictures is mainly the pictures in the material library and the video frames in the corresponding segmented video.

[0099] If the edited manuscript is set as a news article with text and video content, the text part comes from the text editing content, and the source of the video is mainly the corresponding segmented video in the material library.

[0100] If the edited manuscript is set as a news video, the text editing content will be used as the broadcast content in the video, with voice content generated manually or by a machine, and the corresponding segmented videos and video frames will be retrieved from the material library as B-roll or illustrations.

[0101] In one embodiment, Figure 2 As shown, the hybrid expert model includes:

[0102] Initialization layer 1, initialization layer 1 vectorizes the input information to form an initialization vector;

[0103] Attention layer 2, attention layer 2 is connected to initialization layer 1;

[0104] Attention layer 2 calculates the correlation for each initialization vector and forms vector association information;

[0105] A first residual module 3, wherein a first input end of the first residual module 3 is connected to the initialization layer 1, and a second input end of the first residual module 3 is connected to the output end of the attention layer 2;

[0106] The first residual module 3 superimposes the residual information of the initialization vector and the vector association information to form a vector to be processed;

[0107] A routing gateway 4, wherein an input end of the routing gateway 4 is connected to an output end of the first residual module 3;

[0108] The output ends of the routing gateway 4 are respectively connected to a plurality of expert models 6;

[0109] The routing gateway 4 distributes the vectors to be processed to the expert model 6 based on the control gate mechanism;

[0110] A weighted fusion module 5, wherein the input end of the weighted fusion module 5 is respectively connected to each expert model 6;

[0111] The weighted fusion module 5 performs weighted fusion on the output results of the expert model 6 according to the output probability of the expert model 6 to obtain a hidden state tensor;

[0112] A second residual module 7, wherein a first input end of the second residual module 7 is connected to an output end of the first residual module 3, and a second input end of the second residual module is connected to an output end of the weighted fusion module;

[0113] The second residual module 7 forms a model output result after superimposing the residual information of the vector to be processed with the hidden state tensor and outputs it;

[0114] The model output is content summary or video segmentation information.

[0115] Specifically, in order to achieve a better processing effect of the model on a large number of financial news involving many fields, in this embodiment, a hybrid expert model architecture is introduced, and based on the gating mechanism, each expert model 6 can process different subsets of the training samples respectively, and then comprehensively obtain a more accurate result. At the same time, the Transformer structure and attention mechanism are also introduced to enhance the processing ability of the text by capturing the association between different parts of the text.

[0116] Specifically, as shown in the accompanying drawings, when input information appears, the input information is first sent to the initialization layer 1, and the initialization layer 1 vectorizes the input information to form an initialization vector. It should be noted that for multimodal data, additional processing is required before input, including voice-to-text conversion for the audio stream of the video material, sampling the video according to the step size to extract the video frame, and further extracting semantic information.

[0117] In the initialization layer 1, the input information is first segmented to decompose the longer text into the smallest unit words, and then the words are assembled to form a minimum unit sequence.

[0118] For the minimum unit sequence, an embedding layer can be used to map it and convert it into a vector representation, where each minimum unit, that is, a word, corresponds to a vector.

[0119] Finally, the embedded vector represents the corresponding hidden state and is output as the initialization vector.

[0120] In order to capture the contextual relevance in the input text, the attention layer 2 is introduced in this embodiment. The attention layer 2 uses a self-attention mechanism to calculate the correlation between each initialization vector and other initialization vectors, thereby obtaining vector correlation information.

[0121] At the same time, in order to improve the overall expression ability of the model, the residual connection is also introduced in this embodiment to retain the original input information. The residual connection part includes two parts. One part is that after the attention layer 2 captures the correlation between each initialization vector and other initialization vectors, the residual information of the initialization vector is superimposed with the vector association information to form a vector to be processed;

[0122] The other part is the retention of the input information of the expert model 6. Specifically, the second residual module 7 superimposes the residual information of the vector to be processed with the hidden state tensor fused with the expert model 6 to form a model output result and output it, thereby retaining the information before the expert model 6 is judged.

[0123] After the first residual module 3 forms a to-be-processed vector by superimposing the residual information of the initialization vector with the vector association information, the to-be-processed vector is input into the routing gateway 4. The routing gateway 4 constructs weights corresponding to different expert models 6.

[0124] Specifically, each expert model 6 is used to process a subset of a large amount of sample data of financial news to improve the accuracy of the overall judgment of the model.

[0125] To achieve this process, during the training and fine-tuning process, different weights are configured for each expert model 6 for different types of subsets and stored in the routing gateway 4 .

[0126] After receiving the vector to be processed, the routing gateway 4 distributes the vector to be processed to the expert model 6 based on the control gate mechanism.

[0127] Each expert model6 is essentially a multi-layer perceptron (MLP) consisting of three linear layers and one activation layer, which is used to capture the semantic information and classification probability in the input vector to be processed.

[0128] Subsequently, the weighted fusion module 5 performs weighted fusion on the output results of the expert model 6 according to the output probability of the expert model 6 to obtain a hidden state tensor as the comprehensive output of multiple expert models 6.

[0129] At this time, in order to retain part of the original information, the second residual module 7 superimposes the residual information of the vector to be processed with the hidden state tensor fused to the expert model 6 to form a model output result and output it, thereby retaining the information before the expert model 6 is judged, and then generating the corresponding model output result.

[0130] Depending on the input content, the model output is content summary or video segmentation information.

[0131] In one embodiment, Figure 3 As shown, before executing step S1, an expert model generation process is also included, and the expert model generation process includes:

[0132] Step A1: Collect news corpus corresponding to financial news and construct a financial data set;

[0133] Step A2: embed Chinese semantic units into the vocabulary of the Mixtral model to obtain a Chinese adjustment model;

[0134] Step A3: Use the financial data set to train the Chinese adjustment model, and adjust the Chinese adjustment model based on the LoRA fine-tuning technology during the training process to obtain a pre-trained model;

[0135] Step A4: Parameter fusion of the pre-trained model and the Mixtral model to obtain an expert model.

[0136] Specifically, in order to achieve better processing effect on various types of input multimodal data, in this embodiment, the above method is used to construct the expert model part of the hybrid expert model.

[0137] Specifically, in order to achieve a more accurate model feedback effect, the expert model part needs to fine-tune the parameters based on the large language model and the LoRA fine-tuning technology, so that the output of the model is more in line with the vocabulary and usage habits in the financial news field.

[0138] To implement this process, this solution uses news corpus corresponding to financial news, which is cleaned, formatted, and semantically annotated to form a financial data set. The training set, test set, and validation set are set according to the training requirements.

[0139] Then, a Mixtral model, preferably the Mixtra-8*7B model, is selected as the base model, and Chinese semantic units (tokens) are embedded in its vocabulary to expand the original vocabulary so as to improve its encoding efficiency and semantic understanding of Chinese.

[0140] Specifically, to avoid degradation of the original model’s performance, the newly added Chinese semantic units are mainly added to the end of the original embedding matrix to ensure that the embeddings of the tokens in the original vocabulary are not affected.

[0141] After making the above modifications to the model, a financial dataset was used for training. During the training process, the Chinese adjustment model was adjusted based on the LoRA fine-tuning technology to obtain a pre-trained model.

[0142] To reduce the usage of video memory during training, you can use fp16 precision for training to significantly reduce video memory usage.

[0143] At the same time, to avoid model overfitting, after training, the pre-trained model and the original Mixtral model were fused to obtain an expert model. In addition, to achieve control over the model output, about 10,000 fine-tuning data were manually generated and combined with some open source instruction fine-tuning data and value alignment data to align the model with the actual output intent.

[0144] After the above model is deployed, we can continuously collect feedback from users and continuously improve the model. We introduce a reward model training mechanism and use transformers for loading to achieve dynamic optimization of the model.

[0145] In one embodiment, Figure 4 As shown, step S1 includes:

[0146] Step S11: extracting video stream and audio stream from the original video content respectively;

[0147] Step S12: extracting a picture start mark from the video stream, and extracting an audio start mark from the audio stream;

[0148] Step S13: aligning the audio stream and the video stream according to the picture start mark and the audio start mark to obtain a processed video;

[0149] Step S14: input the audio stream into the speech transcription model to obtain text content;

[0150] Step S15: semantically recognize the text content and segment it to obtain segmented semantics, extract picture frames from the video stream for the time interval of the audio stream corresponding to each paragraph of the segmented semantics, and assemble them with the segmented semantics to obtain multimodal input data;

[0151] Step S16: Input the multimodal input data into the hybrid expert model for processing to obtain video segmentation information.

[0152] Specifically, in order to achieve better content extraction effects on video information, in this embodiment, the information in the two dimensions of audio and video is first aligned, and then the multimodal data is organized for input, thereby ensuring the accuracy of the input information of the model and facilitating the input content to have more obvious semantics.

[0153] Specifically, the original video content includes video stream and audio stream. Due to factors such as transmission and rendering, there may be slight asynchrony between audio and video. This phenomenon is relatively minor to the user, but it will cause the two parts of the input information to be asynchronized during the word segmentation process related to timing.

[0154] To this end, we first extract the basic information of the video, such as duration, resolution, frame rate, etc., so that we can adjust and optimize the parameters in the subsequent processing. Then we extract the picture start mark from the video stream, and the audio start mark from the audio stream. Generally speaking, when recording news videos, the filming content will be marked in the form of a clapperboard. The frame with the clapperboard closed is used as the picture start mark, and the sound of the clapperboard closing at the corresponding moment is used as the audio start mark.

[0155] Then, the audio stream and the video stream are aligned according to the picture start mark and the audio start mark to obtain the processed video. At the same time, the audio stream needs to be input into the speech transcription model to remove the information in the sound dimension and only retain the text content of the text type.

[0156] For the processed video, the text content can be semantically recognized and segmented to obtain segmented semantics. Since the text content will retain the corresponding timestamp when it is transcribed, and a single sentence of text will show a more obvious amplitude change in the audio track, the time interval in the audio stream of the processed video can be reversely determined according to the segmented semantics.

[0157] For this time interval, it covers several video frames, from which several frames with better picture quality can be extracted and assembled with segmented semantics to obtain multimodal input data.

[0158] Finally, after feature extraction, the multimodal input data is input into the hybrid expert model to obtain video segmentation information.

[0159] The feature extraction step mainly includes extracting features from audio, such as text content obtained by speech recognition, spectral features of audio, etc. Analyze the subtitle information in the video and convert it into text features. Use image recognition technology to extract video features, including object recognition, scene classification, color distribution, etc.

[0160] Among them, image recognition includes object recognition and scene classification. Object recognition can identify various objects in the video screen. For financial news editing, it may be used to identify objects related to finance, such as stock charts, currency symbols, etc., to help determine the theme and key content of the news. Scene classification is to classify the scene in which the video screen is located, such as office scenes, exchange scenes, etc. Its role is to assist in understanding the background of the news through scene information, and provide more contextual information for news editing.

[0161] In one embodiment, the video segmentation information includes: content-intensive segment intervals, content themes, and content importance scores;

[0162] like Figure 5 As shown, step S2 includes:

[0163] Step S21: extracting pre-cut content from the original video content according to the content-dense segment interval;

[0164] Step S22: offsetting the nodes of the pre-cut content according to the volume amplitude interval of the audio stream to form a modified video content;

[0165] Step S23: annotating the modified video content according to the content theme and the content importance score to form a segmented video.

[0166] Specifically, in order to achieve a better video slicing effect, in this embodiment, when the content input to the hybrid expert model is multimodal data of the original video content, corresponding prompt words can be further constructed to control the output of the hybrid expert model.

[0167] Specifically, to facilitate video segmentation, the hybrid expert model can be asked to give possible splitting points in the video in the prompt words, and set the splitting rules and thresholds, such as the minimum segment length, importance score, etc.

[0168] The hybrid expert model then outputs the content-intensive segment intervals, content topics, and content importance scores corresponding to each segment.

[0169] Based on the content-intensive segment interval, the pre-cut content can be extracted from the original video content. Since the timestamp output by the hybrid expert model is often an integer, in order to make the information recorded by the audio stream complete, the volume amplitude sequence of the audio stream in the pre-cut content is also extracted, and the position of the volume amplitude interval is determined according to the threshold division, which corresponds to the position of the speaking pause.

[0170] The nodes of the pre-cropped content are then offset according to the volume amplitude interval of the audio stream to form a modified video content.

[0171] Finally, to facilitate subsequent retrieval, the revised video content is annotated according to the content theme and content importance score, and video names are added to form segmented videos.

[0172] In some real-time model tuning scenarios, after the segmented video is formed, the video clips after stripping are also evaluated to check the accuracy and effect of the stripping. By manually watching and comparing the videos before and after stripping, it is evaluated whether the stripping meets expectations. Based on the evaluation results, the model and stripping process are optimized.

[0173] In one embodiment, Figure 6As shown, step S3 includes:

[0174] Step S31: pre-processing the news material to be processed to obtain pre-processing information;

[0175] Step S32: classifying the preprocessed information to add classification annotation information;

[0176] Step S33: Input the classification annotation information into the hybrid expert model to obtain a content summary.

[0177] Specifically, in order to achieve effective extraction of text content, in this embodiment, the text-type news material to be processed must first be preprocessed to obtain preprocessing information, thereby achieving formatting of the information content.

[0178] Then, the news text contained in the preprocessed information is classified and screened, such as into categories such as macroeconomics, financial markets, industry trends, and company news.

[0179] Finally, after inputting the trained hybrid expert model, key information and core content can be automatically extracted, and news highlights can be generated using text summarization technology.

[0180] In one embodiment, Figure 7 As shown, step S31 includes:

[0181] Step S311: identifying and removing noise information in the news material to be processed to obtain denoised information;

[0182] Step S312: Formatting the denoised information to obtain pre-processed information;

[0183] Formatting includes unified encoding format and case conversion.

[0184] Specifically, in order to achieve better preprocessing effects, in this embodiment, for the input news material to be processed, the noise part is first identified, including advertisements, irrelevant links, special characters, etc., and then standardized, including unified coding format, case conversion, etc., to obtain preprocessing information.

[0185] In some embodiments, in order to facilitate subsequent recognition of the model, operations such as part-of-speech tagging and named entity recognition are also performed, and the content is annotated and sent to the hybrid expert model.

[0186] In one embodiment, Figure 8 As shown, before executing step S3, a news collection process is also included, and the news collection process includes:

[0187] Step B1: Using crawler software to obtain original web page data from the target website;

[0188] Step B2: extracting original news materials from the original web page data.

[0189] Specifically, in order to achieve more sources of news materials, in this embodiment, channels such as the First Financial News Network and authoritative financial information websites are first set as target websites, and then web crawler technology or related data interfaces are used to collect the latest financial news text data, covering macroeconomic dynamics, stock market conditions, industry development, company financial reports and other aspects as original news materials, thereby obtaining more news material content.

[0190] In one embodiment, Fig. 9 As shown, step S4 includes:

[0191] Step S41: Generate a semantic vector according to the content summary;

[0192] Step S42: extracting the news material to be processed based on the semantic vector to obtain related paragraphs and editing them to obtain text editing content, and searching the material library based on the semantic vector to obtain matching segmented videos;

[0193] Step S43: Generate content to be organized according to the text editing content and the segmented video.

[0194] Specifically, in order to achieve a better retrieval effect, in this embodiment, after extracting the content summary from the text content, a semantic vector is first generated based on the content summary, and then the news material to be processed is extracted based on the semantic vector, and the part directly related to the news description is retained, and the redundant part is removed. At the same time, the material library is searched based on the semantic vector to obtain matching segmented videos, thereby achieving a better retrieval and matching effect.

[0195] In one embodiment, Fig.10 As shown, step S5 includes:

[0196] Step S51: searching for a corresponding news template according to the content summary;

[0197] Step S52: Fill the news template with text editing content and segmented video to form an edited manuscript.

[0198] Specifically, in order to achieve a better manuscript synthesis effect, taking video news broadcast as an example, the special effects department has pre-made news broadcast templates for this type of video, including studio background, bottom banner, special effect subtitle style, etc. There are different templates for different categories. For a piece of news, first find the corresponding news template according to the content summary. Then, fill the text editing content corresponding to the subtitles, banners and other parts in the news template, and add the content of the segmented video at the corresponding pixel position, such as the announcer's broadcast content, to generate a complete financial news morning video, and then render the synthesized video, adjust the resolution, frame rate, color and other parameters, improve the image quality and smoothness, and finally get the edited manuscript.

[0199] In addition, taking web page content as an example, the template of this type of news content is a web page front-end template, in which the placement of different controls is set. For different controls, such as titles, text boxes, picture bars and video players, text editing content and segmented videos are filled in respectively to form the edited manuscript.

[0200] Those skilled in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, may be specifically implemented as systems, methods, or computer program products. Therefore, various aspects of the present invention, or possible implementations of various aspects, may take the form of complete hardware embodiments, complete software embodiments (including firmware, resident software, etc.), or embodiments of combined software and hardware aspects, all collectively referred to herein as "circuits," "modules," or "systems." In addition, various aspects of the present invention, or possible implementations of various aspects, may take the form of computer program products, which refer to computer instructions stored in a memory.

[0201] The memory may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable read-only memory (CD-ROM).

[0202] The processor in the computer reads the computer instructions stored in the memory, so that the processor can execute the functional actions specified in each step or the combination of steps in the flowchart; and generate a device for implementing the functional actions specified in each block or the combination of blocks in the block diagram.

[0203] It should be understood that the processor in the computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components implemented to execute the aforementioned computer instructions.

[0204] The computer instructions may execute entirely on the user's local computer, partly on the user's local computer, as a separate software package, partly on the user's local computer and partly on a remote computer, or entirely on a remote computer or server.

[0205] In some cases, the actions or steps recited in the claims may be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0206] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A financial news editing method based on a large language model, characterized in that: include: Step S1: processing the original video content using a hybrid expert model to obtain video segmentation information; Step S2: splitting the original video content according to the video segmentation information to obtain segmented videos; Step S3: obtaining the news material to be processed and inputting it into the hybrid expert model to obtain a content summary of the news material to be processed; Step S4: Generate text editing content according to the content summary and match the segmented video to form content to be organized; Step S5: forming an edited manuscript based on the content to be organized and outputting it.

2. The financial news editing method according to claim 1, characterized in that: The hybrid expert model includes: An initialization layer, wherein the initialization layer vectorizes the input information to form an initialization vector; an attention layer, the attention layer being connected to the initialization layer; The attention layer calculates the correlation for each of the initialization vectors and forms vector association information; A first residual module, wherein a first input end of the first residual module is connected to the initialization layer, and a second input end of the first residual module is connected to an output end of the attention layer; The first residual module superimposes the residual information of the initialization vector and the vector association information to form a vector to be processed; A routing gateway, wherein an input end of the routing gateway is connected to an output end of the first residual module; The output ends of the routing gateway are respectively connected to a plurality of expert models; The routing gateway allocates the to-be-processed vector to the expert model based on a control gate mechanism; A weighted fusion module, wherein the input end of the weighted fusion module is respectively connected to each of the expert models; The weighted fusion module performs weighted fusion on the output results of the expert model according to the output probability of the expert model to obtain a hidden state tensor; A second residual module, wherein a first input end of the second residual module is connected to an output end of the first residual module, and a second input end of the second residual module is connected to an output end of the weighted fusion module; The second residual module forms a model output result after superimposing the residual information of the vector to be processed with the hidden state tensor and outputs it; The model output result is the content summary or the video segmentation information.

3. The financial news editing method according to claim 2, characterized in that: Before executing step S1, an expert model generation process is also included, and the expert model generation process includes: Step A1: Collect news corpus corresponding to financial news and construct a financial data set; Step A2: embed Chinese semantic units into the vocabulary of the Mixtral model to obtain a Chinese adjustment model; Step A3: using the financial data set to train the Chinese adjustment model, and adjusting the Chinese adjustment model based on the LoRA fine-tuning technology during the training process to obtain a pre-trained model; Step A4: Parameter fusion is performed on the pre-trained model and the Mixtral model to obtain the expert model.

4. The financial news editing method according to claim 1, characterized in that: The step S1 comprises: Step S11: extracting video stream and audio stream from the original video content respectively; Step S12: extracting a picture start marker from the video stream, and extracting an audio start marker from the audio stream; Step S13: aligning the audio stream and the video stream according to the picture start mark and the audio start mark to obtain a processed video; Step S14: inputting the audio stream into a speech transcription model to obtain text content; Step S15: semantically identifying the text content and segmenting it to obtain segmented semantics, extracting picture frames from the video stream for each time interval of the audio stream corresponding to each paragraph of the segmented semantics, and assembling them with the segmented semantics to obtain multimodal input data; Step S16: inputting the multimodal input data into the hybrid expert model for processing to obtain the video segmentation information.

5. The financial news editing method according to claim 1, characterized in that: The video segmentation information includes: content-intensive segment intervals, content themes, and content importance scores; The step S2 comprises: Step S21: extracting pre-cut content from the original video content according to the content-intensive segment interval; Step S22: offsetting the nodes of the pre-cut content according to the volume amplitude interval of the audio stream to form a modified video content; Step S23: labeling the modified video content according to the content theme and the content importance score to form the segmented video.

6. The financial news editing method according to claim 1, characterized in that: The step S3 comprises: Step S31: preprocessing the news material to be processed to obtain preprocessing information; Step S32: classifying the preprocessed information to add classification labeling information; Step S33: inputting the classification annotation information into the hybrid expert model to obtain the content summary.

7. The financial news editing method according to claim 6, characterized in that: The step S31 comprises: Step S311: identifying and removing noise information in the news material to be processed to obtain denoised information; Step S312: Formatting the denoised information to obtain the pre-processed information; The formatting process includes unified coding format and case conversion.

8. The financial news editing method according to claim 1, characterized in that: Before executing step S3, a news collection process is also included, and the news collection process includes: Step B1: Using crawler software to obtain original web page data from the target website; Step B2: extracting the original news material from the original web page data.

9. The financial news editing method according to claim 1, characterized in that: The step S4 comprises: Step S41: generating a semantic vector according to the content summary; Step S42: extracting the news material to be processed based on the semantic vector to obtain related paragraphs and editing them to obtain the text editing content, and searching the material library based on the semantic vector to obtain the matching segmented video; Step S43: generating the content to be organized according to the text editing content and the segmented video.

10. The financial news editing method according to claim 1, characterized in that: The step S5 comprises: Step S51: searching for a corresponding news template according to the content summary; Step S52: Filling the text editing content and the segmented video in the news template to form the edited manuscript.

Images