Athletic contest commentary generation method based on large language model
By applying the combination technology of large language models and video analysis modules in sports event videos, natural language commentary is generated, and the problem of unnatural and time-consuming generation of commentary in the existing technology is solved, and efficient and professional automatic commentary generation is achieved. Improves the audience's viewing experience.
Patent Information
- Application Number
- CN202411537440.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-11
AI Technical Summary
The prior art is difficult to effectively generate sports event commentary in natural language, resulting in the inability to promptly explain a large number of competitions, and the recognition results of existing methods do not have the fluency and easy-to-understand characteristics of natural language.
The sports competition commentary generation method based on the large language model is adopted, and the action information in the sports event video is recognized through the video analysis module, and natural language commentary is generated by combining the large language model commentary module. The method includes a video analysis module, a large language model explanation module and a commentary generation system development module, and optimizes the explanation quality through retrieval and enhances generation technology and prompt word engineering.
The automatic commentary generation of sports event videos is realized, reducing the time and cost of manual commentary, and the generated commentary is more accurate, professional and smooth, significantly improving the audience's viewing experience.
Smart Images

Figure CN119421013A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for generating sports game commentary based on a large language model, and belongs to the fields of computer and sports science and technology. Background Art
[0002] In sports events, commentary is an indispensable part, which can convey information such as event knowledge, real-time game conditions, and technical and tactical analysis to the audience. Experienced audiences of sports events (such as veteran fans) can resonate with the commentary, while audiences who do not understand sports can gain professional event knowledge from it. However, due to the large number of events, the high cost and time-consuming nature of manual commentary result in a large number of games being unable to receive timely commentary, and the contradiction between the supply and demand of event commentary is very prominent.
[0003] Currently, computer vision technology is used to locate and analyze game videos, extract and identify actions in the video, and obtain information on the time and type of actions. However, the recognition result of this method is only the action content in a dictionary format, which does not have the fluent and easy-to-understand characteristics of natural language, and is not friendly to event spectators.
[0004] The existing technology also uses a large language model to read and analyze video content and generate natural language descriptions, but this method transcribes the video sound into text and uses a large language model for analysis. This method has limited ability to analyze video image content such as sports events, so there is currently no suitable automatic commentary generation method that can be applied to the commentary of sports events. Summary of the invention
[0005] The present invention proposes a method for generating sports game commentary based on a large language model, and its purpose is to provide a method for automatically generating commentary for sports event videos, thereby reducing the time and cost of manual event commentary.
[0006] The technical solution of the present invention is: a method for generating sports game commentary based on a large language model, which uses a sports game commentary generation system based on a large language model to perform sports game commentary; the sports game commentary generation system based on a large language model includes a video analysis module, a large language model commentary module, and a commentary generation system development module; the specific process of using the sports game commentary generation system based on a large language model to perform sports game commentary includes: Step 1) inputting the sports event video into the video analysis module, and using the video analysis module to identify action information in the sports event video; Step 2) The action information identified by the video analysis module is transmitted to the large language model interpretation module and the interpretation generation system development module; Step 3) The large language model interpretation module uses the large language model to process the received action information and generate an interpretation; Step 4) Use the explanation generation system development module to combine the action information identified by the video analysis module with the explanation generated by the large language model explanation module to form an interactive explanation generation system.
[0007] Furthermore, the use of the video analysis module to identify action information in sports game videos specifically includes positioning and identifying actions in sports game videos, and analyzing action information in the events; the action information includes the time when the technical action occurs and the action type.
[0008] Furthermore, the large language model is the Zhipu GLM4 large language model.
[0009] Furthermore, the specific process of using the video analysis module to identify action information in sports event videos includes: Step 1-1) Use the two-dimensional convolutional neural network PP-TSM to extract image features from sports game videos, that is, to understand the video and extract dimensionality reduction features; Step 1-2) Use a boundary matching network based on a convolutional neural network structure to locate the start and end time of the action; Steps 1-3) The long short-term memory network based on the attention mechanism recognizes the action type.
[0010] Furthermore, the large language model interpretation module uses the large language model to process the received action information to generate an interpretation, specifically including: Step 3-1) Optimize the event commentary capabilities of the large language model through retrieval-enhanced generation technology; Step 3-2) Use prompt word engineering to iteratively optimize the large language model to generate prompt words for explanation; Step 3-3) Input the prompt words into the large language model to generate the explanation content.
[0011] Furthermore, the optimization of the event commentary capability of the large language model through the retrieval enhancement generation technology specifically includes: Step 3-1-1) Build the development environment; Step 3-1-2) Build an explanation knowledge base; Step 3-1-3) The explanation knowledge base is vectorized and stored in the explanation knowledge vector database; Step 3-1-4) Connect the large language model to the Langchain framework.
[0012] Furthermore, the commentary knowledge base is used to optimize the large language model, provide comprehensive sports commentary knowledge, and improve the professionalism and accuracy of the commentary generated by the large language model; the commentary knowledge base contains the following three types of content: knowledge of competition items from Wikipedia and relevant official websites, game commentary tutorials from major sports event websites and commentary professional platforms, and manual commentary scripts of English broadcast videos of well-known media of high-level events; the content formats of the commentary knowledge base are markdown and pdf; the Jina.ai / Reader API tool is used to extract the html content of the web page and convert it to markdown format, and the commentary voice transcription text of the mp4 game video is extracted through the Tongyi Tingwu tool, and saved in pdf format as a commentary text pdf file; The large language model is connected to the Langchain framework specifically including: obtaining the Zhipu GLM4 large language model API, programming to connect the Zhipu GLM4 large language model API to the Langchain framework, Langchain provides a chain structure from the prompt word input into the large language model to the large language model output result parsing; loading the explanation knowledge vector database stored in step 3-1-3), and using the functions in Langchain to build a retrieval question-answering chain; at this time, the user enters the prompt word, and the retrieval question-answering chain will search for content related to the prompt word in the explanation knowledge vector database, combine it with the prompt word entered by the user, and then input it into the large language model to obtain a reply.
[0013] Furthermore, the vectorization of the explanation knowledge base and storing it in the explanation knowledge vector database specifically includes: First, write code to read and parse the files of the explanation knowledge base. Use UnstructuredMarkdownLoader in the Langchain framework to read the explanation tutorials and knowledge content in markdown format; use PyMuPDFLoader to parse the explanation text pdf file; store the explanation knowledge base content in markdown and pdf formats in a unified data format in python; Then, we wrote code and used a text replacement algorithm to remove unnecessary characters such as line breaks and spaces in the explanation knowledge base content to improve the efficiency of subsequent retrieval. We then segmented the markdown and PDF files in the explanation knowledge base into segments with 1,000 characters per segment to prevent the file content from exceeding the processing capacity of the large language model and improve the speed of subsequent retrieval and information search. Next, we use Zhipu's vectorization API to vectorize the content in the segmented commentary knowledge base to form a vectorized result, that is, a mathematical representation of characters converted into a vector, which is convenient for retrieval using the cosine similarity algorithm; then the vectorized result is stored in the lightweight vector database Chroma integrated in the Langchain framework to obtain the commentary knowledge vector database; when the large language model needs to comment on the event, it first retrieves the most relevant knowledge from the commentary knowledge vector database and generates it based on the current input.
[0014] Furthermore, the use of prompt word engineering to iteratively optimize the large language model to generate prompt words for explanation specifically includes: Step 3-2-1) Design the prompt words for generating the explanation, including the following elements: (1) Role setting: Set the role of the big language model as the commentator of the event, and guide the big language model to generate content that meets the needs; (2) Task information: clearly inform the large language model of the specific steps for interpretation in a step-by-step manner to ensure that the large language model can fully understand the interpretation task; (3) Video Action: The time and type of technical actions identified by the video analysis module are included in the prompt words, so that the large language model can understand the technical actions during the game; (4) Explanation example: Give an example explanation so that the large language model can clearly understand what form of explanation content needs to be generated; Step 3-2-2) Manual evaluation and feedback.
[0015] Furthermore, the step 4) uses the commentary generation system development module to combine the action information identified by the video analysis module with the commentary generated by the large language model commentary module to form an interactive commentary generation system. The specific process includes: after the user uploads the sports event video, the commentary generation system development module calls the action information identified by the video analysis module, and the video analysis module outputs the JSON content that saves the action information; the action information in the prompt word designed in step 3-2) is derived from the JSON content that saves the action information output by the video analysis module, and the prompt word combined with the action information is input into the large language model optimized by the retrieval enhancement generation technology in step 3-1), and the large language model returns the generated commentary content.
[0016] Beneficial effects of the present invention: 1) The present invention uses a high-quality event commentary knowledge base to perform retrieval enhancement generation optimization on a large language model, thereby optimizing the efficiency of automatically generating commentary and language fluency; it innovates the automatic commentary generation technology for sports competitions and can generate more accurate and professional intelligent commentary; 2) Conduct prompt word engineering to further optimize the ability of large language models to generate explanations, improve the quality of explanation generation, and make the generated explanations more fluent, professional, and vivid; facilitate the further design of complete web applications, with complete front-end functions and provide a good user interaction experience; 3) Based on the computer vision method to understand sports events, the present invention uses a large language model to realize event commentary. The commentary is more interactive, smooth and vivid, which significantly improves the audience's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Attached Figure 1 Flow chart of a method for generating sports event commentary by a sports game commentary generation system based on a large language model.
[0018] Attached Figure 2 It is a flowchart for optimizing the interpretation effect of a large language model.
[0019] Attached Figure 3 This is a screenshot of a web page of an interactive explanation generation system developed based on the present invention. DETAILED DESCRIPTION
[0020] The present invention is described in detail below in conjunction with specific implementation modes.
[0021] A method for generating sports game commentary based on a large language model, the method uses a sports game commentary generation system based on a large language model to perform sports game commentary; the sports game commentary generation system based on a large language model includes a video analysis module, a large language model commentary module, and a commentary generation system development module; the specific process of using the sports game commentary generation system based on a large language model to perform sports game commentary includes the following steps: Step 1) inputting the sports event video into the video analysis module, and using the video analysis module to identify action information in the sports event video; Step 2) The action information identified by the video analysis module is transmitted to the large language model interpretation module and the interpretation generation system development module; Step 3) The large language model interpretation module uses the large language model to process the received action information and generate an interpretation; Step 4) Use the explanation generation system development module to combine the action information identified by the video analysis module with the explanation generated by the large language model explanation module to form an interactive explanation generation system.
[0022] The use of the video analysis module to identify action information in sports game videos specifically includes positioning and identifying actions in sports game videos, and analyzing action information in the games; the action information includes the time when technical actions occur and the action type.
[0023] The large language model is preferably the Zhipu GLM4 large language model; the present invention collects a professional commentary knowledge base, and vectorizes it into a commentary knowledge vector database to perform retrieval enhancement generation optimization on the large language model; and performs prompt word engineering design to generate prompt words for the commentary.
[0024] The commentary generation system development module of the present invention combines the video analysis module and the large language model commentary module to develop an interactive commentary generation system, which uses the video analysis module to perform action recognition on sports videos uploaded by users, and uses the large language model commentary module to generate fluent and professional commentary and support real-time interactive questioning.
[0025] The video analysis module is used to identify action information in sports event videos, and preferably the PaddleVideo open source general sports action recognition method is used to identify action information in sports event videos; the PaddleVideo open source general sports action recognition method can accurately identify the type of action in sports event videos and locate the start and end time of the action.
[0026] The specific process of the PaddleVideo open source general sports action recognition method for identifying action information in sports event videos includes the following steps: Step 1-1) Use the two-dimensional convolutional neural network PP-TSM (PaddlePaddle-Temporal ShiftModule, PaddlePaddle-Time Series Offset Module Network) to extract image features from sports game videos, that is, to understand the video and extract dimensionality reduction features; Step 1-2) Use the boundary matching network (BMN) based on the convolutional neural network structure to locate the start and end time of the action; Step 1-3) The long short-term memory network (Attention-LSTM) based on the attention mechanism identifies the action type; the present invention is generally applicable to sports events such as football, basketball, table tennis, etc., and the action positioning and identification of different sports can be performed by replacing the video data set of the corresponding sports.
[0027] Taking football as an example, the data set used by the PaddleVideo open source general sports action recognition method is derived from the complete video files of the official broadcasts of public international football sports games in mp4 format. The video length ranges from 1 hour to 3 hours. The data set used by the PaddleVideo open source general sports action recognition method annotates the start and end time and type of technical actions in the video, and marks six common football technical actions including goals, corner kicks, free kicks, yellow cards, substitutions, and out-of-bounds balls; the football game video is input into the video analysis module, and the video analysis module outputs JSON format content containing the time and action type of the technical action after running.
[0028] In order to optimize the ability of the large language model to explain sports events, this method adopts two technologies: explanation knowledge base retrieval enhancement generation and prompt word engineering. The large language model explanation module uses the large language model to process the received action information to generate explanations, which specifically includes the following steps: Step 3-1) Optimize the event commentary capabilities of the large language model through retrieval-augmented generation (RAG) technology; Step 3-2) Use prompt engineering to iteratively optimize the large language model to generate prompt words for the explanation; Step 3-3) Input the prompt words into the large language model to generate the explanation content.
[0029] Step 3-1) of the present invention optimizes the event commentary capability of the large language model through retrieval augmentation generation (RAG) technology, which means building a comprehensive and professional event commentary knowledge base, and using the commentary knowledge base to tune the large language model, thereby ultimately improving its performance in commenting on sports events.
[0030] The step 3-1) optimizes the event commentary capability of the large language model by using the retrieval augmentation generation (RAG) technology, and specifically includes the following steps: Step 3-1-1) Build the development environment: The video analysis module in the present invention needs to have GPU computing resources and a main memory capacity greater than 16GB. The large language model has relatively low server configuration requirements. Considering all factors, this method uses a cloud server with GPU, 16GB main memory and dual-core CPU as the hardware environment for development. The present invention mainly uses Python for programming, and needs to install the PaddlePaddle deep learning framework required by the video analysis module and the Langchain framework required by the large language model interpretation module, as well as other libraries used. Step 3-1-2) Build the commentary knowledge base: The commentary knowledge base is used to optimize the large language model, provide comprehensive sports commentary knowledge, and improve the professionalism and accuracy of the commentary generated by the large language model; the commentary knowledge base contains the following three types of content: knowledge of competition events from Wikipedia and relevant official websites, game commentary tutorials from major sports event websites and commentary professional platforms, and manual commentary scripts of English broadcast videos of well-known media of high-level events; the content format of the commentary knowledge base is markdown and pdf; use the Jina.ai / Reader API tool to extract the html content of the web page and convert it to markdown format, use the Tongyi Tingwu tool to extract the commentary voice transcription text of the mp4 game video, and save it in pdf format as a commentary text pdf file. This step ensures that knowledge content from different sources can be stored in the commentary knowledge base in a format that is easy to process; Step 3-1-3) Vectorize the explanation knowledge base and store it in the explanation knowledge vector database: First, write code to read and parse the file of the explanation knowledge base, use the UnstructuredMarkdownLoader in the Langchain framework to read the explanation tutorial and knowledge content in markdown format; use PyMuPDFLoader to parse the explanation text pdf file; store the explanation knowledge base content in markdown and pdf formats in a unified data format in python; Then, we wrote code and used a text replacement algorithm to remove unnecessary characters such as line breaks and spaces in the explanation knowledge base content to improve the efficiency of subsequent retrieval. We then segmented the markdown and PDF files in the explanation knowledge base into segments with 1,000 characters per segment to prevent the file content from exceeding the processing capacity of the large language model and improve the speed of subsequent retrieval and information search. Next, we use Zhipu's embedding API to vectorize the contents in the segmented commentary knowledge base to form a vectorized result, that is, a mathematical representation from characters to vectors, which is convenient for retrieval using the cosine similarity algorithm. The vectorized result is then stored in the lightweight vector database Chroma integrated in the Langchain framework to obtain the commentary knowledge vector database. When the large language model needs to comment on a match, it will first retrieve the most relevant knowledge from the commentary knowledge vector database and generate it based on the current input. This retrieval-enhanced generation method ensures that the output of the large language model can reflect real-time scenario requirements while relying on rich background knowledge, thereby significantly improving the professionalism and accuracy of the commentary. Step 3-1-4) Connect the large language model to the Langchain framework: Get the Zhipu GLM4 large language model API, programmatically connect the Zhipu GLM4 large language model API to the Langchain framework, Langchain provides a chain structure from the prompt word input into the large language model to the large language model output result parsing; load the explanation knowledge vector database stored in step 3-1-3), and use the functions in Langchain to build a retrieval question-answering chain; at this time, the user enters the prompt word, and the retrieval question-answering chain will search for content related to the prompt word in the explanation knowledge vector database, combine it with the prompt word entered by the user, and then input it into the large language model to get a reply.
[0031] Step 3-2) Use prompt engineering to iteratively optimize the large language model to generate explanation prompts. This refers to the process of inputting prompts into the large language model through multiple rounds of evaluation and iterative modification to ensure that its structure is clear, its function is clear, and it can generate explanations most effectively.
[0032] The step 3-2) uses prompt engineering to iteratively optimize the large language model to generate prompt words for explanation, which specifically includes the following steps: Step 3-2-1) Design the prompt words for generating the explanation, including the following elements: (1) Role setting: Set the role of the big language model as the commentator of the event, and guide the big language model to generate content that meets the needs; (2) Task information: clearly inform the large language model of the specific steps for interpretation in a step-by-step manner to ensure that the large language model can fully understand the interpretation task; (3) Video Action: The time and type of technical actions identified by the video analysis module are included in the prompt words, so that the large language model can understand the technical actions during the game; (4) Explanation example: Give an example explanation so that the large language model can clearly understand what form of explanation content needs to be generated; Step 3-2-2) Manual evaluation and feedback: In order to quantitatively evaluate the quality of the large language model commentary, a five-level Likert scale (with integer scores from 1 to 5) was designed for evaluation. The main evaluation dimensions of the generated commentary are: accuracy (the degree of match between the commentary content and the actual situation of the event), fluency (the naturalness of the language and logical coherence) and professionalism (whether it reflects the depth and technical analysis that the event commentary should have).
[0033] The manual evaluation and feedback process includes: inputting the designed prompt words into the large language model described in step 3-1-4) to obtain the commentary generated by the large language model; three senior fans score the commentary content based on the above five-level Likert scale. This scoring mechanism can quantify the performance of different commentary words in the above three aspects and provide directions for further improvement of the commentary words. According to the results of each round of evaluation, the prompt words are adjusted in a targeted manner. After a total of 10 rounds of iterative modifications, the prompt words finally designed can stably generate high-quality event commentary.
[0034] The narration generation system development module combines the video analysis module and the large language model narration module to develop an interactive narration generation system with comprehensive functions and a coherent narration process.
[0035] The step 4) uses the commentary generation system development module to combine the action information identified by the video analysis module with the commentary generated by the large language model commentary module to form an interactive commentary generation system. Specifically, the following steps are performed: after the user uploads the sports event video, the commentary generation system development module calls the action information identified by the video analysis module, and the video analysis module outputs the JSON content that stores the action information; the action information in the prompt words designed in step 3-2) is derived from the JSON content that stores the action information output by the video analysis module, and the prompt words combined with the action information are input into the large language model optimized by the retrieval augmented generation (RAG) technology in step 3-1, and the large language model returns the generated commentary content; in addition, the user can also ask questions about the game to the large language model, and the large language model queries the sports commentary knowledge in the commentary knowledge vector database for answers.
[0036] In order to realize the front-end visualization of the above functions, the present invention can further adopt the Python library Streamlit to build a web page, and the web page content includes a button for users to upload videos, a video component, a commentary text output component and a chat box for dialogue interaction with a large language model.
Claims
1. A method for generating sports game commentary based on a large language model, characterized by A sports game commentary generation system based on a large language model is used to perform sports game commentary; the sports game commentary generation system based on a large language model includes a video analysis module, a large language model commentary module, and a commentary generation system development module; The specific process of using the sports game commentary generation system based on the large language model to perform sports game commentary includes: Step 1) inputting the sports event video into the video analysis module, and using the video analysis module to identify action information in the sports event video; Step 2) The action information identified by the video analysis module is transmitted to the large language model interpretation module and the interpretation generation system development module; Step 3) The large language model interpretation module uses the large language model to process the received action information and generate an interpretation; Step 4) Use the explanation generation system development module to combine the action information identified by the video analysis module with the explanation generated by the large language model explanation module to form an interactive explanation generation system.
2. A method for generating sports game commentary based on a large language model according to claim 1, characterized in that The use of the video analysis module to identify action information in sports game videos specifically includes positioning and identifying actions in sports game videos, and analyzing action information in the games; the action information includes the time when technical actions occur and the action type.
3. A method for generating sports game commentary based on a large language model according to claim 1, characterized in that The large language model is the Zhipu GLM4 large language model.
4. A method for generating sports game commentary based on a large language model according to claim 1, characterized in that The specific process of using the video analysis module to identify action information in sports event videos includes: Step 1-1) Use the two-dimensional convolutional neural network PP-TSM to extract image features from sports game videos, that is, to understand the video and extract dimensionality reduction features; Step 1-2) Use a boundary matching network based on a convolutional neural network structure to locate the start and end time of the action; Steps 1-3) The long short-term memory network based on the attention mechanism recognizes the action type.
5. The method for generating sports game commentary based on a large language model according to claim 1, characterized in that The large language model interpretation module uses the large language model to process the received action information to generate an interpretation, specifically including: Step 3-1) Optimize the event commentary capabilities of the large language model through retrieval-enhanced generation technology; Step 3-2) Use prompt word engineering to iteratively optimize the large language model to generate prompt words for explanation; Step 3-3) Input the prompt words into the large language model to generate the explanation content.
6. A method for generating sports game commentary based on a large language model according to claim 5, characterized in that The optimization of the event commentary capability of the large language model through the retrieval enhancement generation technology specifically includes: Step 3-1-1) Build the development environment; Step 3-1-2) Build an explanation knowledge base; Step 3-1-3) The explanation knowledge base is vectorized and stored in the explanation knowledge vector database; Step 3-1-4) Connect the large language model to the Langchain framework.
7. The method for generating sports game commentary based on a large language model according to claim 6, characterized in that The commentary knowledge base is used to optimize the large language model, provide comprehensive sports commentary knowledge, and improve the professionalism and accuracy of the commentary generated by the large language model; the commentary knowledge base contains the following three types of content: knowledge of competition items from Wikipedia and relevant official websites, game commentary tutorials from major sports event websites and commentary professional platforms, and manual commentary scripts of English broadcast videos of well-known media of high-level events; the content formats of the commentary knowledge base are markdown and pdf; the Jina.ai / Reader API tool is used to extract the html content of the web page and convert it to markdown format, and the commentary voice transcription text of the mp4 game video is extracted through the Tongyi Tingwu tool, and saved in pdf format as a commentary text pdf file; The large language model is connected to the Langchain framework specifically including: obtaining the Zhipu GLM4 large language model API, programming to connect the Zhipu GLM4 large language model API to the Langchain framework, Langchain provides a chain structure from the prompt word input into the large language model to the large language model output result parsing; loading the explanation knowledge vector database stored in step 3-1-3), and using the functions in Langchain to build a retrieval question-answering chain; at this time, the user enters the prompt word, and the retrieval question-answering chain will search for content related to the prompt word in the explanation knowledge vector database, combine it with the prompt word entered by the user, and then input it into the large language model to obtain a reply.
8. A method for generating sports game commentary based on a large language model according to claim 6, characterized in that The vectorization of the explanation knowledge base and storage in the explanation knowledge vector database specifically includes: First, write code to read and parse the files of the explanation knowledge base. Use UnstructuredMarkdownLoader in the Langchain framework to read the explanation tutorials and knowledge content in markdown format; use PyMuPDFLoader to parse the explanation text pdf file; store the explanation knowledge base content in markdown and pdf formats in a unified data format in python; Then, we wrote code and used a text replacement algorithm to remove unnecessary characters such as line breaks and spaces in the explanation knowledge base content to improve the efficiency of subsequent retrieval. We then segmented the markdown and PDF files in the explanation knowledge base into segments with 1,000 characters per segment to prevent the file content from exceeding the processing capacity of the large language model and improve the speed of subsequent retrieval and information search. Next, we use Zhipu's vectorization API to vectorize the content in the segmented commentary knowledge base to form a vectorized result, that is, a mathematical representation of characters converted into a vector, which is convenient for retrieval using the cosine similarity algorithm; then the vectorized result is stored in the lightweight vector database Chroma integrated in the Langchain framework to obtain the commentary knowledge vector database; when the large language model needs to comment on the event, it first retrieves the most relevant knowledge from the commentary knowledge vector database and generates it based on the current input.
9. The method for generating sports game commentary based on a large language model according to claim 5, characterized in that The use of prompt word engineering to iteratively optimize the large language model to generate prompt words for explanation specifically includes: Step 3-2-1) Design the prompt words for generating the explanation, including the following elements: (1) Role setting: Set the role of the big language model as the commentator of the event, and guide the big language model to generate content that meets the needs; (2) Task information: clearly inform the large language model of the specific steps for interpretation in a step-by-step manner to ensure that the large language model can fully understand the interpretation task; (3) Video Action: The time and type of technical actions identified by the video analysis module are included in the prompt words, so that the large language model can understand the technical actions during the game; (4) Explanation example: Give an example explanation so that the large language model can clearly understand what form of explanation content needs to be generated; Step 3-2-2) Manual evaluation and feedback.
10. The method for generating sports game commentary based on a large language model according to claim 5, characterized in that The specific process of the step 4) using the commentary generation system development module to combine the action information identified by the video analysis module with the commentary generated by the large language model commentary module to form an interactive commentary generation system includes: after the user uploads the sports event video, the commentary generation system development module calls the action information identified by the video analysis module, and the video analysis module outputs the JSON content that saves the action information; the action information in the prompt word designed in step 3-2) is derived from the JSON content that saves the action information output by the video analysis module, and the prompt word combined with the action information is input into the large language model optimized by the retrieval enhancement generation technology in step 3-1), and the large language model returns the generated commentary content.
Citation Information
Patent Citations
Explanation method and device for sports competition
CN110826361A
Knowledge base knowledge retrieval method based on large language model and related device
CN117851435A
Multi-modal video understanding method based on large language model
CN118675092A
Video based question and answer
US11995412B1
Cited By
Football commentary generation method and device based on visual reasoning and knowledge enhancement
CN120471178A
Video description information generation method and device, equipment and storage medium
CN120499467A
Method, device, equipment and storage medium for generating video description information
CN120499467B