Curling match video description method based on large language model

Abstract

The invention provides a curling match video description method based on a large language model. According to the method, the videos are intelligently analyzed, and detailed text description is generated, so that curling players can more deeply know each detail of a match with less time, detailed tactical analysis and action guidance are provided, and the skill level of the players can be improved more quickly. According to the method, the advantages of a video analysis technology and a large language model are combined, comprehensive, detailed and deep natural language description of curling match videos is achieved, the accuracy and comprehensiveness of curling match video description are improved, and richer match information and analysis means are provided for audiences and coaches.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence and large language models, and in particular to a curling game video description method based on a large language model. Background Art

[0002] As a highly strategic and entertaining winter sport, curling competitions contain a wealth of key information such as tactical layout, details of athletes' movements, and teamwork. However, traditional video analysis technology is often limited to quantitative statistics of game data, and it is difficult to capture and describe these key information comprehensively and deeply. This makes it difficult for coaches to fully and deeply understand the game situation, which affects the formulation of game strategies and the adjustment of tactics.

[0003] In recent years, large language models (LLMs) have made significant progress in the field of natural language processing. Large language models have powerful text generation and understanding capabilities, can understand and generate complex natural language descriptions, and provide new technical means for video description. By applying large language models to the description of curling competition videos, a comprehensive, detailed, and in-depth description of the competition process can be achieved, thus making up for the shortcomings of traditional video analysis technology. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art and propose a curling game video description method based on a large language model. The method combines the advantages of video analysis technology and a large language model to achieve a comprehensive, detailed and in-depth natural language description of the curling game video. The present invention not only improves the accuracy and comprehensiveness of the curling game video description, but also provides viewers and coaches with more abundant game information and analysis methods.

[0005] The present invention is implemented by the following technical scheme. The present invention proposes a curling game video description method based on a large language model, and the method comprises the following steps:

[0006] Step 1: pre-processing the curling game video and extracting feature information in the curling game video; by constructing a feature vector, the feature information in the curling game video is converted into a data form that can be processed by a computer; the feature information includes the action features of the curling players, the trajectory features of the curling ball and the situation features of the game;

[0007] Step 2: Input the extracted feature information into the large language model to generate a natural language description of the curling game process; in the process of generating the natural language description, the attention mechanism, context understanding, entity recognition and relationship extraction natural language processing technology are introduced to improve the accuracy and logic of the generated description;

[0008] Step 3: Post-process and optimize the generated natural language description of the curling game process; use natural language processing technology to perform grammar checking, semantic analysis, and coherence evaluation on the description to ensure that the generated description conforms to natural language specifications and is readable and practical.

[0009] Furthermore, the step 1 is specifically as follows:

[0010] Step 1.1, preprocessing of curling game video: extracting the curling players' movements, the trajectory of the curling ball and the key video clips of the curling game situation by detecting the key frames in the curling game video. At the same time, inter-frame difference and background elimination processing are performed on the key video clips to improve the accuracy and efficiency of subsequent steps;

[0011] Step 1.2, extracting feature information from key video clips: the feature information includes: curling player's action feature A, curling ball's trajectory feature T and curling game situation feature S; curling player's action feature includes: curling player's serial number A i , the force of throwing the curling ball A if , angle A ia , Speed ​​A iv And the frequency of wiping ice A is and intensity A it The trajectory characteristics of the curling ball include: the curling ball serial number T j , speed T jv , Direction T jd and the collision point with other curling balls or the boundary jp ; The characteristics of the curling game include: the number of rounds S k , Score Changes ks , Team Strategy S kg and the athlete's mental state kp ;

[0012] Step 1.3, construct feature vector V: construct feature vectors with various feature information extracted in step 1.2. It is necessary to perform feature normalization to ensure the consistency of numerical range and dimension between different features; design different normalization coefficients according to the importance of each feature, and obtain the normalized curling athlete's serial number A * i , the force of throwing the curling ball A * if , angle A * ia , Speed ​​A * iv And the frequency of wiping ice A * is and intensity A * it 、The curling ball number T* j , speed T * jv , Direction T * jd and the collision point with other curling balls or the boundary * jp 、S * k , Score Changes * ks , Team Strategy S * kg and the athlete's mental state * kp Among them, the throwing angle of the curling ball has the largest normalization coefficient in the action feature, the landing point of the curling ball has the largest normalization coefficient in the trajectory feature, and the score change has the largest normalization coefficient in the situation feature;

[0013] V=[A * i A * if A * ia A * iv A * is A * it T * j T * jv T * jd T * jp S * k S * ks S * kg S * kp ] T .

[0014] Furthermore, in step 2, an attention mechanism for curling game videos is constructed: the weight coefficients of the attention mechanism are introduced, including three categories: the first category is the weight coefficients of the action characteristics of the strength, angle, speed, and ice-rubbing frequency and strength of each athlete throwing the curling ball; the second category is the weight coefficients of the sliding trajectory, collision effect, and final stopping position trajectory characteristics of each curling ball; the third category is the weight coefficients of the situation characteristics of the curling game, such as the score changes of each game, team strategies, and the psychological state of the athletes.

[0015] Furthermore, in step 2, the contextual understanding of the curling game process is optimized: by understanding the rules and spirit of the curling game, accurately interpreting the athletes' actions and strategies, as well as the impact of these actions and strategies on the outcome of the game, and tracking the progress and changes in the game in real time; describing the various stages and key nodes of the game, generating a coherent and logical video description, ensuring that the description is semantically and logically consistent, and avoiding irrelevant or contradictory content.

[0016] Furthermore, in step 2, entity recognition and relationship extraction of the curling game process are specifically as follows: through entity recognition technology, entities identified from the text include: athletes, curling balls, number of rounds, teams and scores; through relationship extraction technology, the relationship between entities is obtained: how many points a team leads another team when a certain player throws a certain ball in a certain round.

[0017] Furthermore, in step 3, the grammar check of the natural language description of the curling game process is specifically as follows: using natural language processing tools to perform grammar check on the generated description to ensure that the sentence structure is correct and the wording is accurate; establishing a curling game terminology library to verify and correct the professional terms involved in the description to ensure the correct use of the terms.

[0018] Furthermore, in step 3, the semantic analysis of the natural language description of the curling game process is specifically: by identifying the entities, events and the relationships between them in the description, the accuracy and completeness of the description content at the semantic level are ensured; at the same time, the description is further enriched and optimized in combination with the rules and context of the curling game, so as to more accurately convey the key information and wonderful moments of the game process.

[0019] Furthermore, in step 3, the coherence evaluation of the natural language description of the curling game process is specifically as follows: the coherence of the generated natural language description is evaluated to ensure that the description content is consistent in logic and chronological order; by evaluating whether the connection and transition between sentences in the description are natural and smooth, and whether there is redundant or missing information, the overall coherence of the description and the reading experience are improved.

[0020] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for describing a curling game video based on a large language model are implemented.

[0021] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the method for describing a curling game video based on a large language model.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention proposes a curling game video description method based on a large language model. By intelligently parsing the video and generating a detailed text description, curling athletes can use less time to understand every detail of the game more deeply, provide detailed tactical analysis and action guidance, and help athletes improve their skills more quickly. By accumulating and analyzing the long-term curling game video and game data, the development law of curling can be revealed, providing valuable reference information for event organizers and coaches. At the same time, it can provide richer and more vivid game commentary content for video broadcast commentators, enhancing the attractiveness and competitiveness of media broadcasts. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0025] Figure 1 This is a flow chart of a curling game video description method based on a large language model described in the present invention. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Combination Figure 1 The present invention proposes a curling game video description method based on a large language model, the method comprising the following steps:

[0028] Step 1: pre-processing the curling game video and extracting feature information in the curling game video; by constructing a feature vector, the feature information in the curling game video is converted into a data form that can be processed by a computer; the feature information includes the action features of the curling players, the trajectory features of the curling ball and the situation features of the game;

[0029] The step 1 is specifically as follows:

[0030] Step 1.1, preprocessing of curling game video: extracting the curling players' movements, the trajectory of the curling ball and the key video clips of the curling game situation by detecting the key frames in the curling game video. At the same time, inter-frame difference and background elimination processing are performed on the key video clips to improve the accuracy and efficiency of subsequent steps;

[0031] Step 1.2, extracting feature information from key video clips: the feature information includes: curling player's action feature A, curling ball's trajectory feature T and curling game situation feature S; curling player's action feature includes: curling player's serial number A i , the force of throwing the curling ball A if , angle A ia , Speed ​​A iv And the frequency of wiping ice A is and intensity A it The trajectory characteristics of the curling ball include: the curling ball serial number T j , speed T jv , Direction T jd and the collision point with other curling balls or the boundary jp ; The characteristics of the curling game include: the number of rounds S k , Score Changes ks , Team Strategy S kg and the athlete's mental state kp ;

[0032] Step 1.3, construct feature vector V: construct feature vectors with various feature information extracted in step 1.2. It is necessary to perform feature normalization to ensure the consistency of numerical range and dimension between different features; design different normalization coefficients according to the importance of each feature, and obtain the normalized curling athlete's serial number A * i , the force of throwing the curling ball A * if , angle A * ia , Speed ​​A * iv And the frequency of wiping ice A * is and intensity A * it 、The curling ball number T * j , speed T * jv , Direction T * jd and the collision point with other curling balls or the boundary * jp 、S * k, Score Changes * ks , Team Strategy S * kg and the athlete's mental state * kp Among them, the throwing angle of the curling ball has the largest normalization coefficient in the action feature, the landing point of the curling ball has the largest normalization coefficient in the trajectory feature, and the score change has the largest normalization coefficient in the situation feature;

[0033] V=[A * i A * if A * ia A * iv A * is A * it T * j T * jv T * jd T * jp S * k S * ks S * kg S * kp ] T .

[0034] Step 2: Input the extracted feature information into the large language model to generate a natural language description of the curling game process; in the process of generating the natural language description, the attention mechanism, context understanding, entity recognition and relationship extraction natural language processing technology are introduced to improve the accuracy and logic of the generated description;

[0035] In step 2, the attention mechanism of the curling game video is constructed: the weight coefficients of the attention mechanism are introduced, including three categories: the first category is the weight coefficients of the strength, angle, speed of each athlete throwing the curling ball, and the frequency and strength of the ice-rubbing action characteristics; the second category is the weight coefficients of the sliding trajectory, collision effect and final stop position trajectory characteristics of each curling ball; the third category is the weight coefficients of the situation characteristics of the curling game, such as the score changes of each game, team strategies and the psychological state of the athletes.

[0036] In step 2, the contextual understanding of the curling game process is optimized: by understanding the rules and spirit of the curling game, accurately interpreting the athletes' actions and strategies, as well as the impact of these actions and strategies on the outcome of the game, and tracking the progress and changes in the game in real time; describing the various stages and key nodes of the game, such as the order in which athletes throw the stones, score changes, and tactical confrontations between teams, etc., to generate a coherent and logical video description, ensuring that the description content is consistent in semantics and logic, and avoiding irrelevant or contradictory content.

[0037] In step 2, entity recognition and relationship extraction of the curling game process are as follows: through entity recognition technology, entities identified from the text include: athletes, curling balls, number of rounds, teams and scores; through relationship extraction technology, the relationship between entities is obtained: how many points a team leads another team when a certain player throws a certain ball in a certain round.

[0038] Step 3: Post-process and optimize the generated natural language description of the curling game process; use natural language processing technology to perform grammar checking, semantic analysis, and coherence evaluation on the description to ensure that the generated description conforms to natural language specifications and is readable and practical.

[0039] In step 3, the grammar check of the natural language description of the curling competition process is as follows: using natural language processing tools to perform grammar check on the generated description to ensure correct sentence structure and accurate wording; establishing a curling competition terminology library to verify and correct the professional terms involved in the description to ensure the correct use of the terms. Through this step, the professionalism and accuracy of the description are improved to make it more in line with the context and norms of the curling competition.

[0040] In step 3, the semantic analysis of the natural language description of the curling game process is specifically as follows: by identifying the entities, events and the relationships between them in the description, the accuracy and completeness of the description content at the semantic level are ensured. At the same time, the description is further enriched and optimized in combination with the rules and context of the curling game to more accurately convey the key information and wonderful moments of the game process.

[0041] In step 3, the coherence evaluation of the natural language description of the curling game process is specifically as follows: the coherence of the generated natural language description is evaluated to ensure that the description content is consistent in logic and chronological order; by evaluating whether the connection and transition between sentences in the description are natural and smooth, and whether there is redundant or missing information, the overall coherence of the description and the reading experience are improved.

[0042] The present invention proposes a curling game video description method based on a large language model. The method combines the advantages of video analysis technology and a large language model to achieve a comprehensive, detailed and in-depth natural language description of the curling game video. Specifically, the method first uses video analysis technology to extract key frames and key information in the video, such as the movements of the players, the trajectory of the curling stone, the situation of the game, etc.; then, these key information are input into the large language model for natural language generation, thereby obtaining a comprehensive description of the game process.

[0043] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for describing a curling game video based on a large language model are implemented.

[0044] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the method for describing a curling game video based on a large language model.

[0045] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0046] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0047] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0048] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0049] The above is a detailed introduction to the curling game video description method based on a large language model proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A curling game video description method based on a large language model, characterized in that: The method comprises the following steps: Step 1: pre-processing the curling game video and extracting feature information in the curling game video; by constructing a feature vector, the feature information in the curling game video is converted into a data form that can be processed by a computer; the feature information includes the action features of the curling players, the trajectory features of the curling ball and the situation features of the game; Step 2: Input the extracted feature information into the large language model to generate a natural language description of the curling game process; in the process of generating the natural language description, the attention mechanism, context understanding, entity recognition and relationship extraction natural language processing technology are introduced to improve the accuracy and logic of the generated description; Step 3: Post-process and optimize the generated natural language description of the curling game process; use natural language processing technology to perform grammar checking, semantic analysis, and coherence evaluation on the description to ensure that the generated description conforms to natural language specifications and is readable and practical.

2. The method according to claim 1, characterized in that The step 1 is specifically as follows: Step 1.1, preprocessing of curling game video: extracting the curling players' movements, the trajectory of the curling ball and the key video clips of the curling game situation by detecting the key frames in the curling game video. At the same time, inter-frame difference and background elimination processing are performed on the key video clips to improve the accuracy and efficiency of subsequent steps; Step 1.2, extracting feature information from key video clips: the feature information includes: curling player's action feature A, curling ball's trajectory feature T and curling game situation feature S; curling player's action feature includes: curling player's serial number A i , the force of throwing the curling ball A if , angle A ia , Speed ​​A iv And the frequency of wiping ice A is and intensity A it The trajectory characteristics of the curling ball include: the curling ball serial number T j , speed T jv , Direction T jd and the collision point with other curling balls or the boundary jp ; The characteristics of the curling game include: the number of rounds S k , Score Changes ks , Team Strategy S kg and the athlete's mental state kp ; Step 1.3, construct feature vector V: construct feature vectors with various feature information extracted in step 1.

2. It is necessary to perform feature normalization to ensure the consistency of numerical range and dimension between different features; design different normalization coefficients according to the importance of each feature, and obtain the normalized curling athlete's serial number A * i , the force of throwing the curling ball A * if , angle A * ia , Speed ​​A * iv And the frequency of wiping ice A * is and intensity A * it 、The curling ball number T * j , speed T * jv , Direction T * jd and the collision point with other curling balls or the boundary * jp 、S * k , Score Changes * ks , Team Strategy S * kg and the athlete's mental state * kp Among them, the throwing angle of the curling ball has the largest normalization coefficient in the action feature, the landing point of the curling ball has the largest normalization coefficient in the trajectory feature, and the score change has the largest normalization coefficient in the situation feature; V=[A * i A * if A * ia A * iv A * is A * it T * j T * jv T * jd T * jp S * k S * ks S * kg S * kp ] T 。 3. The method according to claim 1, characterized in that In step 2, the attention mechanism of the curling game video is constructed: the weight coefficients of the attention mechanism are introduced, including three categories: the first category is the weight coefficients of the strength, angle, speed of each athlete throwing the curling ball, and the frequency and strength of the ice-rubbing action characteristics; the second category is the weight coefficients of the sliding trajectory, collision effect and final stop position trajectory characteristics of each curling ball; the third category is the weight coefficients of the situation characteristics of the curling game, such as the score changes of each game, team strategies and the psychological state of the athletes.

4. The method according to claim 1, characterized in that: In step 2, the contextual understanding of the curling game process is optimized: by understanding the rules and spirit of the curling game, accurately interpreting the players' actions and strategies, as well as the impact of these actions and strategies on the outcome of the game, and tracking the progress and changes in the game in real time; describing the various stages and key nodes of the game, generating a coherent and logical video description, ensuring that the description is semantically and logically consistent, and avoiding irrelevant or contradictory content.

5. The method according to claim 1, characterized in that In step 2, entity recognition and relationship extraction of the curling game process are as follows: through entity recognition technology, entities identified from the text include: athletes, curling balls, number of rounds, teams and scores; through relationship extraction technology, the relationship between entities is obtained: how many points a team leads another team when a certain player throws a certain ball in a certain round.

6. The method according to claim 1, characterized in that In step 3, the grammar check of the natural language description of the curling game process is specifically as follows: using natural language processing tools to check the grammar of the generated description to ensure that the sentence structure is correct and the wording is accurate; establishing a curling game terminology library to verify and correct the professional terms involved in the description to ensure the correct use of the terms.

7. The method according to claim 1, characterized in that In step 3, the semantic analysis of the natural language description of the curling game process is specifically as follows: by identifying the entities, events and the relationships between them in the description, the accuracy and completeness of the description content at the semantic level are ensured. At the same time, the description is further enriched and optimized in combination with the rules and context of the curling game to more accurately convey the key information and wonderful moments of the game process.

8. The method according to claim 1, characterized in that In step 3, the coherence evaluation of the natural language description of the curling game process is specifically as follows: the coherence of the generated natural language description is evaluated to ensure that the description content is consistent in logic and chronological order; by evaluating whether the connection and transition between sentences in the description are natural and smooth, and whether there is redundant or missing information, the overall coherence of the description and the reading experience are improved.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images