Video analysis method and device, equipment and storage medium

Abstract

The invention discloses a video analysis method and device, equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps: responding to a triggered video analysis instruction, and converting the image content and / or audio content in a to-be-analyzed video into text information through a large language model; performing semantic analysis on the text information through a large language model to obtain a semantic analysis result; and based on a semantic analysis result, generating video description information corresponding to the text information by a large language model. According to the method, the text information obtained by converting the image content or the audio content is subjected to semantic analysis by using the large language model, and the video content can be deeply understood and described through the video description information generated according to the semantic analysis result, so that the user demand is met, and the use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video analysis method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology and multimedia technology, personal computers and mobile devices are becoming more and more popular. People watch and shoot more and more videos in their lives. Digital videos have become an indispensable part of people's learning and entertainment.

[0003] At present, in order to improve the user's viewing experience of the video, the existing method usually analyzes the video and extracts intuitive and obvious information in the video, such as subtitles, dialogue text, text in the image and other text information for display, so that users can know the basic information of the video. However, for users who need to deeply understand the video content, the information obtained by the above method cannot meet their needs, resulting in poor user experience.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a video analysis method, device, equipment and storage medium, aiming to solve the technical problem that the existing technology cannot meet user needs, resulting in poor user experience.

[0006] To achieve the above object, the present application provides a video analysis method, the method comprising:

[0007] In response to the triggered video analysis instruction, converting the image content and / or audio content in the video to be analyzed into text information through the large language model;

[0008] Performing semantic analysis on the text information by using the large language model to obtain a semantic analysis result;

[0009] Based on the semantic analysis result, the large language model generates video description information corresponding to the text information.

[0010] In one embodiment, the step of generating video description information corresponding to the text information by the large language model based on the semantic analysis result includes:

[0011] Identifying key picture content in the image content that matches a preset type;

[0012] Determining the picture description information corresponding to the key picture content in the semantic analysis result;

[0013] The picture description information and the text information are merged by using the large language model to generate video description information corresponding to the text information.

[0014] In one embodiment, the step of identifying key screen content that meets a preset type in the video to be analyzed includes:

[0015] Determining visual features of each video frame in the video to be analyzed;

[0016] Obtaining baseline video features associated with a preset type;

[0017] Determining the similarity between each of the visual features and the reference video features;

[0018] The video frames corresponding to the video feature information whose similarity reaches a preset similarity are selected as the key picture content.

[0019] In one embodiment, the step of determining the screen description information corresponding to the key screen content in the semantic analysis result includes:

[0020] Extracting first description information describing the positional relationship between the objects in the key picture content and second description information describing the semantic relationship between the objects in the key picture content from the semantic analysis result;

[0021] The screen description information is generated according to the first description information and the second description information.

[0022] In one embodiment, after the step of generating the video description information corresponding to the text information by the large language model based on the semantic analysis result, the method further includes:

[0023] Receiving a content generation instruction based on the video description information feedback from the user;

[0024] Acquire a target text template corresponding to the content generation instruction from a preset text template;

[0025] Extracting a topic feature matching a page element of the target text template from the video description information;

[0026] Each of the subject features is added to a corresponding page element in the target text template to generate a target text.

[0027] In one embodiment, after the step of generating the video description information corresponding to the text information by the large language model based on the semantic analysis result, the method further includes:

[0028] Obtaining image information of the speaker in each of the videos to be analyzed;

[0029] Determine speaker identification information according to the image information;

[0030] The image information and the speaker identification information are displayed in a window of the video to be analyzed.

[0031] In one embodiment, the step of determining the speaker identification information according to the image information includes:

[0032] Obtain a preset character image library, wherein the preset character image library stores a plurality of character images and corresponding identification information;

[0033] Determining whether the image information matches any character image in the preset character image library;

[0034] If so, the identification information corresponding to the successfully matched character image is obtained as the speaker identification information.

[0035] In one embodiment, after the step of determining whether the image information matches any character image in the preset character image library, the method further includes:

[0036] If not, obtaining the dialogue text in which the speaker participated in the video description information;

[0037] Determining semantic information of the conversation text by using the large language model;

[0038] Determining whether there is identity information identifying the speaker in the conversation text according to the semantic information;

[0039] If it exists, the identity information is used as the speaker identification information;

[0040] If not present, a random keyword matching the semantic information is generated as the speaker identification information.

[0041] In one embodiment, after the step of generating the video description information corresponding to the text information by the large language model based on the semantic analysis result, the method further includes:

[0042] Acquire the speech text of the speaker in the video to be analyzed from the video description information, and extract each feature word in the speech text through the large language model;

[0043] Assigning a corresponding weight to each of the feature words according to the frequency of each of the feature words in the speech text;

[0044] Determine the speaker's opinion text according to each of the feature words and the corresponding weights;

[0045] The opinion text is displayed in the window of the video to be analyzed.

[0046] In one embodiment, the step of determining the speaker's opinion text according to each of the feature words and the corresponding weights includes:

[0047] Determining semantic information between each of the feature words;

[0048] Identify the sentiment type of each of the feature words through the large language model;

[0049] Add feature words belonging to the same sentiment type to the same list to form multiple feature sets;

[0050] Integrate the weights of the feature words in each feature set to obtain the sentiment weight of each feature set;

[0051] A target feature set with the highest sentiment weight is obtained, and an opinion text is generated according to each feature word in the target feature set.

[0052] In one embodiment, after the step of generating the video description information corresponding to the text information by the large language model based on the semantic analysis result, the method further includes:

[0053] Determine the start time and end time of the speech of the speaker in the video to be analyzed according to the video description information;

[0054] Generate a timestamp according to the speech start time and the speech end time;

[0055] Displaying the timestamp in the window of the video to be analyzed, and configuring a click event for each time point in the timestamp;

[0056] When a user triggers a click event corresponding to any time point based on the timestamp, the video position corresponding to the any time point in the video to be analyzed is located.

[0057] In addition, to achieve the above-mentioned purpose, the present application also proposes a video analysis device, the device comprising:

[0058] A text conversion module, configured to convert image content and / or audio content in the video to be analyzed into text information through a large language model in response to a triggered video analysis instruction;

[0059] A semantic analysis module, used to perform semantic analysis on the text information through the large language model to obtain a semantic analysis result;

[0060] A content output module is used to generate video description information corresponding to the text information by the large language model based on the semantic analysis result.

[0061] In one embodiment, the content output module is further used to identify key picture content that meets a preset type in the image content;

[0062] The content output module is further used to determine the picture description information corresponding to the key picture content in the semantic analysis result;

[0063] The content output module is further used to merge the picture description information and the text information using the large language model to generate video description information corresponding to the text information.

[0064] In one embodiment, the content output module is further used to determine the visual features of each video frame in the video to be analyzed;

[0065] The content output module is further used to obtain reference video features associated with a preset type;

[0066] The content output module is further used to determine the similarity between each of the visual features and the reference video features;

[0067] The content output module is further used to select video frames corresponding to video feature information whose similarity reaches a preset similarity as key picture content.

[0068] In one embodiment, the content output module is further used to extract first description information describing the positional relationship between objects in the key picture content and second description information describing the semantic relationship between objects in the key picture content from the semantic analysis result;

[0069] The content output module is further used to generate screen description information according to the first description information and the second description information.

[0070] In one embodiment, the content output module is further used to receive a content generation instruction based on the video description information feedback from the user;

[0071] The content output module is further used to obtain a target text template corresponding to the content generation instruction from a preset text template;

[0072] The content output module is further used to extract the subject features matching the page elements of the target text template from the video description information;

[0073] The content output module is further used to add each of the subject features to the corresponding page elements in the target text template to generate the target text.

[0074] In one embodiment, the content output module is further used to obtain image information of the speaker in each of the videos to be analyzed;

[0075] The content output module is further used to determine the speaker identification information according to the image information;

[0076] The content output module is further used to display the image information and the speaker identification information in the window of the video to be analyzed.

[0077] In one embodiment, the content output module is further used to obtain a preset character image library, wherein the preset character image library contains a plurality of character images and corresponding identification information;

[0078] The content output module is further used to determine whether the image information matches any character image in the preset character image library;

[0079] The content output module is further configured to obtain identification information corresponding to the successfully matched character image as the speaker identification information.

[0080] In addition, to achieve the above-mentioned purpose, the present application also proposes a video analysis device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the video analysis method described above.

[0081] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the video analysis method described above are implemented.

[0082] One or more technical solutions proposed in this application have at least the following technical effects:

[0083] The present application discloses a video analysis method, device, equipment and storage medium, and relates to the field of computer technology. The disclosed method includes: in response to a triggered video analysis instruction, converting the image content and / or audio content in the video to be analyzed into text information through a large language model; performing semantic analysis on the text information through the large language model to obtain a semantic analysis result; based on the semantic analysis result, generating video description information corresponding to the text information by the large language model. The present application uses a large language model to perform semantic analysis on the text information obtained by converting the image content or audio content, and then generates video description information for the text information based on the semantic analysis result. Compared with the prior art that can only extract the text information of the video, the present application can further perform semantic analysis on the text information after obtaining the text information, and obtain more in-depth video description information, thereby achieving a deep understanding and description of the video content, thereby meeting user needs and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0085] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0086] Figure 1 This is a flow chart of the first embodiment of the video analysis method of the present application;

[0087] Figure 2 This is a flow chart of the second embodiment of the video analysis method of the present application;

[0088] Figure 3 This is a flow chart of the third embodiment of the video analysis method of the present application;

[0089] Figure 4 This is a schematic diagram of the module structure of the video analysis device of this application;

[0090] Figure 5 The figure is a schematic diagram of the structure of a video analysis device suitable for implementing the embodiment of the present application.

[0091] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0092] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0093] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0094] The main solution of the embodiment of the present application is: the video analysis device responds to the triggered video analysis instruction, converts the image content and / or audio content in the video to be analyzed into text information through a large language model; performs semantic analysis on the text information through the large language model to obtain a semantic analysis result; based on the semantic analysis result, the large language model generates video description information corresponding to the text information.

[0095] In the embodiments of the present application, for ease of description, the following description is made using a video analysis device as the execution subject.

[0096] The existing technology analyzes the video and extracts text information for display. However, for users who need to deeply understand the video content, the basic information obtained in the above manner cannot meet their needs, resulting in poor user experience.

[0097] The present application provides a solution, which uses a large language model to perform semantic analysis on the text information obtained by converting image content or audio content, and then generates video description information for the text information based on the semantic analysis result. Compared with the existing technology that can only extract text information of the video, the present application can further perform semantic analysis on the text information after obtaining the text information, and obtain more in-depth video description information, thereby achieving a deep understanding and description of the video content, thereby meeting user needs and improving user experience.

[0098] It should be noted that the execution subject of this embodiment can be a computing service device with video analysis, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a video analysis device, etc. The following takes the video analysis device as an example to illustrate this embodiment and the following embodiments.

[0099] Based on this, the present application embodiment provides a video analysis method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the video analysis method of the present application.

[0100] In this embodiment, the video analysis method includes steps S10 to S30:

[0101] Step S10, in response to the triggered video analysis instruction, converting the image content and / or audio content in the video to be analyzed into text information through the large language model.

[0102] It should be noted that the above-mentioned large language model can be a model constructed using deep learning and natural language processing technologies.

[0103] In a specific implementation, the above-mentioned large language model can learn text extraction and text analysis (such as analyzing the language patterns and semantic relationships of the text) from a large amount of text data, image data and audio data through unsupervised learning, so that it has the ability to understand and generate natural language text. After training, the large language model can be saved to the cloud or locally on the video analysis device.

[0104] Further, it is understandable that the video to be analyzed can be actively uploaded by the user, or the user can send a specified path to the video analysis device, and the video analysis device obtains the video at the specified path as the video to be analyzed. When the user does not specify the video to be analyzed, the video analysis device can use the currently playing video as the video to be analyzed. The user can trigger the video analysis instruction by clicking a button, sending a command, etc., and the video analysis device can respond to the video analysis instruction and obtain the large language model from the local or cloud. When the video analysis instruction analyzes the image content in the video to be analyzed, the large language model can extract each video frame in the video to be analyzed, and extract the text information contained in the image content corresponding to each video frame; when the video analysis instruction analyzes the audio content in the video to be analyzed, the large language model can extract the audio content in the video to be analyzed and convert the audio content into corresponding text information; when the video analysis instruction analyzes the image content and audio content in the video to be analyzed, the large language model can respectively extract the text information contained in the image content corresponding to each video frame in the video to be analyzed and convert the audio content into corresponding text information, and integrate them to obtain the final text information.

[0105] Step S20: performing semantic analysis on the text information using the large language model to obtain a semantic analysis result.

[0106] In a specific implementation, after obtaining the text information of the video to be analyzed, the above-mentioned video analysis device can use a large language model to identify the sentence structure of the text information, and then identify the roles of each part of the text information according to the sentence structure of the text information, and add corresponding role labels, and then obtain a pre-configured role library, which includes different role labels and their corresponding intentions. The role labels based on the above-mentioned text information are matched in the pre-built role library to determine the intentions corresponding to each role label, and the semantics of the text information are determined through each intention. The semantics can be used as a semantic analysis result to characterize the relationship between words, phrases, and sentences in the text information, as well as the meaning and context expressed by each.

[0107] Step S30: Based on the semantic analysis result, the large language model generates video description information corresponding to the text information.

[0108] In a specific implementation, the above-mentioned video analysis device can generate a coherent text describing the text information based on the semantic analysis results through a large language model, and combine it with the text information as the video description information of the video to be analyzed. The video description information can be used to describe the meaning of each part of the video to be analyzed, so as to facilitate users to have a deeper understanding of the main content of the video to be analyzed.

[0109] In a feasible implementation manner, step S30 may further include steps A31 to A34:

[0110] Step A31, receiving a content generation instruction from a user based on the video description information feedback.

[0111] It should be noted that the above-mentioned content generation instruction may be an instruction that triggers the generation of user-specified text.

[0112] In a specific implementation, after the user obtains the video description information of the video to be analyzed, if the user wishes to display it through other text, the user can trigger a content generation instruction to the video analysis device by clicking a button, sending a command, and other operations. The above-mentioned video analysis device can receive the content generation instruction and respond to the content generation instruction.

[0113] Step A32, obtaining a target text template corresponding to the content generation instruction from a preset text template.

[0114] It should be noted that the above-mentioned preset text template may be a pre-configured template, such as a presentation template, a mind map template, etc.

[0115] In a specific implementation, after responding to the content generation instruction, the video analysis device may obtain a target text template corresponding to the content generation instruction from a plurality of preset text templates stored locally.

[0116] Step A33: extracting the subject features matching the page elements of the target text template from the video description information.

[0117] It should be noted that the above-mentioned page elements may be elements constituting the page structure of the target text template.

[0118] In a specific implementation, the video analysis device can determine the page structure that constitutes the target text target, the various page elements, and characterize the element features of the page elements, and then match the various role tags in the video description information with the element features, and calculate the similarity between each calculated tag and each element feature. When the similarity between any element feature and any role tag is greater than a preset similarity calibration value, the match is determined to be successful, and any role tag that is successfully matched can be a theme feature. Repeat the above process to extract the theme features that match each page element of the target text template.

[0119] Step A34, adding each of the subject features to the corresponding page element in the target text template to generate the target text.

[0120] In a specific implementation, the video analysis device may add each topic feature to the matching page elements in the target text target to generate a final target text.

[0121] Exemplarily, a presentation template is used as an example of the target text template. The page structure of the presentation template includes a cover page, a directory page and a content page. The subject features in the video description information that match the cover page may be the title, author, and date of the video to be analyzed. The title, author, and date may be added to the corresponding element positions in the cover page. The subject features in the video description information that match the directory page may be the scene descriptions of different scenes of the video to be analyzed. Each scene description may be added to the corresponding element of the directory page. The subject features in the video description information that match the content page may be the specific content descriptions in different scenes. Each specific content description may be added to the content page to form a final presentation.

[0122] In addition, after the above-mentioned video analysis device obtains the video description information, the user can enter the follow-up mode by performing a corresponding click operation. In the follow-up mode, the video analysis device can display an interactive page to the user, and can provide a series of preset follow-up questions or options. The user can enter the corresponding entry by clicking or selecting these questions, or the user can manually enter a custom entry. The video analysis device can respond to the entry entered by the user, and feedback the follow-up results related to the semantics of the entry in the video description information through the large language model. If the follow-up results related to the entry cannot be retrieved in the video description information, the extended search engine can be called to feedback the follow-up results related to the language of the entry through the search engine.

[0123] In this embodiment, in response to a triggered video analysis instruction, the image content and / or audio content in the video to be analyzed is converted into text information through a large language model; the text information is semantically analyzed by the large language model to obtain a semantic analysis result; based on the semantic analysis result, the large language model generates video description information corresponding to the text information. After this embodiment uses a large language model to perform semantic analysis on the text information obtained by converting the image content or audio content, it generates video description information for the text information based on the semantic analysis result. Compared with the prior art that can only extract the text information of the video, this embodiment can further perform semantic analysis on the text information after obtaining the text information to obtain a more in-depth video description information, thereby achieving a deep understanding and description of the video content, thereby meeting user needs and improving the user experience.

[0124] Based on the first embodiment of the present application, the second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the first embodiment can be referred to the above introduction, and will not be repeated later. Figure 2 , Figure 2 This is a flow chart of the second embodiment of the video analysis method of the present application.

[0125] In this embodiment, step S30 may include steps S301 to S302:

[0126] Step S301, identifying key picture contents in the image content that conform to a preset type.

[0127] It should be noted that the above-mentioned key screen content may be a preset type of screen content, and the preset type may be a pre-configured type, for example, a specific person, animal, landmark or scene type.

[0128] In a specific implementation, in order to improve the description accuracy of the video description information for the video to be analyzed, the above-mentioned video analysis device can extract key picture content that meets the preset type from the image content of each video frame of the video to be analyzed, or determine the target text information that meets the preset type from the text information, and obtain the picture content corresponding to the target text information as the key picture content, so as to subsequently generate information describing the key picture content.

[0129] It should be understood that the above-mentioned video analysis device may store multiple type information representing the type of picture content, and after the user inputs a video analysis instruction, each type of information may be displayed to the user. The user may perform a selection operation on each type of information, triggering a type selection instruction. The video analysis device may obtain the type information corresponding to the type selection instruction, and use the type corresponding to the obtained type information as the preset type.

[0130] In a feasible implementation, step S301 may include steps S3011 to S3014:

[0131] Step S3011, determining the visual features of each video frame in the video to be analyzed.

[0132] It should be noted that the above-mentioned visual features may be key visual elements or attributes for describing and identifying the contents of the picture, such as color, shape, and position.

[0133] In a specific implementation, the video analysis device may extract each video frame of the video to be analyzed, and perform feature extraction on the picture content in each video frame to obtain the visual features of each video frame.

[0134] Step S3012, obtaining a reference video feature associated with a preset type.

[0135] It should be noted that the above-mentioned reference video features may be pre-set video features that conform to a preset type.

[0136] In a specific implementation, a corresponding model can be obtained by training and learning a large number of related videos and different feature types in advance, and then the model can be used to output video features corresponding to the preset type as the benchmark video features, or the developer can manually define and construct the benchmark video features in advance. When the above-mentioned video analysis device obtains the visual features of each video frame, it can obtain the benchmark video features associated with the preset type.

[0137] Step S3013: Determine the similarity between each of the visual features and the reference video feature.

[0138] In a specific implementation, the video analysis device may respectively calculate the similarity between each video feature information and the reference video feature to identify whether the picture content corresponding to each video feature information conforms to a preset type.

[0139] Step S3014: Select the video frame corresponding to the video feature information whose similarity reaches a preset similarity as the key picture content.

[0140] It should be noted that the above-mentioned preset similarity may be a pre-set calibration value used to determine whether the similarity between the visual feature and the reference video feature is high.

[0141] In a specific implementation, after calculating the similarity between each video feature and the reference video feature, the video analysis device may select video frames corresponding to all video feature information reaching a preset similarity from each similarity as key picture content.

[0142] Step S302: determining the picture description information corresponding to the key picture content in the semantic analysis result.

[0143] It should be noted that the above screen description information may be text describing key screen contents.

[0144] In a specific implementation, the video analysis device may determine the corresponding semantic information of each key picture content in the semantic analysis result, and then generate picture description information corresponding to the picture content according to each semantic information.

[0145] In a feasible implementation, step S302 may include steps S3021 to S3022:

[0146] Step S3021 : extracting first description information describing the positional relationship between the objects in the key picture content and second description information describing the semantic relationship between the objects in the key picture content from the semantic analysis result.

[0147] In a specific implementation, the above-mentioned large language model can identify all objects (such as people, animals, objects, buildings, etc.) in each video frame, and determine the relative position relationship between them, as well as the attributes, behaviors of each object and the semantic relationships such as causal relationship and time relationship between them, and generate a semantic analysis result containing the above-mentioned relative position relationship and semantic relationship. The above-mentioned video analysis device can extract the first description information describing the position relationship between different objects in the key picture content from the semantic analysis result, and at the same time, extract the second description information describing the semantic relationship between different objects in the key picture content from the semantic analysis result.

[0148] Step S3022: Generate screen description information according to the first description information and the second description information.

[0149] In a specific implementation, the video analysis device may integrate the first description information and the second description information to generate picture description information, through which the attributes of any object in the key picture content, its relative position with other objects, and its semantic relationship with other objects may be described.

[0150] Step S303: merging the picture description information and the text information using the large language model to generate video description information corresponding to the text information.

[0151] In a specific implementation, the above-mentioned video analysis device can use a large language model to identify the contextual relationship between the picture description information and the text information, merge the picture description information and the text information according to the contextual relationship, and generate video description information, so that the generated video description information contains both the text information of each video frame in the video to be analyzed and the description information of the key picture content in the video to be analyzed.

[0152] This embodiment identifies key picture contents that match preset types in image contents; determines picture description information corresponding to the key picture contents in the semantic analysis results; and uses the large language model to merge the picture description information and text information to generate video description information corresponding to the text information, so that the video description information can describe the key picture contents of preset types, thereby improving the description accuracy of the video description information for the video to be analyzed.

[0153] Based on the first and second embodiments of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as those of the first and second embodiments can be referred to the above introduction, and will not be repeated later. Figure 3 , Figure 3 This is a flow chart of the third embodiment of the video analysis method of the present application.

[0154] In this embodiment, in order to improve the user's viewing experience of the video to be analyzed, steps S40 to S60 may be further included after step S30:

[0155] Step S40, obtaining image information of the speaker in each of the videos to be analyzed.

[0156] In a specific implementation, the video analysis device may also perform face recognition on each video frame of the video to be analyzed, identify each person as a spokesperson, and crop a corresponding face image at the position of each spokesperson's head portrait as the image information of the spokesperson.

[0157] Step S50: determining speaker identification information according to the image information.

[0158] It should be noted that the above identification information may be information for identifying or describing the speaker, such as name, nickname, avatar, number, etc.

[0159] In a specific implementation, the video analysis device may randomly generate multiple identification information and configure unique identification information for different speakers to distinguish different speakers.

[0160] In a feasible implementation, step S50 may include steps S501 to S503:

[0161] Step S501: obtaining a preset character image library, wherein the preset character image library stores a plurality of character images and corresponding identification information.

[0162] In a specific implementation, the video generation device may obtain multiple character images and their associated identification information from an online resource library, or may receive a character image input by a user and its associated, user-defined identification information, and then save the associated character images and identification information to the same database to form a preset character image library. After determining the speaker identification information of the video to be analyzed, the video analysis device may obtain the preset character image library to perform subsequent operations.

[0163] Step S502: determine whether the image information matches any character image in the preset character image library.

[0164] In a specific implementation, the video analysis device may traverse each character image in a preset character image library, match the image information of each speaker in the video to be analyzed with each character image, and determine whether the image information of any speaker matches any character image.

[0165] Step S503: If yes, then obtain identification information corresponding to the successfully matched person image as speaker identification information.

[0166] In a specific implementation, when the image information of any speaker matches any character image in a preset character image library, the video analysis device may use the identification information associated with the successfully matched character image as the identification information of the speaker.

[0167] In a feasible implementation manner, step S502 may further include steps S504 to S508:

[0168] Step S504: if not, obtaining the dialogue text in which the speaker participates in the video description information.

[0169] In a specific implementation, the video analysis device may obtain a dialogue text converted from the speaker's voice data in the text information of the video description information when the image information of any speaker does not match any character image in the preset character image library.

[0170] Step S505: determining semantic information of the dialogue text by using the large language model.

[0171] In a specific implementation, the video analysis device may perform semantic analysis on the conversation text through a large language model to obtain semantic information of the conversation text.

[0172] Step S506: determining whether there is identity information identifying the speaker in the conversation text according to the semantic information.

[0173] In a specific implementation, the video analysis device can determine whether there is any word in the dialogue text whose semantics can represent the speaker's identity based on the semantic information, and use the word representing the speaker's identity as the identity information for identifying the speaker's identity.

[0174] Step S507: If yes, use the identity information as speaker identification information.

[0175] In a specific implementation, when the video analysis device detects that there is identity information identifying the speaker's identity in the conversation text, that is, there is any word whose semantics represent the speaker's identity, the word can be used as the identity information identifying the speaker's identity, and the identity information can be configured as the speaker identification information.

[0176] Step S508: If it does not exist, a random keyword matching the semantic information is generated as the speaker identification information.

[0177] In a specific implementation, when the video analysis device detects that there is no identity information identifying the speaker in the conversation text, it can randomly generate random keywords that conform to the semantic information as the speaker identification information.

[0178] It should be understood that when the image information of any speaker does not match any character image in the preset character image library, by judging whether there is identity information identifying the identity of the speaker in the dialogue text, different strategies are adopted according to different judgment results to configure corresponding speaker identification information for the speaker, thereby effectively improving the configuration accuracy of the speaker identification information.

[0179] Step S60: displaying the image information and the speaker identification information in the window of the video to be analyzed.

[0180] In a specific implementation, the video analysis device may display the image information of each speaker and the corresponding speaker identification information in a window of the video to be analyzed, such as the right window.

[0181] This embodiment obtains the image information of the speaker in each video to be analyzed; determines the speaker identification information based on the image information; and displays the image information and the speaker identification information in the window of the video to be analyzed, so that when the user watches the video to be analyzed, the user's understanding of each character in the video to be analyzed is deepened, thereby effectively improving the user's viewing experience of the video to be analyzed.

[0182] In a feasible implementation manner, step S30 may further include steps S71 to S74:

[0183] Step S71, obtaining the speech text of the speaker in the video to be analyzed from the video description information, and extracting each feature word in the speech text through the large language model.

[0184] It should be noted that the above speech text may be a text converted from the speech of the speaker in the video to be analyzed.

[0185] In a specific implementation, the above-mentioned video analysis device can identify each speaker in the video to be analyzed, and then obtain the speech text of each speaker from the video description information, and then use the large language model to segment each speech text, wherein the segmented words of the speech text can be feature words.

[0186] Step S72: assigning a corresponding weight to each of the feature words according to the frequency of each of the feature words in the speech text.

[0187] In a specific implementation, the above-mentioned video analysis device can count the frequency of occurrence of feature words in each speech text as word frequency, and then configure different weights for each feature word according to the frequency, among which feature words with high word frequency are configured with higher weights than feature words with low word frequency.

[0188] Step S73, determining the speaker's opinion text according to each of the feature words and the corresponding weights.

[0189] It should be noted that the above-mentioned opinion text may be a text reflecting the main opinions of the speaker.

[0190] In a specific implementation, for the speech text of any speaker, the above-mentioned video analysis device can obtain all feature words whose weights reach a preset weight reference value, and use a large language model to analyze the semantic relationship of each feature word whose weight reaches the weight reference value, and combine them according to the semantic relationship to generate an opinion text of the speech text. The above process can be repeated to obtain the opinion text of the speech text of each speaker.

[0191] The weight reference value may be a value with a larger weight. When the weight of any feature word reaches the weight reference value, it is determined that the feature word is repeatedly mentioned by the speaker, and it can be determined that the feature word can reflect the speaker's point of view.

[0192] In a feasible implementation manner, step S73 may include steps S731 to S735:

[0193] Step S731, determining the semantic information between the feature words.

[0194] In a specific implementation, the video analysis device may determine the semantic relationship between different feature words as semantic information through a large language model.

[0195] Step S732: identifying the emotion type of each of the feature words through the large language model.

[0196] In a specific implementation, the emotion types can be annotated for multiple training data in advance, and then the large language model can be trained using the annotated training data to fine-tune the large language model so that the large language model has the emotion analysis function. Based on this, the above-mentioned video analysis device can input the feature words of each speech text into the fine-tuned large language model, and output the emotion type corresponding to each feature word through the large language model, and the emotion type includes positive, negative, neutral, etc.

[0197] Step S733, adding feature words belonging to the same emotion type to the same list to form multiple feature sets.

[0198] In a specific implementation, for any speaker's speech text, the video analysis device can add feature words of the same emotion type in the speech text to the same list to obtain multiple feature sets. By repeating the above process, the feature set of each speech text can be obtained.

[0199] Step S734, integrating the weights of the feature words in each of the feature sets to obtain the sentiment weight of each of the feature sets.

[0200] In a specific implementation, for any speaker's speech text, the video analysis device can add the weights of the feature words in each corresponding feature set to obtain the sentiment weight of each feature set. By repeating the above process, the sentiment weight of the feature set of each speech text can be obtained.

[0201] Step S735, obtaining a target feature set with the highest sentiment weight, and generating an opinion text according to each feature word in the target feature set.

[0202] In a specific implementation, for any speaker's speech text, the video analysis device can obtain the target feature set with the highest sentiment weight, and use a large language model to analyze the semantic relationship of each feature word in the target feature set, combine them according to the semantic relationship, and generate the opinion text of the speech text. By repeating the above process, the opinion text of each speaker's speech text can be accurately obtained.

[0203] Step S74, displaying the opinion text in the window of the video to be analyzed.

[0204] In a specific implementation, the video analysis device may display the text of each speaker's opinion in a window of the video to be analyzed, such as a window on the left side of the video to be analyzed.

[0205] This embodiment obtains the speech text of the speaker in the video to be analyzed from the video description information, extracts each feature word in the speech text through a large language model, configures a corresponding weight for each feature word according to the word frequency of each feature word in the speech text, and then determines the speaker's opinion text according to each feature word and the corresponding weight, and displays the opinion text in the window of the video to be analyzed, so that the user can know the opinion text of each speaker when watching the video to be analyzed, thereby making it easier to understand the relationship between the characters in the video to be analyzed, effectively improving the user's viewing experience of the video to be analyzed.

[0206] In a feasible implementation manner, step S30 may further include steps S81 to S84:

[0207] Step S81, determining the speech start time and speech end time of the speaker in the video to be analyzed according to the video description information.

[0208] It should be noted that the above-mentioned speech start time may be the time when the speaker starts speaking, and the above-mentioned speech end time may be the time when the speaker ends speaking.

[0209] In a specific implementation, the video analysis device may extract text information from the video description information, and determine the speech start time and speech end time of each speaker in the video to be analyzed during the speech from the text information.

[0210] Step S82: Generate a timestamp according to the speech start time and the speech end time.

[0211] In a specific implementation, the video analysis device may determine the total speech time according to the speech start time and the speech end time, and then generate a timestamp representing each time point under the total speech time.

[0212] Step S83, displaying the timestamp in the window of the video to be analyzed, and configuring a click event for each time point in the timestamp.

[0213] It should be noted that the above-mentioned click event can be an event at any time point in the user's click event stamp.

[0214] In a specific implementation, the video analysis device may display the timestamp in the window of the video to be analyzed, such as below the progress bar of the window, and configure a corresponding click event for each time point of the timestamp.

[0215] Step S84, when the user triggers a click event corresponding to any time point based on the timestamp, locate the video position corresponding to the any time point in the video to be analyzed.

[0216] In a specific implementation, a user can select any time point according to the timestamp in the window of the video to be analyzed to trigger a corresponding click event. The above-mentioned video analysis device can respond to the click event, locate the video position corresponding to the time point in the video to be analyzed, and then jump the content currently being played in the video to be analyzed to the content of the video position.

[0217] This embodiment determines the speech start time and speech end time of the speaker in the video to be analyzed based on the video description information; generates a timestamp based on the speech start time and speech end time; displays the timestamp in the window of the video to be analyzed, and configures a click event for each time point in the timestamp; when the user triggers a click event corresponding to any time point based on the timestamp, locates the video position corresponding to any time point in the video to be analyzed, so that the user can jump the playback content of the video to be analyzed to the speech time point of the speaker to be viewed, thereby effectively improving the user's viewing experience of the video to be analyzed.

[0218] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the video analysis method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0219] This application also provides a video analysis device, please refer to Figure 4 , Figure 4 This is a schematic diagram of the module structure of the video analysis device of the present application, and the video analysis device includes:

[0220] A text conversion module 10, configured to convert image content and / or audio content in the video to be analyzed into text information through a large language model in response to a triggered video analysis instruction;

[0221] A semantic analysis module 20, configured to perform semantic analysis on the text information using the large language model to obtain a semantic analysis result;

[0222] The content output module 30 is used to generate video description information corresponding to the text information by the large language model based on the semantic analysis result.

[0223] The video analysis device provided by the present application adopts the video analysis method in the above embodiment, which can solve the technical problem that the existing technology cannot meet user needs, resulting in poor user experience. Compared with the existing technology, the beneficial effects of the video analysis device provided by the present application are the same as the beneficial effects of the video analysis method provided by the above embodiment, and the other technical features in the video analysis device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0224] As an implementation manner, the content output module 30 is further used to identify key picture content that meets a preset type in the image content;

[0225] The content output module 30 is further used to determine the picture description information corresponding to the key picture content in the semantic analysis result;

[0226] The content output module 30 is further configured to use the large language model to merge the picture description information and the text information to generate video description information corresponding to the text information.

[0227] As an implementation manner, the content output module 30 is further used to determine the visual features of each video frame in the video to be analyzed;

[0228] The content output module 30 is also used to obtain reference video features associated with a preset type;

[0229] The content output module 30 is further used to determine the similarity between each of the visual features and the reference video features;

[0230] The content output module 30 is further configured to select video frames corresponding to video feature information whose similarity reaches a preset similarity as key picture content.

[0231] As an implementation manner, the content output module 30 is further used to extract first description information describing the positional relationship between the objects in the key picture content and second description information describing the semantic relationship between the objects in the key picture content from the semantic analysis result;

[0232] The content output module 30 is further configured to generate screen description information according to the first description information and the second description information.

[0233] As an implementation mode, the content output module 30 is further used to receive a content generation instruction from a user based on the video description information feedback;

[0234] The content output module 30 is further used to obtain a target text template corresponding to the content generation instruction from a preset text template;

[0235] The content output module 30 is further used to extract the subject features matching the page elements of the target text template from the video description information;

[0236] The content output module 30 is further used to add each of the subject features to the corresponding page elements in the target text template to generate the target text.

[0237] As an implementation manner, the content output module 30 is further used to obtain image information of the speaker in each of the videos to be analyzed;

[0238] The content output module 30 is further used to determine the speaker identification information according to the image information;

[0239] The content output module 30 is further configured to display the image information and the speaker identification information in the window of the video to be analyzed.

[0240] As an implementation manner, the content output module 30 is further used to obtain a preset character image library, wherein the preset character image library stores a plurality of character images and corresponding identification information;

[0241] The content output module 30 is further used to determine whether the image information matches any character image in the preset character image library;

[0242] The content output module 30 is further configured to obtain identification information corresponding to the successfully matched character image as speaker identification information.

[0243] As an implementation manner, the content output module 30 is further configured to, if not, obtain the dialogue text in which the speaker participates in the video description information;

[0244] The content output module 30 is further used to determine the semantic information of the dialogue text through the large language model;

[0245] The content output module 30 is further used to determine whether there is identity information identifying the speaker's identity in the conversation text according to the semantic information;

[0246] The content output module 30 is further configured to use the identity information as speaker identification information if it exists;

[0247] The content output module 30 is further configured to generate a random keyword that matches the semantic information as speaker identification information if the keyword does not exist.

[0248] As an implementation mode, the content output module 30 is further used to obtain the speech text of the speaker in the video to be analyzed from the video description information, and extract each characteristic word in the speech text through the large language model;

[0249] The content output module 30 is further used to configure a corresponding weight for each of the feature words according to the frequency of each of the feature words in the speech text;

[0250] The content output module 30 is further used to determine the speaker's opinion text according to each of the feature words and the corresponding weights;

[0251] The content output module 30 is further configured to display the opinion text in the window of the video to be analyzed.

[0252] As an implementation mode, the content output module 30 is further used to determine the semantic information between the feature words;

[0253] The content output module 30 is further used to identify the emotion type of each of the feature words through the large language model;

[0254] The content output module 30 is further used to add feature words belonging to the same emotion type to the same list to form multiple feature sets;

[0255] The content output module 30 is further used to integrate the weights of the feature words in each feature set to obtain the sentiment weight of each feature set;

[0256] The content output module 30 is further used to obtain a target feature set with the highest sentiment weight, and generate an opinion text according to each feature word in the target feature set.

[0257] As an implementation manner, the content output module 30 is further used to determine the speech start time and speech end time of the speaker in the video to be analyzed according to the video description information;

[0258] The content output module 30 is further used to generate a timestamp according to the speech start time and the speech end time;

[0259] The content output module 30 is further used to display the timestamp in the window of the video to be analyzed, and configure a click event for each time point in the timestamp;

[0260] The content output module is further configured to locate a video position corresponding to any time point in the video to be analyzed when a user triggers a click event corresponding to any time point based on the timestamp.

[0261] The present application provides a video analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video analysis method in the above-mentioned embodiment 1.

[0262] Reference below Figure 5 , Figure 5 Schematic diagram of the structure of a video analysis device suitable for implementing the embodiment of the present application. The video analysis device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The video analysis device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0263] like Figure 5As shown, the video analysis device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the video analysis device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the video analysis device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a video analysis device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0264] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0265] The video analysis device provided by the present application adopts the video analysis method in the above embodiment, which can solve the technical problem that the existing technology cannot meet user needs, resulting in poor user experience. Compared with the prior art, the beneficial effects of the video analysis device provided by the present application are the same as the beneficial effects of the video analysis method provided by the above embodiment, and the other technical features in the video analysis device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0266] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0267] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0268] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the video analysis method in the above-mentioned embodiment.

[0269] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0270] The computer-readable storage medium may be included in the video analysis device; or may exist independently without being assembled into the video analysis device.

[0271] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the video analysis device, the video analysis device: responds to the triggered video analysis instruction, converts the image content and / or audio content in the video to be analyzed into text information through the large language model; performs semantic analysis on the text information through the large language model to obtain a semantic analysis result; based on the semantic analysis result, generates video description information corresponding to the text information by the large language model.

[0272] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0273] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0274] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0275] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned video analysis method, and can solve the technical problem that the prior art cannot meet user needs, resulting in poor user experience. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the video analysis method provided in the above-mentioned embodiment, and will not be repeated here.

[0276] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A video analysis method, characterized in that: The method comprises: In response to the triggered video analysis instruction, converting the image content and / or audio content in the video to be analyzed into text information through the large language model; Performing semantic analysis on the text information by using the large language model to obtain a semantic analysis result; Based on the semantic analysis result, the large language model generates video description information corresponding to the text information.

2. The video analysis method according to claim 1, characterized in that: The step of generating video description information corresponding to the text information by the large language model based on the semantic analysis result includes: Identifying key picture content in the image content that matches a preset type; Determining the picture description information corresponding to the key picture content in the semantic analysis result; The picture description information and the text information are merged by using the large language model to generate video description information corresponding to the text information.

3. The video analysis method according to claim 2, characterized in that: The step of identifying key picture content that meets a preset type in the video to be analyzed includes: Determining visual features of each video frame in the video to be analyzed; Obtaining baseline video features associated with a preset type; Determining the similarity between each of the visual features and the reference video features; The video frames corresponding to the video feature information whose similarity reaches a preset similarity are selected as the key picture content.

4. The video analysis method according to claim 2, characterized in that: The step of determining the picture description information corresponding to the key picture content in the semantic analysis result includes: Extracting first description information describing the positional relationship between the objects in the key picture content and second description information describing the semantic relationship between the objects in the key picture content from the semantic analysis result; The screen description information is generated according to the first description information and the second description information.

5. The video analysis method according to claim 1, characterized in that: After the step of generating video description information corresponding to the text information by the large language model based on the semantic analysis result, the method further includes: Obtaining image information of the speaker in each of the videos to be analyzed; Determine speaker identification information according to the image information; The image information and the speaker identification information are displayed in a window of the video to be analyzed.

6. The video analysis method according to claim 1, characterized in that: After the step of generating video description information corresponding to the text information by the large language model based on the semantic analysis result, the method further includes: Acquire the speech text of the speaker in the video to be analyzed from the video description information, and extract each feature word in the speech text through the large language model; Assigning a corresponding weight to each of the feature words according to the frequency of each of the feature words in the speech text; Determine the speaker's opinion text according to each of the feature words and the corresponding weights; The opinion text is displayed in the window of the video to be analyzed.

7. The video analysis method according to claim 6, characterized in that: The step of determining the speaker's opinion text according to each of the feature words and the corresponding weights includes: Determining semantic information between each of the feature words; Identify the sentiment type of each of the feature words through the large language model; Add feature words belonging to the same sentiment type to the same list to form multiple feature sets; Integrate the weights of the feature words in each feature set to obtain the sentiment weight of each feature set; A target feature set with the highest sentiment weight is obtained, and an opinion text is generated according to each feature word in the target feature set.

8. A video analysis device, characterized in that: The device comprises: A text conversion module, configured to convert image content and / or audio content in the video to be analyzed into text information through a large language model in response to a triggered video analysis instruction; A semantic analysis module, used to perform semantic analysis on the text information through the large language model to obtain a semantic analysis result; A content output module is used to generate video description information corresponding to the text information by the large language model based on the semantic analysis result.

9. A video analysis device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the video analysis method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the video analysis method according to any one of claims 1 to 7 are implemented.

Images