Video processing method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses a video processing method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of multimedia, the method comprises the following steps: obtaining a plurality of videos to be processed, the plurality of videos to be processed being obtained in response to the same retrieval intention; obtaining a sample video in response to a video selection operation for the plurality of to-be-processed videos, and generating reference information corresponding to the sample video in response to a video labeling operation for the sample video; generating role prompt information corresponding to the sample video based on the video content of the sample video; fusing the reference information with the role prompt information to generate target prompt information corresponding to the sample video; and performing video segmentation on each to-be-processed video according to the target prompt information to obtain an event segment corresponding to each to-be-processed video and semantic annotation information corresponding to the event segment. By implementing the technical scheme of the invention, automatic slicing and semantic annotation of the video can be realized, and the processing efficiency is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multimedia, in particular to a video processing method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] In the field of digital content creation such as games and films, creative personnel highly rely on video reference materials to stimulate creative inspiration, unify visual style and improve production efficiency. At present, the demand for content identification, event slicing and semantic labeling of multiple video sources is increasingly prominent, especially in scenarios that require batch processing of similar videos and extraction of structured information. However, related technologies mostly rely on manual methods for video segment arrangement and labeling, which not only has low efficiency, but also is greatly influenced by subjective experience, making it difficult to realize large-scale and high-consistency video content mining and reuse. SUMMARY

[0003] Therefore, the present application provides a video processing method and device, electronic equipment, storage medium and program product to solve the problem of low efficiency and heavy reliance on manual experience of related video labeling methods.

[0004] In a first aspect, the present application provides a video processing method, comprising: acquiring a plurality of to-be-processed videos, the plurality of to-be-processed videos being acquired in response to a same search intent; in response to a video selection operation for the plurality of to-be-processed videos, acquiring a sample video, and in response to a video labeling operation for the sample video, generating reference information corresponding to the sample video; based on the video content of the sample video, generating role prompt information corresponding to the sample video; fusing the reference information and the role prompt information to generate target prompt information corresponding to the sample video; and performing video segmentation on each to-be-processed video according to the target prompt information to obtain event clips corresponding to each to-be-processed video and semantic labeling information corresponding to the event clips.

[0005] In an optional implementation, based on the video content of the sample video, the role prompt information corresponding to the sample video is generated, comprising: performing semantic understanding on the video content of the sample video to obtain semantic features of the sample video; and determining role prompt information corresponding to the semantic features based on a matching relationship between the semantic features and preset roles.

[0006] In an optional implementation, the reference information is fused with the role prompt information to generate target prompt information corresponding to the sample video, including: obtaining an initial prompt word template corresponding to the sample video; instantiating the initial prompt word template based on the role prompt information to generate first prompt information; slicing and labeling the sample video based on the first prompt information to generate a first labeling result of the sample video; comparing the first labeling result and the reference information to obtain a first labeling comparison result; and optimizing the initial prompt word template based on the first labeling comparison result to generate the target prompt information corresponding to the sample video.

[0007] In an optional implementation, the initial prompt word template is instantiated based on the role prompt information to generate the first prompt information, including: obtaining first metadata information of the sample video; and instantiating the initial prompt word template based on the role prompt information and the first metadata information to generate the first prompt information.

[0008] In an optional implementation, the first labeling result and the reference information are compared to obtain the first labeling comparison result, including: updating the reference information based on the role prompt information to obtain target reference information; and comparing the first labeling result and the target reference information to obtain the first labeling comparison result.

[0009] In an optional implementation, the initial prompt word template is optimized based on the first labeling comparison result to generate the target prompt information corresponding to the sample video, including: determining missing information in the first labeling result based on the first labeling comparison result, and generating supplementary prompt information based on the missing information; filling the supplementary prompt information into a target position of the initial prompt word template to obtain a target prompt word template; instantiating the target prompt word template based on the role prompt information to generate second prompt information; slicing and labeling the sample video based on the second prompt information to generate a second labeling result of the sample video; comparing the second labeling result and the reference information to obtain a second labeling comparison result; and optimizing the second prompt information based on the second labeling comparison result to generate the target prompt information corresponding to the sample video.

[0010] In an optional implementation, for any to-be-processed video, event clips, semantic labeling information and second metadata information associated with the to-be-processed video are obtained; the event clips, the semantic labeling information and the second metadata information are fused to generate a target labeling result corresponding to the to-be-processed video.

[0011] In an optional implementation, the semantic labeling information in the target labeling result is converted into a vector representation to obtain a first vector; an association relationship among the first vector, the event clips and the semantic labeling information is established, and the association relationship is stored in a preset vector database.

[0012] In an optional implementation, in response to a video retrieval operation on the vector database, the video retrieval operation corresponding retrieval information is converted into a vector representation to obtain a second vector; similarity of the second vector with each first vector in the vector database is determined; each event clip in the vector database is sorted based on the similarity to generate a candidate event clip set corresponding to the retrieval information.

[0013] In a second aspect, the present application provides a video processing apparatus, comprising: a first obtaining module, configured to obtain a plurality of to-be-processed videos, the plurality of to-be-processed videos being obtained in response to a same retrieval intention; a second obtaining module, configured to obtain a sample video in response to a video selection operation on the plurality of to-be-processed videos, and generate reference information corresponding to the sample video in response to a video labeling operation on the sample video; a first generating module, configured to generate role prompt information corresponding to the sample video based on video content of the sample video; a second generating module, configured to fuse the reference information and the role prompt information to generate target prompt information corresponding to the sample video; and a processing module, configured to perform video segmentation on each to-be-processed video according to the target prompt information to obtain an event clip corresponding to each to-be-processed video and semantic labeling information corresponding to the event clip.

[0014] In a third aspect, the present application provides an electronic device, comprising: a memory and a processor, which are communicatively connected with each other, and the memory stores computer instructions; the processor executes the computer instructions to perform the video processing method of the first aspect or any of the corresponding implementation manners thereof.

[0015] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the video processing method of the first aspect or any of the corresponding implementation manners thereof.

[0016] In a fifth aspect, the present application provides a computer program product, which comprises computer instructions, and the computer instructions are used to make a computer execute the video processing method of the first aspect or any of the corresponding implementation manners thereof.

[0017] The video processing method provided by the embodiment of the application ensures that the subsequent processing objects have high relevance in content theme by obtaining a plurality of to-be-processed videos based on the same search intention, thereby laying a solid foundation for batch automatic processing. On this basis, reference information is generated by means of user selection and labeling operation on a sample video, which contains clear semantic elements and intention information, thereby providing high-quality intention definition for subsequent processing. Furthermore, role prompt information conforming to the style and context of the sample video content is automatically generated according to the sample video content, so that the semantic description is more professional and consistent, and the self-adaptation capability for different video types is enhanced. By fusing the reference information provided by the user and the automatically generated role prompt, target prompt information is formed, thereby effectively combining artificial priori and automatic capability, improving the intelligent level of the prompt engineering, and reducing the dependence on artificial writing. According to the fused target prompt information, all to-be-processed videos are uniformly sliced and semantically labeled, thereby ensuring the time accuracy and description consistency of the output event clips, and significantly improving the efficiency of structured processing of large-scale video content and the reliability of the results. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the specific embodiments or related art, the following will briefly introduce the drawings needed to be used in the specific embodiments or related art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present application; Figure 2 is a first flowchart of a video processing method according to an embodiment of the present application; Figure 3 is a second flowchart of a video processing method according to an embodiment of the present application; Figure 4 is a third flowchart of a video processing method according to an embodiment of the present application; Figure 5 is a fourth flowchart of a video processing method according to an embodiment of the present application; Figure 6 is a structural block diagram of a video processing apparatus according to an embodiment of the present application; Figure 7 is a hardware structure schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0021] It can be understood that, before using the technical solutions disclosed in the embodiments of the present application, the type of personal information involved in the present application, the use range, the use scenario, and the like should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0022] The terms “first”, “second” are only for descriptive purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first”, “second” can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of “multiple” is two or more than two, unless otherwise specifically limited.

[0023] As an optional application scenario of the embodiments of the present application, Figure 1 An application scenario of a video processing system is shown. As Figure 1 shown, the system can include at least one terminal device and at least one server, Figure 1 The system includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0024] The terminal device can be specifically a smart phone, a tablet computer, a notebook computer, a palm computer, and can also be a desktop computer, a game console, a smart television, a smart wearable device, a vehicle-mounted terminal, a VR (Virtual Reality) device, an AR (Augmented Reality) device, and the like. The server 103 can be an independent physical server, can also be a server cluster or a distributed system, and can also be a cloud server providing cloud services. The network 110 can be a wired network or a wireless network, and examples thereof include but are not limited to the Internet, an enterprise intranet, a local area network, a wide area network, a mobile communication network, and a combination thereof.

[0025] For example, a designer may need to find a specific segment of a game video where a character uses a giant sword to perform a powerful swing. The traditional method requires the designer to manually watch a large number of videos on a terminal device, manually screen and mark each segment that meets the requirements, record the time point, and add a description text. This process is not only tedious and time-consuming, but also the annotation results of different personnel are easily affected by subjective differences, making it difficult to ensure consistency and accuracy.

[0026] When the number of video resources is large, this manual annotation method is particularly inefficient. For example, a large game project may need to establish a reference library containing tens of thousands of action segments, and it is almost impossible to complete the manual annotation. Even if the initial annotation is completed, when new search requirements are needed, such as "find all rotation attack actions with fire effects", the tedious manual screening and annotation process still needs to be started again. This traditional video processing method not only has low efficiency, but also relies too much on the professional experience and subjective judgment of the operator, resulting in unstable annotation quality, making it difficult to form a large-scale and systematic video database, and greatly limiting the work efficiency and resource reuse ability of the creative team.

[0027] Although some current automatic video processing technologies have made some progress, they still have obvious limitations. Current technical solutions mainly focus on two directions: one is based on traditional computer vision and feature engineering, which relies on manually designed feature extraction algorithms and can achieve basic content matching, but it is difficult to understand the complex semantic information in game videos and cannot distinguish between actions with similar underlying visual features but completely different semantics, such as "elegant dodge" and "frantic roll"; the second is a multi-modal machine learning method, such as image-text matching transfer annotation or using a vision language model (VLM) to generate video titles. However, these solutions cannot accurately capture dynamic information in video segments, resulting in noisy and biased annotations. At the same time, their effectiveness is highly dependent on the carefully designed prompts, and when dealing with different styles of game videos, professional personnel still need to adjust and debug, which is tedious and technically demanding.

[0028] More importantly, these technical solutions lack adaptive and self-optimizing capabilities and cannot dynamically adjust processing strategies according to new video content or annotation requirements, still requiring a large amount of manual intervention, making it difficult to meet the game industry's demand for large-scale and high-quality video content processing automation.

[0029] Therefore, the technical scheme of the present application realizes preliminary screening of target video resources by obtaining a plurality of to-be-processed videos in response to the same search intent, thereby reducing the workload of manual searching from the source. The reference information is generated by the user's labeling operation on a single sample video, which contains explicit semantic elements and intent information, providing high-quality intent definition for subsequent processing. At the same time, the tedious batch labeling task is simplified to one-time example provision, significantly improving efficiency and ensuring the consistency of the labeling source. The role prompt information is automatically generated based on the content of the sample video, setting a professional analysis perspective for subsequent processing, making the output more consistent with the field requirements. By integrating the reference information provided by the user with the generated role prompt information, the present application forms accurate target prompt information, overcomes the dependence on pre-set fixed prompt templates, and has the ability to adapt to different styles of content. According to the optimized target prompt information, the to-be-processed videos are automatically batch processed, and structured results with accurate event segments and deep semantic labeling are output, which not only liberates manual repetitive labor and realizes large-scale processing, but also ensures the consistency, accuracy and professionalism of the labeling results through unified standards.

[0030] According to an embodiment of the present application, a video processing method embodiment is provided. It should be noted that the steps shown in the flowchart of the drawing can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0031] In this embodiment, a video processing method is provided, which can be used in electronic devices such as computers 101, mobile terminals 102, etc. Figure 2 is a flowchart of the video processing method according to an embodiment of the present application, as shown in Figure 2 The flowchart includes the following steps: Step S201, obtaining a plurality of to-be-processed videos, the plurality of to-be-processed videos being obtained in response to the same search intent.

[0032] The to-be-processed video refers to an original video obtained from a public video platform, which needs to be automatically sliced and semantically labeled. The search intent refers to the query requirement expressed by the user in natural language, for example, "the main character uses a giant sword in the fight in XX game". Specifically, a built-in web crawler module can be used to search and batch download from a specified video platform according to the search intent input by the user, i.e. a keyword or natural language sentence describing the requirement, thereby obtaining a series of original videos related to the intent and available for processing.

[0033] The video to be processed can be pre-processed, for example, decoded into a sequence of static image frames at a preset frame rate (such as 10 fps) to prepare for subsequent artificial intelligence model (such as visual language model VLM) analysis.

[0034] In step S202, in response to a video selection operation on the plurality of videos to be processed, a sample video is obtained, and in response to a video annotation operation on the sample video, reference information corresponding to the sample video is generated.

[0035] The sample video refers to an example video provided by a user to guide the generation of an annotation style. Specifically, a user interface is provided to display a list or thumbnail interface of the plurality of videos to be processed obtained. When the user selects one of the videos from the list, the selected video is determined as the sample video for the current optimization process, which is used for subsequent annotation and prompt information generation process.

[0036] The reference information refers to a natural language example or a preliminary annotation file provided by a user as a reference standard for optimizing the annotation style. Specifically, when the user performs an annotation operation on the sample video, a description can be input through natural language, such as "the main character holds a giant sword and charges down", or a preliminary manual annotation file (GT) can be uploaded. After receiving the input, it is used as the initial reference information.

[0037] In step S203, role prompt information corresponding to the sample video is generated based on the video content of the sample video.

[0038] The video content refers to the visual information contained in the sample video, such as scenes, characters, actions, special effects, etc. The role prompt information refers to the role setting automatically generated according to the video style, which is used to generate professional style annotation text. Specifically, the visual content of the sample video is analyzed, such as picture style, character type, action effect, etc., and according to the built-in rules or knowledge base, these visual features are mapped to a most suitable "expert role", for example, "you are a senior action effect designer". The role prompt information (role_prompt) will be used as the basis for subsequent interaction with the artificial intelligence model, ensuring that the generated annotation text has a professional, unified, and high-quality style.

[0039] In step S204, the reference information and the role prompt information are fused to generate target prompt information corresponding to the sample video.

[0040] The target prompt information is generated by fusing the reference information and the role prompt information, and is used to guide the video slicing and the final prompt instruction of the labeling. Specifically, the professional perspective and the style keynote defined by the role prompt information are taken as the macro basis and background setting of the entire instruction. Then, the specific examples or quality requirements contained in the reference information are taken as the detail standards and style templates that should be followed when the artificial intelligence model outputs the results. By combining the macro role setting with the micro reference example, a target prompt information with comprehensive content and clear requirements is automatically generated.

[0041] In step S205, the video slicing is performed on each to-be-processed video according to the target prompt information, to obtain the event segments corresponding to each to-be-processed video and the semantic labeling information corresponding to the event segments.

[0042] The event segment refers to a short video paragraph with independent semantic meaning cut from the to-be-processed video, with start and end time stamps. The semantic labeling information refers to the natural language description of the event segment, including action, role, scene, style and other deep semantic information. Specifically, the target prompt information is used to input the complete to-be-processed video and the prompt information to the artificial intelligence model through one API (Application Programming Interface) call. The artificial intelligence model automatically analyzes the video content of the to-be-processed video as a whole according to the logic in the target prompt information, identifies the start and end time points of the key events, completes the video slicing, and simultaneously generates a natural language deep description for each identified event segment.

[0043] The video processing method provided by the embodiment of the present application ensures that the subsequent processing objects have high relevance in content theme by obtaining a plurality of to-be-processed videos based on the same retrieval intention, thereby laying a solid foundation for batch automatic processing. On this basis, the reference information is generated by the user's selection and labeling operation of the sample video, which contains clear semantic elements and intention information, and provides high-quality intention definition for subsequent processing. Further, the role prompt information conforming to the style and context of the sample video content is automatically generated, which makes the semantic description more professional and consistent, and enhances the adaptive ability to different video types. By fusing the reference information provided by the user and the automatically generated role prompt, the target prompt information is formed, which realizes the effective combination of artificial priori and automation ability, improves the intelligent level of the prompt engineering, and reduces the dependence on artificial writing. According to the fused target prompt information, the unified slicing and semantic labeling are performed on all to-be-processed videos, which ensures the time accuracy and description consistency of the output event segments, and significantly improves the efficiency of structured processing of large-scale video content and the reliability of the results.

[0044] A video processing method is provided in the embodiment, which can be used in electronic devices such as computer 101, mobile terminal 102, etc. Figure 3 A flowchart of the video processing method according to the embodiment of the present application is shown in FIG. 3, which includes the following steps: Figure 3 In step S301, a plurality of to-be-processed videos are acquired, which are acquired in response to a same search intention. For details, refer to step S201 of the embodiment shown in FIG. 2, which will not be repeated here. Figure 2

[0045] In step S302, a sample video is acquired in response to a video selection operation on the plurality of to-be-processed videos, and reference information corresponding to the sample video is generated in response to a video labeling operation on the sample video. For details, refer to step S202 of the embodiment shown in FIG. 2, which will not be repeated here. Figure 2

[0046] In step S303, role prompt information corresponding to the sample video is generated based on the video content of the sample video.

[0047] Specifically, the above step S303 includes: In step S3031, semantic understanding is performed on the video content of the sample video to obtain semantic features of the sample video.

[0048] The semantic features refer to semantic information extracted after analysis of the video content by an artificial intelligence model, such as visual style, content theme, etc. of the video. Specifically, the artificial intelligence model performs multi-frame sampling and analysis on the sample video, comprehensively understands visual elements (such as characters, costumes, weapons, environment, special effects, etc.) and dynamic information (such as action type, rhythm, interaction, etc.) thereof, and induces and refines them into a series of high-level and understandable semantic concepts. These semantic concepts constitute the semantic features of the sample video, which are no longer original pixel data, but an abstract description of the video content, for example, identifying that the sample video belongs to “wuxia style”, “science fiction theme”, or contains “high-speed combat”, “magic special effects”, “cold weapon combat” and other key themes.

[0049] In step S3032, role prompt information corresponding to the semantic features is determined based on a matching relationship between the semantic features and preset roles.

[0050] ​​​The preset role refers to a pre-defined virtual expert identity template with a specific professional background or style attribute, which is used to standardize the professionalism and style uniformity of the annotation content generated by the artificial intelligence model. The matching relationship refers to a pre-set corresponding rule system for associating semantic features and preset roles. Specifically, an internal preset role knowledge base is maintained, in which each preset role defines a specific expert identity and its corresponding description style, such as "a martial arts director proficient in medieval swordsmanship", "a visual effects artist with in-depth research on magic special effects", etc. Each role is associated with a set of keywords or semantic tags, i.e. the matching relationship. The extracted semantic features are compared with the tags associated with all preset roles in this role library, and by calculating the similarity or according to the pre-set rules, the most suitable expert role is automatically matched. Once the matching is successful, the definition of the preset role (such as "you are a [role description]") is output as the final role prompt information, thereby ensuring that the subsequent generated annotations are highly relevant to the video content in terms of style and professionalism.

[0051] In step S304, the reference information is fused with the role prompt information to generate target prompt information corresponding to the sample video. For details, please refer to Figure 2 The step S204 of the embodiment shown in

[0052] In step S305, the target prompt information is used to perform video segmentation on each to-be-processed video to obtain event segments corresponding to each to-be-processed video and semantic annotation information corresponding to the event segments. For details, please refer to Figure 2 The step S205 of the embodiment shown in

[0053] The video processing method provided by the embodiment of the present application realizes the intelligentization of role setting and content adaptation by automatically analyzing the semantic features of the sample video and generating role prompt information based on the preset role library, avoids the subjectivity and tediousness of manual role setting, and thus improves the accuracy and efficiency of prompt information generation.

[0054] In the embodiment, a video processing method is provided, which can be used in electronic devices such as computers 101, mobile terminals 102, etc. Figure 4 is a flowchart of the video processing method according to the embodiment of the present application, as shown in Figure 4 The flowchart includes the following steps: In step S401, a plurality of to-be-processed videos are obtained, which are obtained in response to the same search intent. For details, please refer to Figure 3 The step S301 of the embodiment shown in

[0055] Step S402: In response to the video selection operation for multiple videos to be processed, a sample video is obtained, and in response to the video annotation operation for the sample video, reference information corresponding to the sample video is generated. For details, please refer to [link to relevant documentation]. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0056] Step S403: Based on the video content of the sample video, generate character prompt information corresponding to the sample video. For details, please refer to [link to relevant documentation]. Figure 3 Step S303 of the illustrated embodiment will not be described again here.

[0057] Step S404: The reference information and the role prompt information are fused to generate the target prompt information corresponding to the sample video.

[0058] Specifically, step S404 includes: Step S4041: Obtain the initial prompt word template corresponding to the sample video.

[0059] The initial prompt template refers to a pre-defined, structured framework of prompts used to guide AI models in video analysis. This template includes, for example,... Figure 5 The multi-round thinking logic is illustrated, such as ROUND1 focusing on the scene, ROUND2 focusing on actions, and ROUND3 focusing on details. Specifically, one or more general, structured initial cue word templates are preset. These templates typically contain placeholders (such as [ROLE]) and a general thought chain (CoT) reasoning framework (e.g., "Step 1: Analyze the scene, Step 2: Analyze the actions..."). When processing sample videos, it automatically selects the most basic, unoptimized template from a built-in template library as a starting point, based on the video's type or format; this template is the initial cue word template.

[0060] Step S4042: Instantiate the initial prompt word template based on the role prompt information to generate the first prompt information.

[0061] The first prompt is an executable prompt with a clear professional style, generated by instantiating the initial prompt template by substituting the role-based prompt information. Specifically, instantiation is the process of filling an abstract template with concrete instructions. The generated role-based prompt information is obtained, and then this concrete text replaces the corresponding placeholder in the initial prompt template (e.g., [ROLE]). After this simple text replacement or concatenation, a specific, executable instruction that incorporates the expert role setting is generated—the first prompt.

[0062] In some optional implementations, step S4042 above includes: Step a1: Obtain the first metadata information of the sample video.

[0063] First-level metadata is obtained by automatically extracting structured data from the sample video itself or its associated information. Specifically, basic information can be obtained by analyzing the video's filename, file attributes (such as creation date and resolution), or metadata tags embedded in the video container; alternatively, artificial intelligence models can be used to identify keyframe content in the video and automatically extract higher-level semantic information, such as the game name, main character names, weapon types, and scene environment appearing in the video. This automatically identified and standardized set of information constitutes the first-level metadata used to optimize prompts.

[0064] Step a2: Based on the role prompt information and the first metadata information, instantiate the initial prompt word template to generate the first prompt information.

[0065] The default initial prompt template is a structured text framework containing specific placeholders (such as [ROLE], [METADATA]). When generating the first prompt, a simple text replacement operation is performed, filling the character prompt information (such as "You are a senior martial arts action director") into the corresponding [ROLE] position in the template; simultaneously, the first metadata information (such as "Game Name: Netherworld, Weapon: Katana") is formatted and filled into the corresponding [METADATA] or other preset positions in the template. Through this replacement, a general template is transformed into a highly customized, executable first prompt that integrates specific expert personas and video context information.

[0066] In the above implementation, based on the role-based prompting information, the first metadata information of the sample video is further introduced to jointly instantiate the initial prompting template. This design ensures that the generated first prompting information not only has the perspective of a professional role but also incorporates the specific contextual information of the video itself, making the prompting information more specific and aligned with the actual video content. This effectively improves the targeting and accuracy of subsequent video slicing and annotation operations.

[0067] Step S4043: Based on the first prompt information, the sample video is sliced ​​and annotated to generate the first annotation result of the sample video.

[0068] The first annotation result refers to the preliminary result generated by the artificial intelligence model after initially slicing and annotating the sample video based on the first prompt information. Specifically, the first prompt information and the sample video are input into the artificial intelligence model together. According to the instructions, the artificial intelligence model autonomously analyzes the video content, identifies the start and end times of action events, and generates a natural language description for each identified segment. These raw results, containing timestamps and descriptive text, output by the artificial intelligence model constitute the first annotation result.

[0069] Step S4044: Compare the first annotation result with the reference information to obtain the first annotation comparison result.

[0070] The first annotation comparison result refers to the differences in various aspects, such as the accuracy of time points, the richness of description, and the precision of details, analyzed by comparing the first annotation result with the reference information. Specifically, the AI ​​model compares the first annotation result and the user-provided reference information side by side. The AI ​​model analyzes the differences between the two from multiple dimensions, such as the accuracy of time point segmentation, the level of detail in action description, the accuracy of the use of professional terminology, and whether the style meets the requirements. This difference is the first annotation comparison result.

[0071] In some optional implementations, step S4044 above includes: Step b1: Update the reference information based on the role prompt information to obtain the target reference information.

[0072] By leveraging the expert perspective and professional style defined in the generated role prompts, the original reference information provided by the user (which may be somewhat colloquial or simple) is refined and enhanced. Specifically, the role prompt and reference information are input into the AI ​​model, which is then instructed to "rewrite or enrich the following description in the manner and tone of [role prompt] to make it more professional and vivid." The higher-quality, more stylistically consistent descriptive text generated by the AI ​​model accordingly becomes the target reference information.

[0073] Step b2: Compare the first annotation result with the target reference information to obtain the first annotation comparison result.

[0074] The system takes the initial annotation results and target reference information as input and instructs them to perform a refined comparison across multiple preset dimensions. These dimensions include time segmentation accuracy (e.g., whether the start and end points of the action are aligned), descriptive richness (whether it includes details such as power, rhythm, and special effects), terminology professionalism (whether the vocabulary used conforms to domain norms), and semantic consistency (whether it accurately understands the intention and context of the action). The system analyzes the specific differences between the two in these dimensions to generate the initial annotation comparison results.

[0075] In the above implementation, before performing multi-dimensional comparisons, the original reference information is updated using role-based prompts to generate target reference information. This step injects a professional role perspective and standards into the reference information, making the evaluation criteria used when comparing the first annotation result with the target reference information more professional and consistent. This ensures the quality and consistency of the comparison results themselves, providing a more reliable and accurate basis for subsequent optimization of the initial prompt word template.

[0076] Step S4045: Optimize the initial prompt word template based on the first annotation comparison result to generate target prompt information corresponding to the sample video.

[0077] The initial annotation comparison results are input into the AI ​​model, which is then instructed to "generate a supplementary requirement to improve the initial prompt based on these differences." The AI ​​model generates optimized text, such as "Pay more attention to the spark effects of weapon collisions." This optimized text is then added as a supplementary requirement to a specific part of the initial prompt template, resulting in a more powerful and accurate new prompt. This final optimized version is the target prompt message.

[0078] The video processing method provided in this application instantiates an initial template using role-based prompting information to generate executable first prompting information and obtain preliminary annotation results. Subsequently, this preliminary result is compared with manually provided reference information from multiple dimensions to objectively and quantitatively identify deficiencies in the accuracy of the current prompting information or the quality of the annotation. Based on this comparison result, the initial template is optimized in a targeted manner to generate the final target prompting information. This method can automatically and specifically optimize the initial template based on the differences between the initial prompting word template and the first annotation results and the reference information, thereby dynamically generating high-quality target prompting information. This effectively improves the accuracy of the final video slicing and annotation and reduces reliance on external manual adjustments.

[0079] In some optional implementations, step S4045 above includes: Step c1: Determine the missing information in the first annotation result based on the first annotation comparison result, and generate supplementary prompt information based on the missing information.

[0080] The AI ​​model automatically analyzes the differences between the initial annotation results and the target reference information, identifying missing information in the initial annotation results in areas such as the accuracy of segmentation timing, the richness of action description, and the precision of key object identification. Subsequently, a supplementary prompt text is generated based on this missing information. For example, if the description lacks representation of the attack force, an instruction is generated such as, "Please observe the visual effects and impact feedback at the moment the weapon contacts the target, and reflect the sense of power and impact in the description."

[0081] Step c2: Fill the target position of the initial prompt word template with the supplementary prompt information to obtain the target prompt word template.

[0082] The initial prompt template is a structured text containing a specific target location. This is typically an explicit placeholder, such as `[ADDITIONAL_REQUIREMENTS]` or `##Supplementary Requirement##`. Specifically, newly generated supplementary prompts are precisely inserted or appended to this predefined target location using simple text find and replace (or string manipulation). Once completed, the initial prompt template evolves into a more powerful target prompt template containing the added optimization instructions, ready for the next iteration.

[0083] Step c3: Instantiate the target prompt word template based on the role prompt information to generate the second prompt information.

[0084] By utilizing the identified role-based prompts and the first metadata extracted from the sample video, the corresponding placeholders in the target prompt template, such as [ROLE] and [METADATA], are populated. Through this text replacement or concatenation, an abstract template containing optimization instructions is transformed into an executable second prompt that integrates the expert role, specific context, and new optimization requirements.

[0085] Step c4: Slice and annotate the sample video based on the second prompt information to generate the second annotation result of the sample video.

[0086] The second cue message and the frame sequence data of the sample video are then input into the AI ​​model again. Based on this more refined and optimized instruction, the AI ​​model re-analyzes the video content. Following the requirements of the new instruction, the AI ​​model will again identify the start and end times of action events and generate higher-quality natural language descriptions for each segment. The output this time is expected to be of higher quality than the initial second annotation result.

[0087] Step c5: Compare the second annotation result with the reference information to obtain the second annotation comparison result.

[0088] The second annotation result is compared side-by-side with the reference information (or its optimized target reference information). The dimensions of comparison remain the same: temporal accuracy, descriptive richness, and professionalism. The purpose of this comparison is to evaluate the quality improvement of the output results after one round of optimization and to identify any remaining or newly emerging differences.

[0089] Step c6: Optimize the second prompt information based on the second annotation comparison results to generate the target prompt information corresponding to the sample video.

[0090] Analyzing the comparison results of the second annotation, if the difference is within an acceptable threshold, the current second prompt message can be directly determined as the final target prompt message. If the difference is outside the acceptable threshold, a new round of supplementary prompt messages will be generated based on the newly discovered differences, and the prompt word template will be updated again, or the second prompt message itself will be fine-tuned. This process may repeat several times until the output quality meets the standard or the preset maximum number of iterations is reached. Finally, the prompt message determined after iterative optimization that can produce high-quality results is output as the target prompt message for processing all videos to be processed.

[0091] In the above implementation, a clear multi-round iterative optimization mechanism enables continuous refinement and calibration of the prompt information. Based on the comparison results of the previous round, targeted supplementary prompt information is generated, and the prompt word template is updated accordingly. Then, a new round of slicing, annotation, and result comparison is performed, which can automatically and gradually narrow the gap between the annotation results and the reference standard, thereby systematically improving the quality of the final generated target prompt information and the accuracy of the resulting video slicing and annotation results.

[0092] Step S405: Based on the target prompt information, segment each video to be processed to obtain the event segments corresponding to each video and the semantic annotation information corresponding to the event segments. For details, please refer to [link to relevant documentation]. Figure 3 Step S305 of the illustrated embodiment will not be described again here.

[0093] In some optional implementations, the video processing method described above further includes: Step d1: For any video to be processed, obtain the event segments, semantic annotation information, and second metadata information associated with the video to be processed.

[0094] All videos to be processed were batch-processed using target-specific prompts, generating structured output for each video. These outputs are typically saved in formats such as JSON, explicitly containing multiple event segments for each video (including start and end timestamps), semantic annotation information (text description) for each segment, and secondary metadata information automatically identified or generated by the Agent in previous steps, such as game name, character name, and style type. By reading this final result file, all relevant information can be retrieved for any specified video to be processed.

[0095] Step d2 involves fusing event fragments, semantic annotation information, and secondary metadata information to generate target annotation results for the video to be processed.

[0096] A structured data object (such as a JSON object) is created to associate the start and end times (event segment), text description (semantic annotation information), and global information extracted from the video (secondary metadata information) of the same event segment. Ultimately, all segments and descriptions of a video are organized into a complete, context-rich structured data file, which is the target annotation result for that video.

[0097] In the above implementation, by fusing automatically generated event fragments, semantic annotation information, and secondary metadata information associated with the video, a complete and information-rich target annotation result is generated. This process organically combines dynamic action content, in-depth semantic description, and metadata information, greatly enriching the information dimensions and application value of the final video annotation, and providing a more comprehensive and accurate data foundation for subsequent video retrieval, management, and analysis.

[0098] Step d3: Convert the semantic annotation information in the target annotation result into a vector representation to obtain the first vector.

[0099] Call a pre-trained text embedding model (such as BERT). Apply this model to each textual semantic annotation in the target annotation results. The model will convert the meaning of each text segment into a high-dimensional, machine-readable numerical array. This array is the first vector representing the semantics of that text segment. Text descriptions with similar semantics will have their corresponding vectors closer together in the vector space.

[0100] Step d4: Establish the association between the first vector, event fragment, and semantic annotation information, and store the association in a preset vector database.

[0101] Create a record containing at least three key fields: the generated first vector, the actual video file path or access address of the corresponding event segment and its timestamp, and the original semantic annotation text. Then, store this complete record in the vector database. This allows you to index the corresponding video segment and the original text using the vector.

[0102] In the above implementation, by converting semantic annotation information into vector representations and establishing their association with event segments and original annotation information, and then storing them in a vector database, unstructured natural language descriptions are transformed into structured data forms that can be efficiently computed by machines. This step provides a core data foundation for achieving rapid retrieval, clustering, and analysis of large-scale video segments based on semantic similarity, greatly improving the efficiency and accuracy of subsequent video content retrieval and applications.

[0103] Step d5: In response to the video retrieval operation for the vector database, the retrieval information corresponding to the video retrieval operation is converted into a vector representation to obtain the second vector.

[0104] When a user enters natural language search information into the search interface, such as "finding slow motion of double swords crossing and blocking," the text embedding model is invoked to vectorize the user's query. This process is completely consistent with the processing of semantic annotation information. The resulting high-dimensional vector representing the user's search intent is the second vector.

[0105] Step d6: Determine the similarity between the second vector and each of the first vectors in the vector database.

[0106] The core function of a vector database is to perform approximate nearest neighbor search. When a second vector is submitted to the vector database, the database uses an efficient similarity calculation algorithm to quickly calculate the similarity score between the second vector and all first vectors in the database. This score quantifies the semantic match between the user's query and the description of each video segment.

[0107] Step d7: Sort the event fragments in the vector database based on similarity to generate a set of candidate event fragments corresponding to the search information.

[0108] The vector database sorts all matching video clip records from highest to lowest based on the calculated similarity score. It then returns a sorted list containing references (such as links and timestamps) to a series of video clips (event clips) most relevant to the user's query, along with their original descriptions. This list, the candidate event clip set, is ultimately presented to the user.

[0109] In the above implementation, by converting the user's natural language retrieval information into vectors and performing efficient similarity matching and sorting with a large number of pre-stored labeled vectors in the database, a video retrieval method based on deep semantic understanding rather than simple keyword matching is realized. This method can more accurately capture the user's retrieval intent and quickly and accurately filter out the most relevant results from multiple video clips, thereby greatly improving the intelligence level of video content retrieval and user experience.

[0110] In the following embodiment, the above video processing method will be illustrated by way of a specific application scenario.

[0111] like Figure 5 As shown, for the input raw video (such as game combat or stealth videos), users only need to provide a few annotated examples in natural language (such as examples 1-3 and their corresponding videos 1-3), without requiring additional training for the model. Subsequently, VLM automatically generates prompts based on these natural language examples and gradually aligns the language style and key event segmentation attributes of the examples through multiple rounds of optimization. For example, in the iteration example of "round 1-round 3" in the figure, the description gradually refines from the brief statement "a male samurai stealthily approaches the enemy" to the precise and detailed expression "a ninja approaches the enemy on a bridge, grabs the enemy with his left hand, and stabs him repeatedly with a short sword in his right hand, causing the enemy to fall to the ground." At the same time, the "supplementary requirements" module provides a refined direction for the Prompt iteration through phases (ROUND1 focuses on the continuity of actions and the use of key props, ROUND2 focuses on the distinction between character types and action details, and ROUND3 accurately describes the scene background and action details, etc.), ensuring that the Prompt can accurately guide video processing. After the Prompt iterations were improved, VLM performed two core operations on the video based on the "segmentation and annotation Prompt": first, segmentation, automatically breaking down long videos into short segments containing independent key events (such as the thumbnails of multi-frame video clips shown); second, annotation, generating structured semantic annotation information (such as results presented in JSON format, with each segment containing "start_time" and "end_time" timestamps, and detailed natural language descriptions of scenes, characters, and actions under the "clip_annotation" field). Ultimately, this process achieves "full output after iteration" of video slices and corresponding semantic annotations, efficiently generating game video reference assets that can be used for motion design, special effects reference, and other scenarios. This reduces manual annotation costs while ensuring the accuracy and stylistic consistency of annotations through VLM's automated iterations.

[0112] This embodiment also provides a video processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0113] This embodiment provides a video processing device, such as... Figure 6 As shown, it includes: The first acquisition module 601 is used to acquire multiple videos to be processed, which are acquired in response to the same search intent; The second acquisition module 602 is used to acquire sample videos in response to a video selection operation for multiple videos to be processed, and to generate reference information corresponding to the sample videos in response to a video annotation operation for the sample videos. The first generation module 603 is used to generate character prompt information corresponding to the sample video based on the video content of the sample video. The second generation module 604 is used to fuse reference information with role prompt information to generate target prompt information corresponding to the sample video; The processing module 605 is used to segment each video to be processed according to the target prompt information, and obtain the event segments corresponding to each video to be processed and the semantic annotation information corresponding to the event segments.

[0114] In some alternative implementations, the first generation module 603 includes: The understanding submodule is used to perform semantic understanding on the video content of the sample video to obtain the semantic features of the sample video; The matching submodule is used to determine the role prompt information corresponding to the semantic features based on the matching relationship between semantic features and preset roles.

[0115] In some alternative implementations, the second generation module 604 includes: The acquisition submodule is used to obtain the initial prompt word template corresponding to the sample video; The instantiation submodule is used to instantiate the initial prompt word template based on the role prompt information and generate the first prompt information; The generation submodule is used to slice and annotate the sample video based on the first prompt information, and generate the first annotation result of the sample video; The comparison submodule is used to compare the first annotation result with the reference information to obtain the first annotation comparison result; The optimization submodule is used to optimize the initial prompt word template based on the first annotation comparison results, and generate target prompt information corresponding to the sample video.

[0116] In some alternative implementations, the instantiated submodule includes: The acquisition unit is used to acquire the first metadata information of the sample video. The first instantiation unit is used to instantiate the initial prompt word template based on the role prompt information and the first metadata information, and generate the first prompt information.

[0117] In some optional implementations, the comparison submodule includes: The update unit is used to update the reference information based on the role prompt information to obtain the target reference information; The first comparison unit is used to compare the first annotation result with the target reference information to obtain the first annotation comparison result.

[0118] In some alternative implementations, the optimization submodule includes: The generation unit is used to determine the missing information in the first annotation result based on the first annotation comparison result, and to generate supplementary prompt information based on the missing information; The fill unit is used to fill the supplementary prompt information into the target position of the initial prompt word template to obtain the target prompt word template; The second instantiation unit is used to instantiate the target prompt word template based on the role prompt information and generate the second prompt information; The annotation unit is used to slice and annotate the sample video based on the second prompt information, and generate the second annotation result of the sample video; The second comparison unit is used to compare the second annotation result with the reference information to obtain the second annotation comparison result; The optimization unit is used to optimize the second prompt information based on the second annotation comparison results, and generate the target prompt information corresponding to the sample video.

[0119] In some alternative embodiments, the video processing apparatus further includes: The third acquisition module is used to acquire, for any video to be processed, the event segments associated with the video, semantic annotation information and second metadata information; The fusion module is used to fuse event fragments, semantic annotation information, and secondary metadata information to generate target annotation results for the video to be processed.

[0120] In some alternative embodiments, the video processing apparatus further includes: The first conversion module is used to convert the semantic annotation information in the target annotation result into a vector representation to obtain the first vector; The module is used to establish the association between the first vector, event fragments, and semantic annotation information, and to store the association in a preset vector database.

[0121] In some alternative embodiments, the video processing apparatus further includes: The second conversion module is used to respond to video retrieval operations on the vector database by converting the retrieval information corresponding to the video retrieval operation into a vector representation to obtain a second vector. The determination module is used to determine the similarity between the second vector and each first vector in the vector database; The sorting module is used to sort the event fragments in the vector database based on similarity and generate a set of candidate event fragments corresponding to the search information.

[0122] The video processing apparatus provided in this application can execute the video processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0123] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0124] The following is a detailed reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0125] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0126] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the video processing method of embodiments of this application.

[0127] Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0128] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the video processing method shown in the above embodiments is implemented.

[0129] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0130] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A video processing method, characterized in that, The method includes: Multiple videos to be processed are obtained, wherein the multiple videos to be processed are obtained in response to the same search intent; In response to a video selection operation for the plurality of videos to be processed, a sample video is obtained, and in response to a video annotation operation for the sample video, reference information corresponding to the sample video is generated; Based on the video content of the sample video, generate character prompt information corresponding to the sample video; The reference information and the role prompt information are fused together to generate the target prompt information corresponding to the sample video; Based on the target prompt information, each of the videos to be processed is segmented to obtain the event segments corresponding to each video to be processed and the semantic annotation information corresponding to the event segments.

2. The method according to claim 1, characterized in that, The step of generating character prompt information corresponding to the sample video based on the video content of the sample video includes: Semantic understanding is performed on the video content of the sample video to obtain the semantic features of the sample video; Based on the matching relationship between the semantic features and the preset roles, the role prompt information corresponding to the semantic features is determined.

3. The method according to claim 1 or 2, characterized in that, The step of fusing the reference information with the role prompt information to generate the target prompt information corresponding to the sample video includes: Obtain the initial prompt word template corresponding to the sample video; The initial prompt word template is instantiated based on the character prompt information to generate the first prompt information; Based on the first prompt information, the sample video is sliced ​​and annotated to generate the first annotation result of the sample video; The first annotation result and the reference information are compared to obtain the first annotation comparison result; Based on the first annotation comparison result, the initial prompt word template is optimized to generate the target prompt information corresponding to the sample video.

4. The method according to claim 3, characterized in that, The instantiation of the initial prompt word template based on the role prompt information to generate the first prompt information includes: Obtain the first metadata information of the sample video; Based on the role prompt information and the first metadata information, the initial prompt word template is instantiated to generate the first prompt information.

5. The method according to claim 3, characterized in that, The step of comparing the first annotation result and the reference information to obtain the first annotation comparison result includes: The reference information is updated based on the role prompt information to obtain the target reference information; The first annotation result and the target reference information are compared to obtain the first annotation comparison result.

6. The method according to claim 5, characterized in that, The step of optimizing the initial prompt word template based on the first annotation comparison result to generate the target prompt information corresponding to the sample video includes: Based on the first annotation comparison result, the missing information in the first annotation result is determined, and supplementary prompt information is generated based on the missing information; The supplementary prompt information is filled into the target position of the initial prompt word template to obtain the target prompt word template; Based on the role prompt information, the target prompt word template is instantiated to generate second prompt information; Based on the second prompt information, the sample video is sliced ​​and annotated to generate a second annotation result for the sample video; The second annotation result is compared with the reference information to obtain the second annotation comparison result; Based on the second annotation comparison results, the second prompt information is optimized to generate the target prompt information corresponding to the sample video.

7. The method according to claim 1, characterized in that, The method further includes: For any of the videos to be processed, obtain the event segment associated with the video to be processed, the semantic annotation information, and the second metadata information; The event fragment, the semantic annotation information, and the second metadata information are fused to generate the target annotation result corresponding to the video to be processed.

8. The method according to claim 7, characterized in that, The method further includes: The semantic annotation information in the target annotation result is converted into a vector representation to obtain the first vector; Establish the association between the first vector, the event fragment, and the semantic annotation information, and store the association in a preset vector database.

9. The method according to claim 8, characterized in that, The method further includes: In response to a video retrieval operation on the vector database, the retrieval information corresponding to the video retrieval operation is converted into a vector representation to obtain a second vector; Determine the similarity between the second vector and each of the first vectors in the vector database; Based on the similarity, the event fragments in the vector database are sorted to generate a set of candidate event fragments corresponding to the search information.

10. A video processing apparatus, characterized in that, The device includes: The first acquisition module is used to acquire multiple videos to be processed, wherein the multiple videos to be processed are acquired in response to the same search intent; The second acquisition module is used to acquire sample videos in response to a video selection operation for the plurality of videos to be processed, and to generate reference information corresponding to the sample videos in response to a video annotation operation for the sample videos. The first generation module is used to generate character prompt information corresponding to the sample video based on the video content of the sample video; The second generation module is used to fuse the reference information with the role prompt information to generate target prompt information corresponding to the sample video; The processing module is used to segment each of the videos to be processed according to the target prompt information, and obtain the event segments corresponding to each video to be processed and the semantic annotation information corresponding to the event segments.

11. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the video processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the video processing method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the video processing method according to any one of claims 1 to 9.

Images