A media information processing method, device, system, equipment and storage medium based on AI big model

Abstract

The embodiment of the present invention provides a media information processing method based on an AI large model, the method comprising: generating an Nth input according to a first constraint and a second constraint, transmitting the Nth input to a target large model; obtaining an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set is composed of M text content items; sending the M text content items of the Nth text content information set to a first terminal, so that the first terminal displays the target text content items matching the target media segment. Through this scheme, it is possible to automatically obtain constraints from multiple dimensions, not only supporting the acquisition of explicit first constraints, but also obtaining implicit second constraints from the video content itself, and combining the first constraints and the second constraints as inputs to the large model, so that the video description copy can be obtained more efficiently and accurately.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a media information processing method, device, system, equipment and storage medium based on an AI big model. Background Art

[0002] Currently in the field of video editing, the processing methods for automatically generating text based on videos are relatively rough, requiring users to provide more external manual input constraints, and are highly dependent on external manual input constraints. The manual constraints are also highly uncertain, and once the external manual constraints are input inaccurately, it is difficult to obtain better video text, and the generation efficiency is low. Summary of the invention

[0003] An embodiment of the present invention provides a media information processing method based on an AI big model, which is used to use the AI ​​big model on the server side to quickly generate short video copy content for video materials shot by users and return it to the terminal.

[0004] In a first aspect, an embodiment of the present invention provides a media information processing method based on an AI big model, which is applied to a server, and the method includes:

[0005] Step 1: In response to obtaining an Nth content prompt for a target media segment, the Nth content prompt includes a first constraint and a second constraint, wherein the first constraint includes a custom constraint and / or a formatting constraint, wherein the custom constraint is a constraint extracted by parsing a constraint content input by a user through a first terminal using a first language model, and the formatting constraint is a formatting condition extracted by obtaining a sample video specified by the user through the first terminal and parsing the sample video using a third language model, and

[0006] The second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition.

[0007] generating an Nth input according to the first constraint condition and the second constraint condition, and transmitting the Nth input to the target large model;

[0008] Step 2: obtaining an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set consists of M text content items;

[0009] Step 3: sending the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and

[0010] So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

[0011] In a second aspect, an embodiment of the present invention provides a media information processing device based on an AI large model, which is applied to a server, and the device includes:

[0012] An input module, in response to obtaining an Nth content prompt for a target media segment, the Nth content prompt comprising a first constraint and a second constraint, wherein the first constraint comprises a custom constraint and / or a formatting constraint, wherein the custom constraint is a constraint input by a user through a first terminal obtained through a first large model, and the formatting constraint is a formatting condition extracted by obtaining a sample video specified by a user through the first terminal and parsing the sample video selected by the user through a third large model, and

[0013] The second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition;

[0014] generating an Nth input according to the first constraint condition and the second constraint condition, and transmitting the Nth input to the target large model;

[0015] An acquisition module, acquiring an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set consists of M text content items;

[0016] a sending module, sending the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and

[0017] So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

[0018] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the media information processing method described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the media information processing method described in the first aspect.

[0020] In the solution provided in the embodiment of the present invention, an Nth input is generated according to the first constraint condition and the second constraint condition, and the Nth input is transmitted to the target large model; an Nth text content information set generated by the target large model and matching the Nth input is obtained, wherein the Nth text content information set is composed of M text content items; and the M text content items of the Nth text content information set are sent to the first terminal, so that the first terminal displays the target text content item matching the target media segment.

[0021] Based on the solution provided by the embodiment of the present invention, it can be seen that the calculation solution of the present invention automatically obtains constraints from multiple dimensions, not only supports obtaining explicit first constraints, but also obtains implicit second constraints from the video content itself, and combines the first constraints and the second constraints as the input of the large model, so that the video description copy can be obtained more efficiently and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a media information processing method based on an AI big model provided in an embodiment of the present invention;

[0024] Figure 2 A flow chart of information interaction between a server and a client provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the structure of a media information processing device based on an AI large model provided by an embodiment of the present invention;

[0026] Figure 4 For Figure 3 The illustrated embodiment provides a schematic structural diagram of an electronic device corresponding to a media information processing device based on an AI big model. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0029] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0030] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0031] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0032] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0033] Embodiment 1

[0034] The data governance method provided in the embodiment of the present invention can be executed by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server or a server cluster.

[0035] Figure 1 A flowchart of a media information processing method based on an AI big model provided in an embodiment of the present invention, such as Figure 1 As shown, the following steps may be included:

[0036] Step 1: In response to obtaining an Nth content prompt for a target media segment, the Nth content prompt includes a first constraint and a second constraint, wherein the first constraint includes a custom constraint and / or a formatting constraint, wherein the custom constraint is a constraint extracted by parsing a constraint content input by a user through a first terminal using a first language model, and the formatting constraint is a formatting condition extracted by obtaining a sample video specified by the user through the first terminal and parsing the sample video using a third language model, and

[0037] The second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition.

[0038] generating an Nth input according to the first constraint condition and the second constraint condition, and transmitting the Nth input to the target large model;

[0039] Step 2: obtaining an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set consists of M text content items;

[0040] Step 3: sending the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and

[0041] So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

[0042] Specifically, in this embodiment, the first terminal may set a first input control in the user interaction interface to receive the constraint content described by the user input through the user interaction interface, and send the constraint content input by the user to the server after obtaining it. The first language model deployed by the server parses the constraint content and extracts the custom constraint condition. Specifically, the information interaction between the first terminal and the server is as shown in the attached figure. Figure 2 As shown;

[0043] Specifically, the first large language model is obtained after training with a first training set consisting of specific content input, and can be used to extract a multi-dimensional description tag of a custom description content (i.e., a content constraint prompt word generated by the first large language model) as a first constraint condition. Specifically, the large model for generating prompts according to text descriptions belongs to the category of open source, and will not be described in detail in this embodiment;

[0044] Alternatively, the first terminal may receive a sample video selected by a user through a selection control in a pushed sample video stream through a user interaction interface, or may obtain an address of a sample video pasted by a user through a second input control set in the user interface, extract the sample video according to the address of the sample video, and send it to the server, where the sample video is processed by a third language model deployed on the server, the sample video is parsed, and formatting conditions are extracted.

[0045] Specifically, the third largest language model is a large language model that is obtained after training with a third training set consisting of a large number of video sample sets and has the ability to understand video content. Unlike the above-mentioned second largest language model, the third largest language model is mainly used to perceive and identify the type of sample videos, such as identifying the emotional type of the video, whether it is a funny type or a literary type, etc., identifying the attribute style classification of the video, whether it is a daily life record or oral knowledge, etc. The third largest language model can be used to extract multi-dimensional style labels for sample videos (i.e., the prompt word of the style type generated by the third largest language model) as formatting constraints.

[0046] It should be noted that the first constraint condition includes a custom constraint condition and / or a formatted constraint condition. Therefore, in this embodiment, the first constraint condition may be simply a first language model that parses the constraint content and extracts the custom constraint condition, or a third language model that parses the sample video and extracts the formatted constraint condition. It may also include both the custom constraint condition and the formatted constraint condition. This aspect can be achieved by designing the input controls on the user interaction interface of the first terminal to obtain different user input content and then send it to the server for corresponding processing, which will not be elaborated here.

[0047] Furthermore, after receiving the explicit first constraint condition, it is necessary to obtain the implicit second constraint condition according to the target video segment. Specifically, the second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition, wherein the content constraint condition is a constraint condition on the text content extracted according to the content of each frame of the target media segment, and the length constraint condition is a constraint condition on the text length extracted according to the duration of the target media segment;

[0048] Preferably, the target video segment can be sent to the server, and the target video segment can be processed by the second largest language model deployed on the server to obtain the second constraint. Specifically, the second largest language model is a large language model that is obtained after training with a second training set consisting of a specific video set and has the ability to understand video content, and is used to identify the specific content of the video. The second largest language model can be used to extract the second constraint for the target video segment. Specifically, the second largest language model can be used to extract the multi-dimensional content label of each frame (i.e., the content label prompt word generated by the second largest language model) as the content constraint, and then the length of the text (i.e., the text length prompt word prompt) can be determined according to the length of the video segment as a text length constraint;

[0049] It should be noted that the length constraint of the text determined according to the length of the video clip can be identified according to the second largest language model, or it can be directly obtained according to the video length. The target video clip can be sent to the server, and the target video clip can be processed by the second largest language model deployed on the server to obtain the content constraint. In addition, the length constraint can be directly obtained according to the video length. In addition, the length constraint can be directly obtained according to the video length, which will not be elaborated here.

[0050] Further, after the server obtains the first constraint and the second constraint, an Nth input is generated according to the first constraint and the second constraint, and the input is input into a target large language model deployed on the server to obtain an Nth text content information set corresponding to the target media segment, wherein the Nth text content information set includes M text content items;

[0051] Further, M text content items of the Nth text content information set are sent to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and

[0052] So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

[0053] It should be noted that the target large language model involved in this embodiment is obtained after training with a specific data set based on an open source model. In order to facilitate user selection, a text content set can be obtained after one input, and a text content set can include multiple text content items.

[0054] Specifically, the first constraint condition obtained through the first largest language model (i.e., the multi-dimensional description label of the custom description content (which is the content constraint prompt generated by the first largest language model)) and the second constraint condition obtained through the second largest language model (the multi-dimensional content label of each frame of the target video clip (which is the content label prompt generated by the second largest language model), the length of the text (which is the prompt of the text length)) are combined as the prompt of the target large language model to obtain a text content set adapted to the target video clip.

[0055] The calculation scheme of the present invention automatically obtains constraints from multiple dimensions, and not only supports obtaining explicit first constraints, but also obtains implicit second constraints from the video content itself. The first constraints and the second constraints are combined as the input of the large model, so that the video description text can be obtained more efficiently and accurately.

[0056] Embodiment 2

[0057] With respect to the above-mentioned first embodiment, all the contents generated by the Nth content prompt are sent to the first terminal. However, when the user is still not satisfied after switching to view all the contents and needs to continue to generate new target text contents for presentation, in this embodiment, a preset number threshold can be set for the number of text content items without display marks in the M text content items on the first terminal. When the number of text content items without display marks in the M text content items reaches the preset number threshold, an instruction to update the text content is automatically triggered to be sent to the server, and the original first constraint condition and the second constraint condition are again used as the N+1th input, and the N+1th input is transmitted to the target large model.

[0058] Acquire a text content information set that matches the N+1th input, wherein the N+1th text content information set consists of M text content items;

[0059] updating the N+1th text content information set to the Nth text content information set, and then sending the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and

[0060] So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

[0061] Embodiment 3

[0062] In response to obtaining an adjustment of any one of the first constraint conditions, obtaining an adjusted third constraint condition,

[0063] For example, in the above embodiment, the user's custom constraint condition is re-acquired, or the sample video selected by the user is re-acquired to re-acquire the formatting constraint condition, or the user's custom constraint condition is newly acquired, and the sample video selected by the user is re-acquired to re-acquire the formatting constraint condition; both involve adjusting the first constraint condition to the third constraint condition to obtain the adjusted third constraint condition;

[0064] A Kth input is generated according to the third constraint and the second constraint, and is input into a target large language model to obtain a Kth text content set corresponding to the target media segment, wherein the Kth text content set includes M text content items.

[0065] Using the Kth text content set to update the Nth text content information set,

[0066] A target text content item is displayed, wherein the target text content item is at least one text content item in the candidate text content item set, and the candidate text content item is a text content item without a display mark among the M text content items in the updated Nth text content information set.

[0067] Furthermore, a display mark is added to the displayed text content items among the M text content items, and in response to receiving a switch instruction for the target text content item, at least one text content item is selected from the candidate text content item set as the target text content item.

[0068] Exemplarily, in response to receiving a switch instruction for a target text content item sent by a user through a switch control of a first terminal, the displayed text content item is marked as displayed. At this time, at least one text content item is selected from the set of candidate text content items as the target text content item for display.

[0069] Further, in response to the number of candidate text content items in the candidate text content set reaching a preset number threshold;

[0070] Exemplarily, when only 2 of the M content items are left without display marks, it can be considered that the number of candidate text content items is 2, reaching the preset number threshold, and it is necessary to obtain more text content items again, and obtain the N+1th text content information set again based on the Nth input, wherein the N+1th text content information set includes M text content items; the M text content items in the N+1 text content set are used and added to the candidate text content item set for displaying the content items after the switching instruction is obtained later. In this way, by performing the Nth input once, multiple results can be obtained at one time, avoiding frequent calls to the model, simplifying the content generation process, and improving the content generation efficiency.

[0071] Embodiment 4

[0072] In response to obtaining an adjustment of any constraint condition in the second constraint condition, obtaining an adjusted fourth constraint condition,

[0073] For example, in the first embodiment, if the user deletes the target media segment, the content and duration of the target media segment will be reduced, so that the content constraint and the text length constraint will change. If the user speeds up or slows down the target media segment, the duration of the target media segment will be reduced, so that the text length constraint will also change. Both involve adjusting the second constraint to the fourth constraint to obtain the adjusted fourth constraint.

[0074] generating a Pth input according to the first constraint and the fourth constraint, and inputting the input into a target large language model to obtain a Pth text content set corresponding to the target media segment, wherein the Pth text content set includes M text content items,

[0075] Using the Pth text content set to update the Nth text content information set,

[0076] A target text content item is displayed, wherein the target text content item is at least one text content item in the candidate text content item set, and the candidate text content item is a text content item without a display mark among the M text content items in the updated Nth text content information set.

[0077] Furthermore, a display mark is added to the displayed text content items among the M text content items, and in response to receiving a switch instruction for the target text content item, at least one text content item is selected from the candidate text content item set as the target text content item.

[0078] Exemplarily, in response to receiving a switch instruction for a target text content item sent by the user through a switch control, the displayed text content item is marked as displayed. At this time, at least one text content item is selected from the candidate text content item set as the target text content item for display.

[0079] Further, in response to the number of candidate text content items in the candidate text content set reaching a preset number threshold;

[0080] Exemplarily, when only 2 of the M content items are left without display marks, it can be considered that the number of candidate text content items is 2, reaching the preset number threshold, and it is necessary to obtain more text content items again, and obtain the N+1th text content information set again based on the Nth input, wherein the N+1th text content information set includes M text content items; the M text content items in the N+1 text content set are used and added to the candidate text content item set for displaying the content items after the switching instruction is obtained later. In this way, by performing the Nth input once, multiple results can be obtained at one time, avoiding frequent calls to the model, simplifying the content generation process, and improving the content generation efficiency.

[0081] Embodiment 5

[0082] In response to obtaining an adjustment of any constraint in the first constraint, obtaining an adjusted third constraint, for example, in one of the above embodiments, re-obtaining a user's custom constraint, or re-obtaining a sample video selected by the user to re-obtain a formatted constraint, or re-obtaining a user's custom constraint, and re-obtaining a sample video selected by the user to re-obtain a formatted constraint; both involve adjusting the first constraint to the third constraint to obtain the adjusted third constraint;

[0083] and obtaining an adjustment of any constraint condition in the second constraint condition, obtaining an adjusted fourth constraint condition,

[0084] For example, in the first embodiment, if the user deletes the target media segment, the content and duration of the target media segment will be reduced, so that the content constraint and the text length constraint will change. If the user speeds up or slows down the target media segment, the duration of the target media segment will be reduced, so that the text length constraint will also change. Both involve adjusting the second constraint to the fourth constraint to obtain the adjusted fourth constraint.

[0085] A Qth input is generated according to the third constraint condition and the fourth constraint condition, and is input into a target large language model to obtain a Qth text content set corresponding to the target media segment, wherein the Qth text content set includes M text content items,

[0086] Using the Qth text content set to update the Nth text content information set,

[0087] A target text content item is displayed, wherein the target text content item is at least one text content item in the candidate text content item set, and the candidate text content item is a text content item without a display mark among the M text content items in the updated Qth text content set.

[0088] Furthermore, a display mark is added to the displayed text content items among the M text content items, and in response to receiving a switch instruction for the target text content item, at least one text content item is selected from the candidate text content item set as the target text content item.

[0089] Exemplarily, in response to receiving a switch instruction for a target text content item sent by the user through a switch control, the displayed text content item is marked as displayed. At this time, at least one text content item is selected from the candidate text content item set as the target text content item for display.

[0090] Further, in response to the number of candidate text content items in the candidate text content set reaching a preset number threshold;

[0091] Exemplarily, when only 2 of the M content items are left without display marks, it can be considered that the number of candidate text content items is 2, reaching the preset number threshold, and it is necessary to obtain more text content items again, and obtain the N+1th text content information set again based on the Nth input, wherein the N+1th text content information set includes M text content items; the M text content items in the N+1 text content set are used and added to the candidate text content item set for displaying the content items after the switching instruction is obtained later. In this way, by performing the Nth input once, multiple results can be obtained at one time, avoiding frequent calls to the model, simplifying the content generation process, and improving the content generation efficiency.

[0092] The data management device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0093] Figure 3 A structural diagram of a media information processing device based on an AI large model provided by an embodiment of the present invention, such as Figure 3 As shown, the device includes: an input module 11, an acquisition module 12, and a sending module 13.

[0094] The input module 11 is responsive to obtaining an Nth content prompt for a target media segment, wherein the Nth content prompt includes a first constraint and a second constraint, wherein the first constraint includes a custom constraint and / or a formatting constraint, wherein the custom constraint is a constraint input by a user through a first terminal obtained through a first large model, and the formatting constraint is a formatting condition extracted by obtaining a sample video specified by a user through the first terminal and parsing the sample video selected by the user through a third large model, and

[0095] The second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition;

[0096] generating an Nth input according to the first constraint condition and the second constraint condition, and transmitting the Nth input to the target large model;

[0097] An acquisition module 12, acquiring an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set consists of M text content items;

[0098] The sending module 13 sends the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and

[0099] So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

[0100] The acquisition module is further specifically configured to respond to receiving an instruction to update the text content sent by the first terminal, wherein the instruction to update the text content is sent by the first terminal in response to the number of text content strips without display marks in the M text content strips reaching a preset number threshold;

[0101] Again using the original first constraint condition and the second constraint condition as the N+1th input, and transmitting the N+1th input to the target large model;

[0102] Acquire a text content information set that matches the N+1th content input, wherein the N+1th text content information set consists of M text content items;

[0103] The sending module is also specifically used to update the N+1th text content information set to the Nth text content information set, and to send the M text content items of the updated Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and so that the first terminal marks the displayed text content item as a displayed text content item, and after the first terminal receives a switching instruction for the target text content item, at least one text content item is selected from the text content items without a display mark among the M text content items as the target text content item.

[0104] Figure 3 The device shown can execute the steps introduced in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments, which will not be repeated here.

[0105] In one possible design, the above Figure 3 The structure of the data management device shown can be implemented as an electronic device, such as Figure 4 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable codes, and when the executable codes are executed by the processor 22, the processor 22 can at least implement the data governance method provided in the above-mentioned embodiment.

[0106] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the media information processing method provided in the aforementioned embodiment.

[0107] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0108] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A media information processing method based on an AI large model, which is applied to a server, characterized in that: The method comprises: Step 1: In response to obtaining an Nth content prompt for a target media segment, the Nth content prompt includes a first constraint and a second constraint, wherein the first constraint includes a custom constraint and / or a formatting constraint, wherein the custom constraint is a constraint extracted by parsing a constraint content input by a user through a first terminal using a first language model, and the formatting constraint is a formatting condition extracted by obtaining a sample video specified by the user through the first terminal and parsing the sample video using a third language model, and The second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition. generating an Nth input according to the first constraint condition and the second constraint condition, and transmitting the Nth input to the target large model; Step 2: obtaining an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set consists of M text content items; Step 3: sending the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

2. The media information processing method according to claim 1, characterized in that: The content constraint condition is a constraint condition on text content extracted from each frame of the target media segment through the second largest language model, and the length constraint condition is a constraint condition on text length extracted according to the duration of the target media segment.

3. The media information processing method according to claim 1, characterized in that: The method comprises: In response to receiving an instruction to update the text content sent by the first terminal, the instruction to update the text content is sent by the first terminal in response to the number of text content items without display marks in the M text content items reaching a preset number threshold; Again using the original first constraint condition and the second constraint condition as the N+1th input, and transmitting the N+1th input to the target large model; Acquire a text content information set that matches the N+1th content input, wherein the N+1th text content information set consists of M text content items; The N+1th text content information set is updated to the Nth text content information set, and step 3 is repeated.

4. The media information processing method according to claim 1, characterized in that: In response to obtaining an adjustment of any one of the first constraints, obtaining an adjusted third constraint, A Kth input is generated according to the third constraint and the second constraint, and is input into a target large language model to obtain a Kth text content set corresponding to the target media segment, wherein the Kth text content set includes M text content items. The Kth text content set is used to update the Nth text content information set, and step 3 is repeated.

5. The media information processing method according to claim 1, characterized in that: In response to obtaining an adjustment of any one of the second constraints, obtaining an adjusted fourth constraint, generating a Pth input according to the first constraint and the fourth constraint, and inputting the input into a target large language model to obtain a Pth text content set corresponding to the target media segment, wherein the Pth text content set includes M text content items, The Pth text content set is used to update the Nth text content information set, and step 3 is repeated.

6. The media information processing method according to claim 1, characterized in that: In response to obtaining an adjustment of any one of the first constraints, obtaining an adjusted third constraint, and obtaining an adjustment of any constraint condition in the second constraint condition, obtaining an adjusted fourth constraint condition, A Qth input is generated according to the third constraint condition and the fourth constraint condition, and is input into a target large language model to obtain a Qth text content set corresponding to the target media segment, wherein the Qth text content set includes M text content items, The Qth text content set is used to update the Nth text content information set, and step 3 is repeated.

7. A media information processing device based on an AI large model, applied to a server, characterized in that: The device comprises: An input module, in response to obtaining an Nth content prompt for a target media segment, the Nth content prompt comprising a first constraint and a second constraint, wherein the first constraint comprises a custom constraint and / or a formatting constraint, wherein the custom constraint is a constraint input by a user through a first terminal obtained through a first large model, and the formatting constraint is a formatting condition extracted by obtaining a sample video specified by a user through the first terminal and parsing the sample video selected by the user through a third large model, and The second constraint condition is generated according to the target media segment, and the second constraint condition at least includes a content constraint condition and a text length constraint condition; generating an Nth input according to the first constraint condition and the second constraint condition, and transmitting the Nth input to the target large model; An acquisition module, acquiring an Nth text content information set generated by the target large model and matching the Nth input, wherein the Nth text content information set consists of M text content items; a sending module, sending the M text content items of the Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

8. The media information processing device according to claim 7, characterized in that: The acquisition module is further specifically configured to respond to receiving an instruction to update the text content sent by the first terminal, wherein the instruction to update the text content is sent by the first terminal in response to the number of text content items without display marks in the M text content items reaching a preset number threshold; Again using the original first constraint condition and the second constraint condition as the N+1th input, and transmitting the N+1th input to the target large model; Acquire a text content information set that matches the N+1th content input, wherein the N+1th text content information set consists of M text content items; The sending module is further specifically configured to update the N+1th text content information set to the Nth text content information set, and send the M text content items of the updated Nth text content information set to the first terminal, so that the first terminal displays the target text content item matching the target media segment, and So that the first terminal marks the displayed text content strip as a displayed text content strip, and after the first terminal receives a switching instruction for the target text content strip, at least one text content strip among the M text content strips without a display mark is selected as the target text content strip.

9. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the media information processing method as described in any one of claims 1 to 6.

10. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the media information processing method as described in any one of claims 1 to 6.

Images