Generative large model controllable text generation method

Abstract

The invention provides a method for generating a controllable text of a generative large model, which relates to a data processing technology and comprises the following steps of: processing the attribute of a first modal file to be analyzed by the generative large model to generate a corresponding analysis controllable template; interacting with the analysis controllable template, and determining controllable information of a controllable text of a first modal file; segmenting the first modal file according to corresponding attributes based on the controllable information to obtain a plurality of segmented fragments and controllable sub-information of each segmented fragment; and performing content extraction and conversion processing on each segmented fragment to generate a corresponding initial text, and processing the initial text based on the controllable sub-information to obtain a multi-level controllable text of the first modal file.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a generative large-model controllable text generation method. Background Art

[0002] At present, text generation technology for images and videos mainly relies on traditional text generation models. These models often use relatively general algorithms in the processing process, lacking precise control of image and video content and in-depth understanding of user-specific needs. That is, although the existing generative models can output relatively fluent descriptive text based on the input image or video content, most of them remain at the "overall description" or "automatic summary" stage, lacking fine-grained support for user-controllable needs. It is unable to meet user-controllable and personalized text output.

[0003] Therefore, how to combine user needs to produce controllable and personalized text output has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiment of the present invention provides a generative large-model controllable text generation method, which can perform controllable and personalized text output in combination with user needs.

[0005] A first aspect of an embodiment of the present invention provides a method for generating controllable text using a generative large model, comprising: The generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding analytical controllable template; interacting with the parsed controllable template to determine controllable information of the controllable text of the first modal file; Based on the controllable information, the first modal file is segmented according to corresponding attributes to obtain a plurality of segmented segments and controllable sub-information of each segmented segment; Content extraction and conversion processing are performed on each segmented segment to generate a corresponding initial text, and the initial text is processed based on the controllable sub-information to obtain a multi-level controllable text of the first modal file.

[0006] Optionally, the generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding parsing controllable template, including: A1. If the attributes of the first modal file include image attributes, generating an image interactive preview box corresponding to the image attributes; A2. Adaptively process the interactive preview frame based on the specifications in the first modal file attributes, so that the interactive preview frame fits the first modal file; A3. After extracting the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection lines, a corresponding analytical controllable template is generated.

[0007] Optionally, the adaptive processing of the interactive preview frame based on the specification in the first modal file attribute so that the interactive preview frame is aligned with the first modal file includes: Obtain the number of horizontal pixels and the number of vertical pixels of the first modal file; The interactive preview frame is adaptively processed based on the specifications of the number of horizontal pixels and the number of vertical pixels to make the interactive preview frame fit the boundary of the first modal file.

[0008] Optionally, after extracting the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection line, a corresponding analytical controllable template is generated, including: Determine the target horizontal frame and the target vertical frame in the interactive preview frame, and take the intersection of the target horizontal frame and the target vertical frame as the starting point; Taking the starting point as the initial point, corresponding coordinate lines are established according to the target horizontal frame and the target vertical frame respectively, and the end points of the target horizontal frame and the target vertical frame are the end points; The coordinates of the unit number of pixels are calculated based on the number of pixels between the initial point and the cutoff point, and the pixel points corresponding to the preset coordinate values ​​are selected to add coordinate selection lines to obtain a parsable controllable template.

[0009] Optionally, the interacting with the parsed controllable template to determine the controllable information of the controllable text of the first modal file includes: After determining that the user has selected any one of the coordinate selection lines, receiving a first mark added by the user to the coordinate selection line, and generating a first controllable slot and a first confirmation slot corresponding to the first mark at the parsed controllable template; After determining that the first confirmation slot is triggered, counting the first closed area formed by all coordinate selection lines having the same first mark; If it is determined that the user has selected a first closed area, it is used as a pre-divided area, and controllable sub-information about the pre-divided area is received based on the first controllable slot, and the controllable information at least includes text requirements of the controllable text.

[0010] Optionally, the first modal file is segmented according to corresponding attributes based on the controllable information to obtain a plurality of segmented segments and controllable sub-information of each segmented segment, including: Segmenting the first modal file according to the pre-segmented regions to obtain a plurality of segmented segments; Controllable sub-information of each segmented segment is extracted, wherein the controllable sub-information includes at least one of a text threshold quantity and a text ratio.

[0011] Optionally, the extracting and converting each segmented segment to generate a corresponding initial text, and processing the initial text based on the controllable sub-information to obtain a multi-level controllable text of the first modal file, includes: Extracting content from each segmented segment to obtain first text content and / or image content, converting the image content into a preset second text content, wherein the initial text includes the first text content and the second text content; Based on the text threshold number and text ratio corresponding to the controllable sub-information, the initial text is fused to obtain the hierarchy and controllable sub-text of each segmented segment; The levels and controllable sub-texts of the segmented segments are counted and filled into a preset area of ​​the output template to obtain multi-level controllable texts of the first modal file.

[0012] Optionally, the initial text is fused based on the text threshold quantity and text ratio corresponding to the controllable sub-information to obtain the level and controllable sub-text of each segmented segment, including: If it is determined that the text threshold number of the controllable sub-information is set without setting the text ratio, and the number of controllable sub-texts is less than or equal to the first text threshold number, then save; If the number of controllable subtexts is greater than the first threshold number, the controllable subtexts are input into the text simplification model to be reduced to a number less than or equal to the first threshold number of texts and then saved; A corresponding level is generated according to the number of pixels in each segmented segment, and the level and controllable sub-text of each segmented segment are stored correspondingly.

[0013] Optionally, the initial text is fused based on the text threshold quantity and text ratio corresponding to the controllable sub-information to obtain the level and controllable sub-text of each segmented segment, including: If it is determined that the controllable sub-information does not set the text threshold number and the text ratio, the pixel ratio of all the segmented segments is calculated according to the number of pixels in each segmented segment; Extracting the highest ratio value in the pixel point ratio and the corresponding controllable sub-text quantity as the highest quantity, and determining the second threshold quantity of controllable sub-information of other segmented segments based on the highest quantity and the pixel point ratio; Based on the second threshold number and the number of pixels in each segmented segment, the level of each segmented segment and the corresponding storage of the controllable sub-text are obtained.

[0014] Optionally, obtaining the level of each segmented segment and the corresponding storage of the controllable subtext based on the second threshold number and the number of pixels in each segmented segment includes: If the number of controllable sub-texts of the segmented fragment is less than or equal to the second threshold number of texts, save; If the number of controllable subtexts is greater than the second threshold number, the controllable subtexts are input into the text simplification model to be reduced to a number less than or equal to the second threshold number of texts and then saved; A corresponding level is generated according to the number of pixels in each segmented segment, and the level and controllable sub-text of each segmented segment are stored correspondingly.

[0015] Optionally, the generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding parsing controllable template, including: If the attributes of the first modal file include video attributes, generating a video interaction preview frame corresponding to the video attributes, wherein the video interaction preview frame includes a video interaction axis; Adaptively process the interactive preview frame based on the specifications in the first modal file attributes, so that the interactive preview frame fits and corresponds to the first modal file; After extracting the video interaction axis to generate the corresponding time point selection line, a corresponding analytically controllable template is generated.

[0016] Optionally, after extracting the video interaction axis to generate a corresponding time point selection line, generating a corresponding analytical controllable template includes: The video interaction axis takes point 0 as the starting point, determines a time point at a preset time interval, and generates a corresponding time point selection line.

[0017] Optionally, the interacting with the parsed controllable template to determine the controllable information of the controllable text of the first modal file includes: After determining that the user selects any time point selection line, receiving a second mark added by the user to the time point selection line, and generating a second controllable slot and a second confirmation slot corresponding to the second mark at the parsed controllable template; After determining that the second confirmation slot is triggered, the minimum time and the maximum time with the same second mark are counted to obtain a time period; The controllable sub-information about the corresponding time period is received based on the first controllable slot, and the controllable information at least includes text requirements of the controllable text.

[0018] Optionally, also include: If it is determined that the user selects a frame image at any time in the video interaction axis and inputs image attributes, steps A1 to A3 are executed.

[0019] This solution generates an image interaction preview frame for the first modal file with image attributes, and performs adaptive processing based on the image specifications to make it fit the image accurately. At the same time, the preview frame specifications are extracted to generate coordinate selection lines to obtain a parsable controllable template. When processing video attribute files, a video interaction preview frame containing a video interaction axis is generated, and a parsable controllable template is formed by generating time point selection lines for the video interaction axis according to a preset time period. This method provides users with an intuitive and accurate operation interface. Users can easily specify areas of interest or time periods by clicking on coordinate selection lines or time point selection lines, which greatly improves the accuracy and interactivity of the operation and provides a solid foundation for the subsequent determination of controllable information.

[0020] The user of this solution interacts with the parsed controllable template and determines the controllable information of the controllable text by adding tags, generating controllable slots, and other operations. Based on this information, the system segments the first modal file according to the attributes to obtain multiple segmented segments and extract the controllable sub-information of each segment. In the image field, the text requirements can be determined based on the user-marked area, such as the text threshold number, text ratio, etc.; in the video field, the text generation requirements can be clarified based on the time period marked by the user. This mechanism enables customized processing of text generation for different areas or time periods, realizes refined operations on multimedia files, and improves the fit between the generated text and user needs.

[0021] This solution extracts and converts the content of each segmented segment to generate the initial text, and then processes the initial text based on the controllable sub-information. According to the number of text thresholds and the text ratio, the text simplification model and other technologies are used to obtain the hierarchy and controllable sub-text of each segmented segment. Finally, the text is counted and filled into the output template to obtain multi-level controllable text. This process fully considers the diverse needs of users. The generated text not only accurately reflects the actual situation of each part of the image or video in terms of content, but also has good hierarchy and rationality in structure and length. Taking images as an example, the segmented segments of important areas can generate detailed and high-level text descriptions; for videos, the text length and level of key plot time periods can be reasonably set according to their importance. It greatly improves the quality and practicality of text generation, provides users with more expected text output results, and meets the complex needs of text generation in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a method for generating controllable text of a generative large model provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of an image interactive preview frame provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 , is a flow chart of a method for generating controllable text using a generative large model provided by an embodiment of the present invention, the method comprising: S1, the generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding analytical controllable template.

[0025] In the generative large model controllable text generation method of the present invention, performing attribute processing on the first modal file and generating a parsed controllable template is a key initial step. For the first modal file with image attributes, a parsed controllable template is generated by generating an image interactive preview frame, performing adaptive processing, and extracting coordinate selection lines, which provides a basis and an operation interface for subsequent determination of controllable information, file segmentation, and generation of controllable text, so that the entire text generation process can be carried out under the precise control of the user.

[0026] The generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding analytical controllable template, including: A1. If the attributes of the first modal file include image attributes, an image interactive preview box corresponding to the image attributes is generated.

[0027] See also Figure 2 ,When the attributes of the first modal file to be analyzed include image attributes, the generative ,large model will generate an image interactive preview box corresponding to the image attribute. ,This image interactive preview box is a visual interface element, and its main ,function is to provide users with an intuitive operation platform to facilitate ,the user’s subsequent interactive operations on the image.

[0028] A2. Adaptively process the interactive preview frame based on the specifications in the first modal file attributes so that the interactive preview frame fits the first modal file.

[0029] The image interactive preview box has the same shape as the image, but is slightly larger than the image to wrap the image. For example, for a common landscape image, the generative large model will recognize its format and generate an image interactive preview box that wraps the landscape image according to the system's display settings and user interface layout requirements.

[0030] The step of adaptively processing the interactive preview frame based on the specification in the first modal file attribute so that the interactive preview frame is aligned with the first modal file includes: Obtain the number of horizontal pixels and the number of vertical pixels of the first modal file; The interactive preview frame is adaptively processed based on the specifications of the number of horizontal pixels and the number of vertical pixels to make the interactive preview frame fit the boundary of the first modal file.

[0031] In order to ensure that the image interactive preview frame can accurately reflect the actual range and content of the image, it is necessary to perform adaptive processing on the interactive preview frame based on the specifications in the first modal file attributes so that the interactive preview frame fits the first modal file. The specific implementation method is as follows: First, the system obtains the number of horizontal pixels and the number of vertical pixels of the first modality file. These two parameters are key information describing the image size and can be accurately obtained by reading the metadata of the image file or using an image analysis algorithm.

[0032] Then, based on the obtained specifications of the number of horizontal pixels and the number of vertical pixels, the interactive preview frame is adaptively adjusted. The adjustment process mainly involves dynamically modifying the size and proportion of the interactive preview frame so that its boundary fits the boundary of the first modal file.

[0033] For example, if the horizontal pixel count of an image is 1920 and the vertical pixel count is 1080, the system will adjust the width and height of the image interactive preview box according to this specification, ensuring that the size of the preview box is consistent with the shape of the image, but it needs to be slightly larger, for example, 10 pixels in each direction, to wrap the image so that the image can be displayed in the preview box completely and without deformation. The advantage of this is that users can more accurately select and process various parts of the image in subsequent operations.

[0034] A3. After extracting the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection lines, a corresponding analytical controllable template is generated.

[0035] After completing the adaptive processing of the interactive preview frame, it is necessary to extract the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection lines, and then generate the corresponding analytical controllable template.

[0036] In some embodiments, after extracting the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection line, generating the corresponding analytical controllable template includes: Determine the target horizontal frame and target vertical frame in the interactive preview frame, and take the intersection of the target horizontal frame and the target vertical frame as the starting point. It can be understood that the target horizontal frame and the target vertical frame can be the lower horizontal line and the left vertical line of the interactive preview frame, and the starting point is the point at the lower left corner of the interactive preview frame.

[0037] With the starting point as the initial point, corresponding coordinate lines are established according to the target horizontal frame and the target vertical frame, and the end points of the target horizontal frame and the target vertical frame are the cutoff points. In this solution, the end points (end points of the straight lines) of the target horizontal frame and the target vertical frame are the cutoff points. In this way, a two-dimensional coordinate system is formed for locating various positions in the image.

[0038] The coordinates of the unit number of pixels are calculated based on the number of pixels between the initial point and the cutoff point, and the pixel points corresponding to the preset coordinate values ​​are selected to add coordinate selection lines to obtain a parsable controllable template.

[0039] For example, if the number of horizontal pixels between the initial point and the cutoff point is 1000 and its length is 10 cm, then one pixel corresponds to 0.01 cm, and the preset coordinate value is, for example, 0.5 cm, then the number of corresponding pixels is 50. At this time, a coordinate selection line can be added every 50 pixels. By adding coordinate selection lines, users can more conveniently select specific areas in the image.

[0040] After adding the coordinate selection line, you get a parsable controllable template containing coordinate information and selection functions. This template provides the basis for subsequent interaction with the user and determining the controllable information. The user can specify the image area to be processed by clicking the coordinate selection line.

[0041] S2, interacting with the parsed controllable template to determine controllable information of the controllable text of the first modal file.

[0042] In the generative large model controllable text generation method of the present invention, interacting with the parsed controllable template and determining the controllable information is a key link that connects the previous and the next. The parsed controllable template generated in the previous step provides an operation interface for the user, and this step clarifies the processing requirements for the specific area in the first modal file through the interaction between the user and the template, that is, determines the controllable information of the controllable text. These controllable information will guide the subsequent segmentation of the first modal file and the text generation process to ensure that the final generated text meets the specific needs of the user.

[0043] In some embodiments, the interacting with the parsed controllable template to determine the controllable information of the controllable text of the first modality file includes: S21, after determining that the user selects any one of the coordinate selection lines, receiving a first mark added by the user to the coordinate selection line, and generating a first controllable slot and a first confirmation slot corresponding to the first mark at the parsed controllable template.

[0044] When it is determined that the user has selected any coordinate selection line, the system will enter the information interaction process. The user's selection of the coordinate selection line indicates that the user has begun to specify the part of the first modal file that they are interested in. At this time, the system receives the first mark added by the user to the coordinate selection line. The first mark is a user-defined identifier used to distinguish different areas or contents. For example, the user may add marks such as "1" and "2".

[0045] Next, the system generates a first controllable slot and a first confirmation slot corresponding to the first mark at the parsed controllable template. The first controllable slot is an input area in which the user can enter specific requirements for the marked area, such as the number of words to be generated in the text. The first confirmation slot is used by the user to confirm whether the input information is correct and complete. From the perspective of technical implementation, the system will dynamically create these two slots on the interface of the parsed controllable template and associate them with the first mark added by the user.

[0046] S22, after determining that the first confirmation slot is triggered, counting the first closed area formed by all coordinate selection lines having the same first mark.

[0047] After the user completes the input of the first controllable slot information, the first confirmation slot is triggered. After receiving the confirmation signal, the system will count the first closed area formed by all coordinate selection lines with the same first mark. This statistical process involves geometric analysis and logical judgment of the coordinate selection lines. The system will determine which coordinate selection lines form a closed figure based on the position and connection relationship of the coordinate selection lines.

[0048] For example, in an interactive preview frame of an image, if the user adds a "1" mark to multiple coordinate selection lines, the system will analyze the positions of these coordinate selection lines through algorithms to determine whether they can enclose a closed area. If so, this closed area will be recorded as the first closed area. This closed area will serve as a basic unit for subsequent processing.

[0049] It can be understood that several coordinate selection lines marked “2” can also form a first closed area, that is, there can be multiple first closed areas.

[0050] S23, if it is determined that the user has selected a first closed area, use it as a pre-divided area, and receive controllable sub-information about the pre-divided area based on the first controllable slot, wherein the controllable information at least includes text requirements of the controllable text.

[0051] The pre-segmented area is the basis for the subsequent segmentation of the first modal file, and it clarifies the specific scope where specific text generation is required. Then, the controllable sub-information about the pre-segmented area is received based on the first controllable slot. The controllable information at least includes the text requirements of the controllable text, such as the word range of the text, the focus of the content (such as describing the appearance and function of an object, etc.). The system will associate these controllable sub-information with the pre-segmented area so that it can be processed according to these requirements in the subsequent text generation process.

[0052] Through the above-mentioned interactive process with the parsed controllable template, the user's processing requirements for specific areas in the first modal file, that is, the controllable information of the controllable text, can be accurately determined. This interactive method gives users control, and users can flexibly specify processing areas and text requirements according to their needs. At the same time, the system can accurately identify and record this information, providing clear guidance for subsequent file segmentation and text generation, greatly improving the pertinence and accuracy of generated text, and meeting the diverse needs of different users in different scenarios.

[0053] S3, based on the controllable information, the first modal file is segmented according to corresponding attributes to obtain a plurality of segmented segments and controllable sub-information of each segmented segment.

[0054] In the generative large model controllable text generation method of the present invention, segmenting the first modal file based on controllable information is a key step that connects the previous and the next. The previous steps have determined the controllable information and the pre-segmented area, and this step actually segments the first modal file based on this information to obtain multiple segmented segments and extract the controllable sub-information of each segment. This lays the foundation for the subsequent content extraction and conversion processing of each segmented segment, and then generating the corresponding controllable text, so that text generation can more accurately process each specific area according to user needs.

[0055] In some embodiments, the first modal file is segmented according to the corresponding attributes based on the controllable information to obtain a plurality of segmented segments and controllable sub-information of each segmented segment, including: S31, segmenting the first modal file according to the pre-segmented areas to obtain a plurality of segmented segments.

[0056] After determining the pre-segmented areas, the system will segment the first modal file according to these pre-segmented areas to obtain multiple segmented segments. The system will cut the image out of the whole according to the coordinate range corresponding to the pre-segmented areas. For example, if the pre-segmented area is a rectangular area determined by the coordinate selection line, the system will extract the image pixel information within the rectangular area as an independent segmented segment. This process involves an image cropping algorithm, and the system will accurately extract the corresponding image part according to the boundary coordinates of the pre-segmented area.

[0057] S32, extracting controllable sub-information of each segmented segment, wherein the controllable sub-information includes at least one of a text threshold quantity and a text ratio.

[0058] After obtaining a plurality of segmented segments, the system extracts controllable sub-information of each segmented segment. The controllable sub-information includes at least one of a text threshold quantity and a text ratio.

[0059] The text threshold refers to the maximum or minimum number of characters allowed when generating text for the segmented segment. For example, for a segment of a person's face in an image, the user may set the text threshold to 200 characters, requiring that the generated description text cannot exceed this number of characters or is 200 characters. When the system subsequently processes the segmented segment, it will control the length of the generated text based on this threshold.

[0060] The text ratio is used to determine the length of the generated text according to the ratio of the number of pixels between the segmented segments. For example, some segmented segments do not have a text threshold number set, while some do. In this case, this solution can determine the number of words in the text of the segmented segments that have not been set based on the set text threshold number and the text ratio.

[0061] The system accurately extracts the controllable sub-information corresponding to each segment by associating it with the previously recorded controllable information. For each segment, these controllable sub-information will serve as important constraints for subsequent content extraction and text generation.

[0062] By segmenting the first modal file according to the pre-segmented regions and extracting the controllable sub-information of each segmented segment, this step achieves refined processing of the first modal file. The file is segmented into multiple segments so that subsequent text generation can be performed for different local regions, improving the pertinence of text generation. At the same time, the extraction of controllable sub-information provides clear constraints and guidance for the text generation process, ensuring that the generated text meets the user's expectations in terms of word count, length, etc., and better meets the diverse needs of users.

[0063] S4, extracting and converting the content of each segmented segment to generate a corresponding initial text, and processing the initial text based on the controllable sub-information to obtain a multi-level controllable text of the first modal file.

[0064] In the complete process of the generative large model controllable text generation method of the present invention, this step is in the critical final stage and undertakes the important task of converting the various segmented segments obtained by the previous segmentation process into controllable text. By extracting and converting the segmented segments and processing them based on controllable sub-information, a multi-level controllable text that meets the specific needs of the user is finally generated, ensuring that the generated text not only accurately reflects the content of the first modal file, but also meets the diverse requirements of the user in terms of structure and length.

[0065] In some embodiments, the extracting and converting each segmented segment to generate a corresponding initial text, and processing the initial text based on the controllable sub-information to obtain a multi-level controllable text of the first modal file, includes: S41, extracting the content of each segmented segment to obtain first text content and / or image content, converting the image content into a preset second text content, wherein the initial text includes the first text content and the second text content.

[0066] For each segmented segment, the system will perform a comprehensive content extraction operation. This process will obtain the first text content and / or image content according to the actual situation of the segmented segment. If the segmented segment itself contains text information, such as subtitles in the image, paragraphs in the document, etc., these contents will be directly extracted as the first text content. When there are image elements in the segmented segment, the system will use advanced image recognition and understanding technology to convert the image content into the preset second text content.

[0067] For example, when processing an image segment containing an animal, the image recognition algorithm will identify the animal's species, posture, color and other features, and convert this information into a text description to form the second text content. Finally, the first text content and the second text content are combined to form the initial text of each segment.

[0068] S42, based on the text threshold quantity and text ratio corresponding to the controllable sub-information, the initial text is fused to obtain the level and controllable sub-text of each segmented segment.

[0069] This step uses different strategies to fuse the initial text according to the different settings of the controllable sub-information to obtain the controllable sub-text of each segmented segment.

[0070] The initial text is fused based on the text threshold quantity and text ratio corresponding to the controllable sub-information to obtain the level and controllable sub-text of each segmented segment, including: S421: If it is determined that the text threshold number of the controllable sub-information is set but the text ratio is not set, and the controllable sub-text number is less than or equal to the first text threshold number, then save.

[0071] When it is determined that the controllable sub-information has set the text threshold number but not the text ratio, the system will first check the relationship between the controllable sub-text number and the text first threshold number. If the controllable sub-text number is less than or equal to the text first threshold number, it means that the current generated text length meets the user's set requirements, and the system will directly save the controllable sub-text.

[0072] S422, if the number of controllable sub-texts is greater than the first threshold number, the controllable sub-texts are input into the text simplification model to perform text reduction processing until the number of controllable sub-texts is less than or equal to the first threshold number and then saved.

[0073] If the number of controllable sub-texts is greater than the first threshold number, in order to meet the text length limit set by the user, the system will input the controllable sub-text into the text simplification model. The text simplification model will use natural language processing technology to simplify and refine the text, remove redundant information, simplify expressions, and reduce the number of texts to less than or equal to the first threshold number of texts before saving. For example, the initial text is: "This eye-catching new mobile phone has an extremely fashionable and unique appearance design. Its body lines are smooth and elegant, like smart notes jumping in space." The simplified text is "This new mobile phone has a fashionable and unique appearance and smooth body lines." This involves removing redundant modifiers, such as "eye-catching", "extremely", "like smart notes jumping in space" and other expressions that are highly modified but have little impact on the core information, making the text more concise and clear. Of course, it can also be processed by manual removal. This is a prior art and will not be repeated here.

[0074] S423, generating a corresponding level according to the number of pixels in each segmented segment, and storing the level and controllable subtext of each segmented segment in correspondence.

[0075] After the text quantity processing is completed, the system will generate the corresponding level according to the number of pixels in each segment. Generally speaking, segments with more pixels may represent more important and complex content, so they will be assigned a higher level. Finally, the level and controllable sub-text of each segment are stored accordingly for subsequent integration and output.

[0076] On the basis of the above embodiment, the initial text is fused based on the text threshold quantity and text ratio corresponding to the controllable sub-information to obtain the level and controllable sub-text of each segmented segment, including: S424: If it is determined that the controllable sub-information does not set the text threshold number and the text ratio, the pixel ratio of all the segmented segments is calculated based on the number of pixels in each segmented segment.

[0077] When the controllable sub-information does not set the text threshold but sets the text ratio, the system will calculate the pixel ratio of all the segmented segments based on the number of pixels in each segmented segment. This ratio reflects the proportion of each segmented segment in the entire first modal file. The larger the ratio, the more words will be generated in the final text.

[0078] S425, extracting the highest ratio value in the pixel point ratio and the corresponding controllable sub-text quantity as the highest quantity, and determining the second threshold quantity of controllable sub-information of other segmented segments based on the highest quantity and the pixel point ratio.

[0079] S426: Based on the second threshold number and the number of pixels in each segmented segment, the level of each segmented segment and the corresponding storage of the controllable sub-text are obtained.

[0080] The system will extract the highest ratio value in the pixel ratio and the corresponding number of controllable sub-texts as the highest number. Then, based on this highest number and the pixel ratio of each segmented segment, the second threshold number of controllable sub-information of other segmented segments is calculated. In this way, the length of text generation can be reasonably allocated according to the relative importance and scale of the segmented segments. Suppose a large event poster image with multiple elements is processed, which is divided into 5 segments A, B, C, D, and E. The number of pixels in each segment is counted and the pixel ratio is calculated. For example, segment A has 20,000 pixels, accounting for 0.2. After comparison, segment C has the highest pixel ratio of 0.3, and its controllable sub-text number of 300 words is set as the highest number. According to the formula "second threshold number = maximum number × pixel ratio of the segmented segment ÷ maximum ratio value", the second threshold number of segment A is calculated to be 200 words, segment B is 150 words, segment D is 100 words, and segment E is 250 words. In this way, the text length is reasonably allocated according to the relative importance and size of each segment.

[0081] The step of obtaining the level of each segmented segment and the corresponding storage of the controllable subtext based on the second threshold number and the number of pixels in each segmented segment includes: S4261: If the number of controllable sub-texts of the segmented fragment is less than or equal to the second threshold number of texts, save it.

[0082] If the number of controllable sub-texts of the segmented fragment is less than or equal to the second threshold number of texts, it means that the length of the text meets the requirements calculated based on the ratio, and the system will directly save the controllable sub-text.

[0083] S4262, if the number of controllable sub-texts is greater than the second threshold number, the controllable sub-texts are input into the text simplification model for text reduction processing until the number of texts is less than or equal to the second threshold number and then saved.

[0084] If the number of controllable sub-texts is greater than the second threshold number, the system will input the controllable sub-texts into the text simplification model and perform text reduction processing to make their number less than or equal to the second threshold number of texts before saving.

[0085] S4263, generating a corresponding level according to the number of pixels in each segmented segment, and storing the level and controllable sub-text of each segmented segment accordingly.

[0086] Similarly, a corresponding level is generated according to the number of pixels in each segmented segment, and the level and the controllable sub-text are stored correspondingly.

[0087] S43, counting the levels and controllable sub-texts of the segmented segments, and filling them into a preset area of ​​the output template to obtain multi-level controllable texts of the first modal file.

[0088] After completing the processing and storage of the hierarchy and controllable subtext of each segmented segment, the system will count the hierarchy and controllable subtext of all segmented segments. Then, this information is filled into the preset area of ​​the output template according to certain rules. The output template is pre-designed to standardize the presentation format of multi-level controllable text. By filling the information of each segmented segment into the template in an orderly manner, the multi-level controllable text of the first modal file is finally obtained.

[0089] Through the above steps, the present invention realizes deep content mining and accurate text generation of the first modal file. From content extraction and conversion to fine processing based on controllable sub-information, and then to the final text integration and output, the whole process fully considers the diverse needs of users, and can flexibly adjust the text generation strategy according to different controllable sub-information settings. The generated multi-level controllable text not only accurately reflects the various parts of the first modal file in terms of content, but also has good hierarchy and rationality in structure and length, which greatly improves the quality and practicality of text generation, and provides users with text output results that are more in line with their expectations.

[0090] It can be understood that the above embodiment processes the first modal file as the image attribute. In other embodiments, the present solution can also process the video attribute. The generative large model processes the attribute of the first modal file to be analyzed and generates a corresponding analytical controllable template, including: If the attributes of the first modal file include video attributes, a video interaction preview box corresponding to the video attributes is generated, and the video interaction preview box includes a video interaction axis.

[0091] This embodiment expands the scope of application of the generative large model controllable text generation method of the present invention, extending from the first modal file processing of image attributes to the processing of video attribute files. By generating a specific parsed controllable template for video attributes and implementing interaction with users to determine controllable information, the method can more comprehensively process different types of multimedia data, laying the foundation for subsequent video content segmentation and controllable text generation, further improving the versatility and practicality of the invention.

[0092] In this embodiment, when the attributes of the first modal file to be analyzed include video attributes, the generative large model generates a video interaction preview box corresponding to the video attributes. This preview box includes a video interaction axis, which can be located below the video interaction preview box. The video interaction axis presents the time dimension of the video in a visual manner, providing an interface basis for users to operate and select video time points.

[0093] The interactive preview frame is adaptively processed based on the specifications in the first modal file attributes so that the interactive preview frame corresponds to the first modal file.

[0094] Based on the specifications in the first modal file attributes, such as the size of the video window, the interactive preview frame is adaptively processed so that the interactive preview frame wraps the video window. This process is similar to the processing of the image interactive preview frame, and the purpose is to make the interactive preview frame fit the video content, ensure that the video display effect seen by the user in the preview frame is accurate and complete, and provide a good visual basis for subsequent time point selection operations.

[0095] After extracting the video interaction axis to generate the corresponding time point selection line, a corresponding analytically controllable template is generated.

[0096] After extracting the video interaction axis to generate the corresponding time point selection line, a corresponding analytical controllable template is generated, including: The video interaction axis takes point 0 as the starting point, determines a time point at a preset time interval, and generates a corresponding time point selection line.

[0097] Extract the video interaction axis, take 0 as the starting point, determine the time point according to the preset time period and generate the corresponding time point selection line. For example, if the preset time period is 1 minute, then a time point will be determined every 1 minute on the video interaction axis and a time point selection line will be generated. These time point selection lines constitute the time segmentation mark of the video. Users can specify the video time period of interest by clicking on these selection lines, and finally form a parsable controllable template.

[0098] The step of interacting with the parsed controllable template to determine the controllable information of the controllable text of the first modal file includes: After determining that the user selects any time point selection line, a second mark added by the user to the time point selection line is received, and a second controllable slot and a second confirmation slot corresponding to the second mark are generated at the parsed controllable template.

[0099] When the system determines that the user has selected any time point selection line, it will receive the second mark added by the user to the time point selection line. The second mark is used to distinguish different video time periods. For example, the user may add marks such as "1" and "2". At the same time, a second controllable slot and a second confirmation slot corresponding to the second mark are generated at the parsing controllable template. The second controllable slot allows the user to enter specific requirements for the time period, such as the number of text words, key description content, etc.; the second confirmation slot is used for the user to confirm the input information.

[0100] After determining that the second confirmation slot is triggered, the minimum time and the maximum time with the same second mark are counted to obtain a time period.

[0101] When the user triggers the second confirmation slot, the system will count the minimum and maximum times corresponding to all time point selection lines with the same second mark to determine a complete time period. For example, if the user adds a "1" mark to two interval time point selection lines, the system will find the earliest and latest times among these marked time points to determine a complete time period.

[0102] The controllable sub-information about the corresponding time period is received based on the second controllable slot, and the controllable information at least includes text requirements of the controllable text.

[0103] The controllable sub-information about the corresponding time period is received based on the second controllable slot. The controllable information at least includes text requirements of the controllable text, such as the word range of the text, the content focus, etc. This information will guide the subsequent text generation of the video content in the time period.

[0104] It is worth emphasizing that after obtaining controllable information, text data can be generated in combination with the video content of the corresponding time period. Taking a 10-second time period as an example, if the video content of this time period shows a dog playing with a football, the generated text data will revolve around this scene. It should be noted that regarding the technology for identifying video content such as "dog playing football", this solution adopts existing mature technology without specific limitations and improvements. The core highlight of this solution is that after completing the video segmentation processing, it can accurately meet the diverse customized text generation needs based on the characteristics of different time periods.

[0105] In the above embodiment, it also includes: If it is determined that the user selects a frame image at any time in the video interaction axis and inputs image attributes, steps A1 to A3 are executed.

[0106] If it is determined that the user has selected a frame of image at any time in the video interaction axis and has entered the image attributes, the system will execute steps A1 to A3 for the previous image attribute processing. That is, an image interaction preview frame corresponding to the frame of image is generated, the preview frame is adaptively processed to fit the image, and then the preview frame specifications are extracted to generate coordinate selection lines, and finally a parsable controllable template for the frame of image is generated, which organically combines video processing with image processing to achieve more flexible content processing.

[0107] The present invention also provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the methods provided by the various embodiments described above.

[0108] Among them, the storage medium can be a computer storage medium or a communication medium. The communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The computer storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, the storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also exist in a communication device as discrete components. The storage medium can be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0109] The present invention also provides a program product, which includes an execution instruction, which is stored in a storage medium. At least one processor of a device can read the execution instruction from the storage medium, and at least one processor executes the execution instruction so that the device implements the methods provided in the above various embodiments.

[0110] In the above-mentioned terminal or server embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A generative large model controllable text generation method, characterized in that: include: The generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding analytical controllable template; interacting with the parsed controllable template to determine controllable information of the controllable text of the first modal file; Based on the controllable information, the first modal file is segmented according to corresponding attributes to obtain a plurality of segmented segments and controllable sub-information of each segmented segment; Content extraction and conversion processing are performed on each segmented segment to generate a corresponding initial text, and the initial text is processed based on the controllable sub-information to obtain a multi-level controllable text of the first modal file.

2. The method according to claim 1, characterized in that The generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding analytical controllable template, including: A1. If the attributes of the first modal file include image attributes, generating an image interactive preview box corresponding to the image attributes; A2. Adaptively process the interactive preview frame based on the specifications in the first modal file attributes, so that the interactive preview frame fits the first modal file; A3. After extracting the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection lines, a corresponding analytical controllable template is generated.

3. The method according to claim 2, characterized in that The adaptive processing of the interactive preview frame based on the specification in the first modal file attribute so that the interactive preview frame is aligned with the first modal file includes: Obtain the number of horizontal pixels and the number of vertical pixels of the first modal file; The interactive preview frame is adaptively processed based on the specifications of the number of horizontal pixels and the number of vertical pixels to make the interactive preview frame fit the boundary of the first modal file.

4. The method according to claim 2, characterized in that: After extracting the corresponding specifications of the interactive preview frame to generate the corresponding coordinate selection line, a corresponding analytical controllable template is generated, including: Determine the target horizontal frame and the target vertical frame in the interactive preview frame, and take the intersection of the target horizontal frame and the target vertical frame as the starting point; Taking the starting point as the initial point, corresponding coordinate lines are established according to the target horizontal frame and the target vertical frame, and the end points of the target horizontal frame and the target vertical frame are the end points; The coordinates of the unit number of pixels are calculated based on the number of pixels between the initial point and the cutoff point, and the pixel points corresponding to the preset coordinate values ​​are selected to add coordinate selection lines to obtain a parsable controllable template.

5. The method according to claim 2, characterized in that: The interacting with the parsed controllable template to determine the controllable information of the controllable text of the first modality file includes: After determining that the user has selected any one of the coordinate selection lines, receiving a first mark added by the user to the coordinate selection line, and generating a first controllable slot and a first confirmation slot corresponding to the first mark at the parsed controllable template; After determining that the first confirmation slot is triggered, counting the first closed area formed by all coordinate selection lines having the same first mark; If it is determined that the user has selected a first closed area, it is used as a pre-divided area, and controllable sub-information about the pre-divided area is received based on the first controllable slot, and the controllable information at least includes text requirements of the controllable text.

6. The method according to claim 5, characterized in that The first modal file is segmented and processed according to corresponding attributes based on the controllable information to obtain a plurality of segmented segments and controllable sub-information of each segmented segment, including: Segmenting the first modal file according to the pre-segmented regions to obtain a plurality of segmented segments; Controllable sub-information of each segmented segment is extracted, wherein the controllable sub-information includes at least one of a text threshold quantity and a text ratio.

7. The method according to claim 1, characterized in that The extracting and converting of content from each segmented segment generates a corresponding initial text, and the processing of the initial text based on the controllable sub-information obtains a multi-level controllable text of the first modal file, including: Extracting content from each segmented segment to obtain first text content and / or image content, converting the image content into a preset second text content, wherein the initial text includes the first text content and the second text content; Based on the text threshold number and text ratio corresponding to the controllable sub-information, the initial text is fused to obtain the hierarchy and controllable sub-text of each segmented segment; The levels and controllable sub-texts of the segmented segments are counted and filled into a preset area of ​​the output template to obtain multi-level controllable texts of the first modal file.

8. The method according to claim 7, characterized in that The initial text is fused based on the text threshold quantity and text ratio corresponding to the controllable sub-information to obtain the level and controllable sub-text of each segmented segment, including: If it is determined that the text threshold number of controllable sub-information is set without setting the text ratio, and the number of controllable sub-texts is less than or equal to the first text threshold number, then save; If the number of controllable subtexts is greater than the first threshold number, the controllable subtexts are input into the text simplification model to be reduced to a number less than or equal to the first threshold number of texts and then saved; A corresponding level is generated according to the number of pixels in each segmented segment, and the level and controllable sub-text of each segmented segment are stored correspondingly.

9. The method according to claim 7, characterized in that: The initial text is fused based on the text threshold quantity and text ratio corresponding to the controllable sub-information to obtain the level and controllable sub-text of each segmented segment, including: If it is determined that the controllable sub-information does not set the text threshold number and the text ratio, the pixel ratio of all the segmented segments is calculated according to the number of pixels in each segmented segment; Extracting the highest ratio value in the pixel point ratio and the corresponding controllable sub-text quantity as the highest quantity, and determining the second threshold quantity of controllable sub-information of other segmented segments based on the highest quantity and the pixel point ratio; Based on the second threshold number and the number of pixels in each segmented segment, the level of each segmented segment and the corresponding storage of the controllable sub-text are obtained.

10. The method according to claim 9, characterized in that The step of obtaining the level of each segmented segment and the corresponding storage of the controllable subtext based on the second threshold number and the number of pixels in each segmented segment includes: If the number of controllable sub-texts of the segmented fragment is less than or equal to the second threshold number of texts, save; If the number of controllable subtexts is greater than the second threshold number, the controllable subtexts are input into the text simplification model to be reduced to a number less than or equal to the second threshold number of texts and then saved; A corresponding level is generated according to the number of pixels in each segmented segment, and the level and controllable sub-text of each segmented segment are stored correspondingly.

11. The method according to claim 2, characterized in that The generative large model processes the attributes of the first modal file to be analyzed and generates a corresponding analytical controllable template, including: If the attributes of the first modal file include video attributes, generating a video interaction preview frame corresponding to the video attributes, wherein the video interaction preview frame includes a video interaction axis; Adaptively process the interactive preview frame based on the specifications in the first modal file attributes, so that the interactive preview frame fits and corresponds to the first modal file; After extracting the video interaction axis to generate the corresponding time point selection line, a corresponding analytically controllable template is generated.

12. The method according to claim 11, characterized in that After extracting the video interaction axis to generate the corresponding time point selection line, a corresponding analytical controllable template is generated, including: The video interaction axis takes point 0 as the starting point, determines a time point at a preset time interval, and generates a corresponding time point selection line.

13. The method according to claim 11, characterized in that The interacting with the parsed controllable template to determine the controllable information of the controllable text of the first modality file includes: After determining that the user selects any time point selection line, receiving a second mark added by the user to the time point selection line, and generating a second controllable slot and a second confirmation slot corresponding to the second mark at the parsed controllable template; After determining that the second confirmation slot is triggered, the minimum time and the maximum time with the same second mark are counted to obtain a time period; The controllable sub-information about the corresponding time period is received based on the first controllable slot, and the controllable information at least includes text requirements of the controllable text.

14. The method according to claim 13, characterized in that Also includes: If it is determined that the user selects a frame image at any time in the video interaction axis and inputs image attributes, steps A1 to A3 are executed.

Images