Method, system and equipment for optimizing text graph model and medium

Abstract

The invention is suitable for the technical field of computers, and provides a text graph model optimization method, which comprises the following steps: inputting text data into a pre-training text graph model to obtain a training image, obtaining text features of the text data based on a text encoder, and obtaining image features of the training image based on an image encoder; establishing an evaluation function based on preset evaluation parameters according to the text features and the image features, and obtaining a preference image of the text data based on the evaluation function; using an evaluation training loss function to calculate and obtain a loss function value based on the preference image; and training the pre-trained text graph model based on the loss function value to obtain a target text graph model. According to the method and the device, the evaluation parameters are set, so that the user can set the corresponding evaluation parameters according to own requirements in the optimization process of the target text graph model, the participation degree of the user is ensured, and the text graph meeting the expectation of the user is generated.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a method, system, device and medium for optimizing a cultural graph model. Background Art

[0002] Wenshengtu technology refers to the AI ​​model generating a corresponding picture for a descriptive text given by the user. In the vast field of digital creativity and artistic creation, the image quality assessment of Wenshengtu technology occupies a pivotal position. It is not only directly related to the artistic value and viewing experience of the generated image, but also a key indicator for measuring technological progress and practical application effects. High-quality Wenshengtu works can accurately capture the essence of the text description and present the visual details delicately, so that the audience can feel the strong emotional resonance and visual impact at first sight. Through a scientific and objective quality assessment system, developers and artists can continuously optimize the algorithm model, improve the realism, clarity and creativity of the image, and promote the development of Wenshengtu technology to a higher level. In addition, for commercial applications, high-quality Wenshengtu output can significantly enhance the brand image, enrich the content creation form, and bring unprecedented innovation opportunities to multiple industries such as advertising, games, and film and television. Therefore, establishing a sound Wenshengtu image quality assessment mechanism is of immeasurable value in promoting technological iteration, stimulating creative inspiration, and broadening application scenarios.

[0003] The existing text graph model evaluates the generated images by simply comparing the similarity between the generated images and the text, while ignoring the aesthetics and creativity of the images. For example, the Fréchet distance is used to evaluate the similarity between images and texts. Although it can reflect the statistical characteristics of the images, it cannot reflect the user's subjective feelings and lacks the user's actual preference data. In addition, the existing text graph evaluation methods often only focus on one aspect of the image (such as diversity or quality), ignoring the comprehensive evaluation of multiple dimensions, and usually perform static evaluation on the generated images after the text graph model is trained, which cannot reflect the changes in user preferences in real time and realize the dynamic improvement needs of the text graph model. Summary of the invention

[0004] The embodiments of the present application provide a method, system, device and medium for optimizing a cultural graph model, which can solve one of the above-mentioned problems in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for optimizing a cultural graph model, including:

[0006] Input text data into a pre-trained text-image model to obtain a training image, obtain text features of the text data based on a text encoder, and obtain image features of the training image based on the image encoder;

[0007] According to the text features and the image features, an evaluation function is established based on preset evaluation parameters, wherein the evaluation parameters include a similarity expectation value, a creativity expectation value, and evaluation strictness;

[0008] Based on the evaluation function, obtaining a preference image of the text data;

[0009] Using an evaluation training loss function, calculating a loss function value based on the preferred image;

[0010] Based on the loss function value, the pre-trained text graph model is trained to obtain a target text graph model.

[0011] Furthermore, the similarity expectation value represents the user's expectation value of the similarity of the output image of the target text graph model, the creativity expectation value represents the user's expectation value of the creativity of the output image of the target text graph model, and the evaluation strictness represents the evaluation strictness of the evaluation function on the training image;

[0012] The specific calculation formula of the evaluation function is as follows:

[0013]

[0014] Among them, x represents text data, y represents training images, E(x) represents text features, E(y) represents image features, α1 represents the expected value of similarity, α2 represents the expected value of creativity, and τ represents the strictness of evaluation.

[0015] Furthermore, obtaining the preference image of the text data based on the evaluation function includes:

[0016] Based on the evaluation function, the pre-trained viz-image model is trained to obtain a preferred image generation model;

[0017] Inputting text data into the preference image generation model to generate a plurality of preference prediction images;

[0018] Inputting the plurality of preference prediction images into a prediction value calculation function respectively to obtain a plurality of preference prediction values ​​corresponding one to one with the preference prediction images;

[0019] The preference image of the text data is determined by comparing the numerical values ​​of a plurality of preference prediction values.

[0020] Furthermore, the calculation formula of the predicted value calculation function is as follows:

[0021]

[0022] in, represents the preference prediction value of the i-th preference prediction image, n represents the number of preference prediction images, x represents text data, y i and j Both indicate preference for predicted images.

[0023] Furthermore, the use of the evaluation training loss function to calculate the loss function value based on the preferred image includes:

[0024] Based on a preset corresponding rule, obtaining a true value of the preferred image;

[0025] Inputting the true value into the evaluation training loss function to obtain the loss function value of the preferred image;

[0026] The calculation formula of the evaluation training loss function is as follows:

[0027]

[0028] Among them, β1 and β2 are hyperparameters, and 0<β1<1, 0<β2<1, p i represents the true value of the i-th preference prediction image, represents the preference prediction value of the i-th preference prediction image.

[0029] Furthermore, after obtaining the target document graph model, the following steps are included:

[0030] Presetting the output quantity of the target text graph model, inputting text data into the target text graph model, and generating a plurality of basic text graphs, wherein the quantity of the basic text graphs is greater than the output quantity;

[0031] Inputting a plurality of the basic cultural graphs into the evaluation function to generate an evaluation score corresponding to each of the basic cultural graphs;

[0032] The plurality of evaluation scores are sorted in descending order, so that the target text graph model outputs a plurality of preferred text graphs in the order of arrangement, wherein the number of the preferred text graphs corresponds to the output number.

[0033] Furthermore, after the plurality of evaluation scores are sorted in descending order so that the target text graph model outputs a plurality of preferred text graphs in the order of arrangement, the method further comprises:

[0034] Based on the plurality of the preferred context diagrams, modifying the preset evaluation parameters in the evaluation function;

[0035] Based on the modified evaluation parameters, the target text graph model is updated so that the target text graph model regenerates a plurality of preferred text graphs.

[0036] In a second aspect, an embodiment of the present application provides a cultural graph model optimization system, including:

[0037] The first processing module is used for inputting text data into a pre-trained text-image model to obtain a training image, obtaining text features of the text data based on a text encoder, and obtaining image features of the training image based on the image encoder;

[0038] A second processing module is used to establish an evaluation function according to the text features and the image features based on preset evaluation parameters, wherein the evaluation parameters include a similarity expectation value, a creativity expectation value, and evaluation strictness;

[0039] A third processing module: used for obtaining a preference image of the text data based on the evaluation function;

[0040] A fourth processing module: used for adopting an evaluation training loss function to calculate a loss function value based on the preferred image;

[0041] The fifth processing module is used to train the pre-trained text graph model based on the loss function value to obtain a target text graph model.

[0042] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned Wensheng graph model optimization method when executing the computer program.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including a computer-readable storage medium storing a computer program, and the computer program implements the above-mentioned Wensheng graph model optimization method when executed by a processor.

[0044] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0045] The present application discloses a text graph model optimization method, which pre-trains the text graph model, introduces an evaluation function, generates a loss function value of a preferred image of text data, and then trains the pre-trained text graph model through the loss function value to obtain a target text graph model after optimization training, thereby continuously improving the generation capability and personalization level of the target text graph model through multi-layer feedback and loop training; in addition, by setting evaluation parameters, the user can set the corresponding evaluation parameters of the target text graph model according to his own needs during the optimization training process, thereby ensuring the user's participation; in addition, by introducing similarity expectation value and creativity expectation value, the target text graph model can score the generated text graph from two dimensions of creativity and similarity, and the user can continuously adjust the similarity expectation value and creativity expectation value according to the scoring result of the generated text graph and the image content, so that the target text graph model can generate an output image that meets the user's expectations. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 It is a flow chart of a method for optimizing a cultural graph model provided by an embodiment of the present invention;

[0048] Figure 2 It is a structural schematic diagram of a cultural graph model optimization system provided by an embodiment of the present invention;

[0049] Figure 3 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0051] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0052] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0054] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0056] See also Figure 1 As shown, the present invention is a method for optimizing a cultural graph model, comprising the following steps:

[0057] S100, inputting text data into a pre-trained text-image model to obtain a training image, obtaining text features of the text data based on a text encoder, and obtaining image features of the training image based on the image encoder;

[0058] A text graph model optimization method of the present application is trained and optimized on the basis of an existing text graph model. By conducting targeted training on the existing text graph model, it can better understand the intention of the text description, thereby generating a more accurate, delicate and text description-compliant image, thereby enhancing user experience and satisfaction. Therefore, the pre-trained text graph model in this embodiment is pre-trained to generate a training image corresponding to the input text when text data is input, which is used for subsequent training and optimization of the target text graph model.

[0059] In this embodiment, before inputting the text data into the pre-trained text-generated graph model to obtain the training image, the text data is also pre-processed to obtain the keywords in the text data, thereby improving the pre-trained text-generated graph model's ability to understand and parse the content of the text data, ensuring that the generated training image can closely fit the text content. Specifically, the pre-processing process usually includes text cleaning of the text data, removing irrelevant symbols, stop words and noise data to make the text purer. In addition, it also includes the use of natural language processing technology, such as word segmentation, part-of-speech tagging, etc., to deeply analyze the text structure, and at the same time, through keyword extraction algorithms, such as TF-IDF, TextRank or models based on deep learning, accurately identify and extract keywords in the text data.

[0060] In this embodiment, keywords include image object nouns, image style modifiers, image feature keywords, etc. For example, for a text data "a cat with flowers on its head." After the text data is preprocessed, the extracted keywords are "one", "on the head", "wearing flowers" and "cat" respectively.

[0061] In this embodiment, for text data, a text encoder such as BERT (Bidirectional Encoder Representations from Transformers) or GPT (Generative Pre-trained Transformer) model is used. The above model can capture rich semantic information and contextual relationships of language by learning on massive text data. After the pre-processed text data is processed by the above text encoder, it will be converted into a vector representation in a high-dimensional vector space, that is, a text feature. The above text features not only contain vocabulary-level information, but also contain the semantic connotation of sentences and even paragraph levels, providing a powerful input for subsequent tasks. In addition, for training images, convolutional neural networks such as ResNet, VGG or more advanced image encoders such as Vision Transformer (ViT) are used for feature extraction. The above image encoder can automatically learn hierarchical feature representations from images through stacked convolutional layers or self-attention mechanisms, from low-level edges and texture information to high-level object shapes, spatial layouts, etc. The training image is processed layer by layer by the encoder and finally converted into an image feature vector that can describe the image content, structure and even emotional tendency, providing a basis for the training of the subsequent target text image model.

[0062] S200, establishing an evaluation function according to the text features and the image features based on preset evaluation parameters, wherein the evaluation parameters include a similarity expectation value, a creativity expectation value, and evaluation strictness;

[0063] In this embodiment, the evaluation function can score the training images generated by the pre-trained text graph model, thereby realizing iterative training of the pre-trained text graph model to generate a target text graph model. In addition, the evaluation function can also score the output image of the target text graph model, thereby making the output image of the target text graph model more in line with user expectations.

[0064] In some embodiments, the step S200 includes:

[0065] The expected similarity value represents the user's expected value of the similarity of the output image of the target text graph model, the expected creativity value represents the user's expected value of the creativity of the output image of the target text graph model, and the evaluation strictness represents the evaluation strictness of the evaluation function on the output image of the target text graph model;

[0066] The specific calculation formula of the evaluation function is as follows:

[0067]

[0068] Among them, x represents text data, y represents training images, E(x) represents text features, E(y) represents image features, α1 represents the expected value of similarity, α2 represents the expected value of creativity, and τ represents the strictness of evaluation.

[0069] In this embodiment, the value of the evaluation parameter is set by the user, and the user adjusts the value of the evaluation parameter according to the output image requirements and personal preferences, so that the evaluation function can be applied to different evaluation standards and goals. Based on this, the pre-trained text graph model is trained, and the output image of the target text graph model is reflected by the training image, so that the target text graph model can generate an output image that meets the user's expectations.

[0070] In this embodiment, the evaluation parameters include similarity expectation value, creativity expectation value and evaluation strictness. The similarity expectation value is used to evaluate the degree of match between the output image and the real image. In the image description generation task, the user can balance the degree of match and novelty between the text data and the output image content by adjusting the similarity expectation value. In addition, the creativity expectation value is used to evaluate the degree of creativity between the output image and the real image. For example, in a text data, a cat is required to be output. At this time, the output image of the target text image model is an ordinary cat or a robot cat. The degree of creativity of the two is different. It can be understood that the creativity expectation value is usually used in creative advertising design. By increasing the value of the creativity expectation value, a more innovative design can be generated.

[0071] In this embodiment, by setting the expected similarity value and the expected creativity value, the matching degree and creativity between the output image content generated by the target text-based graph model and the real image content can be balanced, thereby generating a high-quality image that meets the user's expectations.

[0072] In this embodiment, by setting the evaluation strictness, it reflects the strictness of the evaluation function when evaluating the output image, and it is very important to ensure the accuracy and fairness of the evaluation result. Specifically, the level of evaluation strictness determines the standards and thresholds adopted by the evaluation function when measuring the consistency between the output image and the text description, image quality, creativity, etc. When the value of the evaluation strictness is smaller, it means that the strictness of the evaluation of the output image is higher, and the evaluation function will examine every detail of the training image more carefully, not only requiring the image content to be highly matched with the text description, but also putting forward higher requirements on the clarity, color reproduction, composition creativity, etc. of the image, thereby helping to screen out truly high-quality and compliant output images. On the contrary, when the value of the evaluation strictness is larger, it means that the strictness of the evaluation of the output image is lower, and the evaluation function will be relatively loose in evaluating the output image. It may pay more attention to the basic consistency between the image and the text description, while the requirements for image quality, creativity, etc. will be reduced. In practical applications, the setting of the evaluation strictness needs to be weighed and adjusted according to the needs of the specific task and the output image target. By setting the evaluation strictness reasonably, we can ensure that the evaluation function can accurately reflect the quality and value of the output image, while avoiding the adverse effects of overly harsh or loose evaluation criteria on the evaluation results.

[0073] S300, obtaining a preference image of the text data based on the evaluation function;

[0074] In some embodiments, the step S300 includes:

[0075] Based on the evaluation function, the pre-trained viz-image model is trained to obtain a preferred image generation model;

[0076] Inputting text data into the preference image generation model to generate a plurality of preference prediction images;

[0077] Inputting the plurality of preference prediction images into a prediction value calculation function respectively to obtain a plurality of preference prediction values ​​corresponding one to one with the preference prediction images;

[0078] The preference image of the text data is determined by comparing the numerical values ​​of a plurality of preference prediction values.

[0079] In this embodiment, the pre-trained text image model is combined with the evaluation function for further training and optimization to generate a preference image generation model. Specifically, historical user preference data is obtained, and the pre-trained text image model is used as a starting point. In combination with the refined evaluation function and user preference data, training is performed. During the training process, the pre-trained text image model takes text data and user preference information as input to generate images that match them. At the same time, the evaluation function instantly scores the generated images, and uses the scoring results as feedback signals to guide the model to continuously optimize its internal parameters and generation strategies. Through multiple rounds of iterative training, a preference image generation model is generated, and arbitrary text data is input into the preference image generation model, and the preference image generation model generates multiple preference prediction images.

[0080] In this embodiment, the preference prediction values ​​of multiple preference prediction images can be obtained through the prediction value calculation function, and the preference images can be determined based on the preference prediction values. Through the calculation of the preference prediction values, it can further promote the generation of images that meet the user's unique aesthetics and preferences, which are used for training the target text image model. Through multi-layer feedback and loop training, the generation ability and personalization level of the target text image model are continuously improved.

[0081] In some embodiments, the calculation formula of the prediction value calculation function is as follows:

[0082]

[0083] in, represents the preference prediction value of the i-th preference prediction image, n represents the number of preference prediction images, x represents text data, y i and j Both indicate preference for predicted images.

[0084] In this embodiment, when the preference image generation model generates n preference prediction images, the preference prediction value of each preference prediction image can be obtained through the above-mentioned prediction value calculation function, and then the preference image is determined by comparing the preference prediction values ​​of each preference prediction image for the subsequent training of the target text image model.

[0085] S400, using the evaluation training loss function to calculate and obtain a loss function value based on the preferred image;

[0086] In some embodiments, the step S400 includes:

[0087] Based on a preset corresponding rule, obtaining a true value of the preferred image;

[0088] Inputting the true value into the evaluation training loss function to obtain the loss function value of the preferred image;

[0089] The calculation formula of the evaluation training loss function is as follows:

[0090]

[0091] Among them, β1 and β2 are hyperparameters, and 0<β1<1, 0<β2<1, p i represents the true value of the i-th preference prediction image, represents the preference prediction value of the i-th preference prediction image.

[0092] In this embodiment, the true value of the preference image is a result vector. For multiple preference prediction images generated by the preference image generation model, each one corresponds to a true value. When the preference image is determined by the preference prediction value, the true value of the preference image is determined synchronously.

[0093] Specifically, in a preferred embodiment, two preference prediction images are generated by the preference image generation model, namely, the first preference prediction image y1 and the second preference prediction image y2, and the preset corresponding rules are as follows: when the preference prediction value of the first preference prediction image is greater than the prediction value of the second preference prediction image, the preference image is determined to be the first preference prediction image y1, and its corresponding true value is [1,0]; when the preference prediction value of the second preference prediction image is greater than the prediction value of the first preference prediction image, the preference image is determined to be the second preference prediction image y2, and its corresponding true value is [0,1]; and when the calculated preference prediction values ​​of the two preference prediction images are equal, it means that the preference degrees of the n preference prediction images generated by the preference image generation model are the same, and the corresponding true value is [0.5,0.5]. Finally, the corresponding true value is input into the evaluation training loss function to obtain the loss function value of the preference image.

[0094] S500: Based on the loss function value, the pre-trained text graph model is trained to obtain a target text graph model.

[0095] In this embodiment, the true value and the predicted value of the preference image are input into the evaluation training loss function, and the loss function value of the pre-trained text graph model can be obtained. Based on the loss function value, the pre-trained text graph model is trained to obtain the target text graph model. Specifically, the loss function value calculated by the above steps S100-S400 is used as a feedback signal to guide the pre-trained text graph model to continuously optimize its internal parameters and generation strategy. Through multiple rounds of iterative training, the target text graph model is generated. Arbitrary text data is input into the target text graph model to generate multiple preferred text graphs for users to choose from.

[0096] In some embodiments, after obtaining the target text graph model, the method further includes:

[0097] Presetting the output quantity of the target text graph model, inputting text data into the target text graph model, and generating a plurality of basic text graphs, wherein the quantity of the basic text graphs is greater than the output quantity;

[0098] Inputting a plurality of the basic cultural graphs into the evaluation function to generate an evaluation score corresponding to each of the basic cultural graphs;

[0099] The plurality of evaluation scores are sorted in descending order, so that the target text graph model outputs a plurality of preferred text graphs in the order of arrangement, wherein the number of the preferred text graphs corresponds to the output number.

[0100] In this embodiment, after any text data is input into the target text graph model, the target text graph model first generates more basic text graphs than the preset output quantity, and then scores each basic text graph through an evaluation function to generate an evaluation score, and obtains a basic text graph table by sorting the evaluation scores. At this time, the target text graph model selects the basic text graphs arranged in front and corresponding to the preset output quantity from the basic text graph table as the preferred text graph outputs. At this time, multiple preferred text graphs are the output images of the target text graph model, providing users with the opportunity to choose.

[0101] Specifically, in one embodiment, the user presets the output number as 4, then the target text map model generates 8 or more basic text maps, and scores each basic text map through an evaluation function, and then sorts them according to the evaluation scores, and finally selects the top 4 basic text maps as the preferred text map outputs and submits them to the user for selection.

[0102] In some embodiments, after the plurality of evaluation scores are sorted in descending order so that the target text graph model outputs a plurality of preferred text graphs in the order of arrangement, the method further comprises:

[0103] Based on the plurality of the preferred context diagrams, modifying the preset evaluation parameters in the evaluation function;

[0104] Based on the modified evaluation parameters, the target text graph model is updated so that the target text graph model regenerates a plurality of preferred text graphs.

[0105] In this embodiment, the preferred text image will synchronously generate its evaluation score in the evaluation function when it is output. The user determines whether the multiple generated preferred text images meet the expected expectations by judging the corresponding evaluation scores and the image content of the preferred text image. Based on this, if the user believes that the output image generated by the target text image model does not meet the expectations, the preferred image obtained based on the evaluation function can be adjusted by modifying the similarity expected value and the creativity expected value in the evaluation parameters, and then the loss function value calculated by the evaluation training loss function can be modified, so as to update the target text image model and finally generate a preferred text image that better meets the user's expectations.

[0106] See also Figure 2 As shown, the present invention also provides a cultural graph model optimization system, the system comprising:

[0107] The first processing module 201 is used to input text data into a pre-trained text-image model to obtain a training image, obtain text features of the text data based on a text encoder, and obtain image features of the training image based on the image encoder;

[0108] The second processing module 202 is used to establish an evaluation function according to the text features and the image features based on preset evaluation parameters, wherein the evaluation parameters include a similarity expectation value, a creativity expectation value, and evaluation strictness;

[0109] The third processing module 203 is used to obtain the preferred image of the text data based on the evaluation function;

[0110] The fourth processing module 204 is used to adopt the evaluation training loss function to calculate the loss function value based on the preferred image;

[0111] The fifth processing module 205 is used to train the pre-trained text graph model based on the loss function value to obtain a target text graph model.

[0112] It is understandable that if Figure 1 The contents of the embodiment of the raw graph model optimization method shown in the text are all applicable to the embodiment of the raw graph model optimization system in this text. The functions specifically implemented by the embodiment of the raw graph model optimization system in this text are similar to those in the embodiment of the raw graph model optimization method shown in the text. Figure 1 The embodiment of the Wensheng graph model optimization method shown in FIG. 1 is the same as that shown in FIG. 1 , and the beneficial effects achieved are the same as those of FIG. Figure 1 The beneficial effects achieved by the embodiment of the Wensheng graph model optimization method shown are also the same.

[0113] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0114] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0115] See also Figure 3 As shown, an embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, the Wensheng graph model optimization method described in any one of the above methods is implemented.

[0116] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0117] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0118] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit of the computer device 3 and an external storage device. The memory 302 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0119] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for optimizing the Wensheng graph model as described in any one of the above methods is implemented.

[0120] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0121] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for optimizing a cultural graph model, characterized in that: include: Input text data into a pre-trained text-image model to obtain a training image, obtain text features of the text data based on a text encoder, and obtain image features of the training image based on the image encoder; According to the text features and the image features, an evaluation function is established based on preset evaluation parameters, wherein the evaluation parameters include a similarity expectation value, a creativity expectation value, and evaluation strictness; Based on the evaluation function, obtaining a preference image of the text data; Using an evaluation training loss function, calculating a loss function value based on the preferred image; Based on the loss function value, the pre-trained text graph model is trained to obtain a target text graph model.

2. The method according to claim 1, characterized in that The similarity expectation value represents the user's expectation value of the similarity of the output image of the target text graph model, the creativity expectation value represents the user's expectation value of the creativity of the output image of the target text graph model, and the evaluation strictness represents the evaluation strictness of the evaluation function on the training image; The specific calculation formula of the evaluation function is as follows: Among them, x represents text data, y represents training images, E(x) represents text features, E(y) represents image features, α1 represents the expected value of similarity, α2 represents the expected value of creativity, and τ represents the strictness of evaluation.

3. The method according to claim 2, characterized in that The step of obtaining the preference image of the text data based on the evaluation function includes: Based on the evaluation function, the pre-trained viz-image model is trained to obtain a preference image generation model; Inputting text data into the preference image generation model to generate a plurality of preference prediction images; Inputting the plurality of preference prediction images into a prediction value calculation function respectively to obtain a plurality of preference prediction values ​​corresponding one to one with the preference prediction images; The preference image of the text data is determined by comparing the numerical values ​​of a plurality of preference prediction values.

4. The method according to claim 3, characterized in that The calculation formula of the predicted value calculation function is as follows: in, represents the preference prediction value of the i-th preference prediction image, n represents the number of preference prediction images, x represents text data, y i and j Both indicate preference for predicted images.

5. The method according to claim 4, characterized in that The adopting of the evaluation training loss function to calculate the loss function value based on the preferred image includes: Based on a preset corresponding rule, obtaining a true value of the preferred image; Inputting the true value into the evaluation training loss function to obtain the loss function value of the preferred image; The calculation formula of the evaluation training loss function is as follows: Among them, β1 and β2 are hyperparameters, and 0<β1<1, 0<β2<1, p i represents the true value of the i-th preference prediction image, represents the preference prediction value of the i-th preference prediction image.

6. The method according to claim 1, characterized in that After obtaining the target document graph model, including: Presetting the output quantity of the target text graph model, inputting text data into the target text graph model, and generating a plurality of basic text graphs, wherein the quantity of the basic text graphs is greater than the output quantity; Inputting a plurality of the basic cultural graphs into the evaluation function to generate an evaluation score corresponding to each of the basic cultural graphs; The plurality of evaluation scores are sorted in descending order, so that the target text graph model outputs a plurality of preferred text graphs in the order of arrangement, wherein the number of the preferred text graphs corresponds to the output number.

7. The method according to claim 6, characterized in that After the plurality of evaluation scores are sorted in descending order so that the target text graph model outputs a plurality of preferred text graphs in the order of arrangement, the method further comprises: Based on the plurality of the preferred context diagrams, modifying the preset evaluation parameters in the evaluation function; Based on the modified evaluation parameters, the target text graph model is updated so that the target text graph model regenerates a plurality of preferred text graphs.

8. A cultural graph model optimization system, characterized in that: include: The first processing module is used for inputting text data into a pre-trained text-image model to obtain a training image, obtaining text features of the text data based on a text encoder, and obtaining image features of the training image based on the image encoder; A second processing module is used to establish an evaluation function according to the text features and the image features based on preset evaluation parameters, wherein the evaluation parameters include an expected value of similarity, an expected value of creativity, and evaluation strictness; A third processing module: used for obtaining a preference image of the text data based on the evaluation function; A fourth processing module: used for adopting an evaluation training loss function to calculate a loss function value based on the preferred image; The fifth processing module is used to train the pre-trained text graph model based on the loss function value to obtain a target text graph model.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images