Model training methods, information generation methods, devices, equipment and media

Abstract

This disclosure presents embodiments of a model training method, an information generation method, an apparatus, a device, and a medium. One specific implementation of the method includes: acquiring a training image dataset; for each training image data, performing a sample pair generation step: determining the dynamic contextual similarity between each remaining image data and the training image data; selecting image data whose corresponding dynamic contextual similarity with the training image data meets a preset similarity condition as target image data; forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with the training image data and each remaining image data, obtaining a negative sample image data pair group; and training an initial image-to-text semantic conversion model to obtain an image-to-text semantic conversion model. This implementation is related to artificial intelligence; utilizing the image-to-text semantic conversion model, images can be accurately converted into corresponding textual semantic information.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, specifically to model training methods, information generation methods, apparatus, devices, and media. Background Technology

[0002] Currently, image-to-text semantic translation models have been widely applied in numerous downstream tasks of computer vision and natural language processing. The typical approach to training image-to-text semantic translation models is to annotate the semantics of both images and text, enabling subsequent comparative learning training of the initial model.

[0003] However, the inventors discovered that when using the above method to train the model, the following technical problems often occur:

[0004] Existing image-to-text semantic conversion models often treat two images with inconsistent content events as positive sample pairs. This results in the learned sample features being unable to distinguish subtle differences between images, making the trained image-to-text semantic conversion models less accurate in performing text semantic conversion.

[0005] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of this disclosure provide model training methods, information generation methods, apparatuses, devices, and media to address the technical problems mentioned in the background section above.

[0008] In a first aspect, some embodiments of this disclosure provide a model training method, including: acquiring a training image dataset; for each training image data in the training image dataset, performing a sample pair generation step: using a pre-trained dynamic context similarity generation model to determine the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data, wherein the remaining image dataset is the image dataset in the training image dataset after removing the training image data; selecting image data from the remaining image dataset whose corresponding dynamic context similarity with the training image data meets a preset similarity condition as target image data; forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with each remaining image data in the training image data and the subset of remaining image data, to obtain a negative sample image data pair set; and training an initial image-to-text semantic conversion model based on the obtained positive sample image data pair set and negative sample image data pair set, to obtain an image-to-text semantic conversion model.

[0009] Optionally, the aforementioned dynamic context similarity generation model includes: an associated image representation vector generation model; and the determination of the dynamic context similarity between each remaining image data in the remaining image dataset and the aforementioned training image data using the pre-trained dynamic context similarity generation model includes: vectorizing the aforementioned training image data and the aforementioned remaining image data to obtain training image vectors and remaining image vectors; inputting the aforementioned training image vectors into the aforementioned associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector; inputting the aforementioned remaining image vectors into the aforementioned associated image representation vector generation model to output at least one second historical associated image vector and at least one second future associated image vector; and generating dynamic context similarity based on the aforementioned at least one first historical associated image vector, the aforementioned at least one first future associated image vector, the aforementioned at least one second historical associated image vector, and the aforementioned at least one second future associated image vector.

[0010] Optionally, the aforementioned associated image representation vector generation model includes: a historical associated image representation vector generation model and a future associated image representation vector generation model, wherein the semantic association learning parameters of the historical associated image generation model and the future associated image generation model are different; and the aforementioned inputting the training image vector into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector includes: inputting the training image vector into the historical associated image representation vector generation model to generate the at least one first historical associated image vector; and inputting the training image vector into the future associated image representation vector generation model to generate the at least one first future associated image vector.

[0011] Optionally, generating dynamic contextual similarity based on at least one first historical associated image vector, at least one first future associated image vector, at least one second historical associated image vector, and at least one second future associated image vector includes: performing vector weighted summation on the at least one first historical associated image vector to obtain a first weighted summation vector; performing vector weighted summation on the at least one second historical associated image vector to obtain a second weighted summation vector; performing vector weighted summation on the at least one first future associated image vector to obtain a third weighted summation vector; and performing vector weighted summation on the at least one second future associated image vector to obtain a fourth weighted summation vector; and generating the dynamic contextual similarity based on the first weighted summation vector, the second weighted summation vector, the third weighted summation vector, and the fourth weighted summation vector.

[0012] Optionally, the aforementioned associated image representation vector generation model is trained through the following steps: obtaining an image data set, wherein the image data set includes: current image data, at least one historical semantically associated image data for the current image data, and at least one future semantically associated image data for the current image data; based on the current image data, at least one historical semantically associated image data, and at least one future semantically associated image data included in each image data set, performing data pairing on the image data in the image data set to generate a positive image data set and a negative image data set; and based on the positive image data set and the negative image data set, performing parameter learning on at least one initial semantic association learning parameter in the aforementioned initial associated image representation vector generation model to obtain an associated image generation model, wherein the aforementioned associated image representation vector generation model includes: at least one learned semantic association learning parameter.

[0013] Optionally, the aforementioned associated image representation vector generation model includes: at least one learned semantic association learning parameter; and the input of the aforementioned training image vector into the aforementioned associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector includes: multiplying the aforementioned training image vector with at least one parameter transpose matrix corresponding to the aforementioned at least one first semantic association learning parameter to obtain at least one first multiplication result, which serves as the aforementioned at least one first historical associated image vector; and multiplying the aforementioned training image vector with at least one parameter matrix corresponding to the aforementioned at least one second semantic association learning parameter to obtain at least one second multiplication result, which serves as the aforementioned at least one first future associated image vector.

[0014] Secondly, some embodiments of this disclosure provide a model training apparatus, comprising: a first acquisition unit configured to acquire a training image dataset; an execution unit configured to perform a sample pair generation step for each training image data in the training image dataset: using a pre-trained dynamic context similarity generation model to determine the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data, wherein the remaining image dataset is the image dataset after removing the training image data from the training image dataset; selecting image data whose corresponding dynamic context similarity with the training image data meets a preset similarity condition from the remaining image dataset as target image data; forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with each remaining image data in the subset of the training image data and the remaining image data, to obtain a negative sample image data pair set; and a training unit configured to train an initial image-to-text semantic conversion model based on the obtained positive sample image data pair set and negative sample image data pair set, to obtain an image-to-text semantic conversion model.

[0015] Optionally, the dynamic context similarity generation model includes: an associated image representation vector generation model; and the execution unit can be configured to: vectorize the training image data and the remaining image data respectively to obtain training image vectors and remaining image vectors; input the training image vectors into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector; input the remaining image vectors into the associated image representation vector generation model to output at least one second historical associated image vector and at least one second future associated image vector; and generate dynamic context similarity based on the at least one first historical associated image vector, the at least one first future associated image vector, the at least one second historical associated image vector, and the at least one second future associated image vector.

[0016] Optionally, the aforementioned associated image representation vector generation model includes: a historical associated image representation vector generation model and a future associated image representation vector generation model, wherein the semantic association learning parameters of the historical associated image generation model and the future associated image generation model are different; and the execution unit can be configured to: input the aforementioned training image vector into the aforementioned historical associated image representation vector generation model to generate the aforementioned at least one first historical associated image vector; and input the aforementioned training image vector into the aforementioned future associated image representation vector generation model to generate the aforementioned at least one first future associated image vector.

[0017] Optionally, the execution unit can be configured to: perform vector weighted summation on the at least one first historical associated image vector to obtain a first weighted summation vector; perform vector weighted summation on the at least one second historical associated image vector to obtain a second weighted summation vector; perform vector weighted summation on the at least one first future associated image vector to obtain a third weighted summation vector; and perform vector weighted summation on the at least one second future associated image vector to obtain a fourth weighted summation vector; and generate the dynamic context similarity based on the first weighted summation vector, the second weighted summation vector, the third weighted summation vector, and the fourth weighted summation vector.

[0018] Optionally, the aforementioned associated image representation vector generation model is trained through the following steps: obtaining an image data set, wherein the image data set includes: current image data, at least one historical semantically associated image data for the current image data, and at least one future semantically associated image data for the current image data; based on the current image data, at least one historical semantically associated image data, and at least one future semantically associated image data included in each image data set, performing data pairing on the image data in the image data set to generate a positive image data set and a negative image data set; and based on the positive image data set and the negative image data set, performing parameter learning on at least one initial semantic association learning parameter in the aforementioned initial associated image representation vector generation model to obtain an associated image generation model, wherein the aforementioned associated image representation vector generation model includes: at least one learned semantic association learning parameter.

[0019] Optionally, the aforementioned associated image representation vector generation model includes: at least one learned semantic association learning parameter; and the execution unit can be configured to: multiply the aforementioned training image vector with at least one parameter transpose matrix corresponding to the aforementioned at least one first semantic association learning parameter to obtain at least one first multiplication result as the aforementioned at least one first historical associated image vector; and multiply the aforementioned training image vector with at least one parameter matrix corresponding to the aforementioned at least one second semantic association learning parameter to obtain at least one second multiplication result as the aforementioned at least one first future associated image vector.

[0020] Thirdly, some embodiments of this disclosure provide an information generation method, including: acquiring a target image; inputting the target image into a pre-trained image-to-text semantic conversion model to output text semantic information, wherein the image-to-text semantic conversion model is generated by the method described in the first aspect of this disclosure.

[0021] Fourthly, some embodiments of this disclosure provide an information generation apparatus, including: a second acquisition unit configured to acquire a target image; and an input unit configured to input the target image into a pre-trained image-to-text semantic conversion model to output text semantic information, wherein the image-to-text semantic conversion model is generated by the method described in the first aspect of this disclosure.

[0022] Fifthly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any of the implementations of the first and third aspects.

[0023] Sixthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any of the implementations of the first and third aspects.

[0024] In a seventh aspect, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0025] The various embodiments of this disclosure have the following beneficial effects: Through the model training methods of some embodiments of this disclosure, image-to-text semantic conversion models can accurately convert images into corresponding text semantic information. Specifically, the reason for the inaccuracy of related text semantic information conversion is that existing image-to-text semantic conversion models often treat two images with inconsistent image content events as positive sample pairs, resulting in the learned sample features being unable to distinguish subtle differences between images, making the trained image-to-text semantic conversion model less accurate in text semantic conversion. Based on this, the model training method of some embodiments of this disclosure first obtains a training image dataset for the generation of subsequent sample image data pairs. Then, for each training image data in the training image dataset, a sample pair generation step is performed: First, using a pre-trained dynamic context similarity generation model, the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data can be accurately determined. The remaining image dataset is the image dataset after removing the training image data from the training image dataset. Here, by determining the dynamic contextual similarity between the remaining image data and the training image data, we can not only compare the feature similarity between the intrinsic features of the remaining image data and the intrinsic features of the training image data, but also indirectly compare the similarity between the contextual features corresponding to the remaining image data and the contextual features corresponding to the training image data. This subtle comparison of the similarity of various aspects of the features between the remaining image data and the training image data makes the subsequently generated sample image data pairs more accurate. The second step involves selecting image data from the remaining image dataset that meets the preset similarity condition for dynamic contextual similarity with the training image data, as target image data, for use in the generation of subsequent sample image data pairs. Here, by selecting target image data, we can effectively select the remaining image data that is most similar to the training image data in various aspects (including dynamic contextual features). Thus, accurate positive sample image data pairs can be generated subsequently. The third step involves forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with each remaining image data in the subset of the training image data and the remaining image data, resulting in a negative sample image data pair group. Here, based on dynamic contextual similarity, image data with the most similar detailed features can be paired, making the generated positive and negative sample image data pairs more accurate. Finally, based on the obtained sets of positive and negative sample image data pairs, the initial image-to-text semantic conversion model is trained to obtain the image-to-text semantic conversion model.Here, using precise sets of positive and negative image data pairs for model training can indirectly improve the performance of the image-to-text semantic conversion model, enabling it to accurately convert images into corresponding textual semantic information. In summary, the positive and negative image data pairs generated based on dynamic contextual similarity make subsequent image-to-text semantic conversion model training more effective and the conversion of textual semantic information more accurate. Attached Figure Description

[0026] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0027] Figure 1 This is a schematic diagram illustrating an application scenario of a model training method according to some embodiments of the present disclosure;

[0028] Figure 2 This is a flowchart of some embodiments of the model training method according to this disclosure;

[0029] Figure 3 These are flowcharts of other embodiments of the model training method according to this disclosure;

[0030] Figure 4 This is a flowchart of some embodiments of the information generation method according to this disclosure;

[0031] Figure 5 These are schematic diagrams illustrating the structure of some embodiments of the model training apparatus according to this disclosure;

[0032] Figure 6 These are schematic diagrams illustrating the structure of some embodiments of the information generation apparatus according to this disclosure;

[0033] Figure 7 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0034] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0035] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0036] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0037] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0038] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0039] Before performing any operations involving the collection, storage, or use of data (such as training image data) as disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting data security impact assessments, informing data subjects, and obtaining prior authorization and consent from data subjects.

[0040] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] Figure 1 This is a schematic diagram illustrating an application scenario of a model training method according to some embodiments of the present disclosure.

[0042] exist Figure 1In this application scenario, firstly, the electronic device 101 can acquire a training image dataset 102. In this application scenario, the training image dataset 102 includes training image data 1021, training image data 1022, training image data 1023, and training image data 1024. Then, for each training image data in the training image dataset 102, the electronic device 101 can perform a sample pair generation step: First, using a pre-trained dynamic context similarity generation model 104, determine the dynamic context similarity between each remaining image data in the remaining image dataset 103 and the aforementioned training image data. The remaining image dataset 103 is the image dataset in the training image dataset 102 after removing the aforementioned training image data. In this application scenario, for training image data 1021, the remaining image dataset 103 can include training image data 1022, training image data 1023, and training image data 1024. The dynamic context similarity set 105 corresponding to the remaining image dataset 103 includes: dynamic context similarity 1051 between training image data 1021 and training image data 1022, dynamic context similarity 1052 between training image data 1021 and training image data 1023, and dynamic context similarity 1053 between training image data 1021 and training image data 1024. Dynamic context similarity 1051 can be "0.75". Dynamic context similarity 1052 can be "0.64". Dynamic context similarity 1053 can be "0.72". The second step is to select image data from the remaining image dataset 103 whose corresponding dynamic context similarity with the training image data meets the preset similarity condition, and use this as the target image data. In this application scenario, the target image data can be training image data 1022. The third step involves forming positive sample image data pairs from the target image data and the training image data, and forming negative sample image data pairs from the training image data and each remaining image data in the remaining image data subset, resulting in a negative sample image data pair set. In this application scenario, positive sample image data pair 1061 includes training image data 1021 and training image data 1022. The negative sample image data pair set includes negative sample image data pair 1063 and negative sample image data pair 1062. Negative sample image data pair 1062 includes training image data 1021 and training image data 1023. Negative sample image data pair 1063 includes training image data 1021 and training image data 1024. Finally, based on the obtained positive sample image data pair set and negative sample image data pair set, the initial image-to-text semantic conversion model is trained to obtain the image-to-text semantic conversion model.

[0043] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0044] It should be understood that Figure 1 The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0045] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a model training method according to the present disclosure. This model training method includes the following steps:

[0046] Step 201: Obtain the training image dataset.

[0047] In some embodiments, the execution entity of the above model training method (e.g. Figure 1 The electronic device 101 shown can acquire the training image dataset via a wired or wireless connection. The training image dataset can be an image dataset used for subsequent training of the initial image and text semantic conversion model.

[0048] Step 202: For each training image in the above training image dataset, perform the sample pair generation step:

[0049] Step 2021: Using a pre-trained dynamic context similarity generation model, determine the dynamic context similarity between each remaining image data in the remaining image dataset and the aforementioned training image data.

[0050] In some embodiments, the executing entity may utilize a pre-trained dynamic context similarity generation model to determine the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data. The remaining image dataset is the image dataset after removing the training image data from the training image dataset. The dynamic context similarity generation model can be a model that generates the degree of similarity between the dynamic contexts of two images. In practice, the dynamic context similarity generation model can be a generative or adversarial neural network model. Dynamic context similarity can characterize the degree of similarity between the corresponding image content contexts of two image data. In practice, dynamic context similarity can be a value between 0 and 1. The closer to 1, the higher the degree of similarity between the corresponding image content contexts of the two image data.

[0051] Step 2022: Select image data from the remaining image dataset that meets the preset similarity condition for the dynamic context similarity with the training image data, and use them as target image data.

[0052] In some embodiments, the executing entity may select image data from the remaining image dataset that meets a preset similarity condition in terms of dynamic context similarity with the training image data, and use these images as target image data. The preset similarity condition may be that the remaining image data is the data with the highest corresponding dynamic context similarity.

[0053] Step 2023: The target image data and the training image data are combined to form a positive sample image data pair, and the training image data and each remaining image data in the remaining image data subset are combined to form a negative sample image data pair, thus obtaining a negative sample image data pair group.

[0054] In some embodiments, the execution entity may form a positive sample image data pair by combining the target image data and the training image data, and form a negative sample image data pair by combining the training image data and each remaining image data in the subset of remaining image data, thereby obtaining a negative sample image data pair group.

[0055] Step 203: Based on the obtained positive sample image data pair set and negative sample image data pair set, train the initial image-to-text semantic conversion model to obtain the image-to-text semantic conversion model.

[0056] In some embodiments, the aforementioned execution entity can train the initial image-to-text semantic conversion model based on the obtained positive sample image data pairs and negative sample image data pairs to obtain the image-to-text semantic conversion model.

[0057] As an example, the aforementioned execution entity can use the target contrastive learning loss function to train the initial image-to-text semantic conversion model based on the obtained positive sample image data pairs and negative sample image data pairs, thereby obtaining the image-to-text semantic conversion model.

[0058] The various embodiments of this disclosure have the following beneficial effects: Through the model training methods of some embodiments of this disclosure, image-to-text semantic conversion models can accurately convert images into corresponding text semantic information. Specifically, the reason for the inaccuracy of related text semantic information conversion is that existing image-to-text semantic conversion models often treat two images with inconsistent image content events as positive sample pairs, resulting in the learned sample features being unable to distinguish subtle differences between images, making the trained image-to-text semantic conversion model less accurate in text semantic conversion. Based on this, the model training method of some embodiments of this disclosure first obtains a training image dataset for the generation of subsequent sample image data pairs. Then, for each training image data in the training image dataset, a sample pair generation step is performed: First, using a pre-trained dynamic context similarity generation model, the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data can be accurately determined. The remaining image dataset is the image dataset after removing the training image data from the training image dataset. Here, by determining the dynamic contextual similarity between the remaining image data and the training image data, we can not only compare the feature similarity between the intrinsic features of the remaining image data and the intrinsic features of the training image data, but also indirectly compare the similarity between the contextual features corresponding to the remaining image data and the contextual features corresponding to the training image data. This subtle comparison of the similarity of various aspects of the features between the remaining image data and the training image data makes the subsequently generated sample image data pairs more accurate. The second step involves selecting image data from the remaining image dataset that meets the preset similarity condition for dynamic contextual similarity with the training image data, as target image data, for use in the generation of subsequent sample image data pairs. Here, by selecting target image data, we can effectively select the remaining image data that is most similar to the training image data in various aspects (including dynamic contextual features). Thus, accurate positive sample image data pairs can be generated subsequently. The third step involves forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with each remaining image data in the subset of the training image data and the remaining image data, resulting in a negative sample image data pair group. Here, based on dynamic contextual similarity, image data with the most similar detailed features can be paired, making the generated positive and negative sample image data pairs more accurate. Finally, based on the obtained sets of positive and negative sample image data pairs, the initial image-to-text semantic conversion model is trained to obtain the image-to-text semantic conversion model.Here, using precise sets of positive and negative image data pairs for model training can indirectly improve the performance of the image-to-text semantic conversion model, enabling it to accurately convert images into corresponding textual semantic information. In summary, the positive and negative image data pairs generated based on dynamic contextual similarity make subsequent image-to-text semantic conversion model training more effective and the conversion of textual semantic information more accurate.

[0059] Further reference Figure 3 The diagram illustrates flow 300 of some other embodiments of the model training method according to this disclosure. This model training method includes the following steps:

[0060] Step 301: Obtain the training image dataset.

[0061] Step 302: For each training image in the above training image dataset, perform the sample pair generation step:

[0062] Step 3021: Perform data vectorization on the above training image data and the above remaining image data to obtain training image vectors and remaining image vectors respectively.

[0063] In some embodiments, the executing entity (e.g. Figure 1 The electronic device 101 shown can vectorize the training image data and the remaining image data respectively to obtain training image vectors and remaining image vectors. The dynamic context similarity generation model includes: an associated image representation vector generation model. The training image vector can represent the feature information of the training image data. The remaining image vector can represent the feature information of the remaining image data. The associated image representation vector model can be a model that generates representation vectors for at least one context-related image corresponding to the training image data.

[0064] As an example, the aforementioned associated image representation vector model can be a multi-layer convolutional neural network model. The aforementioned execution entity can input the training image data and the aforementioned remaining image data into the multi-layer convolutional neural network model respectively to extract image feature information and obtain training image vectors and remaining image vectors.

[0065] In some optional implementations of certain embodiments, the above-described associated image representation vector generation model is trained through the following steps:

[0066] The first step is to acquire the image data set.

[0067] The image data group includes: current image data, at least one historical semantically associated image data related to the current image data, and at least one future semantically associated image data related to the current image data. The image events corresponding to the historical semantically associated image data and the image events corresponding to the current image data have a chronological order of association. Similarly, the image events corresponding to the current image data and the corresponding times of the future semantically associated image data have a chronological order of association.

[0068] The second step involves combining the image data in each image data group with the current image data, at least one historical semantically related image data, and at least one future semantically related image data to generate positive image data group sets and negative image data group sets.

[0069] As an example, firstly, the aforementioned executing entity can determine each image data group as a positive image data group set. Then, the image data included in different image data groups are randomly combined to ensure that the combined image data group is not an image data group from the aforementioned image data group set, thus obtaining a combined image data group set, which serves as a negative image data group set.

[0070] The third step involves learning parameters for at least one initial semantic association learning parameter in the initial associated image representation vector generation model based on the aforementioned positive and negative image datasets, thereby obtaining the associated image generation model.

[0071] The aforementioned associated image representation vector generation model includes at least one semantic association learning parameter after learning.

[0072] As an example, the aforementioned execution entity can perform parameter comparison learning and update at least one initial semantic association learning parameter in the aforementioned initial associated image representation vector generation model based on the aforementioned positive image data set and negative image data set, thereby obtaining the associated image generation model.

[0073] Step 3022: Input the above-mentioned training image vector into the above-mentioned associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector.

[0074] In some embodiments, the execution entity may input the training image vectors into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector. The first historical associated image vector can represent the feature information of the historical associated image. The contextual information of the historical event corresponding to the historical associated image is closely related to the contextual information of the event corresponding to the training image data. That is, the historical event corresponding to the historical associated image occurred first, and the event corresponding to the training image data occurred later. Similarly, the first future associated image vector can represent the feature information of the future associated image. The contextual information of the historical event corresponding to the future associated image is closely related to the contextual information of the event corresponding to the training image data. That is, the event corresponding to the training image data occurred first, and the historical event corresponding to the future associated image occurred later.

[0075] In some optional implementations of certain embodiments, the aforementioned associated image representation vector generation model includes: a historical associated image representation vector generation model and a future associated image representation vector generation model. The semantic association learning parameters of the historical associated image generation model and the future associated image generation model are different. Specifically, the historical associated image representation vector generation model can be a model that generates representation vectors corresponding to historical associated images. The future associated image representation vector generation model can be a model that generates representation vectors corresponding to future associated images.

[0076] Optionally, inputting the trained image vectors into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector may include the following steps:

[0077] The first step is to input the above-mentioned training image vectors into the above-mentioned historical associated image representation vector generation model to generate at least one first historical associated image vector.

[0078] The second step is to input the above-mentioned training image vectors into the above-mentioned future associated image representation vector generation model to generate at least one of the above-mentioned first future associated image vectors.

[0079] Step 3023: Input the remaining image vectors into the associated image representation vector generation model to output at least one second historical associated image vector and at least one second future associated image vector.

[0080] In some embodiments, the execution entity may input the remaining image vectors into the associated image representation vector generation model to output at least one second historical associated image vector and at least one second future associated image vector.

[0081] In some optional implementations of certain embodiments, the aforementioned associated image representation vector generation model includes at least one learned semantic association learning parameter.

[0082] Optionally, inputting the trained image vectors into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector may include the following steps:

[0083] The first step is to multiply the above-mentioned training image vector with at least one parameter transpose matrix corresponding to at least one first semantic association learning parameter to obtain at least one first multiplication result, which is used as the above-mentioned at least one first historical association image vector.

[0084] Specifically, there is a one-to-one correspondence between at least one first semantic association learning parameter and the corresponding parameter transpose matrix of at least one parameter transpose matrix. The parameter transpose matrix can be the transpose of the parameter matrix corresponding to the semantic association learning parameter.

[0085] The second step involves multiplying the aforementioned training image vector with at least one parameter matrix corresponding to the aforementioned at least one second semantic association learning parameter to obtain at least one second multiplication result, which serves as the aforementioned at least one first future associated image vector. Specifically, there is a one-to-one correspondence between the second semantic association learning parameter in the at least one second semantic association learning parameter and the parameter transpose matrix in the corresponding at least one parameter transpose matrix.

[0086] Step 3024: Generate dynamic context similarity based on the at least one first historical associated image vector, the at least one first future associated image vector, the at least one second historical associated image vector, and the at least one second future associated image vector.

[0087] In some embodiments, the executing entity may generate dynamic contextual similarity based on the at least one first historical associated image vector, the at least one first future associated image vector, the at least one second historical associated image vector, and the at least one second future associated image vector.

[0088] As an example, firstly, the aforementioned executing entity can perform a weighted summation of at least one historically associated image vector and at least one first future associated image vector to obtain a first summation vector. Then, it can perform a weighted summation of at least one second historically associated image vector and at least one second future associated image vector to obtain a second summation vector. Finally, it can average the first and second summation vectors to obtain an average value, which is used as the voice-context similarity.

[0089] In some optional implementations of certain embodiments, generating dynamic contextual similarity based on at least one first historical associated image vector, at least one first future associated image vector, at least one second historical associated image vector, and at least one second future associated image vector may include the following steps:

[0090] The first step is to perform vector weighted summation on at least one first historical associated image vector to obtain a first weighted summation vector, and to perform vector weighted summation on at least one second historical associated image vector to obtain a second weighted summation vector.

[0091] The second step involves performing a weighted summation on at least one first future associated image vector to obtain a third weighted summation vector, and performing a weighted summation on at least one second future associated image vector to obtain a fourth weighted summation vector.

[0092] The third step is to generate the dynamic context similarity based on the first weighted summation vector, the second weighted summation vector, the third weighted summation vector, and the fourth weighted summation vector.

[0093] As an example, firstly, the aforementioned execution entity can multiply the first weighted summation vector and the third weighted summation vector to obtain the first multiplication result. Then, the aforementioned execution entity can multiply the second weighted summation vector and the fourth weighted summation vector to obtain the second multiplication result. Next, the training image vector and the remaining image vector are multiplied to obtain the third multiplication result. Finally, the first, second, and third multiplication results are added together to obtain the summed result, which serves as the dynamic contextual similarity.

[0094] Step 3025: Select image data from the remaining image dataset that meets the preset similarity condition for the dynamic context similarity with the training image data, and use them as target image data.

[0095] Step 3026: The target image data and the training image data are combined to form a positive sample image data pair, and the training image data and each remaining image data in the remaining image data subset are combined to form a negative sample image data pair, thus obtaining a negative sample image data pair group.

[0096] Step 303: Based on the obtained positive sample image data pair set and negative sample image data pair set, train the initial image-to-text semantic conversion model to obtain the image-to-text semantic conversion model.

[0097] In some embodiments, the specific implementation of steps 301, 3024-3025, and 303, and their resulting technical effects, can be found in [reference needed]. Figure 2Steps 201, 2022-2023 and 203 in the corresponding embodiments will not be repeated here.

[0098] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 In some corresponding embodiments, the model training method process 300 utilizes an associated image representation vector generation model to accurately generate at least one first historical associated image vector, at least one first future associated image vector, at least one second historical associated image vector, and at least one second future associated image vector. Therefore, the similarity between the contextual features of the remaining image data and the training image data can be accurately determined subsequently.

[0099] Further reference Figure 4 The diagram illustrates a flow 400 of another embodiment of the information generation method according to the present disclosure. This information generation method includes the following steps:

[0100] Step 401: Obtain the target image.

[0101] In some embodiments, the executing entity (e.g., an electronic device) may acquire a target image. The target image may be an image to be converted into text semantic information. The text semantic information may be semantic information describing the image content.

[0102] Step 402: Input the target image into a pre-trained image-to-text semantic conversion model to output text semantic information.

[0103] In some embodiments, the execution entity may input the target image into a pre-trained image-to-text semantic conversion model to output textual semantic information. The image-to-text semantic conversion model is generated using the model training method described above in this disclosure. The textual semantic information may be textual information representing the image semantics of the target image.

[0104] The above embodiments of this disclosure have the following beneficial effects: by using the information generation method of some embodiments of this disclosure, and utilizing the image and text semantic conversion model, images can be accurately converted into corresponding text semantic information.

[0105] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a model training apparatus, which are similar to... Figure 2 Corresponding to the method embodiments shown, this model training device can be specifically applied to various electronic devices.

[0106] like Figure 5As shown, a model training device 500 includes: a first acquisition unit 501, an execution unit 502, and a training unit 503. The first acquisition unit 501 is configured to acquire a training image dataset; the execution unit 502 is configured to perform a sample pair generation step for each training image data in the training image dataset: using a pre-trained dynamic context similarity generation model, determine the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data, wherein the remaining image dataset is the image dataset after removing the training image data from the training image dataset; select image data whose corresponding dynamic context similarity with the training image data meets a preset similarity condition from the remaining image dataset as target image data; form positive sample image data pairs with the target image data and the training image data, and form negative sample image data pairs with each remaining image data in the training image data and the subset of remaining image data, to obtain a negative sample image data pair set; the training unit 503 is configured to train the initial image-to-text semantic conversion model based on the obtained positive sample image data pair set and negative sample image data pair set, to obtain the image-to-text semantic conversion model.

[0107] In some optional implementations of certain embodiments, the dynamic context similarity generation model includes: an associated image representation vector generation model; and the execution unit 502 in the model training device 500 can be further configured to: vectorize the training image data and the remaining image data respectively to obtain training image vectors and remaining image vectors; input the training image vectors into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector; input the remaining image vectors into the associated image representation vector generation model to output at least one second historical associated image vector and at least one second future associated image vector; and generate dynamic context similarity based on the at least one first historical associated image vector, the at least one first future associated image vector, the at least one second historical associated image vector, and the at least one second future associated image vector.

[0108] In some optional implementations of some embodiments, the aforementioned associated image representation vector generation model includes: a historical associated image representation vector generation model and a future associated image representation vector generation model, wherein the semantic association learning parameters of the historical associated image generation model and the future associated image generation model are different; and the execution unit 502 in the aforementioned model training device 500 may be further configured to: input the aforementioned training image vector into the aforementioned historical associated image representation vector generation model to generate the aforementioned at least one first historical associated image vector; and input the aforementioned training image vector into the aforementioned future associated image representation vector generation model to generate the aforementioned at least one first future associated image vector.

[0109] In some optional implementations of certain embodiments, the execution unit 502 in the model training apparatus 500 may be further configured to: perform vector weighted summation on the at least one first historical associated image vector to obtain a first weighted summation vector; perform vector weighted summation on the at least one second historical associated image vector to obtain a second weighted summation vector; perform vector weighted summation on the at least one first future associated image vector to obtain a third weighted summation vector; and perform vector weighted summation on the at least one second future associated image vector to obtain a fourth weighted summation vector; and generate the dynamic context similarity based on the first weighted summation vector, the second weighted summation vector, the third weighted summation vector, and the fourth weighted summation vector.

[0110] In some optional implementations of some embodiments, the aforementioned associated image representation vector generation model includes: at least one learned semantic association learning parameter; and the execution unit 502 in the aforementioned model training device 500 may be further configured to: multiply the aforementioned training image vector with at least one parameter transpose matrix corresponding to the aforementioned at least one first semantic association learning parameter to obtain at least one first multiplication result as the aforementioned at least one first historical associated image vector; multiply the transpose matrix corresponding to the aforementioned training image vector with at least one parameter matrix corresponding to the aforementioned at least one second semantic association learning parameter to obtain at least one second multiplication result as the aforementioned at least one first future associated image vector.

[0111] In some optional implementations of certain embodiments, the aforementioned associated image representation vector generation model is trained through the following steps: obtaining an image data set, wherein the image data set includes: current image data, at least one historical semantically associated image data for the current image data, and at least one future semantically associated image data for the current image data; performing data pairing on the image data in each image data set based on the current image data, at least one historical semantically associated image data, and at least one future semantically associated image data included in each image data set to generate a positive image data set and a negative image data set; and performing parameter learning on at least one initial semantic association learning parameter in the aforementioned initial associated image representation vector generation model based on the aforementioned positive image data set and negative image data set to obtain an associated image generation model, wherein the aforementioned associated image representation vector generation model includes: at least one learned semantic association learning parameter.

[0112] It is understandable that the units described in the model training device 500 are related to the reference... Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the model training device 500 and the units contained therein, and will not be repeated here.

[0113] Further reference Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an information generation apparatus, which are similar to... Figure 4 Corresponding to the method embodiments shown, this information generation device can be specifically applied to various electronic devices.

[0114] like Figure 6 As shown, an information generation device 600 includes a second acquisition unit 601 and an input unit 602. The second acquisition unit 601 is configured to acquire a target image; the input unit 602 is configured to input the target image into a pre-trained image-to-text semantic conversion model to output text semantic information. The image-to-text semantic conversion model is generated based on the model training method of this disclosure.

[0115] It is understandable that the units described in the information generation device 600 and the reference Figure 4 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the information generation device 600 and the units contained therein, and will not be repeated here.

[0116] The following is for reference. Figure 7 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1A schematic diagram of the structure of electronic device 101)700 in the middle. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0117] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory 702 or a program loaded from a storage device 708 into a random access memory 703. The random access memory 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, the read-only memory 702, and the random access memory 703 are interconnected via a bus 704. An input / output interface 705 is also connected to the bus 704.

[0118] Typically, the following devices can be connected to the input / output interface 705: input devices 706 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 708 including, for example, magnetic tape, hard disk, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 7 Each box shown can represent a device or multiple devices as needed.

[0119] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a storage device 708, or installed from a read-only memory 702. When the computer program is executed by the processing device 701, it performs the functions defined in the methods of some embodiments of this disclosure.

[0120] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0121] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0122] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs. When the electronic device executes the one or more programs, the electronic device causes the following actions: acquiring a training image dataset; for each training image data in the training image dataset, performing a sample pair generation step: using a pre-trained dynamic context similarity generation model, determining the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data, wherein the remaining image dataset is the image dataset after removing the training image data from the training image dataset; selecting image data from the remaining image dataset whose corresponding dynamic context similarity with the training image data meets a preset similarity condition, as target image data; forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with each remaining image data in the subset of the training image data and the remaining image data, obtaining a negative sample image data pair set; and training an initial image-to-text semantic conversion model based on the obtained positive sample image data pair set and negative sample image data pair set, obtaining an image-to-text semantic conversion model. Acquire the target image; input the target image into a pre-trained image-to-text semantic conversion model to output text semantic information, wherein the image-to-text semantic conversion model is generated based on the model training method of this disclosure.

[0123] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a first acquisition unit, an execution unit, and a training unit. The names of these units do not necessarily limit the specific unit; for example, the first acquisition unit may also be described as a "unit for acquiring a training image dataset."

[0126] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0127] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the model training methods and information generation methods described above.

[0128] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A model training method, comprising: Obtain the training image dataset; For each training image in the training image dataset, perform the sample pair generation step: Using a pre-trained dynamic context similarity generation model, the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data is determined, wherein the remaining image dataset is the image dataset after removing the training image data from the training image dataset; Image data whose dynamic context similarity with the training image data meets the preset similarity condition is selected from the remaining image dataset and used as target image data; The target image data and the training image data are combined to form a positive sample image data pair, and the training image data and each remaining image data in the remaining image data subset are combined to form a negative sample image data pair, thus obtaining a negative sample image data pair group; Based on the obtained positive sample image data pairs and negative sample image data pairs, the initial image-to-text semantic conversion model is trained to obtain the image-to-text semantic conversion model, which is used to convert images into corresponding text semantic information.

2. The method according to claim 1, wherein, The dynamic context similarity generation model includes: an associated image representation vector generation model; and The method of using a pre-trained dynamic context similarity generation model to determine the dynamic context similarity between each remaining image in the remaining image dataset and the training image data includes: The training image data and the remaining image data are respectively vectorized to obtain training image vectors and remaining image vectors; The training image vectors are input into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector; The remaining image vectors are input into the associated image representation vector generation model to output at least one second historical associated image vector and at least one second future associated image vector; Dynamic context similarity is generated based on the at least one first historical associated image vector, the at least one first future associated image vector, the at least one second historical associated image vector, and the at least one second future associated image vector.

3. The method according to claim 2, wherein, The associated image representation vector generation model includes: a historical associated image representation vector generation model and a future associated image representation vector generation model, wherein the semantic association learning parameters of the historical associated image generation model and the future associated image generation model are different; and The step of inputting the training image vector into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector includes: The training image vector is input into the historical associated image representation vector generation model to generate the at least one first historical associated image vector; The training image vector is input into the future associated image representation vector generation model to generate the at least one first future associated image vector.

4. The method according to claim 2, wherein, The step of generating dynamic contextual similarity based on the at least one first historical associated image vector, the at least one first future associated image vector, the at least one second historical associated image vector, and the at least one second future associated image vector includes: The at least one first historical associated image vector is subjected to vector weighted summation to obtain a first weighted summation vector, and the at least one second historical associated image vector is subjected to vector weighted summation to obtain a second weighted summation vector; The at least one first future associated image vector is subjected to vector weighted summation to obtain a third weighted summation vector, and the at least one second future associated image vector is subjected to vector weighted summation to obtain a fourth weighted summation vector; The dynamic context similarity is generated based on the first weighted summation vector, the second weighted summation vector, the third weighted summation vector, and the fourth weighted summation vector.

5. The method according to claim 2, wherein, The associated image representation vector generation model includes: at least one learned semantic association learning parameter; and The step of inputting the training image vector into the associated image representation vector generation model to output at least one first historical associated image vector and at least one first future associated image vector includes: The training image vector is multiplied by at least one parameter transpose matrix corresponding to at least one first semantic association learning parameter to obtain at least one first multiplication result, which is used as the at least one first historical association image vector. The training image vector is multiplied by at least one parameter matrix corresponding to the at least one second semantic association learning parameter to obtain at least one second multiplication result, which is used as the at least one first future association image vector.

6. The method according to claim 2, wherein, The associated image representation vector generation model is trained through the following steps: Obtain an image data set, wherein the image data set includes: current image data, at least one historical semantically associated image data for the current image data, and at least one future semantically associated image data for the current image data; Based on the current image data, at least one historical semantically related image data, and at least one future semantically related image data included in each image data group, the image data in the image data group is matched to generate a positive image data group set and a negative image data group set; Based on the positive image dataset and the negative image dataset, at least one initial semantic association learning parameter in the initial associated image representation vector generation model is learned to obtain the associated image generation model, wherein the associated image representation vector generation model includes at least one learned semantic association learning parameter.

7. An information generation method, comprising: Acquire the target image; The target image is input into a pre-trained image-to-text semantic conversion model to output text semantic information, wherein the image-to-text semantic conversion model is generated based on the method described in any one of claims 1-6.

8. A model training device, comprising: The first acquisition unit is configured to acquire the training image dataset; The execution unit is configured to perform a sample pair generation step for each training image data in the training image dataset: using a pre-trained dynamic context similarity generation model, determining the dynamic context similarity between each remaining image data in the remaining image dataset and the training image data, wherein the remaining image dataset is the image dataset after removing the training image data from the training image dataset; selecting image data from the remaining image dataset whose corresponding dynamic context similarity with the training image data meets a preset similarity condition as target image data; forming positive sample image data pairs with the target image data and the training image data, and forming negative sample image data pairs with each remaining image data in the training image data and the subset of remaining image data, to obtain a negative sample image data pair group; The training unit is configured to train the initial image-to-text semantic conversion model based on the obtained positive sample image data pairs and negative sample image data pairs, thereby obtaining the image-to-text semantic conversion model, which is used to convert images into corresponding text semantic information.

9. An information generation device, comprising: The second acquisition unit is configured to acquire the target image; An input unit is configured to input the target image into a pre-trained image-to-text semantic conversion model to output text semantic information, wherein the image-to-text semantic conversion model is generated based on the method described in any one of claims 1-6.

10. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

11. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images