Scene description text generation method and device and computer readable storage medium

Abstract

The invention relates to a scene description text generation method and device and a computer readable storage medium, and relates to the field of artificial intelligence. The method for generating the description text of the scene comprises the steps that a relation network of categories of articles is obtained, vertexes in the relation network represent the categories, and edges connecting the two categories represent common attributes of the two categories; the relation network is divided into a plurality of communities, categories in each community are associated with one another, and at least one community corresponds to one scene; determining text features corresponding to the target scene according to categories in a community corresponding to the target scene and common attributes of the categories; and generating a description text of the target scene according to the text features corresponding to the target scene. According to the invention, the text for accurately describing the target scene can be generated.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a method and device for generating a description text of a scene, and a computer-readable storage medium. Background Art

[0002] Users' needs for multiple items in the same scene are usually related. For example, in a camping scene, users will need clothing (e.g., jackets, sun protection clothing, sun hats, gloves, waterproof bags, etc.), food equipment (e.g., barbecue stoves, lighters, gas cylinders, pot sets, kettles, car refrigerators, outdoor incubators, etc.), accommodation equipment (e.g., tents, sleeping bags, canopies, folding beds, folding chairs, moisture-proof mats, etc.), and other tools (e.g., basins, towels, flashlights, camp lights, large mobile power supplies, first aid kits, tool boxes, etc.).

[0003] These items involve dozens or even hundreds of categories. By learning the potential connections between products across categories, we can identify a series of potential needs and motivations of users and recommend related items to users. Summary of the invention

[0004] According to a first aspect of the present disclosure, a method for generating a description text of a scene is provided, comprising: obtaining a relationship network of categories of objects, wherein vertices in the relationship network represent categories, and edges connecting two categories represent common attributes of the two categories; dividing the relationship network into multiple communities, wherein categories within each community are interrelated, and at least one community corresponds to a scene; determining text features corresponding to a target scene based on categories in the community corresponding to the target scene and common attributes of the categories; and generating a description text of the target scene based on the text features corresponding to the target scene.

[0005] In some embodiments, dividing a relationship network into multiple communities includes: dividing the vertices of the relationship network based on a quality function of the relationship network to obtain a first community set; for at least one community in the first community set, re-dividing the vertices in each community based on the quality function of each community in at least one community in the first community set to obtain a second community set, wherein the second community set includes the re-divided communities and the communities in the first community set that have not been re-divided; taking each community in the second community set as a vertex, iteratively executing to obtain the first community set and the second community set until an iteration termination condition is reached, wherein at least one community in the second community set obtained after the iteration termination corresponds to a scenario.

[0006] In some embodiments, the first set is represented as {community A, community B, community C, ...}, and for at least one community in the first community set, based on the quality function of each community in at least one community in the first community set, the vertices in each community are re-divided to obtain a second community set, including: any community C in at least one community in the first community set includes vertices (v1, v2, v3, ...), each vertex in community C is a subcommunity, community C includes {subcommunity C1, subcommunity C2, ...}, traversing the vertices in community C, for vertex v in community C j and vertex v j Connected subcommunity C j , before merging the vertex into subcommunity C j After that, if the value of the quality function of community C increases, the vertex is merged into subcommunity C j , the multiple subcommunities obtained after the traversal are taken as the communities in the second community set, where the larger the value of the quality function is, the higher the association between the vertices in the community C is, and i and j are natural numbers.

[0007] In some embodiments, before the vertex v j Merge into subcommunity C of community C j After that, if the quality function of community C does not increase, move vertex v j Remain in the subcommunity to which it originally belonged.

[0008] In some embodiments, the quality function of community C is calculated according to the sum of the weights of the internal edges of each subcommunity of community C and the sum of the degrees of the vertices of each subcommunity.

[0009] In some embodiments, based on the quality function of the relationship network, the vertices of the relationship network are divided to obtain a first community set, including: taking each vertex in the relationship network as a community, traversing the vertices in the relationship network, and after merging the vertex into the community connected to the vertex, if the quality function of the relationship network increases, merging the vertex into the community connected to the vertex to obtain the first community set.

[0010] In some embodiments, the quality function of the relationship network is calculated according to the sum of the weights of the internal edges of each community in the relationship network and the sum of the degrees of the internal vertices of each community.

[0011] In some embodiments, text features corresponding to the target scene are determined based on categories in the community corresponding to the target scene and common attributes of the categories, including: determining candidate text features corresponding to the target scene based on categories in the community corresponding to the target scene and common attributes of the categories; selecting text features corresponding to the target scene from the candidate text features based on at least one of an attribution value of the candidate text feature, a target group index of the candidate text feature, and a proportion of users in the target group having the candidate text feature to all users of the target group, wherein the target group represents a user group associated with the category in the community corresponding to the target scene, and the attribution value of the text feature represents a contribution of the text feature to the prediction result when the machine learning model predicts whether a user will select an item in the target scene based on the text feature.

[0012] In some embodiments, a description text of the target scene is generated based on text features corresponding to the target scene, including: determining keywords of the designated object based on the description text of the designated object of the target scene; and generating a description text of the target scene based on the text features corresponding to the target scene and the keywords of the designated object.

[0013] In some embodiments, a description text of the target scene is generated according to text features corresponding to the target scene and keywords of the specified object, including: generating a description text of the target scene using a text generation model according to the text features corresponding to the target scene and keywords of the specified object.

[0014] In some embodiments, a description text of the target scene is generated based on text features corresponding to the target scene and keywords of the specified object, including: generating a description text of the target scene based on the intersection of the text features corresponding to the target scene and the keywords of the specified object; or generating a description text of the target scene based on the union of the text features corresponding to the target scene and the keywords of the specified object.

[0015] According to a second aspect of the present disclosure, a device for generating a description text of a scene is provided, comprising: an acquisition module, configured to acquire a relationship network of categories of objects, wherein vertices in the relationship network represent categories, and edges connecting two categories represent common attributes of the two categories; a division module, configured to divide the relationship network into multiple communities, wherein categories within each community are interrelated, and at least one community corresponds to a scene; a determination module, configured to determine text features corresponding to a target scene based on categories in the community corresponding to the target scene and common attributes of the categories; and a generation module, configured to generate a description text of the target scene based on the text features corresponding to the target scene.

[0016] According to a third aspect of the present disclosure, a device for generating a description text of a scene is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute a method for generating a description text of a scene according to any embodiment of the present disclosure based on instructions stored in the memory.

[0017] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the instructions are executed by a processor, a method for generating a description text of a scene according to any embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0020] Figure 1 A flowchart showing a method for generating a description text of a scene according to some embodiments of the present disclosure;

[0021] Figure 2 A flowchart showing community division according to some embodiments of the present disclosure;

[0022] Figure 3 A flowchart showing iterations according to some embodiments of the present disclosure;

[0023] Figure 4 A schematic diagram showing community division according to some embodiments of the present disclosure is shown;

[0024] Figure 5 A schematic diagram of an encoder-decoder according to some embodiments of the present disclosure is shown;

[0025] Figure 6 shows a converter structure according to some embodiments of the present disclosure;

[0026] Figure 7 A schematic diagram of generating a scene description text according to some embodiments of the present disclosure is shown;

[0027] Figure 8 A block diagram showing a device for generating a description text of a scene according to some embodiments of the present disclosure;

[0028] Fig. 9 A block diagram showing a device for generating a description text of a scene according to some other embodiments of the present disclosure;

[0029] Fig.10 A block diagram of a computer system for implementing some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0031] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0032] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0033] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0034] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0035] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0036] In related technologies, user behavior and association rule algorithms are used to calculate the association between two categories. To build the relationship between two categories, the target category needs to be input in advance. Without a description of the entire scene, it is impossible to draw a conclusion about which type of people the scene is targeting.

[0037] The present disclosure provides a method for generating a description text of a scene, which can generate text that accurately describes the scene.

[0038] Figure 1 A flowchart showing a method for generating a description text of a scene according to some embodiments of the present disclosure is shown.

[0039] like Figure 1 As shown, the method for generating a description text of a scene includes steps S1 to S4. In some embodiments, the method for generating a description text of a scene is performed by a device for generating a description text of a scene.

[0040] In step S1, a relationship network of categories of items is obtained, wherein vertices in the relationship network represent categories, and an edge connecting two categories represents common attributes of the two categories.

[0041] To obtain the relationship network of the categories of items, we must first obtain the associations between categories. The associations between categories are mainly obtained from user behavior. For example, if a user selects (e.g. browses, collects) items of two different categories successively or simultaneously within a period of time, it is considered that there is an association between the two categories. The common attributes of the two categories include the common users of the two categories, the number of times the common users of the two categories select the two categories, etc.

[0042] For a set of categories T, category t (t∈T) is used as the vertex of the relationship network, and the set of vertices is recorded as V. The vertex has a basic attribute V.attr, and V.attr is, for example, the number of users who choose the category within a period of time (user_num). The common data between categories is calculated as the edge between two vertices, and the set of edges is recorded as E, and the edges have a common attribute E.attr. Common attributes include, for example, the number of the same users who choose two categories within a period of time (the number of overlapping users) user_num, the number of times the common users choose items of these two categories, etc.

[0043] Two thresholds θ1 and θ2 are set so that the minimum number of overlapping users is greater than θ1 to avoid errors caused by low confidence due to too little data; and the number of edges connected to the same vertex is less than θ2 to avoid interference caused by too many associated categories. The formula is as follows.

[0044]

[0045]

[0046] In some embodiments, θ1=1000, θ2=100.

[0047] The relational network is represented as an undirected graph G = (V, E), where G includes n = |V| vertices and m = |E| edges.

[0048] In step S2, the relationship network is divided into multiple communities, wherein the categories within each community are interconnected, and at least one community corresponds to one scenario.

[0049] For the relationship network G = (V, E), find a suitable division P of the category so that the internal correlation of the subgroups within the division P is strong, while the correlation between groups is relatively small. This subgroup is also called a community.

[0050] A partition P = {C1,...,C r} contains r = |P| communities, and each vertex in the relationship network satisfies Where V = ∪ i C i Indicates all C i The whole traversal of all vertices, It means that there is no intersection between the vertices of any two communities, which means that the community division is exhaustive and mutually exclusive.

[0051] Figure 2 A flowchart of community division according to some embodiments of the present disclosure is shown.

[0052] like Figure 2 As shown, community division includes steps S21-S23.

[0053] In step S21, based on the quality function of the relationship network, the vertices of the relationship network are divided to obtain a first community set.

[0054] Figure 3 A flowchart illustrating iterations according to some embodiments of the present disclosure.

[0055] like Figure 3 As shown in the figure, during the iteration process, the vertices in the community are moved, and then the divided communities are redefined, and then some communities are aggregated, and the above steps are repeated until the convergence condition is reached and the cycle ends.

[0056] Figure 4 A schematic diagram of community division according to some embodiments of the present disclosure is shown.

[0057] Figure 4 (a) is the relationship network before the vertices are divided. Figure 4 (b) in the figure is the relationship network after executing step s21. The circles in the figure represent vertices, and the different colors of the circles in (b) indicate that the circles belong to different communities. From (b), it can be seen that the vertices are classified into several communities.

[0058] Define the quality function (modularity) H(G, P) to measure the quality of partition P. Modularity means the weight of the edge inside the community minus the weight of the edge connecting the community to the outside. The larger the modularity, the closer the connection inside the community is and the fewer the connection with the outside. H(G, P) is abbreviated as H(P). The definition of the quality function is shown in the following formula.

[0059]

[0060] Among them, e c represents the number of actual edges in community C, m represents the number of edges in the entire network, and K cRepresents the sum of the degrees of the vertices in community C. The degree of a vertex is the sum of the weights of the edges connected to the vertex. The sum symbol represents the sum of all communities. Based on the partition P, the graph G is re-aggregated into a new graph G'. G is called the original graph and G' is called the aggregated graph. The vertices of G' are the communities partitioned in G, that is, V(G') = P. The edges of G' inherit from the edges of G, and one vertex can connect multiple edges. Taking the relationship network containing two communities C and D as an example, the relationship network can be expressed as follows.

[0061] E(G')={(C, D)|(u, v)∈E(G), u∈C∈P, v∈D∈P}

[0062] We use P(v→C) to represent moving vertex v to community C in partition P, and using ΔH p (v→C) represents the change in mass function caused by moving vertex v, then ΔH p (v→C) is expressed as follows.

[0063] ΔH p (v→C)=H(P(v→C))-H(P)

[0064] Step S21 is the process of moving and merging nodes, which improves the quality of community differentiation by initially moving vertices.

[0065] In some embodiments, based on the quality function of the relationship network, the relationship network is divided into multiple communities to obtain a first community set, including: taking each vertex in the relationship network as a community, traversing the vertices in the relationship network, and after merging the vertex into the community connected to the vertex, if the quality function of the relationship network increases, merging the vertex into the community connected to the vertex to obtain the first community set.

[0066] For example, randomly sort all vertices in the relationship network, and calculate the community C that vertex v should be incorporated into one by one according to the sorted order, so that the gain of the quality function is the highest, that is, argmax C∈P ΔH P (v→C). When ΔH P When (v→C)>0 (if moving the community to which vertex v belongs will improve the quality function), change the partition of vertex v. Until all vertices are traversed, H(P) can no longer be improved.

[0067] In some embodiments, the quality function of the relationship network is calculated according to the sum of the weights of the internal edges of each community in the relationship network and the sum of the degrees of the internal vertices of each community.

[0068] For example, first, for each community in the relationship network, the weights of all internal edges of the community are summed up, and the degrees of the vertices within the community are summed up to calculate the community's Then for all communities Sum them up and get the quality function of the entire relationship network.

[0069] In step S22, for at least one community in the first community set, based on the quality function of each community in at least one community in the first community set, the vertices in each community are re-divided to obtain a second community set, wherein the second community set includes the re-divided communities and the communities in the first community set that have not been re-divided. Figure 4 The partial communities in (b) are further divided into multiple communities, and the following are obtained: Figure 4 (c) shows that there are more communities than (b).

[0070] In some embodiments, the first set is represented as {community A, community B, community C, ...}, and for at least one community in the first community set, based on the quality function of each community in the at least one community, the vertices of each community are re-divided to obtain a second community set, including: any community C in at least one community in the first community set includes vertices (v1, v2, v3, ...), each vertex in community C is a subcommunity, and community C includes {subcommunity C1, subcommunity C2, ...}, traversing the vertices in community C, for vertex v in community C j and vertex v j Connected subcommunity C j , before merging the vertex into subcommunity C j After that, if the value of the quality function of community C increases, the vertex is merged into subcommunity C j , the multiple subcommunities obtained after the traversal are taken as the communities in the second community set, where the larger the value of the quality function is, the higher the association between the vertices in the community C is, and i and j are natural numbers.

[0071] For example, each vertex is set as a subcommunity, recorded as partition P1, and P1 is used to redefine the partition P in step S21. For any community C∈P in P, only the vertex v in C is considered. j ∈P1. Recalculate the quality function inside C. If the vertex v j Merging with other vertices (subcommunities) in C will increase the quality function of C, and C will be further subdivided into multiple communities.

[0072] In some embodiments, before the vertex v j Merge into subcommunity C of community C j After that, if the quality function of community C does not increase, move vertex v jRemain in the subcommunity to which it originally belonged. For example, if moving any vertex inside C does not increase the quality function of C, then C remains unchanged and will no longer split into multiple communities.

[0073] Step S22 is a process of redefining the community, which further subdivides the division obtained in step S21, and step S22 is performed within a community C obtained in step S21.

[0074] In some embodiments, the quality function of community C is calculated according to the sum of the weights of the internal edges of each subcommunity of community C and the sum of the degrees of the vertices of each subcommunity.

[0075] For example, first, for each subcommunity in community C, the weights of all internal edges of the subcommunity are summed, and the degrees of the vertices within the subcommunity are summed to calculate the degree of the subcommunity. Then, for all subcommunities Sum them up and get the quality function of community C.

[0076] In step S23, each community in the second community set is taken as a vertex, and the first community set and the second community set are obtained by iteration until the iteration termination condition is reached. At least one community in the second community set obtained after the iteration termination corresponds to a scene.

[0077] For example, according to the redefined partition P1, each community in the second community set is taken as a vertex in the new graph, and the edges between communities are inherited to form a new simplified graph G', such as Figure 4 As shown in (d). By taking each community in the second community set as a vertex and iterating iteratively, some related communities can be aggregated into one community.

[0078] Then, if Figure 4 As shown in (d) to (f), iterate steps S21 and S22, loop merging vertices (communities) and redefining communities, thereby updating the first community set and the second community set until the iteration termination condition is reached. The second community set obtained in the last iteration is used as the final partition result. In the partition result obtained, the categories within a community are highly correlated.

[0079] In some embodiments, the iteration termination condition includes at least one of the following: no communities can be merged, and moving any vertex cannot increase the quality function.

[0080] Compared with other community discovery algorithms (such as the Louvain algorithm), the present disclosure adds an additional redefinition process, that is, adds a step of redividing the community based on the quality function of a single community, so that the community differentiation is more reasonable. In particular, for marginal categories, after adding the redefinition step, the quality of the divided community is higher, and even if the marginal category in the community is removed, the community can be prevented from becoming a non-connected graph.

[0081] In step S3, text features corresponding to the target scene are determined based on the categories in the community corresponding to the target scene and the common attributes of the categories.

[0082] The text features corresponding to the target scene are, for example, keywords of categories and common attributes of categories. For categories and common attributes of categories in the community corresponding to the target scene, based on user portraits and item titles, the text features of common attributes (such as the user's city, points, level, etc.) and item features (such as Christmas gifts, cream style, early education, etc.) are associated.

[0083] In some embodiments, candidate text features corresponding to the target scene are determined based on categories in the community corresponding to the target scene and common attributes of the categories, including: selecting text features corresponding to the target scene from the candidate text features based on at least one of an attribution value of the candidate text feature, a target group index of the candidate text feature, and a proportion of users in the target group having the candidate text feature to all users of the target group, wherein the target group represents a user group associated with the categories in the community corresponding to the target scene, and the attribution value of the text feature represents a contribution of the text feature to the prediction result when the machine learning model predicts whether a user will select an item in the target scene based on the text feature.

[0084] For example, if there are too many candidate text features for categories or common attributes of categories in the community, the candidate texts can be screened first.

[0085] Calculating the attribution value of the candidate text features is a method based on model interpretation. For the user group U and item I in any scenario, the model interpretation method is used to calculate the difference between the features of the user group U and item I in reflecting the characteristics of the scenario and the overall characteristics.

[0086] Machine learning models include random forests, gradient boosted tree (GBDT), etc. By constructing training samples, positive samples are taken as orders under the target scenario, recorded as y=1, and negative samples are randomly sampled orders that are not in the scenario, recorded as y=0. The orders of the training samples are associated with the candidate text features X, and the f(X)~y problem is transformed into a binary classification problem based on the candidate text features to predict the scene corresponding to the order, and then solved.

[0087] The SHAP (Shapley Additive Explanations) model can measure the impact of candidate text features (user profiles, item profiles) on the scene. SHAP is an additive linear attribution method that combines the local explanation model LIME (Local Interpretable Model-Agnostic Explanations) and Shapley value. For each training sample, the Shapley value of the feature is defined as follows.

[0088]

[0089] Among them, g(z′) is the explanatory model, φ j ∈R is the feature attributed Shapley value of feature j, M is the number of features, g(z′) represents the conversion probability of users in this scenario, and φ0 represents the basic conversion probability of all users. For example, for an outdoor scenario, item feature A is camping. Through the Shapley value, φ1 is calculated to be 0.03, which means that camping plays a positive role in the prediction of the machine learning model in the outdoor scenario, and its importance to the prediction is 0.03.

[0090] The SHAP model is used to explain the global importance of the j-th feature to the previously trained machine learning model. That is, the absolute value of Shapley of all samples of each feature is averaged. The calculation formula is as follows.

[0091]

[0092] Among them, n is the number of samples, and i represents the i-th sample.

[0093] According to the descending order of the global importance of each feature, a list of features with the most significant distinction between the target scene and other scenes is obtained. The attribution value has good validity, symmetry, additivity, local accuracy, and consistency. By adding the attribution value of the feature as an evaluation factor for the candidate text feature, the interpretability of the model is improved, which can help find more important features and improve the accuracy of the subsequent generated scene description.

[0094] In some embodiments, the target group index (TGI) of the candidate text feature is calculated using the following formula.

[0095]

[0096] The overall refers to all users, and the target group refers to the user group associated with the category in the community corresponding to the target scenario, for example, users who select items in the category of the community corresponding to the target scenario.

[0097] The calculation formula for the proportion of users in the target group having the candidate text features to all users in the target group is as follows.

[0098]

[0099] The TGI index represents whether the feature is important in the target scenario, and the feature user ratio reflects the relative quantity requirement of the number of users with the feature. Both indexes are between 0-100. In some embodiments, the harmonic mean of TGI and feature user ratio is taken as the evaluation index for sorting to select the feature text that best suits the scenario.

[0100] In some embodiments, the candidate feature texts with the values ​​of the above-mentioned evaluation indicators and the attribution values ​​ranked first are taken as the feature texts corresponding to the target scene, that is, the key features of the target scene, so as to enhance the robustness of the model.

[0101] In step S4, a description text of the target scene is generated according to the text features corresponding to the target scene. The text features corresponding to the target scene are all features highly associated with the target scene, and the description text of the target scene can be obtained according to these text features.

[0102] In some embodiments, a description text of the target scene is generated based on text features corresponding to the target scene, including: determining keywords of the designated object based on the description text of the designated object of the target scene; and generating a description text of the target scene based on the text features corresponding to the target scene and the keywords of the designated object.

[0103] The designated item of the target scene is, for example, an item of high interest or an item selected by an important user, and the description text of the designated item is, for example, the title of the designated item. A keyword extraction algorithm is used to determine the keywords of the designated item. The keyword extraction algorithm is, for example, a summary (TextRank) algorithm, a term frequency-inverse document frequency (TF-IDF) algorithm, or a point mutual information (PMI) algorithm.

[0104] The following uses the TextRank algorithm as an example to describe how to determine the keywords for a specified item.

[0105] Perform word segmentation on the description text of the specified item, remove stop words, take each word as a vertex, take the adjacent words after word segmentation as edges, take the attribute of each specified item (such as the number of views) as the weight, and sum the edges that appear repeatedly in the titles of multiple items as the final weight. Then use the RextRank algorithm to calculate the importance of each word:

[0106]

[0107] Where d is the damping coefficient, usually 0.85, Vi is a word (i.e., a vertex in the graph), w jk In(V i ) is the pointing word vertex V i The collection of Out(V j ) is the word vertex V j The set of outward-pointing vertices, WS(V i ) is the word V i The calculated RextRank score. According to the descending order of the word's textrank score, get the keywords of the specified item.

[0108] In some embodiments, the description text of the target scene is generated based on the intersection of the text features corresponding to the target scene and the keywords of the specified object; or the description text of the target scene is generated based on the union of the text features corresponding to the target scene and the keywords of the specified object. For example, the words after the intersection or union are taken as the keywords of the target scene.

[0109] In some embodiments, a description text of the target scene is generated using a text generation model based on text features corresponding to the target scene and keywords of the specified object.

[0110] For example, a pre-trained model (e.g., Encoder-Decoder, LanguageModel) is selected, and the intersection or union of the text features corresponding to the target scene and the keywords of the specified object are used as the input of the model to generate a coherent description that summarizes the characteristics of the scene, the people it is suitable for, and the objects in the scene.

[0111] During training, the pre-trained model is fine-tuned based on a small amount of pre-labeled sample data (including keywords and descriptions of the target scene), with the keywords of each sentence as the model input and the description text itself as the model output.

[0112] The following takes the model training structure as the Encoder-Decoder structure as an example for introduction.

[0113] Figure 5 A schematic diagram of an encoder-decoder according to some embodiments of the present disclosure is shown.

[0114] like Figure 5 As shown, feature extraction adopts the Transformer structure.

[0115] Figure 6 A converter structure according to some embodiments of the present disclosure is shown.

[0116] like Figure 6The transformer shown includes normalized exponential layers (softmax), linear layers, summation and normalization layers, feedforward neural network layers, multi-head self-attention layers, positional encoding, input and output embedding layers, etc.

[0117] The loss function uses the negative log-likelihood function of the masked words. The set of masked words is M, which is transformed into a multi-classification problem on a dictionary (size is |V|). The loss function is shown below.

[0118]

[0119] Among them, θ is the parameter of the encoder, θ1 is the parameter of the output layer connected to the encoder, and m i Represents the i-th word.

[0120] When generating a scene description, based on the fine-tuned model, keywords (text features corresponding to the target scene, keywords of specified objects, or the intersection or union of the two) are input to obtain a comprehensive description of the scene.

[0121] Figure 7 A schematic diagram of generating scene description text according to some embodiments of the present disclosure is shown.

[0122] like Figure 7 As shown, first, a category relationship diagram is constructed, and then, through community division, a scenario-based category set is generated; statistical methods and explanatory models are used to select text features corresponding to the target scenario; based on the text features corresponding to the target scenario and the keyword extraction method, the scenario definition is obtained; and the scenario definition is used to generate the scenario description text.

[0123] According to the method for generating description text of a scene in some embodiments of the present disclosure, the relationship network of the categories of objects is used; the relationship network is divided into multiple communities; the text features corresponding to the target scene are determined according to the categories in the community corresponding to the target scene and the common attributes of the categories; and the description text of the target scene is generated according to the text features corresponding to the target scene. The present disclosure establishes a correspondence between scenes and communities by dividing the relationship network into multiple communities, thereby constructing the association relationship between multiple categories under the target scene, and finding the categories and attributes that are most closely related to the target scene. Then, the target text features are determined using the categories and attributes in the community corresponding to the target scene, thereby generating a description text of the target scene.

[0124] Compared with only constructing pairwise associations between categories, the category associations obtained by the community segmentation method disclosed in the present invention are more comprehensive and the degree of association between categories is higher, so that the target text features found are more closely associated with the target scene, thereby making the description text of the target scene finally generated more comprehensive and accurate, which is conducive to identifying the segmented user groups and item style attributes suitable for the target scene, and providing a guarantee for subsequent user analysis and item recommendation.

[0125] Figure 8 A block diagram showing a device for generating a description text of a scene according to some embodiments of the present disclosure.

[0126] like Figure 8 As shown, the device 8 for generating a description text of a scene includes an acquisition module 81 , a division module 82 , a determination module 83 , and a generation module 84 .

[0127] The acquisition module 81 is configured to acquire a relationship network of categories of items, wherein the vertices in the relationship network represent categories, and the edges connecting two categories represent the common attributes of the two categories. For example, Figure 1 Step S1 is shown.

[0128] The division module 82 is configured to divide the relationship network into a plurality of communities, wherein the categories in each community are related to each other, and at least one community corresponds to a scenario, for example, Figure 1 Step S2 is shown.

[0129] The determination module 83 is configured to determine the text features corresponding to the target scene according to the categories in the community corresponding to the target scene and the common attributes of the categories, for example, by performing the following steps: Figure 1 Step S3 shown.

[0130] The generating module 84 is configured to generate a description text of the target scene according to the text features corresponding to the target scene, for example, by executing Figure 1 Step S4 is shown.

[0131] In some embodiments, the partitioning module 82 is further configured to partition the vertices of the relationship network based on the quality function of the relationship network to obtain a first community set; for at least one community in the first community set, based on the quality function of each community in at least one community in the first community set, re-partition the vertices in each community to obtain a second community set, wherein the second community set includes the re-partitioned communities and the communities in the first community set that have not been re-partitioned; take each community in the second community set as a vertex, iteratively execute to obtain the first community set and the second community set, until the iteration termination condition is reached, wherein at least one community in the second community set obtained after the iteration termination corresponds to a scene.

[0132] In some embodiments, the partitioning module 82 is further configured to: the first set is represented as {community A, community B, community C, ...}; for at least one community in the first community set, based on the quality function of each community in at least one community in the first community set, re-partition the vertices in each community to obtain a second community set, including: any community C in at least one community in the first community set includes vertices (v1, v2, v3, ...); each vertex in community C is a subcommunity, and community C includes {subcommunity C1, subcommunity C2, ...}; traverse the vertices in community C; for vertex v in community C, j and vertex v j Connected subcommunity C j , before merging the vertex into subcommunity C j After that, if the value of the quality function of community C increases, the vertex is merged into subcommunity C j , the multiple subcommunities obtained after the traversal are taken as the communities in the second community set, where the larger the value of the quality function is, the higher the association between the vertices in the community C is, and i and j are natural numbers.

[0133] In some embodiments, the partitioning module 82 is further configured to divide the vertex v into j Merge into subcommunity C of community C j After that, if the quality function of community C does not increase, move vertex v j Remain in the subcommunity to which it originally belonged.

[0134] In some embodiments, the quality function of community C is calculated according to the sum of the weights of the internal edges of each subcommunity of community C and the sum of the degrees of the vertices of each subcommunity.

[0135] In some embodiments, the partitioning module 82 is further configured to take each vertex in the relationship network as a community, traverse the vertices in the relationship network, and after merging the vertex into the community connected to the vertex, if the quality function of the relationship network increases, merge the vertex into the community connected to the vertex to obtain a first community set.

[0136] In some embodiments, the quality function of the relationship network is calculated according to the sum of the weights of the internal edges of each community in the relationship network and the sum of the degrees of the internal vertices of each community.

[0137] In some embodiments, the determination module 83 is further configured to: determine the candidate text features corresponding to the target scene based on the categories in the community corresponding to the target scene and the common attributes of the categories; select the text features corresponding to the target scene from the candidate text features based on at least one of the attribution value of the candidate text features, the target group index of the candidate text features, and the proportion of users in the target group with the candidate text features to all users of the target group, wherein the target group represents the user group associated with the category in the community corresponding to the target scene, and the attribution value of the text feature represents the contribution of the text feature to the prediction result when the machine learning model predicts whether the user will choose an item in the target scene based on the text feature.

[0138] In some embodiments, the generation module 84 is further configured to: determine keywords of the designated object according to the description text of the designated object in the target scene; and generate description text of the target scene according to text features corresponding to the target scene and the keywords of the designated object.

[0139] In some embodiments, the generation module 84 is further configured to generate a description text of the target scene using a text generation model according to text features corresponding to the target scene and keywords of the specified object.

[0140] In some embodiments, the generation module 84 is further configured to: generate a description text of the target scene according to the intersection of the text features corresponding to the target scene and the keywords of the specified object; or generate a description text of the target scene according to the union of the text features corresponding to the target scene and the keywords of the specified object.

[0141] Fig. 9 A block diagram showing a device for generating a description text of a scene according to some other embodiments of the present disclosure.

[0142] like Fig. 9 As shown, the scene description text generation device 9 includes a memory 91; and a processor 92 coupled to the memory 91, and the memory 91 is used to store and execute the scene description text generation method. The processor 92 is configured to execute the scene description text generation method in any of the embodiments of the present disclosure based on the instructions stored in the memory 91.

[0143] Fig.10 A block diagram of a computer system for implementing some embodiments of the present disclosure is shown.

[0144] like Fig.10 As shown, the computer system 100 may be embodied in the form of a general-purpose computing device. The computer system 100 includes a memory 1010, a processor 1020, and a bus 1000 that connects various system components.

[0145] The memory 1010 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions for executing a method for generating a description text of a scene in any of the embodiments of the present disclosure. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, etc.

[0146] The processor 1020 can be implemented by a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, etc. Discrete hardware components. Accordingly, each module such as the judgment module and the determination module can be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or can be implemented by a dedicated circuit that executes corresponding steps.

[0147] The bus 1000 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0148] The computer system 100 may also include an input / output interface 1030, a network interface 1040, a storage interface 1050, etc. These interfaces 1030, 1040, 1050, the memory 1010, and the processor 1020 may be connected via a bus 1000. The input / output interface 1030 may provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 1040 may provide a connection interface for various networked devices. The storage interface 1050 may provide a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0149] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products for generating description texts of scenes according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0150] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0151] These computer-readable program instructions can also be readable and stored in a computer-readable memory, and these instructions cause the computer to work in a specific manner, thereby producing a product, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0152] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0153] By means of the method and device for generating a description text of a scene and the computer-readable storage medium in the above-mentioned embodiments, a text that accurately describes a target scene can be generated.

[0154] So far, the method and device for generating the description text of the scene according to the present disclosure, and the computer-readable storage medium have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

Claims

1. A method for generating a description text of a scene, comprising: Obtain a relationship network of the categories of items, where the vertices in the relationship network represent the categories, and the edges connecting two categories represent the common attributes of the two categories; Divide the relationship network into multiple communities, where the categories in each community are interconnected, and at least one community corresponds to one scenario; Determine the text features corresponding to the target scenario based on the categories in the community corresponding to the target scenario and the common attributes of the categories; Generate a description text of the target scene based on the text features corresponding to the target scene.

2. The method for generating a description text of a scene according to claim 1, wherein: Divide the relationship network into multiple communities, including: Based on the quality function of the relationship network, the vertices of the relationship network are divided to obtain the first community set; For at least one community in the first community set, based on the quality function of each community in at least one community in the first community set, re-dividing the vertices in each community to obtain a second community set, wherein the second community set includes the re-divided communities and the communities in the first community set that are not re-divided; Each community in the second community set is taken as a vertex, and the first community set and the second community set are obtained by iteration until an iteration termination condition is reached, wherein at least one community in the second community set obtained after the iteration termination corresponds to a scene.

3. The method for generating a description text of a scene according to claim 2, wherein: The first set is represented as {community A, community B, community C, ...}, and for at least one community in the first community set, based on the quality function of each community in at least one community in the first community set, the vertices in each community are re-divided to obtain a second community set, including: Any community C in at least one community in the first community set includes vertices (v1, v2, v3, ...), each vertex in community C is a subcommunity, and community C includes {subcommunity C1, subcommunity C2, ...}. For vertex v in community C, traverse the vertices in community C. j and vertex v j Connected subcommunity C j , before merging the vertex into subcommunity C j After that, if the value of the quality function of community C increases, the vertex is merged into subcommunity C j , the multiple subcommunities obtained after the traversal are taken as the communities in the second community set, where the larger the value of the quality function is, the higher the association between the vertices in the community C is, and i and j are natural numbers.

4. The method for generating a description text of a scene according to claim 3, wherein: In the vertex v j Merge into subcommunity C of community C j After that, if the quality function of community C does not increase, move vertex v j Remain in the subcommunity to which it originally belonged.

5. The method for generating a description text of a scene according to claim 3, wherein: The quality function of community C is calculated based on the sum of the weights of the internal edges of each subcommunity of community C and the sum of the degrees of the vertices of each subcommunity.

6. The method for generating a description text of a scene according to claim 2, wherein: Based on the quality function of the relationship network, the vertices of the relationship network are divided to obtain the first community set, including: Each vertex in the relationship network is regarded as a community. The vertices in the relationship network are traversed. After the vertex is merged into the community connected to the vertex, if the quality function of the relationship network increases, the vertex is merged into the community connected to the vertex to obtain a first community set.

7. The method for generating a description text of a scene according to claim 2, wherein: The quality function of the relationship network is calculated based on the sum of the weights of the internal edges of each community in the relationship network and the sum of the degrees of the internal vertices of each community.

8. The method for generating a description text of a scene according to claim 1, wherein: According to the categories in the community corresponding to the target scenario and the common attributes of the categories, the text features corresponding to the target scenario are determined, including: Determine candidate text features corresponding to the target scenario based on the categories in the community corresponding to the target scenario and the common attributes of the categories; A text feature corresponding to the target scene is selected from the candidate text features based on at least one of the attribution value of the candidate text feature, the target group index of the candidate text feature, and the proportion of users in the target group having the candidate text feature to all users in the target group, wherein the target group represents a user group associated with a category in a community corresponding to the target scene, and the attribution value of the text feature represents a contribution of the text feature to the prediction result when the machine learning model predicts whether a user will select an item in the target scene based on the text feature.

9. The method for generating a description text of a scene according to claim 1, wherein: Generate a description text of the target scene based on the text features corresponding to the target scene, including: Determine keywords of the specified items according to the description text of the specified items in the target scene; Generate a description text of the target scene based on the text features corresponding to the target scene and the keywords of the specified items.

10. The method for generating a description text of a scene according to claim 9, wherein: Generate a description text of the target scene based on the text features corresponding to the target scene and the keywords of the specified object, including: According to the text features corresponding to the target scene and the keywords of the specified objects, the text generation model is used to generate the description text of the target scene.

11. The method for generating a description text of a scene according to claim 9, wherein: Generate a description text of the target scene based on the text features corresponding to the target scene and the keywords of the specified object, including: Generate a description text of the target scene based on the intersection of the text features corresponding to the target scene and the keywords of the specified object; or Generate a description text of the target scene based on the union of the text features corresponding to the target scene and the keywords of the specified object.

12. A device for generating a description text of a scene, comprising: An acquisition module is configured to acquire a relationship network of categories of items, wherein vertices in the relationship network represent categories, and an edge connecting two categories represents a common attribute of the two categories; a partitioning module configured to partition the relationship network into a plurality of communities, wherein the categories within each community are interconnected, and at least one community corresponds to one scenario; A determination module is configured to determine text features corresponding to the target scenario based on categories in the community corresponding to the target scenario and common attributes of the categories; The generation module is configured to generate a description text of the target scene according to text features corresponding to the target scene.

13. A device for generating a description text of a scene, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method for generating a description text of a scene according to any one of claims 1 to 11 based on instructions stored in the memory.

14. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements the method for generating a description text of a scene according to any one of claims 1 to 11.

Images